Re: [dpdk-dev] l3fwd error, port 0 is not present on the board

[Neil Horman <nhorman@tuxdriver.com>]

On Wed, Dec 31, 2014 at 10:50:10AM -0600, Lyn M wrote:
> Neil,
> When I build DPDK 1.8.0 with the default value CONFIG_RTE_BUILD_COMBINE_LIBS=n,
> I am able to run the testpmd successfully.  So, that does appear to be the
> culprit.
> The objdump -t output for testpmd (with COMBINE_LIBS=n and COMBINE_LIBS=y)
> files are here:
> https://www.dropbox.com/sh/6os9yfn22jjcnmy/AAAjEV3u06XQ1t3UXdWcgru0a?dl=0
> 
> Thanks,
> Lyn
> 
I see the problem.  none of the constructors are included in the l3fwd app.  I
think you might need 174572477028b7f04700389e38f87d2ba01a0adc.  Its also
possible something else has gone wrong withthe COMBINE_LIBS option.  Try
building with the head of the tree, and if that doesn't fix it, we can start
debugging in earnest.
Neil

> On Wed, Dec 31, 2014 at 8:48 AM, Neil Horman <nhorman@tuxdriver.com> wrote:
> 
> > On Wed, Dec 31, 2014 at 08:18:34AM -0600, Lyn M wrote:
> > > Neil, to build DPDK, I am doing the following steps:
> > > 1) download DPDK tarball (1.7.1 or 1.8.0) from
> > > http://dpdk.org/browse/dpdk/refs/
> > > 2) untar
> > > 3) edit config/common_linuxapp and change CONFIG_RTE_BUILD_COMBINE_LIBS
> > > from "n" to "y"
> > > 4) follow all of the steps in http://dpdk.org/doc/quick-start
> > >
> > Hmm, I wonder if you're seeing a problem in which the constructors don't
> > get
> > linked in properly.  Does it work if you don't specify
> > CONFIG_RTE_BUILD_COMBINE_LIBS?  I.e. if you build several .a files?
> >
> > Can you objdump -t the resultant binary with and without COMBINE_LIBS and
> > post
> > it?
> >
> > Neil
> >
> > > When I do those steps for DPDK 1.7.1, the output of testpmd is OK, and I
> > > see both of my NICs that are bound to igb_uio.
> > > When I do those steps for DPDK 1.8.0, the output of testpmd is:
> > > EAL: Error - exiting with code: 1
> > >   Cause: No probed ethernet device
> > >
> > > I see the same result when I use the setup.sh script instead of manually
> > > doing the steps in http://dpdk.org/doc/quick-start
> > >
> > > Can you please show me an example of enabling the pmd init debug macros?
> > >
> > > Lyn
> > >
> > > On Tue, Dec 30, 2014 at 3:24 PM, Neil Horman <nhorman@tuxdriver.com>
> > wrote:
> > >
> > > > On Tue, Dec 30, 2014 at 12:54:17PM -0600, Lyn M wrote:
> > > > > Neil, thank you for your reply.
> > > > >
> > > > > I used the DPDK 8.1.0 setup.sh script to clean the system (unbind
> > NICs,
> > > > > rmmod igb_uio and uio, remove hugepage mappings), and then re-built
> > for
> > > > > x86_64-native-linuxapp-gcc, setup hugepages, insmod the modules,
> > bound
> > > > the
> > > > > NICs, but have the same result when trying to run testpmd:
> > > > > EAL: Error - exiting with code: 1
> > > > >   Cause: No probed ethernet device
> > > > >
> > > > >
> > > > > I then cleaned up again, downloaded DPDK 1.7.1 and ran the setup.sh
> > > > script
> > > > > to build for x86_64-native-linuxapp-gcc.  With 1.7.1, testpmd runs
> > > > > successfully, and I am able to run my l3fwd command as well. I also
> > built
> > > > > for x86_64-ivshmem-linuxapp-gcc, and testpmd ran successfully (other
> > than
> > > > > the warning about no IVSHMEM configuration).
> > > > >
> > > > > Thoughts?
> > > > >
> > > > Well, the root cause is still the same, DPDK isn't detecting any
> > devices
> > > > over
> > > > which to forward traffic, which means the nb_port count is zero.  That
> > > > would
> > > > suggest that you didn't find any devices during rte_eal_init.  The
> > > > question is
> > > > why. I'd turn on all the pmd init debug macros to see if all the init
> > > > routines
> > > > are getting called.
> > > >
> > > > Two questions: (1) are you doing a static or DSO build, and (2) did you
> > > > make any
> > > > changes to the make flags?
> > > >
> > > > Neil
> > > >
> > > > > On Mon, Dec 29, 2014 at 3:02 PM, Neil Horman <nhorman@tuxdriver.com>
> > > > wrote:
> > > > >
> > > > > > On Mon, Dec 29, 2014 at 02:18:09PM -0600, Lyn M wrote:
> > > > > > > Hello all,
> > > > > > >
> > > > > > > Rangeley platform (1 CPU with 8 cores, 4x1 GbE, 2x10 GbE, Intel
> > PCIe
> > > > card
> > > > > > > with 2x10 GbE fiber)
> > > > > > > FC20 with hugepages enabled
> > > > > > > DPDK 1.8.0 built with T=x86_64-ivshmem-linuxapp-gcc
> > > > > > >
> > > > > > >  */usr/src/dpdk/tools/dpdk_nic_bind.py --status*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Network devices using DPDK-compatible driver
> > > > > > >
> > > > > > > ============================================
> > > > > > >
> > > > > > > 0000:09:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
> > > > drv=igb_uio
> > > > > > > unused=ixgbe
> > > > > > >
> > > > > > > 0000:09:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
> > > > drv=igb_uio
> > > > > > > unused=ixgbe
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Network devices using kernel driver
> > > > > > >
> > > > > > > ===================================
> > > > > > >
> > > > > > > 0000:00:14.0 'Ethernet Connection I354' if=enp0s20f0 drv=igb
> > > > > > unused=igb_uio
> > > > > > >
> > > > > > > 0000:00:14.1 'Ethernet Connection I354' if=enp0s20f1 drv=igb
> > > > > > unused=igb_uio
> > > > > > >
> > > > > > > 0000:00:14.2 'Ethernet Connection I354' if=enp0s20f2 drv=igb
> > > > > > unused=igb_uio
> > > > > > >
> > > > > > > 0000:00:14.3 'Ethernet Connection I354' if=enp0s20f3 drv=igb
> > > > > > unused=igb_uio
> > > > > > >
> > > > > > > 0000:07:00.0 'Ethernet Controller 10-Gigabit X540-AT2' if=p5p1
> > > > drv=ixgbe
> > > > > > > unused=igb_uio *Active*
> > > > > > >
> > > > > > > 0000:07:00.1 'Ethernet Controller 10-Gigabit X540-AT2' if=p5p2
> > > > drv=ixgbe
> > > > > > > unused=igb_uio
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Other network devices
> > > > > > >
> > > > > > > =====================
> > > > > > >
> > > > > > > <none>
> > > > > > >
> > > > > > > The DPDK helloworld example seems to run OK, other than the
> > warning
> > > > about
> > > > > > > IVSHMEM configuration not found:
> > > > > > > *./helloworld -c 0x3 -n 4*
> > > > > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > > > > EAL: Detected lcore 1 as core 1 on socket 0
> > > > > > > EAL: Detected lcore 2 as core 2 on socket 0
> > > > > > > EAL: Detected lcore 3 as core 3 on socket 0
> > > > > > > EAL: Detected lcore 4 as core 4 on socket 0
> > > > > > > EAL: Detected lcore 5 as core 5 on socket 0
> > > > > > > EAL: Detected lcore 6 as core 6 on socket 0
> > > > > > > EAL: Detected lcore 7 as core 7 on socket 0
> > > > > > > EAL: Support maximum 128 logical core(s) by configuration.
> > > > > > > EAL: Detected 8 lcore(s)
> > > > > > > EAL: Searching for IVSHMEM devices...
> > > > > > > EAL: No IVSHMEM configuration found!
> > > > > > > EAL: Setting up memory...
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcca2c00000 (size = 0x200000)
> > > > > > > EAL: Ask a virtual area of 0x29400000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcc79600000 (size = 0x29400000)
> > > > > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcca3400000 (size = 0x400000)
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcca3000000 (size = 0x200000)
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcc79200000 (size = 0x200000)
> > > > > > > EAL: Ask a virtual area of 0x14d000000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcb2c000000 (size = 0x14d000000)
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > > EAL: Virtual area found at 0x7fcb2bc00000 (size = 0x200000)
> > > > > > > EAL: Requesting 3000 pages of size 2MB from socket 0
> > > > > > > EAL: TSC frequency is ~2393997 KHz
> > > > > > > EAL: Master core 0 is ready (tid=a4ca2840)
> > > > > > > EAL: Core 1 is ready (tid=2bbff700)
> > > > > > > hello from core 1
> > > > > > > hello from core 0
> > > > > > >
> > > > > > >
> > > > > > > I want to run the l3fwd example app, per Appendix A.2 (pg 35-36)
> > in
> > > > the
> > > > > > > DPDK 1.2 Release Benchmark Test Report .pdf:
> > > > > > > *./build/l3fwd -c 0x6 -n 4 --socket-mem 1024,0 -- -p 0x3
> > > > > > > -config="(0,0,1),(1,0,2)"*
> > > > > > > where the "-p option is the hexadecimal bit mask of the ports to
> > > > > > configure"
> > > > > > >
> > > > > > > I am running this command:
> > > > > > >
> > > > > > > *./build/l3fwd -c 0x3 -n 4 --socket-mem 1024,0 -- -p 0x3
> > > > > > > --config="(0,0,0),(1,0,1)" --no-numa*
> > > > > > >
> > > > > > > EAL: Detected lcore 0 as core 0 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 1 as core 1 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 2 as core 2 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 3 as core 3 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 4 as core 4 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 5 as core 5 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 6 as core 6 on socket 0
> > > > > > >
> > > > > > > EAL: Detected lcore 7 as core 7 on socket 0
> > > > > > >
> > > > > > > EAL: Support maximum 128 logical core(s) by configuration.
> > > > > > >
> > > > > > > EAL: Detected 8 lcore(s)
> > > > > > >
> > > > > > > EAL: Searching for IVSHMEM devices...
> > > > > > >
> > > > > > > EAL: No IVSHMEM configuration found!
> > > > > > >
> > > > > > > EAL: Setting up memory...
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f3212600000 (size = 0x200000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x29400000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f31e9000000 (size = 0x29400000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x400000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f3212e00000 (size = 0x400000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f3212a00000 (size = 0x200000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f31e8c00000 (size = 0x200000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x14d000000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f309ba00000 (size = 0x14d000000)
> > > > > > >
> > > > > > > EAL: Ask a virtual area of 0x200000 bytes
> > > > > > >
> > > > > > > EAL: Virtual area found at 0x7f309b600000 (size = 0x200000)
> > > > > > >
> > > > > > > EAL: Requesting 512 pages of size 2MB from socket 0
> > > > > > >
> > > > > > > EAL: TSC frequency is ~2393998 KHz
> > > > > > >
> > > > > > > EAL: Master core 0 is ready (tid=14734840)
> > > > > > >
> > > > > > > EAL: Core 1 is ready (tid=e8bff700)
> > > > > > >
> > > > > > > numa is disabled
> > > > > > >
> > > > > > > port 0 is not present on the board
> > > > > > >
> > > > > > > EAL: Error - exiting with code: 1
> > > > > > >
> > > > > > >   Cause: check_port_config failed
> > > > > > >
> > > > > > > Perhaps the 0x3 (0011) hex portmask is incorrect?  What Linux
> > > > command, or
> > > > > > > combination of commands, would help show the appropriate hex
> > > > portmask to
> > > > > > > use in my case?
> > > > > > >
> > > > > > > Thanks for your time,
> > > > > > > Lyn
> > > > > > >
> > > > > > I don't see the detection of any of your physical devices here.
> > I'm
> > > > > > guessing
> > > > > > your setup is just looking for ivshmem devices, which you won't
> > have
> > > > any
> > > > > > of in
> > > > > > FC20 unless you rebuild qemu.  As such, dpdk doesn't see any ports,
> > > > which
> > > > > > is why
> > > > > > you're getting the error the results from portid 0 being greater
> > than
> > > > the
> > > > > > number
> > > > > > of detected ports on the system
> > > > > >
> > > > > > neil
> > > > > >
> > > > > >
> > > >
> >