[dpdk-users] Issue moving to DPDK 20.11 release

[dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar]

Hi All,

Previously, I had DPDK-19.11 working fine on my server system, which runs
on Ubuntu 18.04.
Have a 4-Port 10 Gigabit Network card, with ports bound to use VFIO-PCI
driver.
But had to move to DPDK-20.11, and running into issues.

I installed on DPDK-20.11 on the server to /usr/local.

And did binding of ports using the dpdk-devbind script to use VFIO-PCI
driver.

But when running DPDK sample application such as->
dpdk-20.11/examples/packet_ordering,
noticed DPDK is NOT able to detect any available ports.
Getting error->  Cause: Error: no ethernet ports detected

Also when I pass parameter to block certain interfaces(Bus:Device.Function)
on a NIC, EAL throws an
Error-> EAL: failed to parse device "0000:31.00.0"

On the same system, using the same 10 Gigbit Network card, DPDK sample
applications were
working fine with older version of DPDK->19.11

Below, I have the output of commands:

~/dpdk-20.11$ sudo ./usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:31:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
unused=i40e
0000:31:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
unused=i40e
0000:31:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
unused=i40e
0000:31:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
unused=i40e

Network devices using kernel driver
===================================
0000:01:00.0 'I210 Gigabit Network Connection 1533' if=enp1s0 drv=igb
unused=vfio-pci *Active*
0000:17:00.0 'Ethernet Controller 10G X550T 1563' if=ens6f0 drv=ixgbe
unused=vfio-pci *Active*
0000:17:00.1 'Ethernet Controller 10G X550T 1563' if=ens6f1 drv=ixgbe
unused=vfio-pci *Active*


cat /proc/meminfo | grep -i huge
AnonHugePages:      8192 kB
ShmemHugePages:        0 kB
HugePages_Total:    2048
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

:~/dpdk-20.11/examples/packet_ordering$ sudo ./build/packet_ordering-shared
-l 4-6 -n 2 --log-level 7 -m 1024
--file-prefix packet_order -- -p 0xf --disable-reorder
EAL: Detected 72 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/packet_order/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: No legacy callbacks, legacy socket not created
reorder disabled
EAL: Error - exiting with code: 1
  Cause: Error: no ethernet ports detected

~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
-n 2 --proc-type auto --log-level 7 -m 1024
--file-prefix packet_ordering -b 0000:31:00.0 -b 0000:31:00.1
-b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p 0xf
--disable-reorder
EAL: Detected 72 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: failed to parse device "0000:31:00.0"
EAL: Unable to parse device '0000:31:00.0'
EAL: Error - exiting with code: 1
  Cause: Invalid EAL arguments


Please let me know, if you have any suggestions.


Thanks,

Sudha

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Stephen Hemminger]

On Wed, 26 May 2021 10:27:56 -0700
Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:

> Hi All,
> 
> Previously, I had DPDK-19.11 working fine on my server system, which runs
> on Ubuntu 18.04.
> Have a 4-Port 10 Gigabit Network card, with ports bound to use VFIO-PCI
> driver.
> But had to move to DPDK-20.11, and running into issues.
> 
> I installed on DPDK-20.11 on the server to /usr/local.
> 
> And did binding of ports using the dpdk-devbind script to use VFIO-PCI
> driver.
> 
> But when running DPDK sample application such as->
> dpdk-20.11/examples/packet_ordering,
> noticed DPDK is NOT able to detect any available ports.
> Getting error->  Cause: Error: no ethernet ports detected
> 
> Also when I pass parameter to block certain interfaces(Bus:Device.Function)
> on a NIC, EAL throws an
> Error-> EAL: failed to parse device "0000:31.00.0"
> 
> On the same system, using the same 10 Gigbit Network card, DPDK sample
> applications were
> working fine with older version of DPDK->19.11
> 
> Below, I have the output of commands:
> 
> ~/dpdk-20.11$ sudo ./usertools/dpdk-devbind.py --status
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:31:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> unused=i40e
> 0000:31:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> unused=i40e
> 0000:31:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> unused=i40e
> 0000:31:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> unused=i40e
> 
> Network devices using kernel driver
> ===================================
> 0000:01:00.0 'I210 Gigabit Network Connection 1533' if=enp1s0 drv=igb
> unused=vfio-pci *Active*
> 0000:17:00.0 'Ethernet Controller 10G X550T 1563' if=ens6f0 drv=ixgbe
> unused=vfio-pci *Active*
> 0000:17:00.1 'Ethernet Controller 10G X550T 1563' if=ens6f1 drv=ixgbe
> unused=vfio-pci *Active*
> 
> 
> cat /proc/meminfo | grep -i huge
> AnonHugePages:      8192 kB
> ShmemHugePages:        0 kB
> HugePages_Total:    2048
> HugePages_Free:     2048
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> 
> :~/dpdk-20.11/examples/packet_ordering$ sudo ./build/packet_ordering-shared
> -l 4-6 -n 2 --log-level 7 -m 1024
> --file-prefix packet_order -- -p 0xf --disable-reorder
> EAL: Detected 72 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/packet_order/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: No legacy callbacks, legacy socket not created
> reorder disabled
> EAL: Error - exiting with code: 1
>   Cause: Error: no ethernet ports detected
> 
> ~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
> -n 2 --proc-type auto --log-level 7 -m 1024
> --file-prefix packet_ordering -b 0000:31:00.0 -b 0000:31:00.1
> -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p 0xf
> --disable-reorder
> EAL: Detected 72 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Auto-detected process type: PRIMARY
> EAL: failed to parse device "0000:31:00.0"
> EAL: Unable to parse device '0000:31:00.0'
> EAL: Error - exiting with code: 1
>   Cause: Invalid EAL arguments
> 
> 
> Please let me know, if you have any suggestions.
> 
> 
> Thanks,
> 
> Sudha

By default, DPDK is now dynamically linked and all the drivers are in shared libraries.
You need to pass the -d flag to indicate which library to load.

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar]

Hi Stephen, Thanks for your reply, I was not aware of that.

I am using 4-Port Intel 10Gigbit network card-> X710 for 10GbE, and this is
currently bound to DPDK using VFIO-PCI driver, but the corresponding
kernel network driver appears to be-> i40e.

Currently I dont see shared libraries built under DPDK's->
build/driver/net/i40e directory.

Should I build those driver libraries, and then provide path to it,
something like this in EAL param -> -d build/drivers/net/i40e/libi40e.so

Please let me know.

Thanks,

Sudha
Regards,

Sudha

On Wed, May 26, 2021 at 10:53 AM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Wed, 26 May 2021 10:27:56 -0700
> Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:
>
> > Hi All,
> >
> > Previously, I had DPDK-19.11 working fine on my server system, which runs
> > on Ubuntu 18.04.
> > Have a 4-Port 10 Gigabit Network card, with ports bound to use VFIO-PCI
> > driver.
> > But had to move to DPDK-20.11, and running into issues.
> >
> > I installed on DPDK-20.11 on the server to /usr/local.
> >
> > And did binding of ports using the dpdk-devbind script to use VFIO-PCI
> > driver.
> >
> > But when running DPDK sample application such as->
> > dpdk-20.11/examples/packet_ordering,
> > noticed DPDK is NOT able to detect any available ports.
> > Getting error->  Cause: Error: no ethernet ports detected
> >
> > Also when I pass parameter to block certain
> interfaces(Bus:Device.Function)
> > on a NIC, EAL throws an
> > Error-> EAL: failed to parse device "0000:31.00.0"
> >
> > On the same system, using the same 10 Gigbit Network card, DPDK sample
> > applications were
> > working fine with older version of DPDK->19.11
> >
> > Below, I have the output of commands:
> >
> > ~/dpdk-20.11$ sudo ./usertools/dpdk-devbind.py --status
> >
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:31:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> > unused=i40e
> > 0000:31:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> > unused=i40e
> > 0000:31:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> > unused=i40e
> > 0000:31:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
> > unused=i40e
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:01:00.0 'I210 Gigabit Network Connection 1533' if=enp1s0 drv=igb
> > unused=vfio-pci *Active*
> > 0000:17:00.0 'Ethernet Controller 10G X550T 1563' if=ens6f0 drv=ixgbe
> > unused=vfio-pci *Active*
> > 0000:17:00.1 'Ethernet Controller 10G X550T 1563' if=ens6f1 drv=ixgbe
> > unused=vfio-pci *Active*
> >
> >
> > cat /proc/meminfo | grep -i huge
> > AnonHugePages:      8192 kB
> > ShmemHugePages:        0 kB
> > HugePages_Total:    2048
> > HugePages_Free:     2048
> > HugePages_Rsvd:        0
> > HugePages_Surp:        0
> > Hugepagesize:       2048 kB
> >
> > :~/dpdk-20.11/examples/packet_ordering$ sudo
> ./build/packet_ordering-shared
> > -l 4-6 -n 2 --log-level 7 -m 1024
> > --file-prefix packet_order -- -p 0xf --disable-reorder
> > EAL: Detected 72 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Multi-process socket /var/run/dpdk/packet_order/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: No available hugepages reported in hugepages-1048576kB
> > EAL: Probing VFIO support...
> > EAL: VFIO support initialized
> > EAL: No legacy callbacks, legacy socket not created
> > reorder disabled
> > EAL: Error - exiting with code: 1
> >   Cause: Error: no ethernet ports detected
> >
> > ~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l
> 4-6
> > -n 2 --proc-type auto --log-level 7 -m 1024
> > --file-prefix packet_ordering -b 0000:31:00.0 -b 0000:31:00.1
> > -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p
> 0xf
> > --disable-reorder
> > EAL: Detected 72 lcore(s)
> > EAL: Detected 2 NUMA nodes
> > EAL: Auto-detected process type: PRIMARY
> > EAL: failed to parse device "0000:31:00.0"
> > EAL: Unable to parse device '0000:31:00.0'
> > EAL: Error - exiting with code: 1
> >   Cause: Invalid EAL arguments
> >
> >
> > Please let me know, if you have any suggestions.
> >
> >
> > Thanks,
> >
> > Sudha
>
> By default, DPDK is now dynamically linked and all the drivers are in
> shared libraries.
> You need to pass the -d flag to indicate which library to load.
>

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar]

Hi Stephen,

Normal for dynamically linked shared libraries, linker should load the
dependant libraries
based on library path under-> /etc/ld.so.conf.d$ cat x86_64-linux-gnu.conf
# Multiarch support
/usr/local/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu

I do have the /usr/local/lib/x86* path, which is where DPDK 20.11
shared(*.so) libraries have been installed.
But during launch of application, application does not seem to load those
libraries. Normally this method works.
Is this a DPDK issue.

In any case, I followed your suggestion, did ldd -v packet_ordering-shared,
to get the dependencies, and manually passed the
paths to the application, but there is another error now-> MBUF: error
setting mempool handler

~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
-n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
-d /usr/local/lib/x86_64-linux-gnu/librte_eal.so -d
/usr/local/lib/x86_64-linux-gnu/librte_net_i40e.so -d
/usr/local/lib/x86_64-linux-gnu/librte_reorder.so -d
/usr/local/lib/x86_64-linux-gnu/librte_ethdev.so -d
/usr/local/lib/x86_64-linux-gnu/librte_mbuf.so -d
/usr/local/lib/x86_64-linux-gnu/librte_mempool.so -d
/usr/local/lib/x86_64-linux-gnu/librte_ring.so -d
/usr/lib/x86_64-linux-gnu/libnuma.so.1 -d
/usr/local/lib/x86_64-linux-gnu/librte_kvargs.so.21 -d
/usr/local/lib/x86_64-linux-gnu/librte_net.so.21 -d
/usr/local/lib/x86_64-linux-gnu/librte_meter.so.21 -b 0000:31:00.0 -b
0000:31:00.1 -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b
0000:08:00.0 -- -p 0xf --disable-reorderEAL: Detected 72 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Multi-process socket /var/run/dpdk/packet_ordering/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
reorder disabled
MBUF: error setting mempool handler
EAL: Error - exiting with code: 1
  Cause: Invalid argument

Please let me know.

Thanks,

Sudha


On Wed, May 26, 2021 at 11:10 AM Sudharshan Krishnakumar <
sudhar.ckrishna@gmail.com> wrote:

> Hi Stephen, Thanks for your reply, I was not aware of that.
>
> I am using 4-Port Intel 10Gigbit network card-> X710 for 10GbE, and this
> is currently bound to DPDK using VFIO-PCI driver, but the corresponding
> kernel network driver appears to be-> i40e.
>
> Currently I dont see shared libraries built under DPDK's->
> build/driver/net/i40e directory.
>
> Should I build those driver libraries, and then provide path to it,
> something like this in EAL param -> -d build/drivers/net/i40e/libi40e.so
>
> Please let me know.
>
> Thanks,
>
> Sudha
> Regards,
>
> Sudha
>
> On Wed, May 26, 2021 at 10:53 AM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
>> On Wed, 26 May 2021 10:27:56 -0700
>> Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:
>>
>> > Hi All,
>> >
>> > Previously, I had DPDK-19.11 working fine on my server system, which
>> runs
>> > on Ubuntu 18.04.
>> > Have a 4-Port 10 Gigabit Network card, with ports bound to use VFIO-PCI
>> > driver.
>> > But had to move to DPDK-20.11, and running into issues.
>> >
>> > I installed on DPDK-20.11 on the server to /usr/local.
>> >
>> > And did binding of ports using the dpdk-devbind script to use VFIO-PCI
>> > driver.
>> >
>> > But when running DPDK sample application such as->
>> > dpdk-20.11/examples/packet_ordering,
>> > noticed DPDK is NOT able to detect any available ports.
>> > Getting error->  Cause: Error: no ethernet ports detected
>> >
>> > Also when I pass parameter to block certain
>> interfaces(Bus:Device.Function)
>> > on a NIC, EAL throws an
>> > Error-> EAL: failed to parse device "0000:31.00.0"
>> >
>> > On the same system, using the same 10 Gigbit Network card, DPDK sample
>> > applications were
>> > working fine with older version of DPDK->19.11
>> >
>> > Below, I have the output of commands:
>> >
>> > ~/dpdk-20.11$ sudo ./usertools/dpdk-devbind.py --status
>> >
>> > Network devices using DPDK-compatible driver
>> > ============================================
>> > 0000:31:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
>> > unused=i40e
>> > 0000:31:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
>> > unused=i40e
>> > 0000:31:00.2 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
>> > unused=i40e
>> > 0000:31:00.3 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=vfio-pci
>> > unused=i40e
>> >
>> > Network devices using kernel driver
>> > ===================================
>> > 0000:01:00.0 'I210 Gigabit Network Connection 1533' if=enp1s0 drv=igb
>> > unused=vfio-pci *Active*
>> > 0000:17:00.0 'Ethernet Controller 10G X550T 1563' if=ens6f0 drv=ixgbe
>> > unused=vfio-pci *Active*
>> > 0000:17:00.1 'Ethernet Controller 10G X550T 1563' if=ens6f1 drv=ixgbe
>> > unused=vfio-pci *Active*
>> >
>> >
>> > cat /proc/meminfo | grep -i huge
>> > AnonHugePages:      8192 kB
>> > ShmemHugePages:        0 kB
>> > HugePages_Total:    2048
>> > HugePages_Free:     2048
>> > HugePages_Rsvd:        0
>> > HugePages_Surp:        0
>> > Hugepagesize:       2048 kB
>> >
>> > :~/dpdk-20.11/examples/packet_ordering$ sudo
>> ./build/packet_ordering-shared
>> > -l 4-6 -n 2 --log-level 7 -m 1024
>> > --file-prefix packet_order -- -p 0xf --disable-reorder
>> > EAL: Detected 72 lcore(s)
>> > EAL: Detected 2 NUMA nodes
>> > EAL: Multi-process socket /var/run/dpdk/packet_order/mp_socket
>> > EAL: Selected IOVA mode 'VA'
>> > EAL: No available hugepages reported in hugepages-1048576kB
>> > EAL: Probing VFIO support...
>> > EAL: VFIO support initialized
>> > EAL: No legacy callbacks, legacy socket not created
>> > reorder disabled
>> > EAL: Error - exiting with code: 1
>> >   Cause: Error: no ethernet ports detected
>> >
>> > ~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l
>> 4-6
>> > -n 2 --proc-type auto --log-level 7 -m 1024
>> > --file-prefix packet_ordering -b 0000:31:00.0 -b 0000:31:00.1
>> > -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p
>> 0xf
>> > --disable-reorder
>> > EAL: Detected 72 lcore(s)
>> > EAL: Detected 2 NUMA nodes
>> > EAL: Auto-detected process type: PRIMARY
>> > EAL: failed to parse device "0000:31:00.0"
>> > EAL: Unable to parse device '0000:31:00.0'
>> > EAL: Error - exiting with code: 1
>> >   Cause: Invalid EAL arguments
>> >
>> >
>> > Please let me know, if you have any suggestions.
>> >
>> >
>> > Thanks,
>> >
>> > Sudha
>>
>> By default, DPDK is now dynamically linked and all the drivers are in
>> shared libraries.
>> You need to pass the -d flag to indicate which library to load.
>>
>

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Stephen Hemminger]

On Wed, 26 May 2021 12:41:21 -0700
Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:

> Hi Stephen,
> 
> Normal for dynamically linked shared libraries, linker should load the
> dependant libraries
> based on library path under-> /etc/ld.so.conf.d$ cat x86_64-linux-gnu.conf
> # Multiarch support
> /usr/local/lib/x86_64-linux-gnu
> /lib/x86_64-linux-gnu
> /usr/lib/x86_64-linux-gnu
> 
> I do have the /usr/local/lib/x86* path, which is where DPDK 20.11
> shared(*.so) libraries have been installed.
> But during launch of application, application does not seem to load those
> libraries. Normally this method works.
> Is this a DPDK issue.
> 
> In any case, I followed your suggestion, did ldd -v packet_ordering-shared,
> to get the dependencies, and manually passed the
> paths to the application, but there is another error now-> MBUF: error
> setting mempool handler
> 

That is true for normal linked in libraries (ie what ldd shows).
The DPDK is doing explicit dlopen to  load poll-mode-drivers.

I would prefer that DPDK was smarter in this area (more like the Linux kernel)
so that it could have a catalog of PCI device vendor/id table and automatically
load the PMD; but that capability is not part of the current versions.

You have to do use -d to load the right .so

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar]

Sure, yes that will be good way, if DPDK maintainers can resolve it.
In the meantime, do you recommend just building all the libraries
statically and have the application link statically to DPDK libraries.

For example, DPDK doc for PMD , says -> When testpmd is built with shared
library, use option -d to load the dynamic PMD for rte_eal_init.
Link-> https://doc.dpdk.org/guides/nics/build_and_test.html

In this case, eventhough I am including path to the PMD-> i40e and ALL the
dependant DPDK libraries(including mbuf and mempool libraries), it is
failing here:
--> RTE_LOG(ERR, MBUF, "error setting mempool handler\n");

if (mp_ops_name == NULL)
mp_ops_name = rte_mbuf_best_mempool_ops();
ret = rte_mempool_set_ops_byname(mp, mp_ops_name, NULL);
if (ret != 0) {
RTE_LOG(ERR, MBUF, "error setting mempool handler\n");
rte_mempool_free(mp);
rte_errno = -ret;
return NULL;
}


~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
-n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
-d /usr/local/lib/x86_64-linux-gnu/librte_eal.so -d
/usr/local/lib/x86_64-linux-gnu/librte_net_i40e.so -d
/usr/local/lib/x86_64-linux-gnu/librte_reorder.so -d
/usr/local/lib/x86_64-linux-gnu/librte_ethdev.so -d
/usr/local/lib/x86_64-linux-gnu/librte_mempool.so.21   -d
/usr/local/lib/x86_64-linux-gnu/librte_mbuf.so  -d
/usr/local/lib/x86_64-linux-gnu/librte_ring.so -d
/usr/lib/x86_64-linux-gnu/libnuma.so.1 -d
/usr/local/lib/x86_64-linux-gnu/librte_kvargs.so.21 -d
/usr/local/lib/x86_64-linux-gnu/librte_net.so.21 -d
/usr/local/lib/x86_64-linux-gnu/librte_meter.so.21 -b 0000:31:00.0 -b
0000:31:00.1 -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b
0000:08:00.0 -- -p 0xf --disable-reorder
EAL: Detected 72 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Multi-process socket /var/run/dpdk/packet_ordering/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
reorder disabled
MBUF: error setting mempool handler
EAL: Error - exiting with code: 1
  Cause: Invalid argument


thanks,

Sudha


On Wed, May 26, 2021 at 1:03 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Wed, 26 May 2021 12:41:21 -0700
> Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:
>
> > Hi Stephen,
> >
> > Normal for dynamically linked shared libraries, linker should load the
> > dependant libraries
> > based on library path under-> /etc/ld.so.conf.d$ cat
> x86_64-linux-gnu.conf
> > # Multiarch support
> > /usr/local/lib/x86_64-linux-gnu
> > /lib/x86_64-linux-gnu
> > /usr/lib/x86_64-linux-gnu
> >
> > I do have the /usr/local/lib/x86* path, which is where DPDK 20.11
> > shared(*.so) libraries have been installed.
> > But during launch of application, application does not seem to load those
> > libraries. Normally this method works.
> > Is this a DPDK issue.
> >
> > In any case, I followed your suggestion, did ldd -v
> packet_ordering-shared,
> > to get the dependencies, and manually passed the
> > paths to the application, but there is another error now-> MBUF: error
> > setting mempool handler
> >
>
> That is true for normal linked in libraries (ie what ldd shows).
> The DPDK is doing explicit dlopen to  load poll-mode-drivers.
>
> I would prefer that DPDK was smarter in this area (more like the Linux
> kernel)
> so that it could have a catalog of PCI device vendor/id table and
> automatically
> load the PMD; but that capability is not part of the current versions.
>
> You have to do use -d to load the right .so
>
>

Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar]

Stephen, Thanks for your help.
Issue is resolved. -d option allows passing directory, where the shared
libraries are located, in this case,
if I pass -d /usr/local/lib/x86_64-linux-gnu/ that is good enough, it is
able to pick the dependent libraries from there.

~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
-n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
-d /usr/local/lib/x86_64-linux-gnu/ -b  0000:31:00.0 -b 0000:31:00.1 -b
0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p 0xf
--disable-reorder
EAL: Detected 72 lcore(s)EAL: Detected 2 NUMA nodesEAL: Auto-detected
process type: PRIMARYEAL: Multi-process socket
/var/run/dpdk/packet_ordering/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
reorder disabled
Initializing port 0... done
Port 0 MAC: 3c fd fe 9c 87 ba
Initializing port 1... done
Port 1 MAC: 3c fd fe 9c 87 bb
REORDERAPP: worker_thread() started on lcore 5
REORDERAPP: tx_thread() started on lcore 6
REORDERAPP: rx_thread() started on lcore 4

Thanks,

Sudha

On Wed, May 26, 2021 at 1:27 PM Sudharshan Krishnakumar <
sudhar.ckrishna@gmail.com> wrote:

> Sure, yes that will be good way, if DPDK maintainers can resolve it.
> In the meantime, do you recommend just building all the libraries
> statically and have the application link statically to DPDK libraries.
>
> For example, DPDK doc for PMD , says -> When testpmd is built with shared
> library, use option -d to load the dynamic PMD for rte_eal_init.
> Link-> https://doc.dpdk.org/guides/nics/build_and_test.html
>
> In this case, eventhough I am including path to the PMD-> i40e and ALL the
> dependant DPDK libraries(including mbuf and mempool libraries), it is
> failing here:
> --> RTE_LOG(ERR, MBUF, "error setting mempool handler\n");
>
> if (mp_ops_name == NULL)
> mp_ops_name = rte_mbuf_best_mempool_ops();
> ret = rte_mempool_set_ops_byname(mp, mp_ops_name, NULL);
> if (ret != 0) {
> RTE_LOG(ERR, MBUF, "error setting mempool handler\n");
> rte_mempool_free(mp);
> rte_errno = -ret;
> return NULL;
> }
>
>
> ~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
> -n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
> -d /usr/local/lib/x86_64-linux-gnu/librte_eal.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_net_i40e.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_reorder.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_ethdev.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_mempool.so.21   -d
> /usr/local/lib/x86_64-linux-gnu/librte_mbuf.so  -d
> /usr/local/lib/x86_64-linux-gnu/librte_ring.so -d
> /usr/lib/x86_64-linux-gnu/libnuma.so.1 -d
> /usr/local/lib/x86_64-linux-gnu/librte_kvargs.so.21 -d
> /usr/local/lib/x86_64-linux-gnu/librte_net.so.21 -d
> /usr/local/lib/x86_64-linux-gnu/librte_meter.so.21 -b 0000:31:00.0 -b
> 0000:31:00.1 -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b
> 0000:08:00.0 -- -p 0xf --disable-reorder
> EAL: Detected 72 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Auto-detected process type: PRIMARY
> EAL: Multi-process socket /var/run/dpdk/packet_ordering/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL:   using IOMMU type 1 (Type 1)
> EAL: Ignore mapping IO port bar(1)
> EAL: Ignore mapping IO port bar(4)
> EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
> EAL: Ignore mapping IO port bar(1)
> EAL: Ignore mapping IO port bar(4)
> EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
> EAL: No legacy callbacks, legacy socket not created
> reorder disabled
> MBUF: error setting mempool handler
> EAL: Error - exiting with code: 1
>   Cause: Invalid argument
>
>
> thanks,
>
> Sudha
>
>
> On Wed, May 26, 2021 at 1:03 PM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
>> On Wed, 26 May 2021 12:41:21 -0700
>> Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:
>>
>> > Hi Stephen,
>> >
>> > Normal for dynamically linked shared libraries, linker should load the
>> > dependant libraries
>> > based on library path under-> /etc/ld.so.conf.d$ cat
>> x86_64-linux-gnu.conf
>> > # Multiarch support
>> > /usr/local/lib/x86_64-linux-gnu
>> > /lib/x86_64-linux-gnu
>> > /usr/lib/x86_64-linux-gnu
>> >
>> > I do have the /usr/local/lib/x86* path, which is where DPDK 20.11
>> > shared(*.so) libraries have been installed.
>> > But during launch of application, application does not seem to load
>> those
>> > libraries. Normally this method works.
>> > Is this a DPDK issue.
>> >
>> > In any case, I followed your suggestion, did ldd -v
>> packet_ordering-shared,
>> > to get the dependencies, and manually passed the
>> > paths to the application, but there is another error now-> MBUF: error
>> > setting mempool handler
>> >
>>
>> That is true for normal linked in libraries (ie what ldd shows).
>> The DPDK is doing explicit dlopen to  load poll-mode-drivers.
>>
>> I would prefer that DPDK was smarter in this area (more like the Linux
>> kernel)
>> so that it could have a catalog of PCI device vendor/id table and
>> automatically
>> load the PMD; but that capability is not part of the current versions.
>>
>> You have to do use -d to load the right .so
>>
>>