Re: [dpdk-users] Issue moving to DPDK 20.11 release

[Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com>]

Stephen, Thanks for your help.
Issue is resolved. -d option allows passing directory, where the shared
libraries are located, in this case,
if I pass -d /usr/local/lib/x86_64-linux-gnu/ that is good enough, it is
able to pick the dependent libraries from there.

~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
-n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
-d /usr/local/lib/x86_64-linux-gnu/ -b  0000:31:00.0 -b 0000:31:00.1 -b
0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b 0000:08:00.0 -- -p 0xf
--disable-reorder
EAL: Detected 72 lcore(s)EAL: Detected 2 NUMA nodesEAL: Auto-detected
process type: PRIMARYEAL: Multi-process socket
/var/run/dpdk/packet_ordering/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
EAL: No legacy callbacks, legacy socket not created
reorder disabled
Initializing port 0... done
Port 0 MAC: 3c fd fe 9c 87 ba
Initializing port 1... done
Port 1 MAC: 3c fd fe 9c 87 bb
REORDERAPP: worker_thread() started on lcore 5
REORDERAPP: tx_thread() started on lcore 6
REORDERAPP: rx_thread() started on lcore 4

Thanks,

Sudha

On Wed, May 26, 2021 at 1:27 PM Sudharshan Krishnakumar <
sudhar.ckrishna@gmail.com> wrote:

> Sure, yes that will be good way, if DPDK maintainers can resolve it.
> In the meantime, do you recommend just building all the libraries
> statically and have the application link statically to DPDK libraries.
>
> For example, DPDK doc for PMD , says -> When testpmd is built with shared
> library, use option -d to load the dynamic PMD for rte_eal_init.
> Link-> https://doc.dpdk.org/guides/nics/build_and_test.html
>
> In this case, eventhough I am including path to the PMD-> i40e and ALL the
> dependant DPDK libraries(including mbuf and mempool libraries), it is
> failing here:
> --> RTE_LOG(ERR, MBUF, "error setting mempool handler\n");
>
> if (mp_ops_name == NULL)
> mp_ops_name = rte_mbuf_best_mempool_ops();
> ret = rte_mempool_set_ops_byname(mp, mp_ops_name, NULL);
> if (ret != 0) {
> RTE_LOG(ERR, MBUF, "error setting mempool handler\n");
> rte_mempool_free(mp);
> rte_errno = -ret;
> return NULL;
> }
>
>
> ~/dpdk-20.11/examples/packet_ordering/build$ sudo ./packet_ordering -l 4-6
> -n 2 --proc-type auto --log-level 7 -m 1024 --file-prefix packet_ordering
> -d /usr/local/lib/x86_64-linux-gnu/librte_eal.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_net_i40e.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_reorder.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_ethdev.so -d
> /usr/local/lib/x86_64-linux-gnu/librte_mempool.so.21   -d
> /usr/local/lib/x86_64-linux-gnu/librte_mbuf.so  -d
> /usr/local/lib/x86_64-linux-gnu/librte_ring.so -d
> /usr/lib/x86_64-linux-gnu/libnuma.so.1 -d
> /usr/local/lib/x86_64-linux-gnu/librte_kvargs.so.21 -d
> /usr/local/lib/x86_64-linux-gnu/librte_net.so.21 -d
> /usr/local/lib/x86_64-linux-gnu/librte_meter.so.21 -b 0000:31:00.0 -b
> 0000:31:00.1 -b 0000:05:00.0 -b 0000:06:00.0 -b 0000:07:00.0  -b
> 0000:08:00.0 -- -p 0xf --disable-reorder
> EAL: Detected 72 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Auto-detected process type: PRIMARY
> EAL: Multi-process socket /var/run/dpdk/packet_ordering/mp_socket
> EAL: Selected IOVA mode 'VA'
> EAL: No available hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL:   using IOMMU type 1 (Type 1)
> EAL: Ignore mapping IO port bar(1)
> EAL: Ignore mapping IO port bar(4)
> EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.2 (socket 0)
> EAL: Ignore mapping IO port bar(1)
> EAL: Ignore mapping IO port bar(4)
> EAL: Probe PCI driver: net_i40e (8086:1572) device: 0000:31:00.3 (socket 0)
> EAL: No legacy callbacks, legacy socket not created
> reorder disabled
> MBUF: error setting mempool handler
> EAL: Error - exiting with code: 1
>   Cause: Invalid argument
>
>
> thanks,
>
> Sudha
>
>
> On Wed, May 26, 2021 at 1:03 PM Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
>> On Wed, 26 May 2021 12:41:21 -0700
>> Sudharshan Krishnakumar <sudhar.ckrishna@gmail.com> wrote:
>>
>> > Hi Stephen,
>> >
>> > Normal for dynamically linked shared libraries, linker should load the
>> > dependant libraries
>> > based on library path under-> /etc/ld.so.conf.d$ cat
>> x86_64-linux-gnu.conf
>> > # Multiarch support
>> > /usr/local/lib/x86_64-linux-gnu
>> > /lib/x86_64-linux-gnu
>> > /usr/lib/x86_64-linux-gnu
>> >
>> > I do have the /usr/local/lib/x86* path, which is where DPDK 20.11
>> > shared(*.so) libraries have been installed.
>> > But during launch of application, application does not seem to load
>> those
>> > libraries. Normally this method works.
>> > Is this a DPDK issue.
>> >
>> > In any case, I followed your suggestion, did ldd -v
>> packet_ordering-shared,
>> > to get the dependencies, and manually passed the
>> > paths to the application, but there is another error now-> MBUF: error
>> > setting mempool handler
>> >
>>
>> That is true for normal linked in libraries (ie what ldd shows).
>> The DPDK is doing explicit dlopen to  load poll-mode-drivers.
>>
>> I would prefer that DPDK was smarter in this area (more like the Linux
>> kernel)
>> so that it could have a catalog of PCI device vendor/id table and
>> automatically
>> load the PMD; but that capability is not part of the current versions.
>>
>> You have to do use -d to load the right .so
>>
>>