[dpdk-users] Running Pktgen with Mellanox CX5

[dpdk-users] Running Pktgen with Mellanox CX5

[Narcisa Ana Maria Vasile]

Hi,

I'm trying to run pktgen (latest 'master' branch) with a Mellanox CX5 NIC on Ubuntu 20.04.
I've installed the latest Mellanox drivers (MLNX_OFED_LINUX-5.3-1.0.0.1-ubuntu20.04-x86_64.iso).
I've compiled and installed DPDK successfully (latest 'main' branch).

As you can see below, I'm getting an error message saying "libmlx4.so.1: cannot open shared object file: No such file or directory".
I am able to run other DPDK applications such as 'testpmd'.

Is pktgen supported with the latest Mellanox drivers on CX5? Thank you!

--------------
pktgen -l 1,3,5 -a 04:00.0 -d librte_net_mlx5.so -- -P -m "[3:5].0" -T

Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 20 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected shared linkage of DPDK
EAL: libmlx4.so.1: cannot open shared object file: No such file or directory
EAL: FATAL: Cannot init plugins
EAL: Cannot init plugins

Thank you,
Narcisa V.

Re: [dpdk-users] Running Pktgen with Mellanox CX5

[Wiles, Keith]

> On Apr 8, 2021, at 1:47 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
> 
> Hi,
>  
> I’m trying to run pktgen (latest ‘master’ branch) with a Mellanox CX5 NIC on Ubuntu 20.04.
> I’ve installed the latest Mellanox drivers (MLNX_OFED_LINUX-5.3-1.0.0.1-ubuntu20.04-x86_64.iso).
> I’ve compiled and installed DPDK successfully (latest ‘main’ branch).
>  
> As you can see below, I’m getting an error message saying “libmlx4.so.1: cannot open shared object file: No such file or directory”.
> I am able to run other DPDK applications such as ‘testpmd’.
>  
> Is pktgen supported with the latest Mellanox drivers on CX5? Thank you!
>  
> --------------
> pktgen -l 1,3,5 -a 04:00.0 -d librte_net_mlx5.so -- -P -m "[3:5].0" -T
>  
> Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
> EAL: Detected 20 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected shared linkage of DPDK
> EAL: libmlx4.so.1: cannot open shared object file: No such file or directory
> EAL: FATAL: Cannot init plugins
> EAL: Cannot init plugins

I have not built anything with mlx in a long time. My guess is the libmix4.so.1 is not located in a place the system can pick up. Maybe you need to set LD_LIBRARY_PATH to the path where this library is located. I see you included the DPDK PMD, but you still need to tell applications where to locate the library. Another option is to add it to the /etc/ld.so.conf.d/ file or use pkg-config to locate the libs may help too. In some cases external packages do not store the libs in a standard place for ldconfig to locate or the package does not provide ldconfig configuration files.
>  
> Thank you,
> Narcisa V.

Re: [dpdk-users] Running Pktgen with Mellanox CX5

[Narcisa Ana Maria Vasile]

Thanks for the quick reply! Just to clarify, the libmlx4 doesn't get installed when installing the latest Mellanox drivers.
Instead, the libmlx5 is installed, which I believe is the right one to use with Cx5s.

It looks like pktgen needs libmlx4 to run, so does this mean that it is only working with older Mellanox tools and NICs?
I could try install the libmlx4, but my understanding was that libmlx4 is for Cx3s and libmlx5 is for Cx4s and Cx5s.

-----Original Message-----
From: Wiles, Keith <keith.wiles@intel.com> 
Sent: Thursday, April 8, 2021 2:12 PM
To: Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com>
Cc: users@dpdk.org; Kevin Daniel (WIPRO LIMITED) <v-kevdan@microsoft.com>; Omar Cardona <ocardona@microsoft.com>
Subject: [EXTERNAL] Re: Running Pktgen with Mellanox CX5



> On Apr 8, 2021, at 1:47 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
> 
> Hi,
>  
> I’m trying to run pktgen (latest ‘master’ branch) with a Mellanox CX5 NIC on Ubuntu 20.04.
> I’ve installed the latest Mellanox drivers (MLNX_OFED_LINUX-5.3-1.0.0.1-ubuntu20.04-x86_64.iso).
> I’ve compiled and installed DPDK successfully (latest ‘main’ branch).
>  
> As you can see below, I’m getting an error message saying “libmlx4.so.1: cannot open shared object file: No such file or directory”.
> I am able to run other DPDK applications such as ‘testpmd’.
>  
> Is pktgen supported with the latest Mellanox drivers on CX5? Thank you!
>  
> --------------
> pktgen -l 1,3,5 -a 04:00.0 -d librte_net_mlx5.so -- -P -m "[3:5].0" -T
>  
> Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
> EAL: Detected 20 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Detected shared linkage of DPDK
> EAL: libmlx4.so.1: cannot open shared object file: No such file or directory
> EAL: FATAL: Cannot init plugins
> EAL: Cannot init plugins

I have not built anything with mlx in a long time. My guess is the libmix4.so.1 is not located in a place the system can pick up. Maybe you need to set LD_LIBRARY_PATH to the path where this library is located. I see you included the DPDK PMD, but you still need to tell applications where to locate the library. Another option is to add it to the /etc/ld.so.conf.d/ file or use pkg-config to locate the libs may help too. In some cases external packages do not store the libs in a standard place for ldconfig to locate or the package does not provide ldconfig configuration files.
>  
> Thank you,
> Narcisa V.

Re: [dpdk-users] Running Pktgen with Mellanox CX5

[Wiles, Keith]

> On Apr 8, 2021, at 7:08 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
> 
> Thanks for the quick reply! Just to clarify, the libmlx4 doesn't get installed when installing the latest Mellanox drivers.
> Instead, the libmlx5 is installed, which I believe is the right one to use with Cx5s.
> 
> It looks like pktgen needs libmlx4 to run, so does this mean that it is only working with older Mellanox tools and NICs?
> I could try install the libmlx4, but my understanding was that libmlx4 is for Cx3s and libmlx5 is for Cx4s and Cx5s.

Pktgen has no requirement to work with any mlx library or hardware for that matter. I do not know what this problem is, but I believe it is not a Pktgen problem. Did you try with testpmd or any of the DPDK examples. If they work with mlx then I can look at why Pktgen does not work in this case. If anything the libmlx5 is referencing the mlx4 library for some reason and when you added the mlx5 PMD it also needed the mlx4.

BTW, notice the error message states "EAL: libmlx4.so.1: cannot open shared object file: No such file or directory” EAL Is DPDK initialization not Pktgen.
> 
> -----Original Message-----
> From: Wiles, Keith <keith.wiles@intel.com> 
> Sent: Thursday, April 8, 2021 2:12 PM
> To: Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com>
> Cc: users@dpdk.org; Kevin Daniel (WIPRO LIMITED) <v-kevdan@microsoft.com>; Omar Cardona <ocardona@microsoft.com>
> Subject: [EXTERNAL] Re: Running Pktgen with Mellanox CX5
> 
> 
> 
>> On Apr 8, 2021, at 1:47 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
>> 
>> Hi,
>> 
>> I’m trying to run pktgen (latest ‘master’ branch) with a Mellanox CX5 NIC on Ubuntu 20.04.
>> I’ve installed the latest Mellanox drivers (MLNX_OFED_LINUX-5.3-1.0.0.1-ubuntu20.04-x86_64.iso).
>> I’ve compiled and installed DPDK successfully (latest ‘main’ branch).
>> 
>> As you can see below, I’m getting an error message saying “libmlx4.so.1: cannot open shared object file: No such file or directory”.
>> I am able to run other DPDK applications such as ‘testpmd’.
>> 
>> Is pktgen supported with the latest Mellanox drivers on CX5? Thank you!
>> 
>> --------------
>> pktgen -l 1,3,5 -a 04:00.0 -d librte_net_mlx5.so -- -P -m "[3:5].0" -T
>> 
>> Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
>> EAL: Detected 20 lcore(s)
>> EAL: Detected 2 NUMA nodes
>> EAL: Detected shared linkage of DPDK
>> EAL: libmlx4.so.1: cannot open shared object file: No such file or directory
>> EAL: FATAL: Cannot init plugins
>> EAL: Cannot init plugins
> 
> I have not built anything with mlx in a long time. My guess is the libmix4.so.1 is not located in a place the system can pick up. Maybe you need to set LD_LIBRARY_PATH to the path where this library is located. I see you included the DPDK PMD, but you still need to tell applications where to locate the library. Another option is to add it to the /etc/ld.so.conf.d/ file or use pkg-config to locate the libs may help too. In some cases external packages do not store the libs in a standard place for ldconfig to locate or the package does not provide ldconfig configuration files.
>> 
>> Thank you,
>> Narcisa V.
> 

Re: [dpdk-users] Running Pktgen with Mellanox CX5

[Narcisa Ana Maria Vasile]

On Fri, Apr 09, 2021 at 02:15:04PM +0000, Wiles, Keith wrote:
> 
> 
> > On Apr 8, 2021, at 7:08 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
> > 
> > Thanks for the quick reply! Just to clarify, the libmlx4 doesn't get installed when installing the latest Mellanox drivers.
> > Instead, the libmlx5 is installed, which I believe is the right one to use with Cx5s.
> > 
> > It looks like pktgen needs libmlx4 to run, so does this mean that it is only working with older Mellanox tools and NICs?
> > I could try install the libmlx4, but my understanding was that libmlx4 is for Cx3s and libmlx5 is for Cx4s and Cx5s.
> 
> Pktgen has no requirement to work with any mlx library or hardware for that matter. I do not know what this problem is, but I believe it is not a Pktgen problem. Did you try with testpmd or any of the DPDK examples. If they work with mlx then I can look at why Pktgen does not work in this case. If anything the libmlx5 is referencing the mlx4 library for some reason and when you added the mlx5 PMD it also needed the mlx4.
> 
  Sure, I understand it's not required to work with any of these HW or libs. I thought I'd ask in case someone in the community tested with a similar
  setup and can confirm whether it works or not. If someone has it working, then it's probably a misconfiguration on my side,
  alternatively if it's known that this is an unsupported setup I can stop investigating.


> BTW, notice the error message states "EAL: libmlx4.so.1: cannot open shared object file: No such file or directory” EAL Is DPDK initialization not Pktgen.

  That's right, it looks like it's required by EAL. However, I am able to run testpmd successfully. I've also tried l2fwd successfully.
  Still, this doesn't necessarily mean the issue is Pktgen-related, that's why I wanted to verify here.

> > 
> > -----Original Message-----
> > From: Wiles, Keith <keith.wiles@intel.com> 
> > Sent: Thursday, April 8, 2021 2:12 PM
> > To: Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com>
> > Cc: users@dpdk.org; Kevin Daniel (WIPRO LIMITED) <v-kevdan@microsoft.com>; Omar Cardona <ocardona@microsoft.com>
> > Subject: [EXTERNAL] Re: Running Pktgen with Mellanox CX5
> > 
> > 
> > 
> >> On Apr 8, 2021, at 1:47 PM, Narcisa Ana Maria Vasile <Narcisa.Vasile@microsoft.com> wrote:
> >> 
> >> Hi,
> >> 
> >> I’m trying to run pktgen (latest ‘master’ branch) with a Mellanox CX5 NIC on Ubuntu 20.04.
> >> I’ve installed the latest Mellanox drivers (MLNX_OFED_LINUX-5.3-1.0.0.1-ubuntu20.04-x86_64.iso).
> >> I’ve compiled and installed DPDK successfully (latest ‘main’ branch).
> >> 
> >> As you can see below, I’m getting an error message saying “libmlx4.so.1: cannot open shared object file: No such file or directory”.
> >> I am able to run other DPDK applications such as ‘testpmd’.
> >> 
> >> Is pktgen supported with the latest Mellanox drivers on CX5? Thank you!
> >> 
> >> --------------
> >> pktgen -l 1,3,5 -a 04:00.0 -d librte_net_mlx5.so -- -P -m "[3:5].0" -T
> >> 
> >> Copyright(c) <2010-2021>, Intel Corporation. All rights reserved. Powered by DPDK
> >> EAL: Detected 20 lcore(s)
> >> EAL: Detected 2 NUMA nodes
> >> EAL: Detected shared linkage of DPDK
> >> EAL: libmlx4.so.1: cannot open shared object file: No such file or directory
> >> EAL: FATAL: Cannot init plugins
> >> EAL: Cannot init plugins
> > 
> > I have not built anything with mlx in a long time. My guess is the libmix4.so.1 is not located in a place the system can pick up. Maybe you need to set LD_LIBRARY_PATH to the path where this library is located. I see you included the DPDK PMD, but you still need to tell applications where to locate the library. Another option is to add it to the /etc/ld.so.conf.d/ file or use pkg-config to locate the libs may help too. In some cases external packages do not store the libs in a standard place for ldconfig to locate or the package does not provide ldconfig configuration files.
> >> 
> >> Thank you,
> >> Narcisa V.
> > 
>