DPDK Mellanox mlx4 and mlx5

DPDK Mellanox mlx4 and mlx5

[Lombardo, Ed]

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

Hi,
I am trying to add support for NIC "MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]" in our application with DPDK 22.11.

I am using meson and ninja and DPDK 22.11.2
I am running Oracle91 in my VM where I build DPDK.

I downloaded the libibverbs (rdma-core-44.0) and built the static libraries from sources.  The static libraries are not in the standard locations but under rpmbuild/SOURCES/rdma-core-44.0/build/lib

In meson I set ibverbs_link=static

When I execute ninja -j 10, I get error "bin/ld: cannot find -l:libibverbs.a"

How do I setup meson and ninja to build DPDK static libraries for Mellanox NIC and use libibverbs libraries.


Thanks,
Ed

[-- Attachment #2: Type: text/html, Size: 2788 bytes --]

RE: DPDK Mellanox mlx4 and mlx5

[Ali Alnubani]

Hi Ed,

Maybe you didn't set PKG_CONFIG_PATH?
 
Also check "5.3.1.1. Upstream Dependencies" section in https://doc.dpdk.org/guides/platform/mlx5.html#linux-prerequisites to build rdma-core statically.
 
Regards,
Ali

RE: DPDK Mellanox mlx4 and mlx5

[Lombardo, Ed]

Hi Ali,
Sorry for getting back to you so late.  Was pulled away for other reasons.  
I found online a way to tell meson and ninja to point to libibverbs.  Now meson and ninja are passing.  

# Define the path where libibverbs.a is located
ibverbs_lib_dir = '/root/rpmbuild/SOURCES/rdma-core-44.0/build/lib'
ibverbs_include_dir = '/root/rpmbuild/SOURCES/rdma-core-44.0/build/include'

# Create an include directory object
inc = include_directories(ibverbs_include_dir)

# Declare a dependency for libibverbs, linking statically
ibverbs_dep = declare_dependency(
  link_args: ['-L' + ibverbs_lib_dir, '-libverbs'],
  include_directories: inc
  )
# End of libibverbs


Next, I copied the DPDK include and lib files to our application build VM.  I built our application and now I see different errors during linking.

lib/dpdk-2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function mlx4_glue_dv_set_context_attr: error: undefined reference to 'mlx4dv_set_context_attr'
lib/dpdk-2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function mlx4_glue_dv_init_obj: error: undefined reference to 'mlx4dv_init_obj'
lib/dpdk-2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function mlx4_glue_dereg_mr: error: undefined reference to 'ibv_dereg_mr'
lib/dpdk-2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function mlx4_glue_reg_mr: error: undefined reference to 'ibv_reg_mr_iova2'
...

Now I need to resolve these issue related to mlx4_glue.c
If I have all the DPDK static libraries copied to my application build VM why does it complain about mlx4_glue.c functions are undefined?

Thanks,
Ed

-----Original Message-----
From: Ali Alnubani <alialnu@nvidia.com> 
Sent: Monday, August 5, 2024 2:55 AM
To: Lombardo, Ed <Ed.Lombardo@netscout.com>; users@dpdk.org
Subject: RE: DPDK Mellanox mlx4 and mlx5

External Email: This message originated outside of NETSCOUT. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hi Ed,

Maybe you didn't set PKG_CONFIG_PATH?
 
Also check "5.3.1.1. Upstream Dependencies" section in https://urldefense.com/v3/__https://doc.dpdk.org/guides/platform/mlx5.html*linux-prerequisites__;Iw!!Nzg7nt7_!AH7GKBDInkMDmwIafll16znS2hljXIeeOXYcfSKp9pAbAiVd-b6QrnHL8J_3Zz1S5ZxZSiB8qo4s2MzmyYQ$  to build rdma-core statically.
 
Regards,
Ali

RE: DPDK Mellanox mlx4 and mlx5

[Bing Zhao]

Have you installed the libs and headers in a standard CMAKE way?

> -----Original Message-----
> From: Lombardo, Ed <Ed.Lombardo@netscout.com>
> Sent: Saturday, August 10, 2024 1:40 AM
> To: Ali Alnubani <alialnu@nvidia.com>; users@dpdk.org
> Subject: RE: DPDK Mellanox mlx4 and mlx5
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Ali,
> Sorry for getting back to you so late.  Was pulled away for other reasons.
> I found online a way to tell meson and ninja to point to libibverbs.  Now
> meson and ninja are passing.
> 
> # Define the path where libibverbs.a is located ibverbs_lib_dir =
> '/root/rpmbuild/SOURCES/rdma-core-44.0/build/lib'
> ibverbs_include_dir = '/root/rpmbuild/SOURCES/rdma-core-
> 44.0/build/include'
> 
> # Create an include directory object
> inc = include_directories(ibverbs_include_dir)
> 
> # Declare a dependency for libibverbs, linking statically ibverbs_dep =
> declare_dependency(
>   link_args: ['-L' + ibverbs_lib_dir, '-libverbs'],
>   include_directories: inc
>   )
> # End of libibverbs
> 
> 
> Next, I copied the DPDK include and lib files to our application build VM.
> I built our application and now I see different errors during linking.
> 
> lib/dpdk-
> 2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function
> mlx4_glue_dv_set_context_attr: error: undefined reference to
> 'mlx4dv_set_context_attr'
> lib/dpdk-
> 2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function
> mlx4_glue_dv_init_obj: error: undefined reference to 'mlx4dv_init_obj'
> lib/dpdk-
> 2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function
> mlx4_glue_dereg_mr: error: undefined reference to 'ibv_dereg_mr'
> lib/dpdk-
> 2211.2/lib/librte_net_mlx4.a(net_mlx4_mlx4_glue.c.o):mlx4_glue.c:function
> mlx4_glue_reg_mr: error: undefined reference to 'ibv_reg_mr_iova2'
> ...
> 
> Now I need to resolve these issue related to mlx4_glue.c If I have all the
> DPDK static libraries copied to my application build VM why does it
> complain about mlx4_glue.c functions are undefined?
> 
> Thanks,
> Ed
> 
> -----Original Message-----
> From: Ali Alnubani <alialnu@nvidia.com>
> Sent: Monday, August 5, 2024 2:55 AM
> To: Lombardo, Ed <Ed.Lombardo@netscout.com>; users@dpdk.org
> Subject: RE: DPDK Mellanox mlx4 and mlx5
> 
> External Email: This message originated outside of NETSCOUT. Do not click
> links or open attachments unless you recognize the sender and know the
> content is safe.
> 
> Hi Ed,
> 
> Maybe you didn't set PKG_CONFIG_PATH?
> 
> Also check "5.3.1.1. Upstream Dependencies" section in
> https://urldefense.com/v3/__https://doc.dpdk.org/guides/platform/mlx5.html
> *linux-
> prerequisites__;Iw!!Nzg7nt7_!AH7GKBDInkMDmwIafll16znS2hljXIeeOXYcfSKp9pAbA
> iVd-b6QrnHL8J_3Zz1S5ZxZSiB8qo4s2MzmyYQ$  to build rdma-core statically.
> 
> Regards,
> Ali