* [dpdk-users] Building mlx5 with rdma-core/OFED @ 2021-08-02 0:23 Gerry Wan 2021-08-02 13:24 ` PATRICK KEROULAS 0 siblings, 1 reply; 7+ messages in thread From: Gerry Wan @ 2021-08-02 0:23 UTC (permalink / raw) To: users I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal machine Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk --upstream-libs --with-mft --with-kernel-mft). However, it now fails with the message Checking SW Requirements... Removing old packages... Uninstalling the previous version of MLNX_OFED_LINUX Installing new packages Installing mft-4.16.1... Installing ofed-scripts-5.2... Installing mstflint-4.15.0... Installing mlnx-ofed-kernel-utils-5.2... Installing mlnx-ofed-kernel-dkms-5.2... Failed to install mlnx-ofed-kernel-dkms DEB Collecting debug info... See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log It appears that my kernel version got upgraded to 5.4.0-80-generic which I think may be the issue. I've also tried to install using --add-kernel-support --kernel 5.4.0-80-generic, with no luck. The DPDK website says that rdma-core is now recommended over OFED, but the documentation for how to use it isn't very clear to me. I've tried to install it via apt-get install rdma-core as well as running the quick start script listed here: https://github.com/linux-rdma/rdma-core, but DPDK installation still fails with ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: infiniband/mlx5dv.h: No such file or directory 15 | #include <infiniband/mlx5dv.h> Any advice on getting the mlx5 drivers to work again would be helpful! ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-02 0:23 [dpdk-users] Building mlx5 with rdma-core/OFED Gerry Wan @ 2021-08-02 13:24 ` PATRICK KEROULAS 2021-08-02 18:35 ` Gerry Wan 0 siblings, 1 reply; 7+ messages in thread From: PATRICK KEROULAS @ 2021-08-02 13:24 UTC (permalink / raw) To: Gerry Wan; +Cc: users Hello Gerry, I went through this a while ago and had to properly remove everything related to OFED with the uninstaller. Then, install rdma-core and libibverbs-dev which includes infiniband/mlx5dv.h IIRC. Regards, Patrick On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> wrote: > I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal machine > Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk > --upstream-libs --with-mft --with-kernel-mft). However, it now fails with > the message > > Checking SW Requirements... > Removing old packages... > Uninstalling the previous version of MLNX_OFED_LINUX > Installing new packages Installing mft-4.16.1... > Installing ofed-scripts-5.2... > Installing mstflint-4.15.0... > Installing mlnx-ofed-kernel-utils-5.2... > Installing mlnx-ofed-kernel-dkms-5.2... > Failed to install mlnx-ofed-kernel-dkms DEB > Collecting debug info... > See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log > > It appears that my kernel version got upgraded to 5.4.0-80-generic which I > think may be the issue. I've also tried to install using > --add-kernel-support --kernel 5.4.0-80-generic, with no luck. > > The DPDK website says that rdma-core is now recommended over OFED, but the > documentation for how to use it isn't very clear to me. I've tried to > install it via > > apt-get install rdma-core > > as well as running the quick start script listed here: > https://github.com/linux-rdma/rdma-core > <https://github.com/linux-rdma/rdma-core>, > but DPDK installation still fails > with > > ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: > infiniband/mlx5dv.h: No such file or directory > 15 | #include <infiniband/mlx5dv.h> > > Any advice on getting the mlx5 drivers to work again would be helpful! > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-02 13:24 ` PATRICK KEROULAS @ 2021-08-02 18:35 ` Gerry Wan 2021-08-02 19:59 ` PATRICK KEROULAS 0 siblings, 1 reply; 7+ messages in thread From: Gerry Wan @ 2021-08-02 18:35 UTC (permalink / raw) To: PATRICK KEROULAS; +Cc: users How did you install these packages? Are there extra environment variables I need to set? After running the OFED uninstall script, apt-get install rdma-core apt-get install libibverbs-dev both work fine, but ninja install now fails on [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. FAILED: drivers/librte_common_mlx5.so.21.1 cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@ @rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map /usr/lib/x86_64-linux-gnu/libmlx5.so /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_aso': mlx5_glue.c:(.text+0x21f): undefined reference to `mlx5dv_dr_action_create_aso' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_action_create_dest_array': mlx5_glue.c:(.text+0x23b): undefined reference to `mlx5dv_dr_action_create_dest_array' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_sampler': mlx5_glue.c:(.text+0x25b): undefined reference to `mlx5dv_dr_action_create_flow_sampler' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_alloc_pp': mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_default_miss': mlx5_glue.c:(.text+0x4bb): undefined reference to `mlx5dv_dr_action_create_default_miss' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_free_pp': mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_reclaim_domain_memory': mlx5_glue.c:(.text+0x2a7): undefined reference to `mlx5dv_dr_domain_set_reclaim_device_memory' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_devx_port_query': mlx5_glue.c:(.text+0x325): undefined reference to `mlx5dv_query_devx_port' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': mlx5_glue.c:(.text+0x545): undefined reference to `mlx5dv_dr_action_create_dest_devx_tir' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_pop_vlan': mlx5_glue.c:(.text+0x655): undefined reference to `mlx5dv_dr_action_create_pop_vlan' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_push_vlan': mlx5_glue.c:(.text+0x665): undefined reference to `mlx5dv_dr_action_create_push_vlan' /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_dest_port': mlx5_glue.c:(.text+0x685): undefined reference to `mlx5dv_dr_action_create_dest_ib_port' collect2: error: ld returned 1 exit status [2/1756] Compiling C object 'drivers/a715181@@tmp_rte_mempool_octeontx@sta /mempool_octeontx_octeontx_fpavf.c.o'. ninja: build stopped: subcommand failed. On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS < patrick.keroulas@radio-canada.ca> wrote: > > Hello Gerry, > I went through this a while ago and had to properly remove everything > related > to OFED with the uninstaller. Then, install rdma-core and libibverbs-dev > which > includes infiniband/mlx5dv.h IIRC. > Regards, > Patrick > > On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> wrote: > >> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal >> machine >> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk >> --upstream-libs --with-mft --with-kernel-mft). However, it now fails with >> the message >> >> Checking SW Requirements... >> Removing old packages... >> Uninstalling the previous version of MLNX_OFED_LINUX >> Installing new packages Installing mft-4.16.1... >> Installing ofed-scripts-5.2... >> Installing mstflint-4.15.0... >> Installing mlnx-ofed-kernel-utils-5.2... >> Installing mlnx-ofed-kernel-dkms-5.2... >> Failed to install mlnx-ofed-kernel-dkms DEB >> Collecting debug info... >> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log >> >> It appears that my kernel version got upgraded to 5.4.0-80-generic which I >> think may be the issue. I've also tried to install using >> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. >> >> The DPDK website says that rdma-core is now recommended over OFED, but the >> documentation for how to use it isn't very clear to me. I've tried to >> install it via >> >> apt-get install rdma-core >> >> as well as running the quick start script listed here: >> https://github.com/linux-rdma/rdma-core, but DPDK installation still >> fails >> with >> >> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: >> infiniband/mlx5dv.h: No such file or directory >> 15 | #include <infiniband/mlx5dv.h> >> >> Any advice on getting the mlx5 drivers to work again would be helpful! >> > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-02 18:35 ` Gerry Wan @ 2021-08-02 19:59 ` PATRICK KEROULAS 2021-08-02 20:18 ` Gerry Wan 0 siblings, 1 reply; 7+ messages in thread From: PATRICK KEROULAS @ 2021-08-02 19:59 UTC (permalink / raw) To: Gerry Wan; +Cc: users I guess you're running Focal or Groovy release so rdma-core and libibverbs are >v28: https://launchpad.net/rdma-core For this version, your missing symbols are defined: https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/mlx5dv.h I expect this header to be installed in `/usr/include/infiniband/`. Please verify. Maybe something wrong with ninja, try meson or make. On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan <gerryw@stanford.edu> wrote: > > How did you install these packages? Are there extra environment variables I need to set? > After running the OFED uninstall script, > > apt-get install rdma-core > apt-get install libibverbs-dev > > both work fine, but ninja install now fails on > > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. > FAILED: drivers/librte_common_mlx5.so.21.1 > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@@rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map /usr/lib/x86_64-linux-gnu/libmlx5.so /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_aso': > mlx5_glue.c:(.text+0x21f): undefined reference to `mlx5dv_dr_action_create_aso' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_action_create_dest_array': > mlx5_glue.c:(.text+0x23b): undefined reference to `mlx5dv_dr_action_create_dest_array' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_sampler': > mlx5_glue.c:(.text+0x25b): undefined reference to `mlx5dv_dr_action_create_flow_sampler' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_alloc_pp': > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_default_miss': > mlx5_glue.c:(.text+0x4bb): undefined reference to `mlx5dv_dr_action_create_default_miss' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_free_pp': > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_reclaim_domain_memory': > mlx5_glue.c:(.text+0x2a7): undefined reference to `mlx5dv_dr_domain_set_reclaim_device_memory' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_devx_port_query': > mlx5_glue.c:(.text+0x325): undefined reference to `mlx5dv_query_devx_port' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': > mlx5_glue.c:(.text+0x545): undefined reference to `mlx5dv_dr_action_create_dest_devx_tir' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_pop_vlan': > mlx5_glue.c:(.text+0x655): undefined reference to `mlx5dv_dr_action_create_pop_vlan' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_push_vlan': > mlx5_glue.c:(.text+0x665): undefined reference to `mlx5dv_dr_action_create_push_vlan' > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_dest_port': > mlx5_glue.c:(.text+0x685): undefined reference to `mlx5dv_dr_action_create_dest_ib_port' > collect2: error: ld returned 1 exit status > [2/1756] Compiling C object 'drivers/a715181@@tmp_rte_mempool_octeontx@sta/mempool_octeontx_octeontx_fpavf.c.o'. > ninja: build stopped: subcommand failed. > > > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS <patrick.keroulas@radio-canada.ca> wrote: >> >> >> Hello Gerry, >> I went through this a while ago and had to properly remove everything related >> to OFED with the uninstaller. Then, install rdma-core and libibverbs-dev which >> includes infiniband/mlx5dv.h IIRC. >> Regards, >> Patrick >> >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> wrote: >>> >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal machine >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk >>> --upstream-libs --with-mft --with-kernel-mft). However, it now fails with >>> the message >>> >>> Checking SW Requirements... >>> Removing old packages... >>> Uninstalling the previous version of MLNX_OFED_LINUX >>> Installing new packages Installing mft-4.16.1... >>> Installing ofed-scripts-5.2... >>> Installing mstflint-4.15.0... >>> Installing mlnx-ofed-kernel-utils-5.2... >>> Installing mlnx-ofed-kernel-dkms-5.2... >>> Failed to install mlnx-ofed-kernel-dkms DEB >>> Collecting debug info... >>> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log >>> >>> It appears that my kernel version got upgraded to 5.4.0-80-generic which I >>> think may be the issue. I've also tried to install using >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. >>> >>> The DPDK website says that rdma-core is now recommended over OFED, but the >>> documentation for how to use it isn't very clear to me. I've tried to >>> install it via >>> >>> apt-get install rdma-core >>> >>> as well as running the quick start script listed here: >>> https://github.com/linux-rdma/rdma-core, but DPDK installation still fails >>> with >>> >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: >>> infiniband/mlx5dv.h: No such file or directory >>> 15 | #include <infiniband/mlx5dv.h> >>> >>> Any advice on getting the mlx5 drivers to work again would be helpful! ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-02 19:59 ` PATRICK KEROULAS @ 2021-08-02 20:18 ` Gerry Wan 2021-08-03 14:23 ` PATRICK KEROULAS 0 siblings, 1 reply; 7+ messages in thread From: Gerry Wan @ 2021-08-02 20:18 UTC (permalink / raw) To: PATRICK KEROULAS; +Cc: users I am running Ubuntu 20.04, and the mlx5dv.h is indeed in `/usr/include/infiniband/`. The steps I take to build DPDK are $ meson --prefix=$DPDK_PATH build $ cd build $ sudo ninja install -> fails here $ sudo ldconfig How can I install the correct versions of rdma-core and libibverbs? On Mon, Aug 2, 2021 at 12:59 PM PATRICK KEROULAS < patrick.keroulas@radio-canada.ca> wrote: > I guess you're running Focal or Groovy release so rdma-core and libibverbs > are >v28: https://launchpad.net/rdma-core > For this version, your missing symbols are defined: > > https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/mlx5dv.h > I expect this header to be installed in `/usr/include/infiniband/`. > Please verify. > > Maybe something wrong with ninja, try meson or make. > > On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan <gerryw@stanford.edu> wrote: > > > > How did you install these packages? Are there extra environment > variables I need to set? > > After running the OFED uninstall script, > > > > apt-get install rdma-core > > apt-get install libibverbs-dev > > > > both work fine, but ninja install now fails on > > > > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. > > FAILED: drivers/librte_common_mlx5.so.21.1 > > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@ > @rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' > -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group > -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl > -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 > lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 > lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 > drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 > lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 > -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map > /usr/lib/x86_64-linux-gnu/libmlx5.so > /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group > '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_aso': > > mlx5_glue.c:(.text+0x21f): undefined reference to > `mlx5dv_dr_action_create_aso' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_action_create_dest_array': > > mlx5_glue.c:(.text+0x23b): undefined reference to > `mlx5dv_dr_action_create_dest_array' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_sampler': > > mlx5_glue.c:(.text+0x25b): undefined reference to > `mlx5dv_dr_action_create_flow_sampler' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_alloc_pp': > > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_default_miss': > > mlx5_glue.c:(.text+0x4bb): undefined reference to > `mlx5dv_dr_action_create_default_miss' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_free_pp': > > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_reclaim_domain_memory': > > mlx5_glue.c:(.text+0x2a7): undefined reference to > `mlx5dv_dr_domain_set_reclaim_device_memory' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_devx_port_query': > > mlx5_glue.c:(.text+0x325): undefined reference to > `mlx5dv_query_devx_port' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': > > mlx5_glue.c:(.text+0x545): undefined reference to > `mlx5dv_dr_action_create_dest_devx_tir' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_pop_vlan': > > mlx5_glue.c:(.text+0x655): undefined reference to > `mlx5dv_dr_action_create_pop_vlan' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_push_vlan': > > mlx5_glue.c:(.text+0x665): undefined reference to > `mlx5dv_dr_action_create_push_vlan' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_dest_port': > > mlx5_glue.c:(.text+0x685): undefined reference to > `mlx5dv_dr_action_create_dest_ib_port' > > collect2: error: ld returned 1 exit status > > [2/1756] Compiling C object 'drivers/a715181@ > @tmp_rte_mempool_octeontx@sta/mempool_octeontx_octeontx_fpavf.c.o'. > > ninja: build stopped: subcommand failed. > > > > > > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS < > patrick.keroulas@radio-canada.ca> wrote: > >> > >> > >> Hello Gerry, > >> I went through this a while ago and had to properly remove everything > related > >> to OFED with the uninstaller. Then, install rdma-core and > libibverbs-dev which > >> includes infiniband/mlx5dv.h IIRC. > >> Regards, > >> Patrick > >> > >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> wrote: > >>> > >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal > machine > >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk > >>> --upstream-libs --with-mft --with-kernel-mft). However, it now fails > with > >>> the message > >>> > >>> Checking SW Requirements... > >>> Removing old packages... > >>> Uninstalling the previous version of MLNX_OFED_LINUX > >>> Installing new packages Installing mft-4.16.1... > >>> Installing ofed-scripts-5.2... > >>> Installing mstflint-4.15.0... > >>> Installing mlnx-ofed-kernel-utils-5.2... > >>> Installing mlnx-ofed-kernel-dkms-5.2... > >>> Failed to install mlnx-ofed-kernel-dkms DEB > >>> Collecting debug info... > >>> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log > >>> > >>> It appears that my kernel version got upgraded to 5.4.0-80-generic > which I > >>> think may be the issue. I've also tried to install using > >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. > >>> > >>> The DPDK website says that rdma-core is now recommended over OFED, but > the > >>> documentation for how to use it isn't very clear to me. I've tried to > >>> install it via > >>> > >>> apt-get install rdma-core > >>> > >>> as well as running the quick start script listed here: > >>> https://github.com/linux-rdma/rdma-core, but DPDK installation still > fails > >>> with > >>> > >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: > >>> infiniband/mlx5dv.h: No such file or directory > >>> 15 | #include <infiniband/mlx5dv.h> > >>> > >>> Any advice on getting the mlx5 drivers to work again would be helpful! > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-02 20:18 ` Gerry Wan @ 2021-08-03 14:23 ` PATRICK KEROULAS 2021-08-13 7:02 ` Gerry Wan 0 siblings, 1 reply; 7+ messages in thread From: PATRICK KEROULAS @ 2021-08-03 14:23 UTC (permalink / raw) To: Gerry Wan; +Cc: users Have you checked that: - the missing symbols like `mlx5dv_dr_action_create_default_miss` are defined in mlx5dv.h? - there is no other mlx5dv.h from previous version installed somewhere? I recommend to keep the package versions provided for your distro and downgrade dpdk if necessary. On Mon, Aug 2, 2021 at 4:18 PM Gerry Wan <gerryw@stanford.edu> wrote: > > I am running Ubuntu 20.04, and the mlx5dv.h is indeed in `/usr/include/infiniband/`. > > The steps I take to build DPDK are > $ meson --prefix=$DPDK_PATH build > $ cd build > $ sudo ninja install -> fails here > $ sudo ldconfig > > How can I install the correct versions of rdma-core and libibverbs? > > On Mon, Aug 2, 2021 at 12:59 PM PATRICK KEROULAS <patrick.keroulas@radio-canada.ca> wrote: >> >> I guess you're running Focal or Groovy release so rdma-core and libibverbs >> are >v28: https://launchpad.net/rdma-core >> For this version, your missing symbols are defined: >> https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/mlx5dv.h >> I expect this header to be installed in `/usr/include/infiniband/`. >> Please verify. >> >> Maybe something wrong with ninja, try meson or make. >> >> On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan <gerryw@stanford.edu> wrote: >> > >> > How did you install these packages? Are there extra environment variables I need to set? >> > After running the OFED uninstall script, >> > >> > apt-get install rdma-core >> > apt-get install libibverbs-dev >> > >> > both work fine, but ninja install now fails on >> > >> > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. >> > FAILED: drivers/librte_common_mlx5.so.21.1 >> > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@@rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map /usr/lib/x86_64-linux-gnu/libmlx5.so /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_aso': >> > mlx5_glue.c:(.text+0x21f): undefined reference to `mlx5dv_dr_action_create_aso' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_action_create_dest_array': >> > mlx5_glue.c:(.text+0x23b): undefined reference to `mlx5dv_dr_action_create_dest_array' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_sampler': >> > mlx5_glue.c:(.text+0x25b): undefined reference to `mlx5dv_dr_action_create_flow_sampler' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_alloc_pp': >> > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_default_miss': >> > mlx5_glue.c:(.text+0x4bb): undefined reference to `mlx5dv_dr_action_create_default_miss' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_free_pp': >> > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_reclaim_domain_memory': >> > mlx5_glue.c:(.text+0x2a7): undefined reference to `mlx5dv_dr_domain_set_reclaim_device_memory' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_devx_port_query': >> > mlx5_glue.c:(.text+0x325): undefined reference to `mlx5dv_query_devx_port' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': >> > mlx5_glue.c:(.text+0x545): undefined reference to `mlx5dv_dr_action_create_dest_devx_tir' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_pop_vlan': >> > mlx5_glue.c:(.text+0x655): undefined reference to `mlx5dv_dr_action_create_pop_vlan' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_push_vlan': >> > mlx5_glue.c:(.text+0x665): undefined reference to `mlx5dv_dr_action_create_push_vlan' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_dest_port': >> > mlx5_glue.c:(.text+0x685): undefined reference to `mlx5dv_dr_action_create_dest_ib_port' >> > collect2: error: ld returned 1 exit status >> > [2/1756] Compiling C object 'drivers/a715181@@tmp_rte_mempool_octeontx@sta/mempool_octeontx_octeontx_fpavf.c.o'. >> > ninja: build stopped: subcommand failed. >> > >> > >> > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS <patrick.keroulas@radio-canada.ca> wrote: >> >> >> >> >> >> Hello Gerry, >> >> I went through this a while ago and had to properly remove everything related >> >> to OFED with the uninstaller. Then, install rdma-core and libibverbs-dev which >> >> includes infiniband/mlx5dv.h IIRC. >> >> Regards, >> >> Patrick >> >> >> >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> wrote: >> >>> >> >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal machine >> >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk >> >>> --upstream-libs --with-mft --with-kernel-mft). However, it now fails with >> >>> the message >> >>> >> >>> Checking SW Requirements... >> >>> Removing old packages... >> >>> Uninstalling the previous version of MLNX_OFED_LINUX >> >>> Installing new packages Installing mft-4.16.1... >> >>> Installing ofed-scripts-5.2... >> >>> Installing mstflint-4.15.0... >> >>> Installing mlnx-ofed-kernel-utils-5.2... >> >>> Installing mlnx-ofed-kernel-dkms-5.2... >> >>> Failed to install mlnx-ofed-kernel-dkms DEB >> >>> Collecting debug info... >> >>> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log >> >>> >> >>> It appears that my kernel version got upgraded to 5.4.0-80-generic which I >> >>> think may be the issue. I've also tried to install using >> >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. >> >>> >> >>> The DPDK website says that rdma-core is now recommended over OFED, but the >> >>> documentation for how to use it isn't very clear to me. I've tried to >> >>> install it via >> >>> >> >>> apt-get install rdma-core >> >>> >> >>> as well as running the quick start script listed here: >> >>> https://github.com/linux-rdma/rdma-core, but DPDK installation still fails >> >>> with >> >>> >> >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: >> >>> infiniband/mlx5dv.h: No such file or directory >> >>> 15 | #include <infiniband/mlx5dv.h> >> >>> >> >>> Any advice on getting the mlx5 drivers to work again would be helpful! >> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dpdk-users] Building mlx5 with rdma-core/OFED 2021-08-03 14:23 ` PATRICK KEROULAS @ 2021-08-13 7:02 ` Gerry Wan 0 siblings, 0 replies; 7+ messages in thread From: Gerry Wan @ 2021-08-13 7:02 UTC (permalink / raw) To: PATRICK KEROULAS; +Cc: users I was able to get things to work with rdma-core by completely deleting my DPDK build directory and building from scratch, not just clean. The latest OFED (5.4) also works with kernel 5.4.0-80-generic, in case others are also having similar issues and cannot use rdma-core. On Tue, Aug 3, 2021 at 7:24 AM PATRICK KEROULAS < patrick.keroulas@radio-canada.ca> wrote: > Have you checked that: > - the missing symbols like `mlx5dv_dr_action_create_default_miss` are > defined in mlx5dv.h? > - there is no other mlx5dv.h from previous version installed somewhere? > > I recommend to keep the package versions provided for your distro and > downgrade dpdk if necessary. > > > On Mon, Aug 2, 2021 at 4:18 PM Gerry Wan <gerryw@stanford.edu> wrote: > > > > I am running Ubuntu 20.04, and the mlx5dv.h is indeed in > `/usr/include/infiniband/`. > > > > The steps I take to build DPDK are > > $ meson --prefix=$DPDK_PATH build > > $ cd build > > $ sudo ninja install -> fails here > > $ sudo ldconfig > > > > How can I install the correct versions of rdma-core and libibverbs? > > > > On Mon, Aug 2, 2021 at 12:59 PM PATRICK KEROULAS < > patrick.keroulas@radio-canada.ca> wrote: > >> > >> I guess you're running Focal or Groovy release so rdma-core and > libibverbs > >> are >v28: https://launchpad.net/rdma-core > >> For this version, your missing symbols are defined: > >> > https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/mlx5dv.h > >> I expect this header to be installed in `/usr/include/infiniband/`. > >> Please verify. > >> > >> Maybe something wrong with ninja, try meson or make. > >> > >> On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan <gerryw@stanford.edu> wrote: > >> > > >> > How did you install these packages? Are there extra environment > variables I need to set? > >> > After running the OFED uninstall script, > >> > > >> > apt-get install rdma-core > >> > apt-get install libibverbs-dev > >> > > >> > both work fine, but ninja install now fails on > >> > > >> > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. > >> > FAILED: drivers/librte_common_mlx5.so.21.1 > >> > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@ > @rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' > -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group > -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl > -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 > lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 > lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 > drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 > lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 > -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map > /usr/lib/x86_64-linux-gnu/libmlx5.so > /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group > '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_aso': > >> > mlx5_glue.c:(.text+0x21f): undefined reference to > `mlx5dv_dr_action_create_aso' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_action_create_dest_array': > >> > mlx5_glue.c:(.text+0x23b): undefined reference to > `mlx5dv_dr_action_create_dest_array' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_sampler': > >> > mlx5_glue.c:(.text+0x25b): undefined reference to > `mlx5dv_dr_action_create_flow_sampler' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_alloc_pp': > >> > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_default_miss': > >> > mlx5_glue.c:(.text+0x4bb): undefined reference to > `mlx5dv_dr_action_create_default_miss' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_free_pp': > >> > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_reclaim_domain_memory': > >> > mlx5_glue.c:(.text+0x2a7): undefined reference to > `mlx5dv_dr_domain_set_reclaim_device_memory' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_devx_port_query': > >> > mlx5_glue.c:(.text+0x325): undefined reference to > `mlx5dv_query_devx_port' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': > >> > mlx5_glue.c:(.text+0x545): undefined reference to > `mlx5dv_dr_action_create_dest_devx_tir' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_pop_vlan': > >> > mlx5_glue.c:(.text+0x655): undefined reference to > `mlx5dv_dr_action_create_pop_vlan' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_push_vlan': > >> > mlx5_glue.c:(.text+0x665): undefined reference to > `mlx5dv_dr_action_create_push_vlan' > >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_dest_port': > >> > mlx5_glue.c:(.text+0x685): undefined reference to > `mlx5dv_dr_action_create_dest_ib_port' > >> > collect2: error: ld returned 1 exit status > >> > [2/1756] Compiling C object 'drivers/a715181@ > @tmp_rte_mempool_octeontx@sta/mempool_octeontx_octeontx_fpavf.c.o'. > >> > ninja: build stopped: subcommand failed. > >> > > >> > > >> > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS < > patrick.keroulas@radio-canada.ca> wrote: > >> >> > >> >> > >> >> Hello Gerry, > >> >> I went through this a while ago and had to properly remove > everything related > >> >> to OFED with the uninstaller. Then, install rdma-core and > libibverbs-dev which > >> >> includes infiniband/mlx5dv.h IIRC. > >> >> Regards, > >> >> Patrick > >> >> > >> >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan <gerryw@stanford.edu> > wrote: > >> >>> > >> >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal > machine > >> >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk > >> >>> --upstream-libs --with-mft --with-kernel-mft). However, it now > fails with > >> >>> the message > >> >>> > >> >>> Checking SW Requirements... > >> >>> Removing old packages... > >> >>> Uninstalling the previous version of MLNX_OFED_LINUX > >> >>> Installing new packages Installing mft-4.16.1... > >> >>> Installing ofed-scripts-5.2... > >> >>> Installing mstflint-4.15.0... > >> >>> Installing mlnx-ofed-kernel-utils-5.2... > >> >>> Installing mlnx-ofed-kernel-dkms-5.2... > >> >>> Failed to install mlnx-ofed-kernel-dkms DEB > >> >>> Collecting debug info... > >> >>> See > /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log > >> >>> > >> >>> It appears that my kernel version got upgraded to 5.4.0-80-generic > which I > >> >>> think may be the issue. I've also tried to install using > >> >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. > >> >>> > >> >>> The DPDK website says that rdma-core is now recommended over OFED, > but the > >> >>> documentation for how to use it isn't very clear to me. I've tried > to > >> >>> install it via > >> >>> > >> >>> apt-get install rdma-core > >> >>> > >> >>> as well as running the quick start script listed here: > >> >>> https://github.com/linux-rdma/rdma-core, but DPDK installation > still fails > >> >>> with > >> >>> > >> >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: > >> >>> infiniband/mlx5dv.h: No such file or directory > >> >>> 15 | #include <infiniband/mlx5dv.h> > >> >>> > >> >>> Any advice on getting the mlx5 drivers to work again would be > helpful! > >> > > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-08-13 7:03 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-02 0:23 [dpdk-users] Building mlx5 with rdma-core/OFED Gerry Wan 2021-08-02 13:24 ` PATRICK KEROULAS 2021-08-02 18:35 ` Gerry Wan 2021-08-02 19:59 ` PATRICK KEROULAS 2021-08-02 20:18 ` Gerry Wan 2021-08-03 14:23 ` PATRICK KEROULAS 2021-08-13 7:02 ` Gerry Wan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).