From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1957EA0547 for ; Tue, 3 Aug 2021 16:24:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AF607411A7; Tue, 3 Aug 2021 16:24:10 +0200 (CEST) Received: from eu-smtp-delivery-104.mimecast.com (eu-smtp-delivery-104.mimecast.com [185.58.85.104]) by mails.dpdk.org (Postfix) with ESMTP id A807440E3C for ; Tue, 3 Aug 2021 16:24:09 +0200 (CEST) Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id uk-mta-262-IvbyJJIzMTW4ew7VTPfz6w-1; Tue, 03 Aug 2021 15:24:07 +0100 X-MC-Unique: IvbyJJIzMTW4ew7VTPfz6w-1 Received: by mail-wm1-f72.google.com with SMTP id d72-20020a1c1d4b0000b029025164ff3ebfso971393wmd.7 for ; Tue, 03 Aug 2021 07:24:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=NHxgMaQ4W93TWiWNc/NWGfcU4MJZ5M18udzsAFQsy30=; b=LrgDB/OnDRMEGsfgBkJTTPUMRbSBzYMOAQGzinXayh7QmrRmXTpk0BI8MYkvGVIUkO cxlyD5WeO7oUuEKK0Iqep98iwJAew3kMFr1Wr/vW+SGp6/u2oPelscszV/g5e6TP/wi6 Vae5JYX72LHzgba6WibQI0xoEGC8M/Lx/QHZwime2Q1vEM6BcKBFIXA5KBHdWDVvOt0x rT3hh/CdMqmzjjn8CHcJ9brsulczrIkpWWnuIy3sWgO0hf/St4PH7vsMeG1uV+2nFxuY CNbLXO0RMEds1HN1RR/BBmKps5t54nKRoX7EM/trGIMLMKGiotMv+3VObUSQi4nUmOf3 Wj4Q== X-Gm-Message-State: AOAM530yh1uu/cXWF+gzYX+qrWoupeGuDmnJFSJCwLK5Pjgv8882ayfz iPLjoIY19npNfm22dKpz/V+qXUwJV80+wLWSAQzj4mxY8R7+5THDhev4FfVYjQQOhN4fmQX4JH2 InQ+Q4vMY1KbJNoL8has= X-Received: by 2002:a5d:424d:: with SMTP id s13mr23609277wrr.356.1628000646741; Tue, 03 Aug 2021 07:24:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzIBZ0sqSHxZYTSNpcPNFd4/l7OqROJEAmsO9PGso5l9AyZsOstWHac2O0Ji330l3Cct65hTN2xCgPzgwWcBOI= X-Received: by 2002:a5d:424d:: with SMTP id s13mr23609248wrr.356.1628000646453; Tue, 03 Aug 2021 07:24:06 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: PATRICK KEROULAS Date: Tue, 3 Aug 2021 10:23:55 -0400 Message-ID: To: Gerry Wan Cc: users@dpdk.org Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUK104A247 smtp.mailfrom=patrick.keroulas@radio-canada.ca X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: radio-canada.ca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-users] Building mlx5 with rdma-core/OFED X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Have you checked that: - the missing symbols like `mlx5dv_dr_action_create_default_miss` are defined in mlx5dv.h? - there is no other mlx5dv.h from previous version installed somewhere? I recommend to keep the package versions provided for your distro and downgrade dpdk if necessary. On Mon, Aug 2, 2021 at 4:18 PM Gerry Wan wrote: > > I am running Ubuntu 20.04, and the mlx5dv.h is indeed in `/usr/include/in= finiband/`. > > The steps I take to build DPDK are > $ meson --prefix=3D$DPDK_PATH build > $ cd build > $ sudo ninja install -> fails here > $ sudo ldconfig > > How can I install the correct versions of rdma-core and libibverbs? > > On Mon, Aug 2, 2021 at 12:59 PM PATRICK KEROULAS wrote: >> >> I guess you're running Focal or Groovy release so rdma-core and libibver= bs >> are >v28: https://launchpad.net/rdma-core >> For this version, your missing symbols are defined: >> https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/m= lx5dv.h >> I expect this header to be installed in `/usr/include/infiniband/`. >> Please verify. >> >> Maybe something wrong with ninja, try meson or make. >> >> On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan wrote: >> > >> > How did you install these packages? Are there extra environment variab= les I need to set? >> > After running the OFED uninstall script, >> > >> > apt-get install rdma-core >> > apt-get install libibverbs-dev >> > >> > both work fine, but ninja install now fails on >> > >> > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. >> > FAILED: drivers/librte_common_mlx5.so.21.1 >> > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@@rte_common= _mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' 'drivers/a715181@@tmp= _rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' 'drivers/a715181@@tmp_= rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' 'drivers/a715181@@tmp_rte_= common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' 'drivers/a715181@@tmp_rte_c= ommon_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' 'drivers/a715181@@tmp_rte_co= mmon_mlx5@sta/common_mlx5_mlx5_malloc.c.o' 'drivers/a715181@@tmp_rte_common= _mlx5@sta/common_mlx5_mlx5_common_pci.c.o' 'drivers/a715181@@tmp_rte_common= _mlx5@sta/common_mlx5_mlx5_common_devx.c.o' 'drivers/a715181@@tmp_rte_commo= n_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' 'drivers/a715181@@tmp_rte_common_= mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' 'drivers/a715181@@tmp_rte_co= mmon_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' 'drivers/a715181@@tm= p_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' -Wl,--as-needed -Wl,= --no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_c= ommon_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_has= h.so.21.1 lib/librte_eal.so.21.1 lib/librte_kvargs.so.21.1 lib/librte_telem= etry.so.21.1 lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.= so.21.1 drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 lib/librte_mb= uf.so.21.1 lib/librte_mempool.so.21.1 -Wl,--version-script=3D/home/gerryw/d= pdk/dpdk-21.02/drivers/common/mlx5/version.map /usr/lib/x86_64-linux-gnu/li= bmlx5.so /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group '-Wl,-rpat= h,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/bui= ld/lib -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_aso': >> > mlx5_glue.c:(.text+0x21f): undefined reference to `mlx5dv_dr_action_cr= eate_aso' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_action_create_dest_array': >> > mlx5_glue.c:(.text+0x23b): undefined reference to `mlx5dv_dr_action_cr= eate_dest_array' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_sampler': >> > mlx5_glue.c:(.text+0x25b): undefined reference to `mlx5dv_dr_action_cr= eate_flow_sampler' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dv_alloc_pp': >> > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_default_miss'= : >> > mlx5_glue.c:(.text+0x4bb): undefined reference to `mlx5dv_dr_action_cr= eate_default_miss' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dv_free_pp': >> > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_reclaim_domain_memory': >> > mlx5_glue.c:(.text+0x2a7): undefined reference to `mlx5dv_dr_domain_se= t_reclaim_device_memory' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_devx_port_query': >> > mlx5_glue.c:(.text+0x325): undefined reference to `mlx5dv_query_devx_p= ort' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dv_create_flow_action_dest_devx_tir= ': >> > mlx5_glue.c:(.text+0x545): undefined reference to `mlx5dv_dr_action_cr= eate_dest_devx_tir' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_pop_vlan': >> > mlx5_glue.c:(.text+0x655): undefined reference to `mlx5dv_dr_action_cr= eate_pop_vlan' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_push_vlan': >> > mlx5_glue.c:(.text+0x665): undefined reference to `mlx5dv_dr_action_cr= eate_push_vlan' >> > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linu= x_mlx5_glue.c.o: in function `mlx5_glue_dr_create_flow_action_dest_port': >> > mlx5_glue.c:(.text+0x685): undefined reference to `mlx5dv_dr_action_cr= eate_dest_ib_port' >> > collect2: error: ld returned 1 exit status >> > [2/1756] Compiling C object 'drivers/a715181@@tmp_rte_mempool_octeontx= @sta/mempool_octeontx_octeontx_fpavf.c.o'. >> > ninja: build stopped: subcommand failed. >> > >> > >> > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS wrote: >> >> >> >> >> >> Hello Gerry, >> >> I went through this a while ago and had to properly remove everything= related >> >> to OFED with the uninstaller. Then, install rdma-core and libibverbs-= dev which >> >> includes infiniband/mlx5dv.h IIRC. >> >> Regards, >> >> Patrick >> >> >> >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan wrote: >> >>> >> >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal = machine >> >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk >> >>> --upstream-libs --with-mft --with-kernel-mft). However, it now fails= with >> >>> the message >> >>> >> >>> Checking SW Requirements... >> >>> Removing old packages... >> >>> Uninstalling the previous version of MLNX_OFED_LINUX >> >>> Installing new packages Installing mft-4.16.1... >> >>> Installing ofed-scripts-5.2... >> >>> Installing mstflint-4.15.0... >> >>> Installing mlnx-ofed-kernel-utils-5.2... >> >>> Installing mlnx-ofed-kernel-dkms-5.2... >> >>> Failed to install mlnx-ofed-kernel-dkms DEB >> >>> Collecting debug info... >> >>> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.= log >> >>> >> >>> It appears that my kernel version got upgraded to 5.4.0-80-generic w= hich I >> >>> think may be the issue. I've also tried to install using >> >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. >> >>> >> >>> The DPDK website says that rdma-core is now recommended over OFED, b= ut the >> >>> documentation for how to use it isn't very clear to me. I've tried t= o >> >>> install it via >> >>> >> >>> apt-get install rdma-core >> >>> >> >>> as well as running the quick start script listed here: >> >>> https://github.com/linux-rdma/rdma-core, but DPDK installation still= fails >> >>> with >> >>> >> >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: >> >>> infiniband/mlx5dv.h: No such file or directory >> >>> 15 | #include >> >>> >> >>> Any advice on getting the mlx5 drivers to work again would be helpfu= l! >>