From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DC78A0A0C for ; Mon, 2 Aug 2021 22:18:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1EB1F4117A; Mon, 2 Aug 2021 22:18:36 +0200 (CEST) Received: from mx0a-00000d04.pphosted.com (mx0a-00000d04.pphosted.com [148.163.149.245]) by mails.dpdk.org (Postfix) with ESMTP id C688041147 for ; Mon, 2 Aug 2021 22:18:34 +0200 (CEST) Received: from pps.filterd (m0102890.ppops.net [127.0.0.1]) by mx0a-00000d04.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 172KHdYx021407 for ; Mon, 2 Aug 2021 13:18:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stanford.edu; h=mime-version : references : in-reply-to : from : date : message-id : subject : to : cc : content-type; s=pps05272020; bh=ZKHiar/c2kq4WNSS6rwTC6OeTKqCnDs2UiR76j2Rib4=; b=DSL+w9KbgcFBZo5R7p5F7C2ZLZv9bMXykljPYuF9+Ic3tL1maNEmyfAAJNqRaV2qSNk6 gL1DpGVsrzJ/RDPnwKpQmgDJyMAAKxv5K1ozgvNVNfenuUV/Vw8LfVpNVp7K6ouUCsH9 NpI+k355qLCLgxIOCp9q4R6J6DckJHCKvcLXVwKOC5t605myvlc57ZAjbFrUte7rtncx 1y0HOvxeShm9OtMGdcwqSMNW/TDwXe+UYOb3EPMiuYZRoAV8pDKuTHmVQI2/PkHeB2BK xfFJWM0gtpuw3809iN5WJQaKhikfPxPFTVzVuTnwW/tESKmYozuOTZMU7vDWabulzgED Pw== Received: from mx0b-00000d03.pphosted.com (mx0b-00000d03.pphosted.com [148.163.153.234]) by mx0a-00000d04.pphosted.com with ESMTP id 3a6dqpuh1b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Aug 2021 13:18:32 -0700 Received: from pps.filterd (m0190090.ppops.net [127.0.0.1]) by mx0a-00000d03.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 172KGbIV011647 for ; Mon, 2 Aug 2021 13:18:31 -0700 Received: from mx0b-00000d06.pphosted.com (mx0b-00000d06.pphosted.com [148.163.139.119]) by mx0a-00000d03.pphosted.com with ESMTP id 3a563dr3ws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Aug 2021 13:18:31 -0700 Received: from pps.filterd (m0167937.ppops.net [127.0.0.1]) by mx0b-00000d06.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 172KCNUr009185 for ; Mon, 2 Aug 2021 13:18:31 -0700 Received: from smtp.stanford.edu (smtp6.stanford.edu [171.67.219.73]) by mx0b-00000d06.pphosted.com with ESMTP id 3a54tvj8kk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 02 Aug 2021 13:18:31 -0700 Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: gerryw) by smtp.stanford.edu (Postfix) with ESMTPSA id 91CB24408D for ; Mon, 2 Aug 2021 13:18:30 -0700 (PDT) Received: by mail-il1-f181.google.com with SMTP id z3so17565954ile.12 for ; Mon, 02 Aug 2021 13:18:30 -0700 (PDT) X-Gm-Message-State: AOAM530pSEZEBsiH/NYPRw+e1Ovmspjy9wuz3h3MmQB5VrnqiJpYYgZX ZJDGY5KedyX3zzBDmzSBGAjZY4KYYOSQCw0MUcU= X-Google-Smtp-Source: ABdhPJwO58BErCXFbT3ta3wUWSFVuO5maZbyZTJvqvtUanjGSdp1em+4sF1Sh0/QSJKvylUxvIg3qYCUCED23oMVj0w= X-Received: by 2002:a92:c10c:: with SMTP id p12mr2514368ile.92.1627935510087; Mon, 02 Aug 2021 13:18:30 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Gerry Wan Date: Mon, 2 Aug 2021 13:18:19 -0700 X-Gmail-Original-Message-ID: Message-ID: To: PATRICK KEROULAS Cc: users@dpdk.org x-proofpoint-stanford-dir: outbound X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-08-02_07:2021-08-02, 2021-08-02 signatures=0 X-Proofpoint-GUID: XpPlqpn2YBExQgZz3-iqLWx1SbY0bFjQ X-Proofpoint-ORIG-GUID: XpPlqpn2YBExQgZz3-iqLWx1SbY0bFjQ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.790 definitions=2021-08-02_07:2021-08-02, 2021-08-02 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 clxscore=1015 bulkscore=0 impostorscore=0 spamscore=0 adultscore=100 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108020131 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-users] Building mlx5 with rdma-core/OFED X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" I am running Ubuntu 20.04, and the mlx5dv.h is indeed in `/usr/include/infiniband/`. The steps I take to build DPDK are $ meson --prefix=$DPDK_PATH build $ cd build $ sudo ninja install -> fails here $ sudo ldconfig How can I install the correct versions of rdma-core and libibverbs? On Mon, Aug 2, 2021 at 12:59 PM PATRICK KEROULAS < patrick.keroulas@radio-canada.ca> wrote: > I guess you're running Focal or Groovy release so rdma-core and libibverbs > are >v28: https://launchpad.net/rdma-core > For this version, your missing symbols are defined: > > https://github.com/linux-rdma/rdma-core/blob/stable-v28/providers/mlx5/mlx5dv.h > I expect this header to be installed in `/usr/include/infiniband/`. > Please verify. > > Maybe something wrong with ninja, try meson or make. > > On Mon, Aug 2, 2021 at 2:35 PM Gerry Wan wrote: > > > > How did you install these packages? Are there extra environment > variables I need to set? > > After running the OFED uninstall script, > > > > apt-get install rdma-core > > apt-get install libibverbs-dev > > > > both work fine, but ninja install now fails on > > > > [1/1756] Linking target drivers/librte_common_mlx5.so.21.1. > > FAILED: drivers/librte_common_mlx5.so.21.1 > > cc -o drivers/librte_common_mlx5.so.21.1 'drivers/a715181@ > @rte_common_mlx5@sha/meson-generated_.._rte_common_mlx5.pmd.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_devx_cmds.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mp.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_mr.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_pci.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_mlx5_common_devx.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_nl.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_os.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_common_verbs.c.o' > 'drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o' > -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group > -Wl,-soname,librte_common_mlx5.so.21 -Wl,--no-as-needed -pthread -lm -ldl > -lnuma lib/librte_hash.so.21.1 lib/librte_eal.so.21.1 > lib/librte_kvargs.so.21.1 lib/librte_telemetry.so.21.1 > lib/librte_ring.so.21.1 lib/librte_rcu.so.21.1 lib/librte_pci.so.21.1 > drivers/librte_bus_pci.so.21.1 lib/librte_net.so.21.1 > lib/librte_mbuf.so.21.1 lib/librte_mempool.so.21.1 > -Wl,--version-script=/home/gerryw/dpdk/dpdk-21.02/drivers/common/mlx5/version.map > /usr/lib/x86_64-linux-gnu/libmlx5.so > /usr/lib/x86_64-linux-gnu/libibverbs.so -Wl,--end-group > '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/lib > -Wl,-rpath-link,/home/gerryw/dpdk/dpdk-21.02/build/drivers > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_aso': > > mlx5_glue.c:(.text+0x21f): undefined reference to > `mlx5dv_dr_action_create_aso' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_action_create_dest_array': > > mlx5_glue.c:(.text+0x23b): undefined reference to > `mlx5dv_dr_action_create_dest_array' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_sampler': > > mlx5_glue.c:(.text+0x25b): undefined reference to > `mlx5dv_dr_action_create_flow_sampler' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_alloc_pp': > > mlx5_glue.c:(.text+0x28b): undefined reference to `mlx5dv_pp_alloc' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_default_miss': > > mlx5_glue.c:(.text+0x4bb): undefined reference to > `mlx5dv_dr_action_create_default_miss' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_free_pp': > > mlx5_glue.c:(.text+0x277): undefined reference to `mlx5dv_pp_free' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_reclaim_domain_memory': > > mlx5_glue.c:(.text+0x2a7): undefined reference to > `mlx5dv_dr_domain_set_reclaim_device_memory' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_devx_port_query': > > mlx5_glue.c:(.text+0x325): undefined reference to > `mlx5dv_query_devx_port' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dv_create_flow_action_dest_devx_tir': > > mlx5_glue.c:(.text+0x545): undefined reference to > `mlx5dv_dr_action_create_dest_devx_tir' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_pop_vlan': > > mlx5_glue.c:(.text+0x655): undefined reference to > `mlx5dv_dr_action_create_pop_vlan' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_push_vlan': > > mlx5_glue.c:(.text+0x665): undefined reference to > `mlx5dv_dr_action_create_push_vlan' > > /usr/bin/ld: drivers/a715181@@tmp_rte_common_mlx5@sta/common_mlx5_linux_mlx5_glue.c.o: > in function `mlx5_glue_dr_create_flow_action_dest_port': > > mlx5_glue.c:(.text+0x685): undefined reference to > `mlx5dv_dr_action_create_dest_ib_port' > > collect2: error: ld returned 1 exit status > > [2/1756] Compiling C object 'drivers/a715181@ > @tmp_rte_mempool_octeontx@sta/mempool_octeontx_octeontx_fpavf.c.o'. > > ninja: build stopped: subcommand failed. > > > > > > On Mon, Aug 2, 2021 at 6:24 AM PATRICK KEROULAS < > patrick.keroulas@radio-canada.ca> wrote: > >> > >> > >> Hello Gerry, > >> I went through this a while ago and had to properly remove everything > related > >> to OFED with the uninstaller. Then, install rdma-core and > libibverbs-dev which > >> includes infiniband/mlx5dv.h IIRC. > >> Regards, > >> Patrick > >> > >> On Sun, Aug 1, 2021 at 8:23 PM Gerry Wan wrote: > >>> > >>> I recently tried to reinstall MLX OFED 5.2-2.2.0.0 on my bare metal > machine > >>> Ubuntu 20.04 (previously using the command ./mlnxofedinstall --dpdk > >>> --upstream-libs --with-mft --with-kernel-mft). However, it now fails > with > >>> the message > >>> > >>> Checking SW Requirements... > >>> Removing old packages... > >>> Uninstalling the previous version of MLNX_OFED_LINUX > >>> Installing new packages Installing mft-4.16.1... > >>> Installing ofed-scripts-5.2... > >>> Installing mstflint-4.15.0... > >>> Installing mlnx-ofed-kernel-utils-5.2... > >>> Installing mlnx-ofed-kernel-dkms-5.2... > >>> Failed to install mlnx-ofed-kernel-dkms DEB > >>> Collecting debug info... > >>> See /tmp/MLNX_OFED_LINUX.9221.logs/mlnx-ofed-kernel-dkms.debinstall.log > >>> > >>> It appears that my kernel version got upgraded to 5.4.0-80-generic > which I > >>> think may be the issue. I've also tried to install using > >>> --add-kernel-support --kernel 5.4.0-80-generic, with no luck. > >>> > >>> The DPDK website says that rdma-core is now recommended over OFED, but > the > >>> documentation for how to use it isn't very clear to me. I've tried to > >>> install it via > >>> > >>> apt-get install rdma-core > >>> > >>> as well as running the quick start script listed here: > >>> https://github.com/linux-rdma/rdma-core, but DPDK installation still > fails > >>> with > >>> > >>> ../drivers/common/mlx5/linux/mlx5_glue.h:15:10: fatal error: > >>> infiniband/mlx5dv.h: No such file or directory > >>> 15 | #include > >>> > >>> Any advice on getting the mlx5 drivers to work again would be helpful! > >