From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F4FBA0577 for ; Mon, 6 Apr 2020 11:14:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B20C82BE9; Mon, 6 Apr 2020 11:14:14 +0200 (CEST) Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) by dpdk.org (Postfix) with ESMTP id 7FC572B96 for ; Mon, 6 Apr 2020 11:14:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3492; q=dns/txt; s=iport; t=1586164452; x=1587374052; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=ubQmR3ei2bbc/jp0N92wtBjoj8IHpbrsEspBK3693Ro=; b=e+dOPCOK1eMYfeQDACD9Ji9w6feZbp5V6llrS4vdYVY40yAb+mjx8UsD m0r75SAjRK072ZChx/qzBA3VjtUC8tw9ydp1rSGZ4smyvqgR53TgtqBhf 6FGLRnfMT/9nN1IeRlMno7273oTPpNTo1Fs4MLUpCk4TpWW5NGQy6Yejq U=; IronPort-PHdr: =?us-ascii?q?9a23=3AMcN5NxUVn9E/wJfLd8g2dXY3B9LV8LGuZFwc94?= =?us-ascii?q?YnhrRSc6+q45XlOgnF6O5wiEPSANSJ8OpK3uzRta2oGXcN55qMqjgjSNRNTF?= =?us-ascii?q?dE7KdehAk8GIiAAEz/IuTtankwE81EUlJN9HCgOk8TE8H7NBXf?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CrAADB8Ype/5JdJa1lHQEBAQkBEQU?= =?us-ascii?q?FAYFoBwELAYFTUAWBRCAECyoKh1YDimaCX5gegS6BJANUCgEBAQwBAS0CBAE?= =?us-ascii?q?BhEQCgkYkNQgOAgMBAQsBAQUBAQECAQUEbYVWDIVwAQEBAQIBEhUTBgEBNwE?= =?us-ascii?q?ECwIBCBQiEDIlAQEEAQ0NGoVQAw4gAQOkEgKBOYhigXQzgn8BAQWFPhiCDQm?= =?us-ascii?q?BOAGMMhqBQT+BEUOCTT6ENhqDQoIsjgKLbJc0CoI9l0eCTog4kHyPN5wlAgQ?= =?us-ascii?q?CBAUCDgEBBYFTATeBV3AVgyRQGA2OHQwXg1CKVXSBKY0fAYEPAQE?= X-IronPort-AV: E=Sophos;i="5.72,350,1580774400"; d="scan'208";a="741969827" Received: from rcdn-core-10.cisco.com ([173.37.93.146]) by rcdn-iport-7.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 06 Apr 2020 09:14:11 +0000 Received: from XCH-ALN-005.cisco.com (xch-aln-005.cisco.com [173.36.7.15]) by rcdn-core-10.cisco.com (8.15.2/8.15.2) with ESMTPS id 0369EBwg025563 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Mon, 6 Apr 2020 09:14:11 GMT Received: from xhs-rcd-002.cisco.com (173.37.227.247) by XCH-ALN-005.cisco.com (173.36.7.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Apr 2020 04:14:11 -0500 Received: from xhs-rtp-002.cisco.com (64.101.210.229) by xhs-rcd-002.cisco.com (173.37.227.247) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Apr 2020 04:14:10 -0500 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (64.101.32.56) by xhs-rtp-002.cisco.com (64.101.210.229) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 6 Apr 2020 05:14:10 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DPxzH6tjBZkzdfjf99B7L2EkrtjxAF5ZtxB36KqbNYW56fl4l5Ay+dqSTjJWhNyHZL5V5MWA8T6Tc0JuIuXQBSGkGxD4D0bydH438kuAm8+h6KPn65i7nFvxIRMROxYmTtntjO72DXMnngFa0tFXX3aaOsBb8qWSUBMCe8Wb7fibLcVDI/O0fKCbX69GfKYMX5VYnkAqZViG9NMcBUpgh0vFvLdEi9hwrvmyjxuhPGi5Uyb5o8TPgg2QoCalBRidt8ZHfSNRhPU6kOFFl+Ac3h7UG9SSN7fOFjc3TkYG1C7HVtPIm/HCMR1HfV2j7gjDePMmGzSsRWrxcoRTyq878g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vOF644aMASDpzJkQFMfdq/m3p3j2tYN7YEcqJ81dD/4=; b=X2PMMl8F+zgJd2xbVpzUm4JU49dNKnzRA/3tFxrDru3u5hQsBO/SZEBok9uTZ39MkREWLkXPzQvptI0nTj2s/QIQxOVugrBxnho0v4enofvzWXRTEK+Bkha9ARSvvCmSF9qF0TE9mmi1UksLqSJ3oMpdlVtCRXAcm+Rt+uHJbTnGggMoZu+isdKoyarbQfA2K8kitVqXp12sF6GgkOv4tZAvkJv5oNblmr+pzJ2t08bJDdm7YS+pDYCCs8ZGLPlFo9VgCN30br+q1WQu+GG3EjwsrflDkIRvNdbRz09zz0fxnQ5l4JAcXc2LUQ+SxJ4SScjybpRsT4IXSUiE8nRLHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vOF644aMASDpzJkQFMfdq/m3p3j2tYN7YEcqJ81dD/4=; b=FwuyW3P6JrvL2ir1pQCnbH4xeBjjdWliW0WS6o5gV46p1Sbi6QV9sr/PRaLMunuturQH26zFeKKHdgT8AEqlCQ6bRM/FjJPPPhTIcnoeabmLcjSyWg1VGl7D7m+jquPFNGMxEjKSpzZcdsTFAxequu1SflEuF+W4IaeX/smcJ50= Received: from CH2PR11MB4327.namprd11.prod.outlook.com (2603:10b6:610:43::30) by CH2PR11MB4325.namprd11.prod.outlook.com (2603:10b6:610:3e::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.20; Mon, 6 Apr 2020 09:14:08 +0000 Received: from CH2PR11MB4327.namprd11.prod.outlook.com ([fe80::2d84:413f:5702:c519]) by CH2PR11MB4327.namprd11.prod.outlook.com ([fe80::2d84:413f:5702:c519%5]) with mapi id 15.20.2878.018; Mon, 6 Apr 2020 09:14:08 +0000 From: "Benoit Ganne (bganne)" To: Matan Azrad , "users@dpdk.org" CC: Shahaf Shuler , Slava Ovsiienko Thread-Topic: mlx5 pmd + rdma-core 28 init failure Thread-Index: AdYIQ9Z8ZbHXLbfSTDeYhwt0ihzoWAAwhXsgAAERNsoAAXwMgACKHK+gAC5NtaA= Date: Mon, 6 Apr 2020 09:14:08 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=bganne@cisco.com; x-originating-ip: [92.154.90.183] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 85593cde-453b-4f0f-5505-08d7da0add08 x-ms-traffictypediagnostic: CH2PR11MB4325: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0365C0E14B x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR11MB4327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(346002)(376002)(39860400002)(396003)(366004)(136003)(9686003)(7696005)(86362001)(8676002)(33656002)(478600001)(55016002)(4326008)(186003)(2906002)(316002)(54906003)(110136005)(76116006)(71200400001)(5660300002)(6506007)(81166006)(81156014)(66476007)(64756008)(66946007)(26005)(52536014)(66446008)(66556008)(8936002); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: gk/BEg1yRkULUBkNLPmqbsmsPzMMQ59kgDxkkyxcANjdJPXLVb8/4NK4Fyukm6RHhFT1yxfrPRYnHMQXKj15cE8CENYk5ZyEXMdC69wUPw9NinKoelc8zMKzIRfS8Iu2PFgCy3l1BJVnyh0Spg+a12IXcIOqcaY0JRqnIs9ppvMhmR6LDJ1Nv4mOurHKzC6H44ziK4x94Ud0G85QrDKelOevkYQxscCw5nw6JbVPacuZNDBltjYCUV5nNaDfETCjyDhJkc4AzcHphTsnxle91/8ROAK0XLdliI9+yJNJ153oqeRwOveZQJWEoRuXhGL1r1BH/oQQotzGIQ4SrlQkZlBvUyfj6uhRSkOE1BCi4USOXA/78DfhRW09uertIz4Snv+3vjt2B9XSblDR+yBRGIBFH7LkEVr1mAtP991nWDlf4Ezvr+CfKqKw8ZNjuaQv x-ms-exchange-antispam-messagedata: 1Gn+fcmJjypFsBnMWiD06Y6IgqxVXsd8lzbWH5+ndPs7o7RxGcoQ0u0rq2mUhHfzkS256D2NNgiREoQtvTvPEZeJFCxrmJ5lh/zxUBQeU3fD0FUYrFIoxLiYgHW5//q831L2YPYdgGnH+lX9SiY63w== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 85593cde-453b-4f0f-5505-08d7da0add08 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2020 09:14:08.7904 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: v5HJ8sh0s6xB6+CVlA5SXW5Gz76KBWCZFVOmpTHaCMSwZ4slP1rMYpfkxPf01quuVsUefI5VZQXRDa8XxlVerg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR11MB4325 X-OriginatorOrg: cisco.com X-Outbound-SMTP-Client: 173.36.7.15, xch-aln-005.cisco.com X-Outbound-Node: rcdn-core-10.cisco.com Subject: Re: [dpdk-users] mlx5 pmd + rdma-core 28 init failure X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi Matan, > Please let us know if working with devargs dv_flow_en=3D0 solves the > issue.... > ./build/app/testpmd --log-level=3D8 --log-level=3Dpmd.common.mlx5:8 - -lo= g- > level=3Dpmd.net.mlx5:8 -w 0000:5e:00.0.dv_flow_en=3D0 -w > 0000:5e:00.1,dv_flow_en=3D0 -l 4,11,35 -- -a -- forward-mode=3Drxonly Yes it does. However it seems to let the device in an unstable state: it doesn't stop pr= operly, and I need to restart the whole server for the interface to come ba= ck (reloading all mlx drivers modules and/or PCIe resetting is not enough).= Here is what happens when I try to start testpmd again after the 1st try: ~# sudo ./build/app/testpmd --log-level=3D8 --log-level=3Dpmd.common.mlx5:8= --log-level=3Dpmd.net.mlx5:8 -w 0000:5e:00.0,dv_flow_en=3D0 -w 0000:5e:00.= 1,dv_flow_en=3D0 -l 4,11,35 -- -a --forward-mode=3Drxonly EAL: Detected 48 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: No available hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:5e:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1013 net_mlx5 net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" net_mlx5: mlx5.c:3084: mlx5_pci_probe(): no Verbs device matches PCI device= 0000:5e:00.0, are kernel drivers loaded? EAL: Requested device 0000:5e:00.0 cannot be used EAL: PCI device 0000:5e:00.1 on NUMA socket 0 EAL: probe driver: 15b3:1013 net_mlx5 net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for device= "mlx5_1" net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device "0000:5e:00= .1" net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported PANIC in mlx5_mr_btree_init(): line 197 assert "!bt->table && !bt->size" failed 14: [./build/app/testpmd(_start+0x2a) [0x560a46a469aa]] 13: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f082ac3ab9= 7]] 12: [./build/app/testpmd(main+0x96) [0x560a46a50da9]] 11: [./build/app/testpmd(rte_eal_init+0x99d) [0x560a46d6f305]] 10: [./build/app/testpmd(rte_bus_probe+0x4c) [0x560a46d898ae]] 9: [./build/app/testpmd(rte_pci_probe+0x96) [0x560a46dcdc0e]] 8: [./build/app/testpmd(+0x4c4b42) [0x560a46dcdb42]] 7: [./build/app/testpmd(+0x4c4952) [0x560a46dcd952]] 6: [./build/app/testpmd(+0xb33efe) [0x560a4743cefe]] 5: [./build/app/testpmd(+0xb2ff4e) [0x560a47438f4e]] 4: [./build/app/testpmd(+0xb2bd88) [0x560a47434d88]] 3: [./build/app/testpmd(mlx5_mr_btree_init+0x90) [0x560a47b7de13]] 2: [./build/app/testpmd(__rte_panic+0xed) [0x560a46d7e046]] 1: [./build/app/testpmd(rte_dump_stack+0x39) [0x560a46d7dea7]] > Please let us know also your FW version... ~# sudo mstflint -d 5e:00.0 q Image type: FS3 FW Version: 12.26.4012 FW Release Date: 10.12.2019 Product Version: 6.4012 Rom Info: type=3DUEFI version=3D14.19.17 cpu=3DAMD64,AARCH64 type=3DPXE version=3D3.5.805 cpu=3DAMD64 Description: UID GuidsNumber Base GUID: 248a0703005b1414 4 Base MAC: 0000248a075b1414 4 Image VSD: N/A Device VSD: N/A PSID: MT_2150110033 Security Attributes: N/A Best, ben