From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 21653A0577 for ; Sun, 5 Apr 2020 13:01:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 893772C5E; Sun, 5 Apr 2020 13:01:52 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40045.outbound.protection.outlook.com [40.107.4.45]) by dpdk.org (Postfix) with ESMTP id 48479FFA for ; Sun, 5 Apr 2020 13:01:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C9i0ztqnRXrQrR2H2XPzq3dV/PgqwjrE5UgwT2qnzZ6c2R2MTCtTbAWeLLjWcRYJkh9m+dlHNNgTH3hy14Dz5CvuygoTkhGnKnsRv5qR5UKRs/b7GX2ukjTDrsn9R/Xp6+9gyXXXQJ05qa3IIwrNbnfzUE137bO5/bqv3GYJ0bcU8v/tq6kIJ76SeRbVxw+HTGpZlUbLI3j3uamlSvhlKET5DUN+gaXIwzWeIStpU/um7/PEvLbqsWArbWJiV7C8fFbLZlILJEfquwKwOmnSwjF4b7a5+PINfQ3vCEX5kAxms0zSZvgo3H/b2sQOZuG197dbgHMGnfJnFsayNikILw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LrZPzJ4t6vzv1eQo564X6F8jTcEZD7TAi1uNI0UzTEs=; b=cz3+4B1ze4KS6B8xv9piKR5UcxpFKgwle4qGeMkQDlxvbl7BdTSQ9Erq5alVUSEzhVngyarG45VSsJEcAFMB0n7jwBnctY9BFMrLblHTCpnKojKh6tJmTmKjJAVcesJTXgiK/hFjQpzXk3X1c2GGw0yjipLAToWcMzuh0e9Qh/YP91OaA0ndTXq3Acz9QhlFPNt2cZSQfW2iS7XxlWJUJ8fxcoJnyNcBuxhHJpmCe/5XYFtK/9ALURiGFEJfA/dLWpIilv6K5RmIY1xROcNj6TvvgszNXKPQdqZapHlwV8sK93391Rs7LLXWl7GTycy8eZwf8NJMLmvxC1BvyFumXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LrZPzJ4t6vzv1eQo564X6F8jTcEZD7TAi1uNI0UzTEs=; b=RQOgjQK+NBN8ynf/pJTkSV71u7CBrK29HYKFCK594RuoMtQF/bSZAoF0ZMprQ8vZoeCDoVzaZWjI0ORfltI4BpmAfU6KrejjQjzzBs7i62VDugeWBh2kKJ2vSonvURmlgV2AnYbSRKWm57IaMV67Im+S3ADVX9JjWWzgqZvhmLw= Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com (2603:10a6:208:f::11) by AM0PR0502MB4068.eurprd05.prod.outlook.com (2603:10a6:208:d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.20; Sun, 5 Apr 2020 11:01:49 +0000 Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::a564:a117:9022:1fee]) by AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::a564:a117:9022:1fee%7]) with mapi id 15.20.2878.018; Sun, 5 Apr 2020 11:01:48 +0000 From: Matan Azrad To: "Benoit Ganne (bganne)" , "users@dpdk.org" CC: Shahaf Shuler , Slava Ovsiienko Thread-Topic: mlx5 pmd + rdma-core 28 init failure Thread-Index: AdYIQ9Z8ZbHXLbfSTDeYhwt0ihzoWAAwhXsgAAERNsoAAXwMgACKHK+g Date: Sun, 5 Apr 2020 11:01:48 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [77.124.80.110] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 2a7f2fa9-cec6-4367-088b-08d7d950bd1c x-ms-traffictypediagnostic: AM0PR0502MB4068:|AM0PR0502MB4068: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:16; x-forefront-prvs: 03648EFF89 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0502MB4019.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(136003)(396003)(346002)(366004)(39860400002)(376002)(54906003)(66946007)(110136005)(81156014)(81166006)(316002)(76116006)(45080400002)(53546011)(6506007)(2906002)(478600001)(86362001)(30864003)(7696005)(8676002)(966005)(26005)(8936002)(33656002)(186003)(66556008)(66476007)(66446008)(64756008)(9686003)(5660300002)(55016002)(52536014)(4326008)(107886003)(71200400001)(579004); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 2DUqgmOEY18T2ILrvGU+gIg/YAtTFKgvxNACUTyEfwhiEdFtzw1YS9Vhsxa1gugdvLFHVFcRgI0/lCMo3JZ946ztoIj/jfYgudsRwS9FJfRjJw1inFJ9fG8/vXU7hX7qC8dHBLtBsHDNBR6uaEMl84csRu9mNYug1dTXMjqf/NhGmlDaIu2zf6L34VZwcA65uMWbwpibUP0q9rbmBU1F39laQVPhJmOLLE5iJ7RSxj5C4DC/fy2cgvTDsE73g21lzAup2pKotKhtNi2ttbBlbKwZcCrfo2aANKhhpSkcn1pO4vB6FLXwS8EyzmhpRDY49AuQm+v1l6BMpjU6Rx7LLDSxdi3kBBPsyfjrMZbyDt014ASbGFxg0ezmWUpNhk+5qUdUEDHhZRPBf0BPL0ps93R0w7r4LrberYkIj7ToplT2tjySxd/LcBq6H2x3WnGw+pA0xMtZbzTXL2rzjR8AE0yofhbpfacAq0tvuwUzUnIUqIGmjZYXzvrCIkTrZgrAWKbH080XhSwasdXKcbslFg== x-ms-exchange-antispam-messagedata: bMsqEqbhXh6iFMo3LQZdwZHQsCSyo8h6XqAqGQNnHfF4JAB2LnEzGxprFjGa5KGYsZbNTR+2Xb4EISL10DDIT1FCYiBEdT4/x8GcZuvcIACrylrhRO/awS8SmqkG6MjPvzPv1N/XYj4Od3KdPw4zbQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2a7f2fa9-cec6-4367-088b-08d7d950bd1c X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Apr 2020 11:01:48.9016 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 23v2ZRZgXPzxqvOCGruLwPWqvfG22f7KlbR07OjegGSS7KqzOLygYJqU2iHJ1ktjt5tqwkXLY0NdCLK2E1b3Og== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4068 Subject: Re: [dpdk-users] mlx5 pmd + rdma-core 28 init failure X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi Benoit Thanks. Please let us know if working with devargs dv_flow_en=3D0 solves the issue.= ... ./build/app/testpmd --log-level=3D8 --log-level=3Dpmd.common.mlx5:8 - -log-= level=3Dpmd.net.mlx5:8 -w 0000:5e:00.0.dv_flow_en=3D0 -w 0000:5e:00.1,dv_fl= ow_en=3D0 -l 4,11,35 -- -a -- forward-mode=3Drxonly Please let us know also your FW version... Matan > -----Original Message----- > From: Benoit Ganne (bganne) > Sent: Thursday, April 2, 2020 8:03 PM > To: Matan Azrad ; users@dpdk.org > Cc: Shahaf Shuler ; Slava Ovsiienko > > Subject: RE: mlx5 pmd + rdma-core 28 init failure >=20 > > Can you run with log level debug and send us the log? >=20 > Here it is: > ~# sudo ./build/app/testpmd --log-level=3D8 --log-level=3Dpmd.common.mlx5= :8 - > -log-level=3Dpmd.net.mlx5:8 -w 0000:5e:00.0 -w 0000:5e:00.1 -l 4,11,35 --= -a -- > forward-mode=3Drxonly > EAL: Detected 48 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: Selected IOVA mode 'VA' > EAL: No available hugepages reported in hugepages-1048576kB > EAL: Probing VFIO support... > EAL: VFIO support initialized > EAL: PCI device 0000:5e:00.0 on NUMA socket 0 > EAL: probe driver: 15b3:1013 net_mlx5 > net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_0" > net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for > device "mlx5_0" > net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" > net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected > net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device > "0000:5e:00.0" > net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x1003dcb20 with table 0x1003d9640 > net_mlx5: mlx5.c:2251: mlx5_dev_spawn(): MPW isn't supported > net_mlx5: mlx5.c:2257: mlx5_dev_spawn(): SWP support: 7 > net_mlx5: mlx5.c:2266: mlx5_dev_spawn(): > min_single_stride_log_num_of_bytes: 0 > net_mlx5: mlx5.c:2268: mlx5_dev_spawn(): > max_single_stride_log_num_of_bytes: 0 > net_mlx5: mlx5.c:2270: mlx5_dev_spawn(): > min_single_wqe_log_num_of_strides: 0 > net_mlx5: mlx5.c:2272: mlx5_dev_spawn(): > max_single_wqe_log_num_of_strides: 0 > net_mlx5: mlx5.c:2274: mlx5_dev_spawn(): supported_qpts: 0 > net_mlx5: mlx5.c:2275: mlx5_dev_spawn(): device supports Multi-Packet RQ > net_mlx5: mlx5.c:2310: mlx5_dev_spawn(): tunnel offloading is supported > net_mlx5: mlx5.c:2322: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel > offloading is not supported > net_mlx5: mlx5.c:2473: mlx5_dev_spawn(): checksum offloading is > supported > net_mlx5: mlx5.c:2493: mlx5_dev_spawn(): maximum Rx indirection table > size is 512 > net_mlx5: mlx5.c:2497: mlx5_dev_spawn(): VLAN stripping is supported > net_mlx5: mlx5.c:2501: mlx5_dev_spawn(): FCS stripping configuration is > supported > net_mlx5: mlx5.c:2531: mlx5_dev_spawn(): MPS is disabled > net_mlx5: mlx5.c:2656: mlx5_dev_spawn(): port 0 MAC address is > 24:8a:07:5b:14:14 > net_mlx5: mlx5.c:2663: mlx5_dev_spawn(): port 0 ifname is "enp94s0f0" > net_mlx5: mlx5.c:2676: mlx5_dev_spawn(): port 0 MTU is 9216 > net_mlx5: mlx5.c:2703: mlx5_dev_spawn(): port 0 forcing Ethernet interfac= e > up > net_mlx5: mlx5.c:1836: mlx5_set_min_inline(): min tx inline configured: 1= 8 > net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with > mlx5_0_flow_table size 0x1000 is created. >=20 > net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_0_tag= s > size 0x2000 is created. >=20 > net_mlx5: mlx5_flow.c:550: mlx5_flow_discover_priorities(): port 0 flow > maximum priority: 3 > net_mlx5: mlx5.c:1887: mlx5_set_metadata_mask(): metadata mode 0 > net_mlx5: mlx5.c:1888: mlx5_set_metadata_mask(): metadata MARK mask > 00FFFFFF > net_mlx5: mlx5.c:1889: mlx5_set_metadata_mask(): metadata META mask > FFFFFFFF > net_mlx5: mlx5.c:1890: mlx5_set_metadata_mask(): metadata reg_c0 mask > FFFFFFFF > net_mlx5: mlx5.c:2771: mlx5_dev_spawn(): port 0 extensive metadata > register is not supported > EAL: PCI device 0000:5e:00.1 on NUMA socket 0 > EAL: probe driver: 15b3:1013 net_mlx5 > net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_0" > net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" > net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for > device "mlx5_1" > net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected > net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device > "0000:5e:00.1" > net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x10037b420 with table 0x100377f40 > net_mlx5: mlx5.c:2251: mlx5_dev_spawn(): MPW isn't supported > net_mlx5: mlx5.c:2257: mlx5_dev_spawn(): SWP support: 7 > net_mlx5: mlx5.c:2266: mlx5_dev_spawn(): > min_single_stride_log_num_of_bytes: 0 > net_mlx5: mlx5.c:2268: mlx5_dev_spawn(): > max_single_stride_log_num_of_bytes: 0 > net_mlx5: mlx5.c:2270: mlx5_dev_spawn(): > min_single_wqe_log_num_of_strides: 0 > net_mlx5: mlx5.c:2272: mlx5_dev_spawn(): > max_single_wqe_log_num_of_strides: 0 > net_mlx5: mlx5.c:2274: mlx5_dev_spawn(): supported_qpts: 0 > net_mlx5: mlx5.c:2275: mlx5_dev_spawn(): device supports Multi-Packet RQ > net_mlx5: mlx5.c:2310: mlx5_dev_spawn(): tunnel offloading is supported > net_mlx5: mlx5.c:2322: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel > offloading is not supported > net_mlx5: mlx5.c:2473: mlx5_dev_spawn(): checksum offloading is > supported > net_mlx5: mlx5.c:2493: mlx5_dev_spawn(): maximum Rx indirection table > size is 512 > net_mlx5: mlx5.c:2497: mlx5_dev_spawn(): VLAN stripping is supported > net_mlx5: mlx5.c:2501: mlx5_dev_spawn(): FCS stripping configuration is > supported > net_mlx5: mlx5.c:2531: mlx5_dev_spawn(): MPS is disabled > net_mlx5: mlx5.c:2656: mlx5_dev_spawn(): port 1 MAC address is > 24:8a:07:5b:14:15 > net_mlx5: mlx5.c:2663: mlx5_dev_spawn(): port 1 ifname is "enp94s0f1" > net_mlx5: mlx5.c:2676: mlx5_dev_spawn(): port 1 MTU is 9216 > net_mlx5: mlx5.c:2703: mlx5_dev_spawn(): port 1 forcing Ethernet interfac= e > up > net_mlx5: mlx5.c:1836: mlx5_set_min_inline(): min tx inline configured: 1= 8 > net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with > mlx5_1_flow_table size 0x1000 is created. >=20 > net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_1_tag= s > size 0x2000 is created. >=20 > net_mlx5: mlx5_flow.c:550: mlx5_flow_discover_priorities(): port 1 flow > maximum priority: 3 > net_mlx5: mlx5.c:1887: mlx5_set_metadata_mask(): metadata mode 0 > net_mlx5: mlx5.c:1888: mlx5_set_metadata_mask(): metadata MARK mask > 00FFFFFF > net_mlx5: mlx5.c:1889: mlx5_set_metadata_mask(): metadata META mask > FFFFFFFF > net_mlx5: mlx5.c:1890: mlx5_set_metadata_mask(): metadata reg_c0 mask > FFFFFFFF > net_mlx5: mlx5.c:2771: mlx5_dev_spawn(): port 1 extensive metadata > register is not supported Auto-start selected Set rxonly packet forwardin= g > mode > testpmd: create a new mbuf pool : n=3D163456, > size=3D2176, socket=3D0 > testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0 > (socket 0) > net_mlx5: mlx5_ethdev.c:424: mlx5_dev_configure(): port 0 Tx queues > number update: 0 -> 1 > net_mlx5: mlx5_ethdev.c:435: mlx5_dev_configure(): port 0 Rx queues > number update: 0 -> 1 > net_mlx5: mlx5_txq.c:172: mlx5_tx_queue_pre_setup(): port 0 configuring > queue 0 for 256 descriptors > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x101051c28 with table 0x1010505c0 > net_mlx5: mlx5_txq.c:225: mlx5_tx_queue_setup(): port 0 adding Tx queue > 0 to list > net_mlx5: mlx5_rxq.c:468: mlx5_rx_queue_pre_setup(): port 0 configuring > Rx queue 0 for 256 descriptors > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x10104fa6c with table 0x10104e500 > net_mlx5: mlx5_rxq.c:1921: mlx5_rxq_new(): port 0 maximum number of > segments per packet: 1 > net_mlx5: mlx5_rxq.c:1759: mlx5_max_lro_msg_size_adjust(): port 0 Rx > Queue 0 max LRO message size adjusted to 1280 bytes > net_mlx5: mlx5_rxq.c:1968: mlx5_rxq_new(): port 0 CRC stripping is enable= d, > 0 bytes will be subtracted from incoming frames to hide it > net_mlx5: mlx5_rxq.c:525: mlx5_rx_queue_setup(): port 0 adding Rx queue > 0 to list > net_mlx5: mlx5_trigger.c:276: mlx5_dev_start(): port 0 starting device > net_mlx5: mlx5_ethdev.c:493: mlx5_dev_configure_rss_reta(): port 0 Rx > queues number update: 1 -> 1 > net_mlx5: mlx5_txq.c:55: txq_alloc_elts(): port 0 Tx queue 0 allocated an= d > configured 256 WRs > net_mlx5: mlx5_txq.c:771: mlx5_txq_obj_new(): port 0: uar_mmap_offset > 0x306000 > net_mlx5: mlx5_trigger.c:145: mlx5_rxq_start(): port 0 Rx queue 0 registe= ring > mp mbuf_pool_socket_0 having 1 chunks > net_mlx5: mlx5_mr.c:600: mlx5_mr_create_primary(): port 0 creating a MR > using address (0x10109b4c0) > net_mlx5: mlx5_mr.c:649: mlx5_mr_create_primary(): port 0 extending > 0x10109b4c0 to [0x100200000, 0x118200000), page_sz=3D0x200000, ms_n=3D192 > net_mlx5: mlx5_mr.c:786: mlx5_mr_create_primary(): port 0 MR CREATED > (0x10104e140) for 0x10109b4c0: > [0x100200000, 0x118200000), lkey=3D0x104b0100 base_idx=3D0 ms_n=3D192, > ms_bmp_n=3D192 > net_mlx5: mlx5_mr.c:345: mr_insert_dev_cache(): device mlx5_0 inserting > MR(0x10104e140) to global cache > net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B- > tree(0x1003dcb20)[1], [0x100200000, 0x118200000) lkey=3D0x104b0100 > net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B- > tree(0x10104fa6c)[1], [0x100200000, 0x118200000) lkey=3D0x104b0100 > net_mlx5: mlx5_rxq.c:257: rxq_alloc_elts_sprq(): port 0 Rx queue 0 alloca= ted > and configured 256 segments (max 256 packets) > net_mlx5: mlx5_rxq.c:1402: mlx5_rxq_obj_new(): port 0 > device_attr.max_qp_wr is 32768 > net_mlx5: mlx5_rxq.c:1404: mlx5_rxq_obj_new(): port 0 > device_attr.max_sge is 30 > net_mlx5: mlx5_rxq.c:1478: mlx5_rxq_obj_new(): port 0 rxq 0 updated with > 0x7ffda24f21f8 > net_mlx5: mlx5_trigger.c:322: mlx5_dev_start(): port 0 failed to set defa= ults > flows > net_mlx5: mlx5_rxq.c:333: rxq_free_elts_sprq(): port 0 Rx queue 0 freeing > WRs Fail to start port 0 Configuring Port 1 (socket 0) > net_mlx5: mlx5_ethdev.c:424: mlx5_dev_configure(): port 1 Tx queues > number update: 0 -> 1 > net_mlx5: mlx5_ethdev.c:435: mlx5_dev_configure(): port 1 Rx queues > number update: 0 -> 1 > net_mlx5: mlx5_txq.c:172: mlx5_tx_queue_pre_setup(): port 1 configuring > queue 0 for 256 descriptors > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x10104d6a8 with table 0x10104c040 > net_mlx5: mlx5_txq.c:225: mlx5_tx_queue_setup(): port 1 adding Tx queue > 0 to list > net_mlx5: mlx5_rxq.c:468: mlx5_rx_queue_pre_setup(): port 1 configuring > Rx queue 0 for 256 descriptors > net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree > 0x10104b4ec with table 0x101049f80 > net_mlx5: mlx5_rxq.c:1921: mlx5_rxq_new(): port 1 maximum number of > segments per packet: 1 > net_mlx5: mlx5_rxq.c:1759: mlx5_max_lro_msg_size_adjust(): port 1 Rx > Queue 0 max LRO message size adjusted to 1280 bytes > net_mlx5: mlx5_rxq.c:1968: mlx5_rxq_new(): port 1 CRC stripping is enable= d, > 0 bytes will be subtracted from incoming frames to hide it > net_mlx5: mlx5_rxq.c:525: mlx5_rx_queue_setup(): port 1 adding Rx queue > 0 to list > net_mlx5: mlx5_trigger.c:276: mlx5_dev_start(): port 1 starting device > net_mlx5: mlx5_ethdev.c:493: mlx5_dev_configure_rss_reta(): port 1 Rx > queues number update: 1 -> 1 > net_mlx5: mlx5_txq.c:55: txq_alloc_elts(): port 1 Tx queue 0 allocated an= d > configured 256 WRs > net_mlx5: mlx5_txq.c:771: mlx5_txq_obj_new(): port 1: uar_mmap_offset > 0x306000 > net_mlx5: mlx5_trigger.c:145: mlx5_rxq_start(): port 1 Rx queue 0 registe= ring > mp mbuf_pool_socket_0 having 1 chunks > net_mlx5: mlx5_mr.c:600: mlx5_mr_create_primary(): port 1 creating a MR > using address (0x10109b4c0) > net_mlx5: mlx5_mr.c:649: mlx5_mr_create_primary(): port 1 extending > 0x10109b4c0 to [0x100200000, 0x118200000), page_sz=3D0x200000, ms_n=3D192 > net_mlx5: mlx5_mr.c:786: mlx5_mr_create_primary(): port 1 MR CREATED > (0x101045dc0) for 0x10109b4c0: > [0x100200000, 0x118200000), lkey=3D0xcac80f00 base_idx=3D0 ms_n=3D192, > ms_bmp_n=3D192 > net_mlx5: mlx5_mr.c:345: mr_insert_dev_cache(): device mlx5_1 inserting > MR(0x101045dc0) to global cache > net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B- > tree(0x10037b420)[1], [0x100200000, 0x118200000) lkey=3D0xcac80f00 > net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B- > tree(0x10104b4ec)[1], [0x100200000, 0x118200000) lkey=3D0xcac80f00 > net_mlx5: mlx5_rxq.c:257: rxq_alloc_elts_sprq(): port 1 Rx queue 0 alloca= ted > and configured 256 segments (max 256 packets) > net_mlx5: mlx5_rxq.c:1402: mlx5_rxq_obj_new(): port 1 > device_attr.max_qp_wr is 32768 > net_mlx5: mlx5_rxq.c:1404: mlx5_rxq_obj_new(): port 1 > device_attr.max_sge is 30 > net_mlx5: mlx5_rxq.c:1478: mlx5_rxq_obj_new(): port 1 rxq 0 updated with > 0x7ffda24f21f8 > net_mlx5: mlx5_trigger.c:322: mlx5_dev_start(): port 1 failed to set defa= ults > flows > net_mlx5: mlx5_rxq.c:333: rxq_free_elts_sprq(): port 1 Rx queue 0 freeing > WRs Fail to start port 1 Please stop the ports first Done No commandline = core > given, start packet forwarding Not all ports were started Press enter to = exit >=20 > Stopping port 0... > Stopping ports... > Done >=20 > Stopping port 1... > Stopping ports... > Done >=20 > Shutting down port 0... > Closing ports... > net_mlx5: mlx5.c:1233: mlx5_dev_close(): port 0 closing device "mlx5_0" > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x10104fa6c with table 0x10104e500 > net_mlx5: mlx5_txq.c:77: txq_free_elts(): port 0 Tx queue 0 freeing WRs > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x101051c28 with table 0x1010505c0 > net_mlx5: mlx5_mr.c:1569: mlx5_mr_dump_dev(): device mlx5_0 MR[0], > LKey =3D 0x104b0100, ms_n =3D 192, ms_bmp_n =3D 192 > net_mlx5: mlx5_mr.c:1579: mlx5_mr_dump_dev(): chunk[0], [0x100200000, > 0x118200000) > net_mlx5: mlx5_mr.c:1582: mlx5_mr_dump_dev(): device mlx5_0 dumping > global cache > net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x1003dcb20)[0], > [0x0, 0x0) lkey=3D0xffffffff > net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x1003dcb20)[1], > [0x100200000, 0x118200000) lkey=3D0x104b0100 > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x1003dcb20 with table 0x1003d9640 > net_mlx5: mlx5_mr.c:459: mr_free(): freeing MR(0x10104e140): > Done >=20 > Shutting down port 1... > Closing ports... > net_mlx5: mlx5.c:1233: mlx5_dev_close(): port 1 closing device "mlx5_1" > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x10104b4ec with table 0x101049f80 > net_mlx5: mlx5_txq.c:77: txq_free_elts(): port 1 Tx queue 0 freeing WRs > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x10104d6a8 with table 0x10104c040 > net_mlx5: mlx5_mr.c:1569: mlx5_mr_dump_dev(): device mlx5_1 MR[0], > LKey =3D 0xcac80f00, ms_n =3D 192, ms_bmp_n =3D 192 > net_mlx5: mlx5_mr.c:1579: mlx5_mr_dump_dev(): chunk[0], [0x100200000, > 0x118200000) > net_mlx5: mlx5_mr.c:1582: mlx5_mr_dump_dev(): device mlx5_1 dumping > global cache > net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x10037b420)[0], > [0x0, 0x0) lkey=3D0xffffffff > net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x10037b420)[1], > [0x100200000, 0x118200000) lkey=3D0xcac80f00 > net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree > 0x10037b420 with table 0x100377f40 > net_mlx5: mlx5_mr.c:459: mr_free(): freeing MR(0x101045dc0): > Done >=20 > Bye... >=20 > Best > ben >=20 > > From: Benoit Ganne (bganne) > > Sent: Thursday, April 2, 2020 6:50:22 PM > > To: users@dpdk.org > > Cc: Matan Azrad ; Shahaf Shuler > > ; Slava Ovsiienko > > Subject: RE: mlx5 pmd + rdma-core 28 init failure > > > > Adding MLX5 PMD maintainers. > > > > I also checked with latest rdma-core master and latest DPDK master and > > it is failing with the same issue. > > Any recommendation? > > > > Best > > ben > > > > > -----Original Message----- > > > From: Benoit Ganne (bganne) > > > Sent: mercredi 1 avril 2020 18:52 > > > To: users@dpdk.org > > > Subject: mlx5 pmd + rdma-core 28 init failure > > > > > > Hi all, > > > > > > I have troubles making DPDK v20.02 MLX5 PMD working with rdma-core > > v28.0: > > > it looks like the flow initialization done while initializing the > > > device fails in rdma-core > > > providers/mlx5/dr_table.c:mlx5dv_dr_table_create() > > > because of unsupported parameters. > > > The issue is the following test in rdma-core > > > providers/mlx5/dr_table.c:mlx5dv_dr_table_create(): > > > if (level && !dmn->info.supp_sw_steering) { > > > errno =3D EOPNOTSUPP; > > > goto dec_ref; > > > } > > > Where level =3D=3D 65534 and dmn->info.supp_sw_steering =3D=3D false,= hence > > > the test is true and the function returns EOPNOTSUPP. > > > > > > Is this expected? It does work fine with ibv instead of dv API. Any > > > help appreciated. > > > > > > Here is what I do: > > > 1) checkout & compile rdma-core v28.0 ~# git clone > > https://eur03.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgit= h > > ub.c > > om%2Flinux-rdma%2Frdma- > > > core&data=3D02%7C01%7Cmatan%40mellanox.com%7C7984449b13af4519 > d70608d > > 7d71 > > > d9000%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637214394282 > 618427& > > amp; > > > sdata=3DDSLvWg3cWwY8prSFOm8BAFf1I2bbKnqPOKBPEiy4b44%3D&rese > rved=3D0 > > > ub. > > com%2Flinux-rdma%2Frdma- > > > core&data=3D02%7C01%7Cmatan%40mellanox.com%7C7984449b13af4519 > d70608d > > 7d71 > > > d9000%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637214394282 > 618427& > > amp; > > > sdata=3DDSLvWg3cWwY8prSFOm8BAFf1I2bbKnqPOKBPEiy4b44%3D&rese > rved=3D0> > > > ~# cd rdma-core > > > ~# git checkout v28.0 > > > ~# mkdir build > > > ~# cd build > > > ~# CFLAGS=3D-fPIC cmake -DIN_PLACE=3D1 -DENABLE_STATIC=3D1 -GNinja .. > > > ~# ninja > > > > > > 2) checkout & compile dpdk v20.02 ~# git clone > > > git://dpdk.org/dpdk ~# cd dpdk ~# make config > > > T=3Dx86_64-native-linuxapp-gcc ~# sed -ri 's,(MLX5_PMD=3D).*,\1y,' > > > build/.config ~# sed -ri 's,(IBVERBS_LINK_STATIC_PMD=3D).*,\1y,' > > > build/.config ~# make > > > EXTRA_CFLAGS=3D-I/home/bganne/src/rdma-core/build/include > > > EXTRA_LDFLAGS=3D-L/home/bganne/src/rdma-core/build/lib > > > PKG_CONFIG_PATH=3D/home/bganne/src/rdma-core/build/lib/pkgconfig > > > ~# sudo gdb --args ./build/app/testpmd -w 0000:5e:00.0 -w > > > 0000:5e:00.1 - > > l > > > 4,11,35 -- -a --forward-mode=3Drxonly -i > > > > > > The backtrace looks like this: > > > #0 mlx5dv_dr_table_create (dmn=3D0x555556c641b0, level=3D65534) at > > > ../providers/mlx5/dr_table.c:183 > > > #1 0x0000555555dfaeaa in flow_dv_tbl_resource_get (dev=3D > > out>, table_id=3D65534, egress=3D, transfer=3D > > out>, > > > error=3D0x7fffffffdca0) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:6746 > > > #2 0x0000555555e02b28 in __flow_dv_translate > > > (dev=3Ddev@entry=3D0x555556bbcdc0 , > > > dev_flow=3D0x100388300, attr=3D, items=3D, > > > actions=3D, error=3D) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:7503 > > > #3 0x0000555555e04954 in flow_dv_translate (dev=3D0x555556bbcdc0 > > > , dev_flow=3D, attr=3D= , > > > items=3D, actions=3D, error=3D > > out>) > > at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:8841 > > > #4 0x0000555555df152f in flow_drv_translate (error=3D0x7fffffffdca0, > > > actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, attr=3D0x7fffffffbb= 88, > > > dev_flow=3D, dev=3D0x555556bbcdc0 ) a= t > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:2571 > > > #5 flow_create_split_inner (error=3D0x7fffffffdca0, external=3Dfalse= , > > > actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, attr=3D0x7fffffffbb= 88, > > > prefix_layers=3D0, sub_flow=3D0x0, flow=3D0x1003885c0, dev=3D0x555556= bbcdc0 > > > ) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3490 > > > #6 flow_create_split_metadata (error=3D0x7fffffffdca0, > > > external=3Dfalse, actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, > > > attr=3D0x7fffffffbb88, prefix_layers=3D0, flow=3D0x1003885c0, > > > dev=3D0x555556bbcdc0 ) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3865 > > > #7 flow_create_split_meter (error=3D0x7fffffffdca0, external=3Dfalse= , > > > actions=3D0x7fffffffdce0, items=3D, attr=3D0x7fffffffd= c94, > > > flow=3D0x1003885c0, dev=3D0x555556bbcdc0 ) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4121 > > > #8 flow_create_split_outer (error=3D0x7fffffffdca0, external=3Dfalse= , > > > actions=3D0x7fffffffdce0, items=3D, attr=3D0x7fffffffd= c94, > > > flow=3D0x1003885c0, dev=3D0x555556bbcdc0 ) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4178 > > > #9 flow_list_create (dev=3Ddev@entry=3D0x555556bbcdc0 > > > , list=3Dlist@entry=3D0x0, > > > attr=3Dattr@entry=3D0x7fffffffdc94, items=3Ditems@entry=3D0x7fffffffd= cc0, > > > actions=3Dactions@entry=3D0x7fffffffdce0, > > > external=3Dexternal@entry=3Dfalse, error=3D0x7fffffffdca0) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4306 > > > #10 0x0000555555df8587 in mlx5_flow_discover_mreg_c > > > (dev=3Ddev@entry=3D0x555556bbcdc0 ) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:5747 > > > #11 0x0000555555d692a6 in mlx5_dev_spawn (config=3D..., > > > spawn=3D0x1003e9e00, > > > dpdk_dev=3D0x555556dd6fe0) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:2763 > > > #12 mlx5_pci_probe (pci_drv=3D, pci_dev=3D > > out>) at > > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:3363 > > > #13 0x0000555555a411c8 in pci_probe_all_drivers () > > > #14 0x0000555555a412f8 in rte_pci_probe () > > > #15 0x0000555555a083da in rte_bus_probe () > > > #16 0x00005555559f204d in rte_eal_init () > > > #17 0x00005555556a0d45 in main () > > > > > > Best > > > ben