From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8CFE3A0562 for ; Thu, 2 Apr 2020 19:03:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C8B951B91B; Thu, 2 Apr 2020 19:03:12 +0200 (CEST) Received: from alln-iport-2.cisco.com (alln-iport-2.cisco.com [173.37.142.89]) by dpdk.org (Postfix) with ESMTP id 2BBA8375B for ; Thu, 2 Apr 2020 19:03:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=21672; q=dns/txt; s=iport; t=1585846991; x=1587056591; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=GuvBWs+4ICmA5bLF5glN4WbiruYe4pEHLMTwLqCOU9k=; b=Z8xQxDa+WvQ3LSEGOw3kZuxKeEOMzMC9viMQZNT6JA9TBfaq9hC6I/3U CjKMf1cFDYAa0ny28iZvBmpKm8x45MLxyH8ilFG0SE1/uBpS4Fp9Efv0t FTvBGLYo0+0kzisZ4PqzvZHq1cT39Fx5akVgR91VzRYueS8gXWf/yKGbg 8=; IronPort-PHdr: =?us-ascii?q?9a23=3AsuqhjBboR5keXeNGXxtxB8T/LSx94ef9IxIV55?= =?us-ascii?q?w7irlHbqWk+dH4MVfC4el20Q6bRp3VvvRDjeee87vtX2AN+96giDgDa9QNMn?= =?us-ascii?q?1NksAKh0olCc+BB1f8KavgYCU8GslqX15+9Hb9Ok9QS47z?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CwAADRGYZe/4sNJK1mGgEBAQEBAQE?= =?us-ascii?q?BAQMBAQEBEQEBAQICAQEBAYF7gVQpJwVsWCAECyoKh1YDimeCX4EBlx2CUgN?= =?us-ascii?q?UCgEBAQwBASUIAgQBAYFngl0CgkEkOBMCAwEBCwEBBQEBAQIBBQRthVYMhXA?= =?us-ascii?q?BAQEBAgESCA0TBgEBNwEEBwQCAQgRBAEBHxAyHQgBAQQBCQQFCBqCfwQCgks?= =?us-ascii?q?DDiABDqRoAoE5iGKBdDOCfwEBBYU5AxWCDAmBOIwxGoFBP4ERR4IYNT6CZwQ?= =?us-ascii?q?agUuDQoIsjgEYi0iXJQqCPYdpj0+CTIEEhzGQcY8pgVKaRAIEAgQFAg4BAQW?= =?us-ascii?q?BaSINgUpwFTuCaQlHGA2OHQkCGIEEAQeCRIUUhUF0AgEBgSWNDgGBDwEB?= X-IronPort-AV: E=Sophos;i="5.72,336,1580774400"; d="scan'208";a="472325421" Received: from alln-core-6.cisco.com ([173.36.13.139]) by alln-iport-2.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 02 Apr 2020 17:03:08 +0000 Received: from XCH-RCD-002.cisco.com (xch-rcd-002.cisco.com [173.37.102.12]) by alln-core-6.cisco.com (8.15.2/8.15.2) with ESMTPS id 032H383p007026 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 2 Apr 2020 17:03:08 GMT Received: from xhs-rtp-003.cisco.com (64.101.210.230) by XCH-RCD-002.cisco.com (173.37.102.12) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 2 Apr 2020 12:03:08 -0500 Received: from xhs-rtp-001.cisco.com (64.101.210.228) by xhs-rtp-003.cisco.com (64.101.210.230) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 2 Apr 2020 13:03:07 -0400 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (64.101.32.56) by xhs-rtp-001.cisco.com (64.101.210.228) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Thu, 2 Apr 2020 13:03:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ftNja21Qq0Js0cAznAZymImRgWXnU57bgfmTaMabmHYWLYo1cjWLCydg+OPZYoRSyjNnCexe/eFfZ70MqEWfxSJ8npsD1CKwMddTRHX/GU/lFpg0ZOePIr7SiauK+jsqFzTkqe84q7YGlhCcPj0gmY+tNO7AznaJMnATJ0VFB0KVE9HobwyIZw14xRTZ3/iMQBofnfN7vzan3beRVfYKo96ZRdQdx69a/rqz264anyGVrR/1WoVgqc4jVkQ8WcWvppDYyn5Wkyay3jyRzrkxTBElIsgGMvxtEl8rOvEHUrQTJ6n3zHuqtV6+PRaV63su4Yq2PgfgMoJV9MMEJ18IQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZcmIeI+3/cLIAxumlBoZBH/Eulahe4rC/KNL76zKMfY=; b=kzhE29fMuebO5lF4V6N0whOjnSInK2mQSV4u8Tk8FffnJYxU6Ch9LhxOsWDr5AfKkjihuNEnz+NIKpIBMZbWEtCLP1xs65JBS1m1h6TthZGcc6AcibWvdvMKJKzX6PQB7D7uf8d1YmiIOxYVFva+3mwDR2zyAgppVZ5AtcpwU7KEq3WAos1qm/bYG59PXDKU3O/PGy0TXDfaprQwL70AJByHjjmJim/AFVzC4K+rqhMema4uuhKnfRnUW+kmzYYKGQbVuk02o1VGPJOc9zZ9u6tHb3SsY818/T3QqljytMhBRV27UNPbKKYlZAb6WaCKJx7gRxeraeKi3flvZR3A7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZcmIeI+3/cLIAxumlBoZBH/Eulahe4rC/KNL76zKMfY=; b=JXff8KDNc+3K8/qTqtpdKWeHsOkYMiGtPzcyNWAvQc6dJP9eo0yjgexzDwKrKmsaz/WWZ3AIFVb0L5nMvaW9/kd354HhFMfkHjPtM6hNdpC3TxhFnBGKAcYgB8QWeiojzikNqF10e1WkNoDxV0ZEzUuXQ98V0Ni6DSgRDxwnkfc= Received: from CH2PR11MB4327.namprd11.prod.outlook.com (2603:10b6:610:43::30) by CH2PR11MB4247.namprd11.prod.outlook.com (2603:10b6:610:42::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.15; Thu, 2 Apr 2020 17:03:05 +0000 Received: from CH2PR11MB4327.namprd11.prod.outlook.com ([fe80::2d84:413f:5702:c519]) by CH2PR11MB4327.namprd11.prod.outlook.com ([fe80::2d84:413f:5702:c519%5]) with mapi id 15.20.2856.019; Thu, 2 Apr 2020 17:03:05 +0000 From: "Benoit Ganne (bganne)" To: Matan Azrad , "users@dpdk.org" CC: Shahaf Shuler , Slava Ovsiienko Thread-Topic: mlx5 pmd + rdma-core 28 init failure Thread-Index: AdYIQ9Z8ZbHXLbfSTDeYhwt0ihzoWAAwhXsgAAERNsoAAXwMgA== Date: Thu, 2 Apr 2020 17:03:04 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=bganne@cisco.com; x-originating-ip: [173.38.220.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 50a9ba51-fe08-477d-68db-08d7d727b5e9 x-ms-traffictypediagnostic: CH2PR11MB4247: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:16; x-forefront-prvs: 0361212EA8 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR11MB4327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(346002)(39860400002)(376002)(396003)(136003)(366004)(4326008)(81166006)(81156014)(6506007)(66556008)(66476007)(2906002)(8936002)(66946007)(66446008)(9686003)(71200400001)(8676002)(76116006)(53546011)(54906003)(55016002)(110136005)(316002)(30864003)(86362001)(64756008)(5660300002)(33656002)(52536014)(966005)(45080400002)(186003)(478600001)(26005)(7696005)(579004); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OYBtHWAL1r+Moz//Itg04Zo/z8/HLL/5NLepy1eXAGFwZLFVAxBV08uObCBDGHHvHqaGwqaHtLtZZE6z7Ye7dLNgoIeWtzylMhbowezqXaH3t2p956xFuTK0fC/bJR66tW66Y+RyPmlmZAPgnARgVpFRvhhh+bzWQ/ZDhK4T7bfAaj917O+IYdShytNnXZ4Mew1Wy9U5GUTfpCKXwGvB+o67L37ySHxYoM4MukjR4ieYO8kDIDJWbU0tL22PwsWSYUns+S5ytVui6uw1Ck52CyUGBD7u07wEHL3cnh6N32eN3kJHdd0IFJAdc+Hs1l4dOGlkJyr9YbBQYz7gHLfR0Yx2B68z+R3tt2//PXNEszRFm4IUCKij2NKHEeSd32ZQte2ADBfo5VGBlcaKfzsQR26/PSWIg6ux4/7D6twxYKKUHPBqR7el2BHR/W6JE1IcTBnnqtLRNCd/KobOuiKQgoolsKoou+13IOsbPR4/RJbiL/sNFB2S2KRUhvFXsw9s x-ms-exchange-antispam-messagedata: jy4N+4QtKOferroQch1KPLGEsv6ElK06ILTYm8vmkgnAnaDVnfV+nPJHiKO/hwYiHUxQXvi8ZNj/cpwfMJyjhg1xByczf6v3me6VOtf8RUH7uh2KcDqXBOvaEdREnBXmNhPvPsqwUsCTpjoq+D6Wug== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 50a9ba51-fe08-477d-68db-08d7d727b5e9 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2020 17:03:04.9225 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1L2kCrHfq/RfsWcA9dmrd4sGTYi90zvqya+wnlazCJjYJVzyZ93V0/ivT/ZY27VxBWKisEm1E5WQFGHqKCJDwg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR11MB4247 X-OriginatorOrg: cisco.com X-Outbound-SMTP-Client: 173.37.102.12, xch-rcd-002.cisco.com X-Outbound-Node: alln-core-6.cisco.com Subject: Re: [dpdk-users] mlx5 pmd + rdma-core 28 init failure X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" > Can you run with log level debug and send us the log? Here it is: ~# sudo ./build/app/testpmd --log-level=3D8 --log-level=3Dpmd.common.mlx5:8= --log-level=3Dpmd.net.mlx5:8 -w 0000:5e:00.0 -w 0000:5e:00.1 -l 4,11,35 --= -a --forward-mode=3Drxonly EAL: Detected 48 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: No available hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:5e:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1013 net_mlx5 net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_0" net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for device= "mlx5_0" net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device "0000:5e:00= .0" net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x1003dcb= 20 with table 0x1003d9640 net_mlx5: mlx5.c:2251: mlx5_dev_spawn(): MPW isn't supported net_mlx5: mlx5.c:2257: mlx5_dev_spawn(): SWP support: 7 net_mlx5: mlx5.c:2266: mlx5_dev_spawn(): min_single_stride_log_num_of_byte= s: 0 net_mlx5: mlx5.c:2268: mlx5_dev_spawn(): max_single_stride_log_num_of_byte= s: 0 net_mlx5: mlx5.c:2270: mlx5_dev_spawn(): min_single_wqe_log_num_of_strides= : 0 net_mlx5: mlx5.c:2272: mlx5_dev_spawn(): max_single_wqe_log_num_of_strides= : 0 net_mlx5: mlx5.c:2274: mlx5_dev_spawn(): supported_qpts: 0 net_mlx5: mlx5.c:2275: mlx5_dev_spawn(): device supports Multi-Packet RQ net_mlx5: mlx5.c:2310: mlx5_dev_spawn(): tunnel offloading is supported net_mlx5: mlx5.c:2322: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel offloadin= g is not supported net_mlx5: mlx5.c:2473: mlx5_dev_spawn(): checksum offloading is supported net_mlx5: mlx5.c:2493: mlx5_dev_spawn(): maximum Rx indirection table size = is 512 net_mlx5: mlx5.c:2497: mlx5_dev_spawn(): VLAN stripping is supported net_mlx5: mlx5.c:2501: mlx5_dev_spawn(): FCS stripping configuration is sup= ported net_mlx5: mlx5.c:2531: mlx5_dev_spawn(): MPS is disabled net_mlx5: mlx5.c:2656: mlx5_dev_spawn(): port 0 MAC address is 24:8a:07:5b:= 14:14 net_mlx5: mlx5.c:2663: mlx5_dev_spawn(): port 0 ifname is "enp94s0f0" net_mlx5: mlx5.c:2676: mlx5_dev_spawn(): port 0 MTU is 9216 net_mlx5: mlx5.c:2703: mlx5_dev_spawn(): port 0 forcing Ethernet interface = up net_mlx5: mlx5.c:1836: mlx5_set_min_inline(): min tx inline configured: 18 net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_0_flow_= table size 0x1000 is created. net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_0_tags = size 0x2000 is created. net_mlx5: mlx5_flow.c:550: mlx5_flow_discover_priorities(): port 0 flow max= imum priority: 3 net_mlx5: mlx5.c:1887: mlx5_set_metadata_mask(): metadata mode 0 net_mlx5: mlx5.c:1888: mlx5_set_metadata_mask(): metadata MARK mask 00FFFFF= F net_mlx5: mlx5.c:1889: mlx5_set_metadata_mask(): metadata META mask FFFFFFF= F net_mlx5: mlx5.c:1890: mlx5_set_metadata_mask(): metadata reg_c0 mask FFFFF= FFF net_mlx5: mlx5.c:2771: mlx5_dev_spawn(): port 0 extensive metadata register= is not supported EAL: PCI device 0000:5e:00.1 on NUMA socket 0 EAL: probe driver: 15b3:1013 net_mlx5 net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_0" net_mlx5: mlx5.c:3040: mlx5_pci_probe(): checking device "mlx5_1" net_mlx5: mlx5.c:3074: mlx5_pci_probe(): PCI information matches for device= "mlx5_1" net_mlx5: mlx5.c:3305: mlx5_pci_probe(): no E-Switch support detected net_mlx5: mlx5.c:2178: mlx5_dev_spawn(): naming Ethernet device "0000:5e:00= .1" net_mlx5: mlx5.c:548: mlx5_alloc_shared_ibctx(): DevX is NOT supported net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x10037b4= 20 with table 0x100377f40 net_mlx5: mlx5.c:2251: mlx5_dev_spawn(): MPW isn't supported net_mlx5: mlx5.c:2257: mlx5_dev_spawn(): SWP support: 7 net_mlx5: mlx5.c:2266: mlx5_dev_spawn(): min_single_stride_log_num_of_byte= s: 0 net_mlx5: mlx5.c:2268: mlx5_dev_spawn(): max_single_stride_log_num_of_byte= s: 0 net_mlx5: mlx5.c:2270: mlx5_dev_spawn(): min_single_wqe_log_num_of_strides= : 0 net_mlx5: mlx5.c:2272: mlx5_dev_spawn(): max_single_wqe_log_num_of_strides= : 0 net_mlx5: mlx5.c:2274: mlx5_dev_spawn(): supported_qpts: 0 net_mlx5: mlx5.c:2275: mlx5_dev_spawn(): device supports Multi-Packet RQ net_mlx5: mlx5.c:2310: mlx5_dev_spawn(): tunnel offloading is supported net_mlx5: mlx5.c:2322: mlx5_dev_spawn(): MPLS over GRE/UDP tunnel offloadin= g is not supported net_mlx5: mlx5.c:2473: mlx5_dev_spawn(): checksum offloading is supported net_mlx5: mlx5.c:2493: mlx5_dev_spawn(): maximum Rx indirection table size = is 512 net_mlx5: mlx5.c:2497: mlx5_dev_spawn(): VLAN stripping is supported net_mlx5: mlx5.c:2501: mlx5_dev_spawn(): FCS stripping configuration is sup= ported net_mlx5: mlx5.c:2531: mlx5_dev_spawn(): MPS is disabled net_mlx5: mlx5.c:2656: mlx5_dev_spawn(): port 1 MAC address is 24:8a:07:5b:= 14:15 net_mlx5: mlx5.c:2663: mlx5_dev_spawn(): port 1 ifname is "enp94s0f1" net_mlx5: mlx5.c:2676: mlx5_dev_spawn(): port 1 MTU is 9216 net_mlx5: mlx5.c:2703: mlx5_dev_spawn(): port 1 forcing Ethernet interface = up net_mlx5: mlx5.c:1836: mlx5_set_min_inline(): min tx inline configured: 18 net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_1_flow_= table size 0x1000 is created. net_mlx5: mlx5_utils.c:41: mlx5_hlist_create(): Hash list with mlx5_1_tags = size 0x2000 is created. net_mlx5: mlx5_flow.c:550: mlx5_flow_discover_priorities(): port 1 flow max= imum priority: 3 net_mlx5: mlx5.c:1887: mlx5_set_metadata_mask(): metadata mode 0 net_mlx5: mlx5.c:1888: mlx5_set_metadata_mask(): metadata MARK mask 00FFFFF= F net_mlx5: mlx5.c:1889: mlx5_set_metadata_mask(): metadata META mask FFFFFFF= F net_mlx5: mlx5.c:1890: mlx5_set_metadata_mask(): metadata reg_c0 mask FFFFF= FFF net_mlx5: mlx5.c:2771: mlx5_dev_spawn(): port 1 extensive metadata register= is not supported Auto-start selected Set rxonly packet forwarding mode testpmd: create a new mbuf pool : n=3D163456, size=3D21= 76, socket=3D0 testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0 (socket 0) net_mlx5: mlx5_ethdev.c:424: mlx5_dev_configure(): port 0 Tx queues number = update: 0 -> 1 net_mlx5: mlx5_ethdev.c:435: mlx5_dev_configure(): port 0 Rx queues number = update: 0 -> 1 net_mlx5: mlx5_txq.c:172: mlx5_tx_queue_pre_setup(): port 0 configuring que= ue 0 for 256 descriptors net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x101051c= 28 with table 0x1010505c0 net_mlx5: mlx5_txq.c:225: mlx5_tx_queue_setup(): port 0 adding Tx queue 0 t= o list net_mlx5: mlx5_rxq.c:468: mlx5_rx_queue_pre_setup(): port 0 configuring Rx = queue 0 for 256 descriptors net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x10104fa= 6c with table 0x10104e500 net_mlx5: mlx5_rxq.c:1921: mlx5_rxq_new(): port 0 maximum number of segment= s per packet: 1 net_mlx5: mlx5_rxq.c:1759: mlx5_max_lro_msg_size_adjust(): port 0 Rx Queue = 0 max LRO message size adjusted to 1280 bytes net_mlx5: mlx5_rxq.c:1968: mlx5_rxq_new(): port 0 CRC stripping is enabled,= 0 bytes will be subtracted from incoming frames to hide it net_mlx5: mlx5_rxq.c:525: mlx5_rx_queue_setup(): port 0 adding Rx queue 0 t= o list net_mlx5: mlx5_trigger.c:276: mlx5_dev_start(): port 0 starting device net_mlx5: mlx5_ethdev.c:493: mlx5_dev_configure_rss_reta(): port 0 Rx queue= s number update: 1 -> 1 net_mlx5: mlx5_txq.c:55: txq_alloc_elts(): port 0 Tx queue 0 allocated and = configured 256 WRs net_mlx5: mlx5_txq.c:771: mlx5_txq_obj_new(): port 0: uar_mmap_offset 0x306= 000 net_mlx5: mlx5_trigger.c:145: mlx5_rxq_start(): port 0 Rx queue 0 registeri= ng mp mbuf_pool_socket_0 having 1 chunks net_mlx5: mlx5_mr.c:600: mlx5_mr_create_primary(): port 0 creating a MR usi= ng address (0x10109b4c0) net_mlx5: mlx5_mr.c:649: mlx5_mr_create_primary(): port 0 extending 0x10109= b4c0 to [0x100200000, 0x118200000), page_sz=3D0x200000, ms_n=3D192 net_mlx5: mlx5_mr.c:786: mlx5_mr_create_primary(): port 0 MR CREATED (0x101= 04e140) for 0x10109b4c0: [0x100200000, 0x118200000), lkey=3D0x104b0100 base_idx=3D0 ms_n=3D192, ms= _bmp_n=3D192 net_mlx5: mlx5_mr.c:345: mr_insert_dev_cache(): device mlx5_0 inserting MR(= 0x10104e140) to global cache net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B-tree(0x1003dcb20)[1]= , [0x100200000, 0x118200000) lkey=3D0x104b0100 net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B-tree(0x10104fa6c)[1]= , [0x100200000, 0x118200000) lkey=3D0x104b0100 net_mlx5: mlx5_rxq.c:257: rxq_alloc_elts_sprq(): port 0 Rx queue 0 allocate= d and configured 256 segments (max 256 packets) net_mlx5: mlx5_rxq.c:1402: mlx5_rxq_obj_new(): port 0 device_attr.max_qp_wr= is 32768 net_mlx5: mlx5_rxq.c:1404: mlx5_rxq_obj_new(): port 0 device_attr.max_sge i= s 30 net_mlx5: mlx5_rxq.c:1478: mlx5_rxq_obj_new(): port 0 rxq 0 updated with 0x= 7ffda24f21f8 net_mlx5: mlx5_trigger.c:322: mlx5_dev_start(): port 0 failed to set defaul= ts flows net_mlx5: mlx5_rxq.c:333: rxq_free_elts_sprq(): port 0 Rx queue 0 freeing W= Rs Fail to start port 0 Configuring Port 1 (socket 0) net_mlx5: mlx5_ethdev.c:424: mlx5_dev_configure(): port 1 Tx queues number = update: 0 -> 1 net_mlx5: mlx5_ethdev.c:435: mlx5_dev_configure(): port 1 Rx queues number = update: 0 -> 1 net_mlx5: mlx5_txq.c:172: mlx5_tx_queue_pre_setup(): port 1 configuring que= ue 0 for 256 descriptors net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x10104d6= a8 with table 0x10104c040 net_mlx5: mlx5_txq.c:225: mlx5_tx_queue_setup(): port 1 adding Tx queue 0 t= o list net_mlx5: mlx5_rxq.c:468: mlx5_rx_queue_pre_setup(): port 1 configuring Rx = queue 0 for 256 descriptors net_mlx5: mlx5_mr.c:214: mlx5_mr_btree_init(): initialized B-tree 0x10104b4= ec with table 0x101049f80 net_mlx5: mlx5_rxq.c:1921: mlx5_rxq_new(): port 1 maximum number of segment= s per packet: 1 net_mlx5: mlx5_rxq.c:1759: mlx5_max_lro_msg_size_adjust(): port 1 Rx Queue = 0 max LRO message size adjusted to 1280 bytes net_mlx5: mlx5_rxq.c:1968: mlx5_rxq_new(): port 1 CRC stripping is enabled,= 0 bytes will be subtracted from incoming frames to hide it net_mlx5: mlx5_rxq.c:525: mlx5_rx_queue_setup(): port 1 adding Rx queue 0 t= o list net_mlx5: mlx5_trigger.c:276: mlx5_dev_start(): port 1 starting device net_mlx5: mlx5_ethdev.c:493: mlx5_dev_configure_rss_reta(): port 1 Rx queue= s number update: 1 -> 1 net_mlx5: mlx5_txq.c:55: txq_alloc_elts(): port 1 Tx queue 0 allocated and = configured 256 WRs net_mlx5: mlx5_txq.c:771: mlx5_txq_obj_new(): port 1: uar_mmap_offset 0x306= 000 net_mlx5: mlx5_trigger.c:145: mlx5_rxq_start(): port 1 Rx queue 0 registeri= ng mp mbuf_pool_socket_0 having 1 chunks net_mlx5: mlx5_mr.c:600: mlx5_mr_create_primary(): port 1 creating a MR usi= ng address (0x10109b4c0) net_mlx5: mlx5_mr.c:649: mlx5_mr_create_primary(): port 1 extending 0x10109= b4c0 to [0x100200000, 0x118200000), page_sz=3D0x200000, ms_n=3D192 net_mlx5: mlx5_mr.c:786: mlx5_mr_create_primary(): port 1 MR CREATED (0x101= 045dc0) for 0x10109b4c0: [0x100200000, 0x118200000), lkey=3D0xcac80f00 base_idx=3D0 ms_n=3D192, ms= _bmp_n=3D192 net_mlx5: mlx5_mr.c:345: mr_insert_dev_cache(): device mlx5_1 inserting MR(= 0x101045dc0) to global cache net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B-tree(0x10037b420)[1]= , [0x100200000, 0x118200000) lkey=3D0xcac80f00 net_mlx5: mlx5_mr.c:173: mr_btree_insert(): inserted B-tree(0x10104b4ec)[1]= , [0x100200000, 0x118200000) lkey=3D0xcac80f00 net_mlx5: mlx5_rxq.c:257: rxq_alloc_elts_sprq(): port 1 Rx queue 0 allocate= d and configured 256 segments (max 256 packets) net_mlx5: mlx5_rxq.c:1402: mlx5_rxq_obj_new(): port 1 device_attr.max_qp_wr= is 32768 net_mlx5: mlx5_rxq.c:1404: mlx5_rxq_obj_new(): port 1 device_attr.max_sge i= s 30 net_mlx5: mlx5_rxq.c:1478: mlx5_rxq_obj_new(): port 1 rxq 0 updated with 0x= 7ffda24f21f8 net_mlx5: mlx5_trigger.c:322: mlx5_dev_start(): port 1 failed to set defaul= ts flows net_mlx5: mlx5_rxq.c:333: rxq_free_elts_sprq(): port 1 Rx queue 0 freeing W= Rs Fail to start port 1 Please stop the ports first Done No commandline core given, start packet forwarding Not all ports were started Press enter to exit Stopping port 0... Stopping ports... Done Stopping port 1... Stopping ports... Done Shutting down port 0... Closing ports... net_mlx5: mlx5.c:1233: mlx5_dev_close(): port 0 closing device "mlx5_0" net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x10104fa6c w= ith table 0x10104e500 net_mlx5: mlx5_txq.c:77: txq_free_elts(): port 0 Tx queue 0 freeing WRs net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x101051c28 w= ith table 0x1010505c0 net_mlx5: mlx5_mr.c:1569: mlx5_mr_dump_dev(): device mlx5_0 MR[0], LKey =3D= 0x104b0100, ms_n =3D 192, ms_bmp_n =3D 192 net_mlx5: mlx5_mr.c:1579: mlx5_mr_dump_dev(): chunk[0], [0x100200000, 0x1= 18200000) net_mlx5: mlx5_mr.c:1582: mlx5_mr_dump_dev(): device mlx5_0 dumping global = cache net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x1003dcb20)[0], [0x0= , 0x0) lkey=3D0xffffffff net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x1003dcb20)[1], [0x1= 00200000, 0x118200000) lkey=3D0x104b0100 net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x1003dcb20 w= ith table 0x1003d9640 net_mlx5: mlx5_mr.c:459: mr_free(): freeing MR(0x10104e140): Done Shutting down port 1... Closing ports... net_mlx5: mlx5.c:1233: mlx5_dev_close(): port 1 closing device "mlx5_1" net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x10104b4ec w= ith table 0x101049f80 net_mlx5: mlx5_txq.c:77: txq_free_elts(): port 1 Tx queue 0 freeing WRs net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x10104d6a8 w= ith table 0x10104c040 net_mlx5: mlx5_mr.c:1569: mlx5_mr_dump_dev(): device mlx5_1 MR[0], LKey =3D= 0xcac80f00, ms_n =3D 192, ms_bmp_n =3D 192 net_mlx5: mlx5_mr.c:1579: mlx5_mr_dump_dev(): chunk[0], [0x100200000, 0x1= 18200000) net_mlx5: mlx5_mr.c:1582: mlx5_mr_dump_dev(): device mlx5_1 dumping global = cache net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x10037b420)[0], [0x0= , 0x0) lkey=3D0xffffffff net_mlx5: mlx5_mr.c:256: mlx5_mr_btree_dump(): B-tree(0x10037b420)[1], [0x1= 00200000, 0x118200000) lkey=3D0xcac80f00 net_mlx5: mlx5_mr.c:230: mlx5_mr_btree_free(): freeing B-tree 0x10037b420 w= ith table 0x100377f40 net_mlx5: mlx5_mr.c:459: mr_free(): freeing MR(0x101045dc0): Done Bye... Best ben > From: Benoit Ganne (bganne) > Sent: Thursday, April 2, 2020 6:50:22 PM > To: users@dpdk.org > Cc: Matan Azrad ; Shahaf Shuler > ; Slava Ovsiienko > Subject: RE: mlx5 pmd + rdma-core 28 init failure >=20 > Adding MLX5 PMD maintainers. >=20 > I also checked with latest rdma-core master and latest DPDK master and it > is failing with the same issue. > Any recommendation? >=20 > Best > ben >=20 > > -----Original Message----- > > From: Benoit Ganne (bganne) > > Sent: mercredi 1 avril 2020 18:52 > > To: users@dpdk.org > > Subject: mlx5 pmd + rdma-core 28 init failure > > > > Hi all, > > > > I have troubles making DPDK v20.02 MLX5 PMD working with rdma-core > v28.0: > > it looks like the flow initialization done while initializing the devic= e > > fails in rdma-core providers/mlx5/dr_table.c:mlx5dv_dr_table_create() > > because of unsupported parameters. > > The issue is the following test in rdma-core > > providers/mlx5/dr_table.c:mlx5dv_dr_table_create(): > > if (level && !dmn->info.supp_sw_steering) { > > errno =3D EOPNOTSUPP; > > goto dec_ref; > > } > > Where level =3D=3D 65534 and dmn->info.supp_sw_steering =3D=3D false, h= ence the > > test is true and the function returns EOPNOTSUPP. > > > > Is this expected? It does work fine with ibv instead of dv API. Any hel= p > > appreciated. > > > > Here is what I do: > > 1) checkout & compile rdma-core v28.0 > > ~# git clone > https://eur03.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.c > om%2Flinux-rdma%2Frdma- > core&data=3D02%7C01%7Cmatan%40mellanox.com%7C7984449b13af4519d70608d7= d71 > d9000%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637214394282618427&= ; > sdata=3DDSLvWg3cWwY8prSFOm8BAFf1I2bbKnqPOKBPEiy4b44%3D&reserved=3D0 > com%2Flinux-rdma%2Frdma- > core&data=3D02%7C01%7Cmatan%40mellanox.com%7C7984449b13af4519d70608d7= d71 > d9000%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C637214394282618427&= ; > sdata=3DDSLvWg3cWwY8prSFOm8BAFf1I2bbKnqPOKBPEiy4b44%3D&reserved=3D0> > > ~# cd rdma-core > > ~# git checkout v28.0 > > ~# mkdir build > > ~# cd build > > ~# CFLAGS=3D-fPIC cmake -DIN_PLACE=3D1 -DENABLE_STATIC=3D1 -GNinja .. > > ~# ninja > > > > 2) checkout & compile dpdk v20.02 > > ~# git clone git://dpdk.org/dpdk > > ~# cd dpdk > > ~# make config T=3Dx86_64-native-linuxapp-gcc > > ~# sed -ri 's,(MLX5_PMD=3D).*,\1y,' build/.config > > ~# sed -ri 's,(IBVERBS_LINK_STATIC_PMD=3D).*,\1y,' build/.config > > ~# make EXTRA_CFLAGS=3D-I/home/bganne/src/rdma-core/build/include > > EXTRA_LDFLAGS=3D-L/home/bganne/src/rdma-core/build/lib > > PKG_CONFIG_PATH=3D/home/bganne/src/rdma-core/build/lib/pkgconfig > > ~# sudo gdb --args ./build/app/testpmd -w 0000:5e:00.0 -w 0000:5e:00.1 = - > l > > 4,11,35 -- -a --forward-mode=3Drxonly -i > > > > The backtrace looks like this: > > #0 mlx5dv_dr_table_create (dmn=3D0x555556c641b0, level=3D65534) at > > ../providers/mlx5/dr_table.c:183 > > #1 0x0000555555dfaeaa in flow_dv_tbl_resource_get (dev=3D, > > table_id=3D65534, egress=3D, transfer=3D, > > error=3D0x7fffffffdca0) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:6746 > > #2 0x0000555555e02b28 in __flow_dv_translate > > (dev=3Ddev@entry=3D0x555556bbcdc0 , dev_flow=3D0x10038= 8300, > > attr=3D, items=3D, actions=3D, > > error=3D) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:7503 > > #3 0x0000555555e04954 in flow_dv_translate (dev=3D0x555556bbcdc0 > > , dev_flow=3D, attr=3D, > > items=3D, actions=3D, error=3D) > at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow_dv.c:8841 > > #4 0x0000555555df152f in flow_drv_translate (error=3D0x7fffffffdca0, > > actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, attr=3D0x7fffffffbb88= , > > dev_flow=3D, dev=3D0x555556bbcdc0 ) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:2571 > > #5 flow_create_split_inner (error=3D0x7fffffffdca0, external=3Dfalse, > > actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, attr=3D0x7fffffffbb88= , > > prefix_layers=3D0, sub_flow=3D0x0, flow=3D0x1003885c0, dev=3D0x555556bb= cdc0 > > ) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3490 > > #6 flow_create_split_metadata (error=3D0x7fffffffdca0, external=3Dfals= e, > > actions=3D0x7fffffffdce0, items=3D0x7fffffffdcc0, attr=3D0x7fffffffbb88= , > > prefix_layers=3D0, flow=3D0x1003885c0, dev=3D0x555556bbcdc0 ) > > at /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:3865 > > #7 flow_create_split_meter (error=3D0x7fffffffdca0, external=3Dfalse, > > actions=3D0x7fffffffdce0, items=3D, attr=3D0x7fffffffdc9= 4, > > flow=3D0x1003885c0, dev=3D0x555556bbcdc0 ) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4121 > > #8 flow_create_split_outer (error=3D0x7fffffffdca0, external=3Dfalse, > > actions=3D0x7fffffffdce0, items=3D, attr=3D0x7fffffffdc9= 4, > > flow=3D0x1003885c0, dev=3D0x555556bbcdc0 ) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4178 > > #9 flow_list_create (dev=3Ddev@entry=3D0x555556bbcdc0 , > > list=3Dlist@entry=3D0x0, attr=3Dattr@entry=3D0x7fffffffdc94, > > items=3Ditems@entry=3D0x7fffffffdcc0, actions=3Dactions@entry=3D0x7ffff= fffdce0, > > external=3Dexternal@entry=3Dfalse, error=3D0x7fffffffdca0) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:4306 > > #10 0x0000555555df8587 in mlx5_flow_discover_mreg_c > > (dev=3Ddev@entry=3D0x555556bbcdc0 ) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5_flow.c:5747 > > #11 0x0000555555d692a6 in mlx5_dev_spawn (config=3D..., spawn=3D0x1003e= 9e00, > > dpdk_dev=3D0x555556dd6fe0) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:2763 > > #12 mlx5_pci_probe (pci_drv=3D, pci_dev=3D) at > > /home/bganne/src/dpdk/drivers/net/mlx5/mlx5.c:3363 > > #13 0x0000555555a411c8 in pci_probe_all_drivers () > > #14 0x0000555555a412f8 in rte_pci_probe () > > #15 0x0000555555a083da in rte_bus_probe () > > #16 0x00005555559f204d in rte_eal_init () > > #17 0x00005555556a0d45 in main () > > > > Best > > ben