From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 63845A00C3 for ; Thu, 20 Jan 2022 08:16:53 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 33CB241C26; Thu, 20 Jan 2022 08:16:53 +0100 (CET) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2081.outbound.protection.outlook.com [40.107.102.81]) by mails.dpdk.org (Postfix) with ESMTP id E6EF840042 for ; Thu, 20 Jan 2022 08:16:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k3qMIUXnpHHEhbCI4ATRCO0tzYOIG9Rv/iZFXl7fXCHpU7v5yiQ46cVCxScLrZiGQEZyPvUQND8CToEZ/yNS4vcNiAZ0l/cQtXtY+SEaTW6xiJ61B85Z2zVSuhaDOCVn9uFWq5DvN1Q+VKL8y+A+OdwL+Y3RgBu+foNeDCfcHpsWZ92SlGlxkZNkBaTpKQa1NxIFVtplGeuognX68wBCOz5pWNV+lRCF2PqoBU46btcHePVq6Gk+Gz3n7+dD8tyvQYrp3CsYbh1iiaHTlyVNv9ZOoYEa2BL7t85dUhzjMw/O54nN5iSNgtJ6rFBCcbzLBqfjoizvlG70zemeYoq+ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hkS8iCIWFR01UKkYGE4XeJ0ps515xgvhF4jhvSCPBew=; b=XnOZLo8Eg44Km34c05SMlYqVu5TY8/gbCoP/PNGd7ILl17XTAyPsrJTAWn5X8qorzF5XSN2cqjnIn05gA3OJ+HUZNBdvDQX2y6ZppB2ZDwoHRTxHgWxFMcC8lH4YMz/lHs/BDL2O8kx/thWjKvqH25hD0TMIvbEuwvbNnDOxgHpejzs8+wBPffCyVh2qwsJAEpNeaztl9+/5ZYwM9MSMY2m5q4D7ziz6nytj6UFEc6xxcMxWYnRrdDuVNYSzDqurJ0Ke+ZIVlqIaYzHlY8xlavdVLc2VPI8+d3xTd22/w1c02IBqgY++q4EHIGx/hCjzI1FKwsRNpYo9PQEnzgwBPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hkS8iCIWFR01UKkYGE4XeJ0ps515xgvhF4jhvSCPBew=; b=NtmB0kfCU7qZxC8p63bK8WzAct+K+5oOlrvMObeUR2VQF7MnR/MPauIAXnuxOaLGZFMmqJF9H5CN+IFy3shc6h7kDz5cwD4oKaD9J6I5+GIFWEnXLKD3hSXdqV9dnYp1kAbF6+3NXG2Ljgr7Inx+AvHlPRykOihOibnZRLO60c1DdrcFkIbec7bYoI8kx3lWzMcKNSFzUBIq9cNxHuacHe4W9fj9ihbiBVLsdqL9lLDOuTz0TW3LfFWljad28JBmobcy0ipcvAPG6D4gZjT044KzC+KoCbd8Q4tPrx7MeRw4BfzemJzqN+QahTqwdtl0Yf5FD+XF2Hv472RbV78TfQ== Received: from DM5PR1201MB2555.namprd12.prod.outlook.com (2603:10b6:3:ea::14) by MWHPR1201MB2558.namprd12.prod.outlook.com (2603:10b6:300:e5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.12; Thu, 20 Jan 2022 07:16:49 +0000 Received: from DM5PR1201MB2555.namprd12.prod.outlook.com ([fe80::e4c8:ad94:bd2f:3fdd]) by DM5PR1201MB2555.namprd12.prod.outlook.com ([fe80::e4c8:ad94:bd2f:3fdd%5]) with mapi id 15.20.4888.014; Thu, 20 Jan 2022 07:16:49 +0000 From: Asaf Penso To: Rocio Dominguez , "NBU-Contact-Thomas Monjalon (EXTERNAL)" CC: "users@dpdk.org" , Matan Azrad , Slava Ovsiienko , Raslan Darawsheh Subject: Re: net_mlx5: unable to recognize master/representors on the multiple IB devices Thread-Topic: net_mlx5: unable to recognize master/representors on the multiple IB devices Thread-Index: AdgJLQTmyQJM7lgBQqSyceNdxLJR4QBr0gyAAARnZ6AAI+HbQACUBFcN Date: Thu, 20 Jan 2022 07:16:49 +0000 Message-ID: References: <37327910.J2Yia2DhmK@thomas> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 61c985c3-87fa-4bf6-7e51-08d9dbe4d378 x-ms-traffictypediagnostic: MWHPR1201MB2558:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: UswtnptP62PyXGCH4Sc/F9YNdyJZyHsNoor+m5cUZxYElI+3QJhQNjRS2f98G00b/C1nl1NoZtbAKFLHOes8hjCfBKmJZk6KKGIjXeXaOc7t7etAjG4KmBhTbFuIrvgbUiD3HwpNXMxPi4JDxxEAfXJt8JC9BlkQi2aJBFqVH5KXJUvKlHOyRvW6ZAOnODMBcPGa7DR4EJy78XkBcztwxOi88p+Vh3gqHCozpMi60PhTy0MY+HQfCba50khk8ucAoLVuRuWSKi9cfetSoeHYBfEwNWTtdKVm/j9RbBhhaZ1qfSsIYp6UBWxPrponNiMPNukiQgWV8+vUpTrxswxIVqOOri6st+EW0f35DpyMr8eQtoT3jqRWC7xVDqtgEZVghBYubUNYGSxNgteTLhCAKA3chZ9uOWxF0i3cMaUT7xQqv75c3x6/k+TgUuXoPPq0uMkEgZVN9l0TyN35F+W2j7JGgrT3+EY2bo8e41m4d7Lo0op1bVlTwCEbMiD4b48Oi4SKm6W3M4L2sVSbYkLE1rXiVl1sgE4fwChE3PGbR3qfbQsM4lHUPw2zh0EPkrfKpR9b8nSH/PDa5ifbXR+WTV578HP0EvGeSnrlAGMUyl2U0AOMqSAcOa69ywvU5YuNIX6Jv4mOAJBthYRHSmr8zVXe4h0dh+L3Itv5M8kl3HXbBUWCQFhnxeuesdBLonnXWVVArIn7xYx9OQMSpHRM1w== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM5PR1201MB2555.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(53546011)(9686003)(66446008)(8676002)(508600001)(2906002)(71200400001)(6506007)(8936002)(64756008)(4326008)(33656002)(5660300002)(66574015)(26005)(66946007)(52536014)(66476007)(38100700002)(316002)(66556008)(38070700005)(7696005)(55016003)(91956017)(83380400001)(76116006)(86362001)(110136005)(107886003)(186003)(54906003)(122000001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?rXMjCHxBY9jkc4Gnru+XCdlPj7X6KNlC+6Pw5BB1qvlG0qf6xTVTABqSNj?= =?iso-8859-1?Q?SsjX/RXzHY9k7ds0oCKMN56oCVEEshoui76nYY7YIhuT6NW7myhEW3SJR0?= =?iso-8859-1?Q?FgonmHiLLuyd5NdnOVv03eYrpplvCcQ9hr6dnxeHiSbQcjL87d6GsNa2t0?= =?iso-8859-1?Q?EmcblvVOSlp3Z9puZzC5Q8JAnCLPXNhp8yS+HtPCqAQBRaWraxSJ1ZXZ6I?= =?iso-8859-1?Q?MnuQUbRYsLV+BQmX9jg3MZrWA8Wvq08TQ++xRYSr+kfK7OL3nZWabwIpZ9?= =?iso-8859-1?Q?6w4xsedWFDnRToY6gU1htF1eSAHvW4qvOSJT5c5YHV6R+kQoGNH/dSWQbM?= =?iso-8859-1?Q?CCwh1eHA24w7ncILPBytV6FPdMw5U5qVuIQ46IgRbAOeJdsKSy9BbvPFrD?= =?iso-8859-1?Q?4Bl1XvXSsbBxFkDcxf1KXhe2jvfnOwRkEMb5FvGtClc4ZjBgsImvsGGyIG?= =?iso-8859-1?Q?0zvM2RwPkjzymzaf04hWiz5M3WRcxSaUoKO/XPy7OSgKwRgJSMJtkxKINF?= =?iso-8859-1?Q?4UhY01P90Wgxl6YmH3lTvkUlp0Mj84yvBQW81OHz2AfRAqMvqoXUcB5YzJ?= =?iso-8859-1?Q?g08MQXm5gd9FTGAyqMS3BFhxurbGq7RA0SdqvijRf1fBcgkfo1OUM84iE4?= =?iso-8859-1?Q?eVZUA8AInEeXU/3wnCI9vq/tAyo7vAv6y/nCO8TK3Z11MtSqZe7csX6Sb7?= =?iso-8859-1?Q?wh/GiBkcCpEkzwsj/dPnlpU0YfhVbLArn2JspWy6KxGMYBjdLEdwWOtiAf?= =?iso-8859-1?Q?x0jJ2jkBZfzSINF2F/FESjYXWzB1yXzm4fWMKdLd0V6wa5SrRv7mGOzebx?= =?iso-8859-1?Q?N1GKxM8koX5KZjzn8N2bXtZ/pzeSShxeeVV0rfAmDF354OYtriFq1I4hNj?= =?iso-8859-1?Q?bLqI34TTyjzj+fPngnwZXHFK56CrmqtS8Jbl7/Nps4lAShsZscHGG0CQzo?= =?iso-8859-1?Q?BbkPDtPwcp3RBq+1QimDZmyXIGcusrZg2s32yyQ9EccGQgyatNT8rjhs6q?= =?iso-8859-1?Q?RnjwDFa7V5KAsjFThYS3muNIKIEEboJyCPE3Vjc4450pY4l0sybPh36kEJ?= =?iso-8859-1?Q?uBxCQLK8HRkVKsXokNkUkDsci/x4FAdym6tRuYYIMo6KK6sUAyiYYyAAoT?= =?iso-8859-1?Q?paFy48b5EN2O3w8xrC4F8uwDPkCWKc7t5h8gNOluVJI/7oH5OMaTmU7gwv?= =?iso-8859-1?Q?Z5Ps95YYOZlbpX6exKy88/r9y3fkl4qyKu/887gUmM3nykR6hVodr6NsbW?= =?iso-8859-1?Q?O/WZuLRdZZJpxL0kg5weBmViwU5olguW2tOhVBDGUsTGawyvlUl+ybnpc/?= =?iso-8859-1?Q?7r9oUGpA9yJelBj5lQ9MwUK85uvS81IwBRAAkm2paHCjZQz0wRgM85krUY?= =?iso-8859-1?Q?Q19GLc+Lev8sNss5VC564zWB0D/Y7Qwq9wtFLm0YK2j+V5uu6lerIykATT?= =?iso-8859-1?Q?acQgwN1nJW82AGyKwgEEA570n0KnQFt4p6PmdslLC5I6BbTcY/YHOwDOEO?= =?iso-8859-1?Q?4K9zmjvCQevySdaPuVfZDf8YgD/qxKvp/M+oIDwTEHsGYVGojXa5wzCwZu?= =?iso-8859-1?Q?Kf25qT3B0Zm4bHwohXyEMHpNUUaiPnIpTYSWyeRcMIaXBZVV/Lcqapm5iu?= =?iso-8859-1?Q?Luu1McH2ULmx7C8fF1qE6QN2uOrPB8rJhA55J9WAXVfU6cSJAHBT43kbd3?= =?iso-8859-1?Q?CJxD+QUlp7xqZ9tUWMk=3D?= Content-Type: multipart/alternative; boundary="_000_DM5PR1201MB25556CD33E69885710EBC247CD5A9DM5PR1201MB2555_" MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM5PR1201MB2555.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 61c985c3-87fa-4bf6-7e51-08d9dbe4d378 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jan 2022 07:16:49.5457 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rgBSX5fwrUdce4jObPU+6IUW0elnqbr0agGdRrVF/rnkPLof3xiUNsEgJqokDAHSPRNAu2BN6WOGS84mnrdlyA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB2558 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --_000_DM5PR1201MB25556CD33E69885710EBC247CD5A9DM5PR1201MB2555_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Although inbox drivers come with a pre installed DPDK, you can manually dow= nload, compile, install, and work with whatever version you wish. Let us know the results, and we'll continue from there. Regards, Asaf Penso ________________________________ From: Rocio Dominguez Sent: Monday, January 17, 2022 10:20:58 PM To: Asaf Penso ; NBU-Contact-Thomas Monjalon (EXTERNAL) <= thomas@monjalon.net> Cc: users@dpdk.org ; Matan Azrad ; Slava = Ovsiienko ; Raslan Darawsheh Subject: RE: net_mlx5: unable to recognize master/representors on the multi= ple IB devices Hi Asaf, Thanks for the prompt answer. I have checked that the latest 19.11 LTS is 19.11.11, but in OpenSUSE repos= itories the corresponding RPM package for SLES 15 SP2 is not available, the= latest one is DPDK 19.11.10. I have installed it but the problem persists. It's probably solved in 19.11= .11. There is a RPM package in SLES 15 SP3 for DPDK 20.11.3, which is LTS also, = not sure if it could be a problem to install it in SLES 15 SP2. I will try = it anyway. Also I will try to find other way to load the 19.11.11 in SLES 15 SP2 apart= from using RPM or zipper, any suggestion is appreciated. Thanks, Roc=EDo -----Original Message----- From: Asaf Penso Sent: Sunday, January 16, 2022 4:31 PM To: NBU-Contact-Thomas Monjalon (EXTERNAL) ; Rocio Dom= inguez Cc: users@dpdk.org; Matan Azrad ; Slava Ovsiienko ; Raslan Darawsheh Subject: RE: net_mlx5: unable to recognize master/representors on the multi= ple IB devices Hello Rocio, IIRC, there was a fix in a recent stable version. Would you please try taking latest 19.11 LTS and tell whether you still see= the issue? Regards, Asaf Penso >-----Original Message----- >From: Thomas Monjalon >Sent: Sunday, January 16, 2022 3:24 PM >To: Rocio Dominguez >Cc: users@dpdk.org; Matan Azrad ; Slava Ovsiienko >; Raslan Darawsheh >Subject: Re: net_mlx5: unable to recognize master/representors on the >multiple IB devices > >+Cc mlx5 experts > > >14/01/2022 11:10, Rocio Dominguez: >> Hi, >> >> I'm doing a setup with Mellanox ConnectX-4 (MCX416A-CCA) NICs. >> >> I'm using: >> >> OS SLES 15 SP2 >> DPDK 19.11.4 (the official supported version for SLES 15 SP2) >> MLNX_OFED_LINUX-5.5-1.0.3.2-sles15sp2-x86_64 (the latest one) >> Mellanox adapters firmware 12.28.2006 (corresponding to this >> MLNX_OFED version) kernel 5.3.18-24.34-default >> >> >> This is my SRIOV configuration for DPDK capable PCI slots: >> >> { >> "resourceName": "mlnx_sriov_netdevice", >> "resourcePrefix": "mellanox.com", >> "isRdma": true, >> "selectors": { >> "vendors": ["15b3"], >> "devices": ["1014"], >> "drivers": ["mlx5_core"], >> "pciAddresses": ["0000:d8:00.2", "0000:d8:00.3", >> "0000:d8:00.4", >"0000:d8:00.5"], >> "isRdma": true >> } >> >> The sriov device plugin starts without problems, the devices are >> correctly >allocated: >> >> { >> "cpu": "92", >> "ephemeral-storage": "419533922385", >> "hugepages-1Gi": "8Gi", >> "hugepages-2Mi": "4Gi", >> "intel.com/intel_sriov_dpdk": "0", >> "intel.com/sriov_cre": "3", >> "mellanox.com/mlnx_sriov_netdevice": "4", >> "mellanox.com/sriov_dp": "0", >> "memory": "183870336Ki", >> "pods": "110" >> } >> >> The Mellanox NICs are binded to the kernel driver mlx5_core: >> >> pcgwpod009-c04:~ # dpdk-devbind --status >> >> Network devices using kernel driver >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> 0000:18:00.0 'Ethernet Controller 10G X550T 1563' if=3Dem1 drv=3Dixgbe >> unused=3Dvfio-pci >> 0000:18:00.1 'Ethernet Controller 10G X550T 1563' if=3Dem2 drv=3Dixgbe >> unused=3Dvfio-pci >> 0000:19:00.0 'Ethernet Controller 10G X550T 1563' if=3Dem3 drv=3Dixgbe >> unused=3Dvfio-pci >> 0000:19:00.1 'Ethernet Controller 10G X550T 1563' if=3Dem4 drv=3Dixgbe >> unused=3Dvfio-pci >> 0000:3b:00.0 'MT27700 Family [ConnectX-4] 1013' if=3Denp59s0f0 >> drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:3b:00.1 'MT27700 Family [ConnectX-4] 1013' if=3Denp59s0f1 >> drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' >> if=3Dp3p1 drv=3Dixgbe unused=3Dvfio-pci >> 0000:5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' >> if=3Dp3p2 drv=3Dixgbe unused=3Dvfio-pci >> 0000:5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' if=3D >> drv=3Dixgbevf unused=3Dvfio-pci >> 0000:5e:10.2 '82599 Ethernet Controller Virtual Function 10ed' >> if=3Dp3p1_1 drv=3Dixgbevf unused=3Dvfio-pci >> 0000:5e:10.4 '82599 Ethernet Controller Virtual Function 10ed' if=3D >> drv=3Dixgbevf unused=3Dvfio-pci >> 0000:5e:10.6 '82599 Ethernet Controller Virtual Function 10ed' >> if=3Dp3p1_3 drv=3Dixgbevf unused=3Dvfio-pci >> 0000:af:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' >> if=3Dp4p1 drv=3Dixgbe unused=3Dvfio-pci >> 0000:af:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' >> if=3Dp4p2 drv=3Dixgbe unused=3Dvfio-pci >> 0000:d8:00.0 'MT27700 Family [ConnectX-4] 1013' if=3Denp216s0f0 >> drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:d8:00.1 'MT27700 Family [ConnectX-4] 1013' if=3Denp216s0f1 >> drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:d8:00.2 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f2 drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:d8:00.3 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f3 drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:d8:00.4 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f4 drv=3Dmlx5_core unused=3Dvfio-pci >> 0000:d8:00.5 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f5 drv=3Dmlx5_core unused=3Dvfio-pci >> >> The interfaces are up: >> >> pcgwpod009-c04:~ # ibdev2netdev -v >> 0000:3b:00.0 mlx5_0 (MT4115 - MT1646K01301) CX416A - ConnectX-4 >QSFP28 >> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp59s0f0 (Up) >> 0000:3b:00.1 mlx5_1 (MT4115 - MT1646K01301) CX416A - ConnectX-4 >QSFP28 >> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp59s0f1 (Up) >> 0000:d8:00.0 mlx5_2 (MT4115 - MT1646K00538) CX416A - ConnectX-4 >QSFP28 >> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp216s0f0 (Up) >> 0000:d8:00.1 mlx5_3 (MT4115 - MT1646K00538) CX416A - ConnectX-4 >QSFP28 >> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp216s0f1 (Up) >> 0000:d8:00.2 mlx5_4 (MT4116 - NA) fw 12.28.2006 port 1 (ACTIVE) =3D=3D> >> enp216s0f2 (Up) >> 0000:d8:00.3 mlx5_5 (MT4116 - NA) fw 12.28.2006 port 1 (ACTIVE) =3D=3D> >> enp216s0f3 (Up) >> 0000:d8:00.4 mlx5_6 (MT4116 - NA) fw 12.28.2006 port 1 (ACTIVE) =3D=3D> >> enp216s0f4 (Up) >> 0000:d8:00.5 mlx5_7 (MT4116 - NA) fw 12.28.2006 port 1 (ACTIVE) =3D=3D> >> enp216s0f5 (Up) pcgwpod009-c04:~ # >> >> >> But when I run my application the Mellanox adapters are probed and I >obtain the following error: >> >> {"proc_id":"6"},"message":"[pio] EAL: Probe PCI driver: mlx5_pci >> (15b3:1014) device: 0000:d8:00.4 (socket 1)"} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","sever >> i >> ty":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id": >> "6"},"message":"[pio] net_mlx5: unable to recognize >> master/representors on the multiple IB devices"} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","sever >> i >> ty":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id": >> "6"},"message":"[pio] common_mlx5: Failed to load driver =3D >> net_mlx5."} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","sever >> i >> ty":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id": >> "6"},"message":"[pio] EAL: Requested device 0000:d8:00.4 cannot be >> used"} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","sever >> i >> ty":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id": >> "6"},"message":"[pio] EAL: Bus (pci) probe failed."} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.890+00:00","sever >> i >> ty":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id": >> "6"},"message":"[pio] USER1: ports init fail in DPDK, expect 1 ports, >> actual 0 ports."} >> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.890+00:00","sever >> i >> ty":"error","service_id":"eric-pc-up-data-plane","metadata":{"proc_id" >> :"6"},"message":"[pktio_libpio_init] No network ports could be >> enabled!"} >> >> Could you please help me with this issue? >> >> >> Thanks, >> >> Roc=EDo >> > > > > --_000_DM5PR1201MB25556CD33E69885710EBC247CD5A9DM5PR1201MB2555_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Although inbox drivers come with a pre installed DPDK, yo= u can manually download, compile, install, and work with whatever version y= ou wish.

Let us know the results, and we'll continue from there.

Regards,
Asaf Penso
From: Rocio Dominguez <r= ocio.dominguez@ericsson.com>
Sent: Monday, January 17, 2022 10:20:58 PM
To: Asaf Penso <asafp@nvidia.com>; NBU-Contact-Thomas Monjalon= (EXTERNAL) <thomas@monjalon.net>
Cc: users@dpdk.org <users@dpdk.org>; Matan Azrad <matan@nvi= dia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; Raslan Darawsh= eh <rasland@nvidia.com>
Subject: RE: net_mlx5: unable to recognize master/representors on th= e multiple IB devices
 
Hi Asaf,

Thanks for the prompt answer.

I have checked that the latest 19.11 LTS is 19.11.11, but in OpenSUSE repos= itories the corresponding RPM package for SLES 15 SP2 is not available, the= latest one is DPDK 19.11.10.

I have installed it but the problem persists. It's probably solved in 19.11= .11.

There is a RPM package in SLES 15 SP3 for DPDK 20.11.3, which is LTS also, = not sure if it could be a problem to install it in SLES 15 SP2. I will try = it anyway.

Also I will try to find other way to load the 19.11.11 in SLES 15 SP2 apart= from using RPM or zipper, any suggestion is appreciated.

Thanks,

Roc=EDo

-----Original Message-----
From: Asaf Penso <asafp@nvidia.com>
Sent: Sunday, January 16, 2022 4:31 PM
To: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Roc= io Dominguez <rocio.dominguez@ericsson.com>
Cc: users@dpdk.org; Matan Azrad <matan@nvidia.com>; Slava Ovsiienko &= lt;viacheslavo@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com><= br> Subject: RE: net_mlx5: unable to recognize master/representors on the multi= ple IB devices

Hello Rocio,
IIRC, there was a fix in a recent stable version.
Would you please try taking latest 19.11 LTS and tell whether you still see= the issue?

Regards,
Asaf Penso

>-----Original Message-----
>From: Thomas Monjalon <thomas@monjalon.net>
>Sent: Sunday, January 16, 2022 3:24 PM
>To: Rocio Dominguez <rocio.dominguez@ericsson.com>
>Cc: users@dpdk.org; Matan Azrad <matan@nvidia.com>; Slava Ovsiien= ko
><viacheslavo@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com= >
>Subject: Re: net_mlx5: unable to recognize master/representors on the <= br> >multiple IB devices
>
>+Cc mlx5 experts
>
>
>14/01/2022 11:10, Rocio Dominguez:
>> Hi,
>>
>> I'm doing a setup with Mellanox ConnectX-4 (MCX416A-CCA) NICs.
>>
>> I'm using:
>>
>> OS SLES 15 SP2
>> DPDK 19.11.4 (the official supported version for SLES 15 SP2)
>> MLNX_OFED_LINUX-5.5-1.0.3.2-sles15sp2-x86_64 (the latest one)
>> Mellanox adapters firmware 12.28.2006 (corresponding to this
>> MLNX_OFED version) kernel 5.3.18-24.34-default
>>
>>
>> This is my SRIOV configuration for DPDK capable PCI slots:
>>
>>           &= nbsp; {
>>           &= nbsp;     "resourceName": "mlnx_sriov_ne= tdevice",
>>           &= nbsp;     "resourcePrefix": "mellanox.co= m",
>>           &= nbsp;     "isRdma": true,
>>           &= nbsp;     "selectors": {
>>           &= nbsp;         "vendors": = ["15b3"],
>>           &= nbsp;         "devices": = ["1014"],
>>           &= nbsp;         "drivers": = ["mlx5_core"],
>>           &= nbsp;         "pciAddresses&qu= ot;: ["0000:d8:00.2", "0000:d8:00.3",
>> "0000:d8:00.4",
>"0000:d8:00.5"],
>>           &= nbsp;         "isRdma": t= rue
>>           &= nbsp;     }
>>
>> The sriov device plugin starts without problems, the devices are <= br> >> correctly
>allocated:
>>
>> {
>>   "cpu": "92",
>>   "ephemeral-storage": "419533922385"= ;,
>>   "hugepages-1Gi": "8Gi",
>>   "hugepages-2Mi": "4Gi",
>>   "intel.com/intel_sriov_dpdk": "0",=
>>   "intel.com/sriov_cre": "3",
>>   "mellanox.com/mlnx_sriov_netdevice": "4= ",
>>   "mellanox.com/sriov_dp": "0",
>>   "memory": "183870336Ki",
>>   "pods": "110"
>> }
>>
>> The Mellanox NICs are binded to the kernel driver mlx5_core:
>>
>> pcgwpod009-c04:~ # dpdk-devbind --status
>>
>> Network devices using kernel driver
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>> 0000:18:00.0 'Ethernet Controller 10G X550T 1563' if=3Dem1 drv=3Di= xgbe
>> unused=3Dvfio-pci
>> 0000:18:00.1 'Ethernet Controller 10G X550T 1563' if=3Dem2 drv=3Di= xgbe
>> unused=3Dvfio-pci
>> 0000:19:00.0 'Ethernet Controller 10G X550T 1563' if=3Dem3 drv=3Di= xgbe
>> unused=3Dvfio-pci
>> 0000:19:00.1 'Ethernet Controller 10G X550T 1563' if=3Dem4 drv=3Di= xgbe
>> unused=3Dvfio-pci
>> 0000:3b:00.0 'MT27700 Family [ConnectX-4] 1013' if=3Denp59s0f0 >> drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:3b:00.1 'MT27700 Family [ConnectX-4] 1013' if=3Denp59s0f1 >> drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'=
>> if=3Dp3p1 drv=3Dixgbe unused=3Dvfio-pci
>> 0000:5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'=
>> if=3Dp3p2 drv=3Dixgbe unused=3Dvfio-pci
>> 0000:5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' if= =3D
>> drv=3Dixgbevf unused=3Dvfio-pci
>> 0000:5e:10.2 '82599 Ethernet Controller Virtual Function 10ed'
>> if=3Dp3p1_1 drv=3Dixgbevf unused=3Dvfio-pci
>> 0000:5e:10.4 '82599 Ethernet Controller Virtual Function 10ed' if= =3D
>> drv=3Dixgbevf unused=3Dvfio-pci
>> 0000:5e:10.6 '82599 Ethernet Controller Virtual Function 10ed'
>> if=3Dp3p1_3 drv=3Dixgbevf unused=3Dvfio-pci
>> 0000:af:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'=
>> if=3Dp4p1 drv=3Dixgbe unused=3Dvfio-pci
>> 0000:af:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb'=
>> if=3Dp4p2 drv=3Dixgbe unused=3Dvfio-pci
>> 0000:d8:00.0 'MT27700 Family [ConnectX-4] 1013' if=3Denp216s0f0 >> drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:d8:00.1 'MT27700 Family [ConnectX-4] 1013' if=3Denp216s0f1 >> drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:d8:00.2 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f2 drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:d8:00.3 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f3 drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:d8:00.4 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f4 drv=3Dmlx5_core unused=3Dvfio-pci
>> 0000:d8:00.5 'MT27700 Family [ConnectX-4 Virtual Function] 1014' >> if=3Denp216s0f5 drv=3Dmlx5_core unused=3Dvfio-pci
>>
>> The interfaces are up:
>>
>> pcgwpod009-c04:~ # ibdev2netdev -v
>> 0000:3b:00.0 mlx5_0 (MT4115 - MT1646K01301) CX416A - ConnectX-4 >QSFP28
>> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp59s0f0 (Up)
>> 0000:3b:00.1 mlx5_1 (MT4115 - MT1646K01301) CX416A - ConnectX-4 >QSFP28
>> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp59s0f1 (Up)
>> 0000:d8:00.0 mlx5_2 (MT4115 - MT1646K00538) CX416A - ConnectX-4 >QSFP28
>> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp216s0f0 (Up)
>> 0000:d8:00.1 mlx5_3 (MT4115 - MT1646K00538) CX416A - ConnectX-4 >QSFP28
>> fw 12.28.2006 port 1 (ACTIVE) =3D=3D> enp216s0f1 (Up)
>> 0000:d8:00.2 mlx5_4 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTI= VE) =3D=3D>
>> enp216s0f2 (Up)
>> 0000:d8:00.3 mlx5_5 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTI= VE) =3D=3D>
>> enp216s0f3 (Up)
>> 0000:d8:00.4 mlx5_6 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTI= VE) =3D=3D>
>> enp216s0f4 (Up)
>> 0000:d8:00.5 mlx5_7 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTI= VE) =3D=3D>
>> enp216s0f5 (Up) pcgwpod009-c04:~ #
>>
>>
>> But when I run my application the Mellanox adapters are probed and= I
>obtain the following error:
>>
>> {"proc_id":"6"},"message":"[pio= ] EAL: Probe PCI driver: mlx5_pci
>> (15b3:1014) device: 0000:d8:00.4 (socket 1)"}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.826+00:00","sever
>> i
>> ty":"info","service_id":"eric-pc-up-= data-plane","metadata":{"proc_id":
>> "6"},"message":"[pio] net_mlx5: unable to= recognize
>> master/representors on the multiple IB devices"}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.826+00:00","sever
>> i
>> ty":"info","service_id":"eric-pc-up-= data-plane","metadata":{"proc_id":
>> "6"},"message":"[pio] common_mlx5: Failed= to load driver =3D
>> net_mlx5."}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.826+00:00","sever
>> i
>> ty":"info","service_id":"eric-pc-up-= data-plane","metadata":{"proc_id":
>> "6"},"message":"[pio] EAL: Requested devi= ce 0000:d8:00.4 cannot be
>> used"}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.826+00:00","sever
>> i
>> ty":"info","service_id":"eric-pc-up-= data-plane","metadata":{"proc_id":
>> "6"},"message":"[pio] EAL: Bus (pci) prob= e failed."}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.890+00:00","sever
>> i
>> ty":"info","service_id":"eric-pc-up-= data-plane","metadata":{"proc_id":
>> "6"},"message":"[pio] USER1: ports init f= ail in DPDK, expect 1 ports,
>> actual 0 ports."}
>> {"version":"0.2.0","timestamp":"= ;2022-01-14T09:51:39.890+00:00","sever
>> i
>> ty":"error","service_id":"eric-pc-up= -data-plane","metadata":{"proc_id"
>> :"6"},"message":"[pktio_libpio_init] No n= etwork ports could be
>> enabled!"}
>>
>> Could you please help me with this issue?
>>
>>
>> Thanks,
>>
>> Roc=EDo
>>
>
>
>
>
--_000_DM5PR1201MB25556CD33E69885710EBC247CD5A9DM5PR1201MB2555_--