From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74AF1A052E; Mon, 3 Feb 2020 13:44:21 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF3961BFB6; Mon, 3 Feb 2020 13:44:20 +0100 (CET) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2076.outbound.protection.outlook.com [40.107.20.76]) by dpdk.org (Postfix) with ESMTP id 3CDCE1BF99 for ; Mon, 3 Feb 2020 13:44:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aLK8m1ln7/FpqClWMoPR9S5NClHvVRN0eOSeN0120oqKGEXhpe4GJJWFb/2TdcygYhYJksOyOLFSCbi/3aOWIS/1Ok7J3h1CALs2XJlBlXutCm4GGTJKyH+EzxYRk3rHUcCQWCPBxKTgnfmrjjhnVYxYclYbxIiIQvneKDrdaVP3Ef4MZftKq0AJXoLFzeSfstdH4df3hqnkqj3rBF8ML94hQMKz7k6a6x9ABw2B6LtLd2TO9rNwUVY5zuNe58pIW9Fu0dM2T3yXGstbbfwAEPP8tU9W+Ikj8bppiH5mJCvc6FKJf/8YRfs0Rv/bEKu/vN20m6EJKs5CZhkziumuSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4u5Qv7U81MJp5kOY/blC8tviFEhMpOMUerQD9DXozNU=; b=QbSVYdEwCbYiIQhh2sBwlvxNCyAWTjlZZOOeMvZt+vEHg4CKHNybF1AvHgBaZi/WllUHFOpK0MpfTX0UVTt/DoQLH9WeMb7i65F63IXzAOf+H04MSAQZrGyzaKn4enHuAr3zPYV/cU0hGB3Vv0Hf96VrXy1QrmtewcRLRlBsw+MDvnXdOTSqf5VFhmDteTtYXw+AGzmf1sHggh2iTtxlSe6meaUXNXXr/341MHCQ22cysof7To62TgRhiTEpONqmBF/4Uon03xIBYuSD+uXTLpHuTZGkJME77RwOSNxwEewHHW4BVlOTZRK3kPoVdokG+yJVb2ymR6uOsaJsuFvsIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4u5Qv7U81MJp5kOY/blC8tviFEhMpOMUerQD9DXozNU=; b=LK3tQ5l9ayMCaySfg5+Ej9EWq51gysaKGg7nW5nCJEZRlPHgJkRXEnGjCkQPAwe+mDsi2BSEw88tiJ7AYyzAJmjEwr3CxJHvUqbQaNo+ICrUr+kTmhUX1Pe4cMbiq6QB2Haxfy+dk2IReC7rnoSJf8JTSOdH6+JXOUhOoJFkqts= Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com (52.133.39.139) by AM0PR0502MB3857.eurprd05.prod.outlook.com (52.133.49.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2686.29; Mon, 3 Feb 2020 12:44:19 +0000 Received: from AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::e495:9258:db09:1de8]) by AM0PR0502MB4019.eurprd05.prod.outlook.com ([fe80::e495:9258:db09:1de8%7]) with mapi id 15.20.2686.031; Mon, 3 Feb 2020 12:44:19 +0000 From: Matan Azrad To: Maxime Coquelin , "dev@dpdk.org" , Slava Ovsiienko Thread-Topic: [PATCH v3 13/13] vdpa/mlx5: disable ROCE Thread-Index: AQHV2nQ6aO83LPmT2UCZ2gGEYLCbj6gJTVaAgAAcjlA= Date: Mon, 3 Feb 2020 12:44:19 +0000 Message-ID: References: <1580292549-27439-1-git-send-email-matan@mellanox.com> <1580659433-25581-1-git-send-email-matan@mellanox.com> <1580659433-25581-14-git-send-email-matan@mellanox.com> <2843355f-407c-1c25-0342-1f26be6eb585@redhat.com> In-Reply-To: Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 4d43315e-98cd-4a46-1a4d-08d7a8a6c967 x-ms-traffictypediagnostic: AM0PR0502MB3857:|AM0PR0502MB3857: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3383; x-forefront-prvs: 0302D4F392 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(136003)(346002)(366004)(396003)(39860400002)(189003)(199004)(81156014)(76116006)(316002)(5660300002)(52536014)(66476007)(81166006)(64756008)(66556008)(66946007)(66446008)(478600001)(33656002)(55016002)(9686003)(86362001)(186003)(26005)(53546011)(8676002)(71200400001)(6506007)(110136005)(6636002)(8936002)(2906002)(7696005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3857; H:AM0PR0502MB4019.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /RicU294OJlIMPpg9Uru1e5QSEN8iv+BTs8dzD2r0xX7qU1sUrAyfWmw5eAx0mD54sx1izyNk06ZZjWSToj34/xHJKFCPmU67+VEZmuveCO3448eNSce9yf4Mtp3xTQOe38iBzKQ0g95+2ro2rmnsWX5lxBYj/ZIyP0ekIxM2TUe8PNOWmWjBNozTB9oAcJ1oTUSRiRZB9oNpalXL9gI/eKE2YCN8ISTUDFD41uDaU0Cl8MKjkJpxuQpjnjZu3voH8uif3mkEm8V37QEufpZ7N4UgmEvZXN66TFGsVUnokwrxvJD6Vuwi2HSPHOp5sKfWrs5uVBlYhfmxuLv6NjZp9Yb5f4/ech4DCDVBqLejIU2qYdcO16wGCegQJ3G0tiny7atY8c3ZjDRzhsbfs7jAe+OHclmOg6eb9KjO9/dqV5jmVVRaAyLPJ2ktI4ZR6M0 x-ms-exchange-antispam-messagedata: ddRvNS15F/RyJXhXRT8vr3VURg+IT7oTEcG6LQOIcX7cRK2IWf4N2B48pZWlQR6OOASFKc6QkWMSRN+oH/md8NUAAHbPe7zKueSNZfWh7KUi4P0P321FlWBJXZcQurdC4OU0zw31rASeOZ0mB/s1Sg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d43315e-98cd-4a46-1a4d-08d7a8a6c967 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Feb 2020 12:44:19.2469 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Ko8pxmmxmUjR5JwsNWiLZbElQIWnuhqN0lkMNjIhpENHjJXIpIVsDtfA8N0/4zYY+gZwLCJD3Kp+8IM7H8PRhw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3857 Subject: Re: [dpdk-dev] [PATCH v3 13/13] vdpa/mlx5: disable ROCE X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Maxime Coquelin > Sent: Monday, February 3, 2020 1:00 PM > To: Matan Azrad ; dev@dpdk.org; Slava Ovsiienko > > Subject: Re: [PATCH v3 13/13] vdpa/mlx5: disable ROCE >=20 >=20 >=20 > On 2/3/20 10:27 AM, Maxime Coquelin wrote: > > Hi Matan, > > > > On 2/2/20 5:03 PM, Matan Azrad wrote: > >> In order to support virtio queue creation by the FW, ROCE mode should > >> be disabled in the device. > >> > >> Do it by netlink which is like the devlink tool commands: > >> 1. devlink dev param set pci/[pci] name enable_roce value false > >> cmode driverinit > >> 2. devlink dev reload pci/[pci] > >> Or by sysfs which is like: > >> echo 0 > /sys/bus/pci/devices/[pci]/roce_enable > >> > >> The IB device is matched again after ROCE disabling. > >> > >> Signed-off-by: Matan Azrad > >> Acked-by: Viacheslav Ovsiienko > >> Acked-by: Maxime Coquelin > >> --- > >> drivers/vdpa/mlx5/Makefile | 2 +- > >> drivers/vdpa/mlx5/meson.build | 2 +- > >> drivers/vdpa/mlx5/mlx5_vdpa.c | 191 > >> ++++++++++++++++++++++++++++++++++-------- > >> 3 files changed, 160 insertions(+), 35 deletions(-) > > ... > >> diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c > >> b/drivers/vdpa/mlx5/mlx5_vdpa.c index 57619d2..710f305 100644 > >> --- a/drivers/vdpa/mlx5/mlx5_vdpa.c > >> +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c > > > > ... > > > >> @@ -246,8 +389,7 @@ > >> mlx5_vdpa_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, > >> struct rte_pci_device *pci_dev __rte_unused) { > >> - struct ibv_device **ibv_list; > >> - struct ibv_device *ibv_match =3D NULL; > >> + struct ibv_device *ibv; > >> struct mlx5_vdpa_priv *priv =3D NULL; > >> struct ibv_context *ctx =3D NULL; > >> struct mlx5_hca_attr attr; > >> @@ -258,42 +400,25 @@ > >> " driver."); > >> return 1; > >> } > >> - errno =3D 0; > >> - ibv_list =3D mlx5_glue->get_device_list(&ret); > >> - if (!ibv_list) { > >> - rte_errno =3D ENOSYS; > >> - DRV_LOG(ERR, "Failed to get device list, is ib_uverbs > loaded?"); > >> + ibv =3D mlx5_vdpa_get_ib_device_match(&pci_dev->addr); > >> + if (!ibv) { > >> + DRV_LOG(ERR, "No matching IB device for PCI slot " > >> + PCI_PRI_FMT ".", pci_dev->addr.domain, > >> + pci_dev->addr.bus, pci_dev->addr.devid, > >> + pci_dev->addr.function); > >> return -rte_errno; > >> - } > >> - while (ret-- > 0) { > >> - struct rte_pci_addr pci_addr; > >> - > >> - DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[ret]- > >name); > >> - if (mlx5_dev_to_pci_addr(ibv_list[ret]->ibdev_path, > &pci_addr)) > >> - continue; > >> - if (pci_dev->addr.domain !=3D pci_addr.domain || > >> - pci_dev->addr.bus !=3D pci_addr.bus || > >> - pci_dev->addr.devid !=3D pci_addr.devid || > >> - pci_dev->addr.function !=3D pci_addr.function) > >> - continue; > >> + } else { > >> DRV_LOG(INFO, "PCI information matches for device > \"%s\".", > >> - ibv_list[ret]->name); > >> - ibv_match =3D ibv_list[ret]; > >> - break; > >> + ibv->name); > >> } > >> - mlx5_glue->free_device_list(ibv_list); > >> - if (!ibv_match) { > >> - DRV_LOG(ERR, "No matching IB device for PCI slot " > >> - "%" SCNx32 ":%" SCNx8 ":%" SCNx8 ".%" SCNx8 ".", > >> - pci_dev->addr.domain, pci_dev->addr.bus, > >> - pci_dev->addr.devid, pci_dev->addr.function); > >> - rte_errno =3D ENOENT; > >> - return -rte_errno; > >> + if (mlx5_vdpa_roce_disable(&pci_dev->addr, &ibv) !=3D 0) { > >> + DRV_LOG(WARNING, "Failed to disable ROCE for \"%s\".", > >> + ibv->name); > >> + //return -rte_errno; > >> } > > > > Is that commented return expected? > > >=20 > Please let me know if I should remove the comment, or remove the return. Sorry, forgot the comment , good catch! It should not be comment just need to remove "//". Can you do it in integration? >=20 > Thanks, > Maxime