From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80079.outbound.protection.outlook.com [40.107.8.79]) by dpdk.org (Postfix) with ESMTP id D033D5F1A for ; Thu, 28 Jun 2018 08:01:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XTO+1VYKvfr/PeNs3YqxBFh0nRBuGtfvVFbqmqGsAcQ=; b=bmD+fwo+EEZFbVWgW1AKvFLxOMdwWwObSUmUqmZqOX4EM7Rf7SLuDWXxGh7FrEZF1Rp1Z0rQr1zLXMegkUfwCzKxjZAIkn7ayvfiNwC7cVwZSMiO2RBe0JSOGC2fNc5SgqKlWSDutoQAXzxucQYbgBetHBKbuFI+k8BFFXD7nTs= Received: from DB7PR05MB4426.eurprd05.prod.outlook.com (52.134.109.15) by DB7PR05MB4444.eurprd05.prod.outlook.com (52.134.109.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.884.23; Thu, 28 Jun 2018 06:01:54 +0000 Received: from DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::4c94:2db9:b8c5:cc6b]) by DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::4c94:2db9:b8c5:cc6b%5]) with mapi id 15.20.0884.025; Thu, 28 Jun 2018 06:01:54 +0000 From: Shahaf Shuler To: Adrien Mazarguil CC: "Xueming(Steven) Li" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2 6/7] net/mlx5: probe all port representors Thread-Index: AQHUA7qglxyeqjm7kUStlpk5b3NHZ6RimMCAgAGgu8CAD/WlAIABE3Lw Date: Thu, 28 Jun 2018 06:01:54 +0000 Message-ID: References: <20180525161814.13873-1-adrien.mazarguil@6wind.com> <20180614083047.10812-1-adrien.mazarguil@6wind.com> <20180614083047.10812-7-adrien.mazarguil@6wind.com> <20180627133228.GV4025@6wind.com> In-Reply-To: <20180627133228.GV4025@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR05MB4444; 7:Sl0MDWJdkdq6web0f8SD3XK0v8h+XxB5vYrqV6y/yaiZ0+t4CqfAHE+uLRl8ChTyiBso9+5CXQCMzbTb+/B2d1b0aHusz9yklky0cCCvzqm8l++yJrZe22ZDUraM1NI3JRhBiYbJiMgQh+8QK2U4t54KAWIsGfbgFGr0tN8XKxqNUOeDiAxlh0P4uNpB8+XR0rJt0LIG6YpBE2QHnRFtes3qVkQ3HYuob9nwh3pgC0RyUBWt+I061c28bCu1krq8 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 192ffb9a-44fe-4410-10c8-08d5dcbca66b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:DB7PR05MB4444; x-ms-traffictypediagnostic: DB7PR05MB4444: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(244540007438412)(278428928389397)(189930954265078)(45079756050767); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(3231254)(944501410)(52105095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:DB7PR05MB4444; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4444; x-forefront-prvs: 0717E25089 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(396003)(136003)(39860400002)(376002)(366004)(189003)(199004)(13464003)(55674003)(11346002)(81166006)(476003)(53546011)(186003)(486006)(54906003)(6506007)(478600001)(6306002)(102836004)(966005)(86362001)(316002)(6916009)(81156014)(4326008)(68736007)(305945005)(8676002)(256004)(446003)(9686003)(99286004)(93886005)(45080400002)(25786009)(66066001)(33656002)(7736002)(2900100001)(55016002)(74316002)(26005)(53936002)(2906002)(76176011)(6246003)(7696005)(106356001)(3846002)(97736004)(14454004)(6436002)(105586002)(5660300001)(6116002)(5250100002)(5024004)(14444005)(8936002)(229853002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4444; H:DB7PR05MB4426.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: OehwagOzK6XLHuS9R6izX4iRAhHD4NsayxL0I1fOallLOaaHEw/hRSEcXf7N/wvZyi/CgfHqfnDzop6+HeXXz3IpVt86+F6rHQZxwr2p11EJQjst8RjtFXbbMbNF5UrZrMXxp79zXVkuwEA1sG8oYzirRsbx2wGgsfmnmBr+w/f+T8ZfMZzqtKrwYCdvZSovnVmyMq1voyJnQf9cNhqQzWFVeQaZT6Du6sh2BsQNbFnj5PN0ltxAdk6+RSpe2S/SAZ5hP4gC03gL73O0bdLux1VeShgg17Akhfi/KPfJbQ7UnwOXemez7YnhNRKS7gb3134IVb/4juVgzPQFlKqgbAILI6I5fsYFedkwfHtY63w= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 192ffb9a-44fe-4410-10c8-08d5dcbca66b X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2018 06:01:54.5239 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4444 Subject: Re: [dpdk-dev] [PATCH v2 6/7] net/mlx5: probe all port representors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jun 2018 06:01:56 -0000 Wednesday, June 27, 2018 4:32 PM, Adrien Mazarguil: > Subject: Re: [dpdk-dev] [PATCH v2 6/7] net/mlx5: probe all port represent= ors >=20 > On Sun, Jun 17, 2018 at 10:15:07AM +0000, Shahaf Shuler wrote: > > Hi Adrien, > > > > Saturday, June 16, 2018 11:58 AM, Xueming(Steven) Li: > > > Subject: RE: [dpdk-dev] [PATCH v2 6/7] net/mlx5: probe all port > > > representors > > > > > > > -----Original Message----- > > > > From: dev On Behalf Of Adrien Mazarguil > > > > Sent: Thursday, June 14, 2018 4:35 PM > > > > To: Shahaf Shuler > > > > Cc: dev@dpdk.org > > > > Subject: [dpdk-dev] [PATCH v2 6/7] net/mlx5: probe all port > > > > representors > > > > > > > > Probe existing port representors in addition to their master > > > > device and > > > associate them automatically. > > > > > > > > To avoid name collision between Ethernet devices, their names use > > > > the same convention as ixgbe and i40e PMDs, that is, instead of > > > > only a PCI > > > address in DBDF notation: > > > > > > > > - "net_{DBDF}_0" for master/switch devices. > > > > This is breaking compatibility for application using the device names i= n > order to attach them to the application (e.g. OVS-DPDK). > > Before this patch the naming scheme for non-representor port is "{DBDF}= ". > > > > Can we preserve the compatibility and add appropriate suffix for the > representor case? >=20 > There's one issue if representors are hot-plugged. The name of the master > device, which happens to be that of the switch domain, cannot be updated. > The form "net_{DBDF}_0" seems expected for PMDs that support > representors (see ixgbe and i40e). >=20 > Now since representor hot-plugging is not supported yet, I guess we could > postpone this problem by keeping the old format in the meantime, however > ideally, these applications should not rely on it. The only safe assumpti= on > they can make is the uniqueness of any given name among ethdevs. >=20 > PCI bus addresses, if needed, should be retrieved by looking at the > underlying bus object. Am not sure I understand. Those application attach the device to the applic= ation based on its name, which happens to be the PCI address in case of mlx= 5.=20 >=20 > By the way, while thinking again about a past comment from Xueming [1], > maybe it's finally time to remove support for multiple Verbs ports on mlx= 5 > after all. This should drop another unnecessary loop and the need for the > unused "port %u" suffix at all while naming the device. >=20 > So how about the following plan for v3: >=20 > - Adding a patch that drops support for multiple Verbs ports (note for > Xueming, yes I changed my mind *again* :) I am OK w/ that.=20 >=20 > - If you really think this will break OVS (please confirm), It will.=20 then when no > "representor" parameter is provided (regardless of the presence of any > representors), name format will use the usual "{DBDF}" notation as you > suggested. >=20 > - Otherwise as soon as a "representor" is found on the command line, the > new > format will be used, again regardless of the presence of any represento= rs. >=20 > - In both cases, representors if any, will be named according to the form= at > specified in this patch. Can we do the following? In case representor is found the naming will be DBDF_representor_%d In case no-representor naming will be DBDF Just removing the net prefix. =20 >=20 > [1] > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fma > ils.dpdk.org%2Farchives%2Fdev%2F2018- > June%2F104015.html&data=3D02%7C01%7Cshahafs%40mellanox.com%7C0037 > 6c6df5044ac9f8f908d5dc32778f%7Ca652971c7d2e4d9ba6a4d149256f461b%7C > 0%7C0%7C636657031665047796&sdata=3DXXYtvW3J3i3Xzkn%2B8YBKYK1b2D6P > 5eUiD2h4VqLUJD8%3D&reserved=3D0 >=20 > > > > > for (i =3D 0; i < attr.orig_attr.phys_port_cnt; ++i) { > > > > - eth_list[i] =3D mlx5_dev_spawn_one(dpdk_dev, ibv_dev, vf, > > > > - &attr, i + 1); > > > > - if (eth_list[i]) > > > > - continue; > > > > - /* Save rte_errno and roll back in case of failure. */ > > > > - ret =3D rte_errno; > > > > - while (i--) { > > > > - mlx5_dev_close(eth_list[i]); > > > > - if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY) > > > > - rte_free(eth_list[i]->data->dev_private); > > > > - claim_zero(rte_eth_dev_release_port(eth_list[i])); > > > > - } > > > > - free(eth_list); > > > > - rte_errno =3D ret; > > > > - return NULL; > > > > + eth_list[n] =3D mlx5_dev_spawn_one(dpdk_dev, ibv_dev[j], > > > vf, > > > > + &attr, i + 1, > > > > + j ? eth_list[0] : NULL, > > > > + j - 1); > > > > The representor id is according to the sort made by qsort (based on dev= ice > names). > > A better way may be to set it according to the sysfs information, like = you do > in the mlx5_get_ifname function. > > What do you think? >=20 > I agree that the current approach sucks, hence the big fat warnings I lef= t > around (see discussion with Xueming [2]). Problem is that the needed > information is not yet known at this stage; there is no private structure= to > rely on to use mlx5_get_ifname() directly. >=20 > I'd also rather see these assumptions go in any case. I'll attempt to imp= rove > things for v3 in preparation of allowing representors to be probed on the= ir > own anytime, possibly even before the master device. >=20 > [2] > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fma > ils.dpdk.org%2Farchives%2Fdev%2F2018- > June%2F104059.html&data=3D02%7C01%7Cshahafs%40mellanox.com%7C0037 > 6c6df5044ac9f8f908d5dc32778f%7Ca652971c7d2e4d9ba6a4d149256f461b%7C > 0%7C0%7C636657031665047796&sdata=3DjWLFP6GMdQ6C88r1v%2BYZx7iKH3k > ZDhVpgP4am9F11PU%3D&reserved=3D0 >=20 > > > > > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h > > > > index > > > > 997b04a33..0fe467140 100644 > > > > --- a/drivers/net/mlx5/mlx5.h > > > > +++ b/drivers/net/mlx5/mlx5.h > > > > @@ -161,6 +161,10 @@ struct priv { > > > > uint16_t mtu; /* Configured MTU. */ > > > > uint8_t port; /* Physical port number. */ > > > > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > > > > + unsigned int representor:1; /* Device is a port representor. */ > > > > Why we need above flag? Why can't we use RTE_ETH_DEV_REPRESENTOR > from eth_dev->data->dev_flags. >=20 > Problem is that this flag can only be set once the ethdev is fully instan= tiated > and can't be relied on internally where needed (e.g. during clean up in e= rror > handling code). It's reported to applications but not used internally. >=20 > As a device property, it's actually pretty similar to the VF bit or offlo= aded > capabilities where checking exposed information would be needlessly > complex. >=20 > Now maybe it could be part of struct mlx5_dev_config as well. I initially > assumed this object was only for user-provided parameters but looks like = it's > not the case. I intend to move it there for v3. >=20 > -- > Adrien Mazarguil > 6WIND