From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 6ED80A05D3 for ; Tue, 26 Mar 2019 13:20:14 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4367537A2; Tue, 26 Mar 2019 13:20:14 +0100 (CET) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130083.outbound.protection.outlook.com [40.107.13.83]) by dpdk.org (Postfix) with ESMTP id 0BBF22BD3 for ; Tue, 26 Mar 2019 13:20:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lmoNIJ3oGYCFq5cmAsqMMN6XUefUxNu6wQjt8WuFMoQ=; b=Kd30dZ3y98FZVtKcTb9gKKXsBSyHCtaLNpugjUXv3F7o7dPyANE7TdDzGoXk0m/8pESONCgzuAI0sxp8gwPvuA1BMpG4PDd1sjpWmeuw/0+ZTcbNwCYzR/I3B//u0eoNB9ooBHSGtPPBpkBae5B4l/pF8c6EFFo1nm3aJB1MuVE= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3633.eurprd05.prod.outlook.com (52.133.46.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.15; Tue, 26 Mar 2019 12:20:10 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Tue, 26 Mar 2019 12:20:10 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH v2 01/13] net/mlx5: add representor recognition on kernels 5.x Thread-Index: AQHU4yzKobarb1smq0Gm7oPy8JAb0qYd1dAQ Date: Tue, 26 Mar 2019 12:20:10 +0000 Message-ID: References: <1553155888-27498-1-git-send-email-viacheslavo@mellanox.com> <1553533414-9911-1-git-send-email-viacheslavo@mellanox.com> <1553533414-9911-2-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1553533414-9911-2-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9278ca1f-b1a7-4644-1f22-08d6b1e56463 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600127)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3633; x-ms-traffictypediagnostic: AM0PR0502MB3633: x-microsoft-antispam-prvs: x-forefront-prvs: 09888BC01D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(396003)(366004)(346002)(376002)(39860400002)(199004)(189003)(26005)(186003)(102836004)(7696005)(6436002)(33656002)(14444005)(5024004)(68736007)(256004)(74316002)(105586002)(66066001)(14454004)(6506007)(76176011)(106356001)(6246003)(305945005)(9686003)(7736002)(55016002)(53936002)(476003)(316002)(110136005)(3846002)(478600001)(2501003)(6116002)(25786009)(99286004)(52536014)(86362001)(229853002)(81156014)(11346002)(446003)(8676002)(71190400001)(81166006)(71200400001)(97736004)(486006)(5660300002)(2906002)(8936002)(309714004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3633; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: AXM4aDLvhohSzJc+LLzK/UCVowDlt9/uwD78cwbNoxPWBjGOPymqxlh/gKzHvaY/kSesTs1GF9u+Ba63lSQ0pGVPgMHir33f1IdDqVyp3BZjZUAhwHN7sYwe15PLNg1sgxUujrl4wYcTO5J8bBSlG0Nlifg1GcfkzbikqOj2AKIhGBcxi6jL1OvxCM4w/JK1bQK4IMoXye0LElPGMTUvOXNYS4NEnDezP3qQgNN5ywenRrEXg/SQqXMe9rr+sOECugdZEfXqPivUjJsS0ixPudtD06zGnAD4JEqVmsDwm7aKi4l+d4QnrOQngMQs3YH8dHrrMMqrxabFF8FcD+rwkHATEic1YGpqkUDFqBtAZkCjb7S+DJYwU06QXONW+yU4R42jFLBeHWJKU0HJ4B+ihrcQpBxRl2Hlp0SYZs9x+R0= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9278ca1f-b1a7-4644-1f22-08d6b1e56463 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Mar 2019 12:20:10.7757 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3633 Subject: Re: [dpdk-dev] [PATCH v2 01/13] net/mlx5: add representor recognition on kernels 5.x X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190326122010.PCLu60UvZAdgx6XnbmlM0zQs8UZ0fPvFXgg9DiMU8Wg@z> Monday, March 25, 2019 7:03 PM, Viacheslav Ovsiienko: > Subject: [PATCH v2 01/13] net/mlx5: add representor recognition on kernel= s > 5.x >=20 > The master device and VF representors were distinguished by presence of > port name, master device did not have one. The new Linux kernels starting > from 5.0 provide the port name for master device and the implemented > representor recognizing method does not work. > The new recognizing method is based on quiering the VF number, created on > the base of the device. >=20 > The IFLA_NUM_VF attribute is returned by kernel if IFLA_EXT_MASK > attribute is specified in the Netlink request message. >=20 > Also the presence of device symlink in device sysfs folder is added to > distinguish representors with sysfs based method. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/Makefile | 10 ++++++++++ > drivers/net/mlx5/meson.build | 4 ++++ > drivers/net/mlx5/mlx5.c | 17 ++++++++++++++--- > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_ethdev.c | 14 ++++++++++++-- > drivers/net/mlx5/mlx5_nl.c | 37 > ++++++++++++++++++++++++++++++++++--- > 6 files changed, 75 insertions(+), 8 deletions(-) >=20 > diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index > 1ed299d..3dd7e38 100644 > --- a/drivers/net/mlx5/Makefile > +++ b/drivers/net/mlx5/Makefile > @@ -231,6 +231,16 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto- > config-h.sh > enum RDMA_NLDEV_ATTR_NDEV_INDEX \ > $(AUTOCONF_OUTPUT) > $Q sh -- '$<' '$@' \ > + HAVE_IFLA_NUM_VF \ > + linux/if_link.h \ > + enum IFLA_NUM_VF \ > + $(AUTOCONF_OUTPUT) > + $Q sh -- '$<' '$@' \ > + HAVE_IFLA_EXT_MASK \ > + linux/if_link.h \ > + enum IFLA_EXT_MASK \ > + $(AUTOCONF_OUTPUT) > + $Q sh -- '$<' '$@' \ > HAVE_IFLA_PHYS_SWITCH_ID \ > linux/if_link.h \ > enum IFLA_PHYS_SWITCH_ID \ > diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build > index 0cf2f08..e3cb9bc 100644 > --- a/drivers/net/mlx5/meson.build > +++ b/drivers/net/mlx5/meson.build > @@ -133,6 +133,10 @@ if build > 'ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT' ], > [ 'HAVE_ETHTOOL_LINK_MODE_100G', 'linux/ethtool.h', > 'ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT' ], > + [ 'HAVE_IFLA_NUM_VF', 'linux/if_link.h', > + 'IFLA_NUM_VF' ], > + [ 'HAVE_IFLA_EXT_MASK', 'linux/if_link.h', > + 'IFLA_EXT_MASK' ], > [ 'HAVE_IFLA_PHYS_SWITCH_ID', 'linux/if_link.h', > 'IFLA_PHYS_SWITCH_ID' ], > [ 'HAVE_IFLA_PHYS_PORT_NAME', 'linux/if_link.h', diff --git > a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index ad1975c..032f1f= 2 > 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -13,7 +13,6 @@ > #include > #include > #include > -#include > #include >=20 > /* Verbs header. */ > @@ -1001,9 +1000,21 @@ > priv->nl_socket_route =3D mlx5_nl_init(NETLINK_ROUTE); > priv->nl_sn =3D 0; > priv->representor =3D !!switch_info->representor; > + priv->master =3D !!switch_info->master; > priv->domain_id =3D RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID; > - priv->representor_id =3D > - switch_info->representor ? switch_info->port_name : -1; > + /* > + * Currently we support sinlge E-Switch per PF configurations > + * only and representors_id field contains the vport index for > + * corresponding VF, deduced from representor port name. > + * For exapmple, let's have the IB device port 10, it has > + * attached network device eth0, which has port name attribute > + * pf0vf2, we can deduce the VF number as 2, and set vport index > + * as 3 (2+1). This assigning schema should be changed if the > + * multiple E-Switch instance per PF configurations or/and PCI > + * subfunctions dded. > + */ > + priv->representor_id =3D switch_info->representor ? > + switch_info->port_name + 1 : -1; The representor_id field is also being used on the info query from user. We= shouldn't report back the vport id of the representor rather its index acc= ording to phys_port_name. You can define a different field, maybe called vport.=20 > /* > * Look for sibling devices in order to reuse their switch domain > * if any, otherwise allocate one. > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > a88cb4a..58bc37f 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -214,6 +214,7 @@ struct mlx5_priv { > uint16_t mtu; /* Configured MTU. */ > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > unsigned int representor:1; /* Device is a port representor. */ > + unsigned int master:1; /* Device is a E-Switch master. */ > uint16_t domain_id; /* Switch domain identifier. */ > int32_t representor_id; /* Port representor identifier. */ > /* RX/TX queues. */ > diff --git a/drivers/net/mlx5/mlx5_ethdev.c > b/drivers/net/mlx5/mlx5_ethdev.c index 2e8a296..64c6b6f 100644 > --- a/drivers/net/mlx5/mlx5_ethdev.c > +++ b/drivers/net/mlx5/mlx5_ethdev.c > @@ -1362,8 +1362,10 @@ int mlx5_fw_version_get(struct rte_eth_dev > *dev, char *fw_ver, size_t fw_size) > .port_name =3D 0, > .switch_id =3D 0, > }; > + DIR *dir; > bool port_name_set =3D false; > bool port_switch_id_set =3D false; > + bool device_dir =3D false; > char c; > int ret; >=20 > @@ -1376,6 +1378,8 @@ int mlx5_fw_version_get(struct rte_eth_dev *dev, > char *fw_ver, size_t fw_size) > ifname); > MKSTR(phys_switch_id, "/sys/class/net/%s/phys_switch_id", > ifname); > + MKSTR(pci_device, "/sys/class/net/%s/device", > + ifname); >=20 > file =3D fopen(phys_port_name, "rb"); > if (file !=3D NULL) { > @@ -1394,8 +1398,14 @@ int mlx5_fw_version_get(struct rte_eth_dev > *dev, char *fw_ver, size_t fw_size) > fscanf(file, "%" SCNx64 "%c", &data.switch_id, &c) =3D=3D 2 && > c =3D=3D '\n'; > fclose(file); > - data.master =3D port_switch_id_set && !port_name_set; > - data.representor =3D port_switch_id_set && port_name_set; > + dir =3D opendir(pci_device); > + if (dir !=3D NULL) { > + closedir(dir); > + device_dir =3D true; > + } > + data.master =3D port_switch_id_set && (!port_name_set || > device_dir); > + data.representor =3D port_switch_id_set && port_name_set && > !device_dir; > + assert(!(data.master && data.representor)); > *info =3D data; > return 0; > } > diff --git a/drivers/net/mlx5/mlx5_nl.c b/drivers/net/mlx5/mlx5_nl.c inde= x > 8a10109..9d0fcba 100644 > --- a/drivers/net/mlx5/mlx5_nl.c > +++ b/drivers/net/mlx5/mlx5_nl.c > @@ -65,6 +65,12 @@ > #endif >=20 > /* These are normally found in linux/if_link.h. */ > +#ifndef HAVE_IFLA_NUM_VF > +#define IFLA_NUM_VF 21 > +#endif > +#ifndef HAVE_IFLA_EXT_MASK > +#define IFLA_EXT_MASK 29 > +#endif > #ifndef HAVE_IFLA_PHYS_SWITCH_ID > #define IFLA_PHYS_SWITCH_ID 36 > #endif > @@ -837,6 +843,7 @@ struct mlx5_nl_ifindex_data { > size_t off =3D NLMSG_LENGTH(sizeof(struct ifinfomsg)); > bool port_name_set =3D false; > bool switch_id_set =3D false; > + bool num_vf_set =3D false; >=20 > if (nh->nlmsg_type !=3D RTM_NEWLINK) > goto error; > @@ -848,6 +855,9 @@ struct mlx5_nl_ifindex_data { > if (ra->rta_len > nh->nlmsg_len - off) > goto error; > switch (ra->rta_type) { > + case IFLA_NUM_VF: > + num_vf_set =3D true; > + break; > case IFLA_PHYS_PORT_NAME: > port_name_set =3D > mlx5_translate_port_name((char *)payload, > @@ -864,8 +874,20 @@ struct mlx5_nl_ifindex_data { > } > off +=3D RTA_ALIGN(ra->rta_len); > } > - info.master =3D switch_id_set && !port_name_set; > - info.representor =3D switch_id_set && port_name_set; > + if (switch_id_set) { > + if (info.port_name_new) { > + /* New representors naming schema. */ > + if (port_name_set) { > + info.master =3D (info.port_name =3D=3D -1); > + info.representor =3D (info.port_name !=3D -1); > + } > + } else { > + /* Legacy representors naming schema. */ > + info.master =3D (!port_name_set || num_vf_set); > + info.representor =3D port_name_set && !num_vf_set; > + } > + } > + assert(!(data.master && data.representor)); > memcpy(arg, &info, sizeof(info)); > return 0; > error: > @@ -893,9 +915,13 @@ struct mlx5_nl_ifindex_data { > struct { > struct nlmsghdr nh; > struct ifinfomsg info; > + struct rtattr rta; > + uint32_t extmask; > } req =3D { > .nh =3D { > - .nlmsg_len =3D NLMSG_LENGTH(sizeof(req.info)), > + .nlmsg_len =3D NLMSG_LENGTH > + (sizeof(req.info) + > + RTA_LENGTH(sizeof(uint32_t))), > .nlmsg_type =3D RTM_GETLINK, > .nlmsg_flags =3D NLM_F_REQUEST | NLM_F_ACK, > }, > @@ -903,6 +929,11 @@ struct mlx5_nl_ifindex_data { > .ifi_family =3D AF_UNSPEC, > .ifi_index =3D ifindex, > }, > + .rta =3D { > + .rta_type =3D IFLA_EXT_MASK, > + .rta_len =3D RTA_LENGTH(sizeof(int32_t)), > + }, > + .extmask =3D RTE_LE32(1), > }; > int ret; >=20 > -- > 1.8.3.1