From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 85A481B4DF for ; Fri, 5 Apr 2019 15:26:10 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Apr 2019 16:26:05 +0300 Received: from pegasus12.mtr.labs.mlnx. (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x35DQ5CS008993; Fri, 5 Apr 2019 16:26:05 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: shahafs@mellanox.com Date: Fri, 5 Apr 2019 13:25:55 +0000 Message-Id: <1554470755-14183-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1554300882-23990-1-git-send-email-viacheslavo@mellanox.com> References: <1554300882-23990-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 1/1] net/mlx5: fix device probing for old kernel drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 13:26:10 -0000 Retrieving network interface index via Netlink fails in case of old ib_core kernel driver installed - mlx5_nl_ifindex() routine fails due to RDMA_NLDEV_ATTR_NDEV_INDEX attribute is not supported by the old driver. The patch allowing to retrieve the network interface index and name via Netlink [1]. So, the problem depends on ib_core module version - 4.16 supports getting ifindex via Netlink, 4.15 does not. This error was ignored in previous versions of MLX5 PMD probing routine. For single device ifindex was retrieved via sysfs and link control was not lost, so problem just was not noticed. In order to support MLX5 PMD functioning over old kernel driver this patch adds ifindex retrieving via sysfs into probing routine. It is worth to note this method works for master/standalone device only. [1] https://www.spinics.net/lists/linux-rdma/msg62948.html Linux tree: 5b2cc79d (Leon Romanovsky 2018-03-27 20:40:49 +0300 270) Fixes: ad74bc619504 ("net/mlx5: support multiport IB device during probing") Signed-off-by: Viacheslav Ovsiienko --- v2: comments and log message update v1: http://patches.dpdk.org/patch/52192/ drivers/net/mlx5/mlx5.c | 36 ++++++++++++++++++++++++++++++++---- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_ethdev.c | 28 ++++++++++++++-------------- 3 files changed, 47 insertions(+), 18 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 09f4a21..b7e6234 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1888,12 +1888,40 @@ struct mlx5_dev_spawn_data { list[ns].ifindex = mlx5_nl_ifindex (nl_rdma, list[ns].ibv_dev->name, 1); if (!list[ns].ifindex) { + char ifname[IF_NAMESIZE]; + /* - * No network interface index found for the - * specified device, it means there it is not - * a representor/master. + * Netlink failed, it may happen with old + * ib_core kernel driver (before 4.16). + * We can assume there is old driver because + * here we are processing single ports IB + * devices. Let's try sysfs to retrieve + * the ifindex. The method works for + * master device only. */ - continue; + if (nd > 1) { + /* + * Multiple devices found, assume + * representors, can not distinguish + * master/representor and retrieve + * ifindex via sysfs. + */ + continue; + } + ret = mlx5_get_master_ifname + (ibv_match[i]->ibdev_path, &ifname); + if (!ret) + list[ns].ifindex = + if_nametoindex(ifname); + if (!list[ns].ifindex) { + /* + * No network interface index found + * for the specified device, it means + * there it is neither representor + * nor master. + */ + continue; + } } ret = -1; if (nl_route >= 0) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index ef05d9f..c9b2251 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -364,6 +364,7 @@ struct mlx5_priv { /* mlx5_ethdev.c */ int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]); +int mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]); unsigned int mlx5_ifindex(const struct rte_eth_dev *dev); int mlx5_ifreq(const struct rte_eth_dev *dev, int req, struct ifreq *ifr); int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu); diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 9ae9ddd..1e6fe19 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -127,21 +127,18 @@ struct ethtool_link_settings { * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static int -mlx5_get_master_ifname(const struct rte_eth_dev *dev, - char (*ifname)[IF_NAMESIZE]) +int +mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]) { - struct mlx5_priv *priv = dev->data->dev_private; DIR *dir; struct dirent *dent; unsigned int dev_type = 0; unsigned int dev_port_prev = ~0u; char match[IF_NAMESIZE] = ""; - assert(priv); - assert(priv->sh); + assert(ibdev_path); { - MKSTR(path, "%s/device/net", priv->sh->ibdev_path); + MKSTR(path, "%s/device/net", ibdev_path); dir = opendir(path); if (dir == NULL) { @@ -161,7 +158,7 @@ struct ethtool_link_settings { continue; MKSTR(path, "%s/device/net/%s/%s", - priv->sh->ibdev_path, name, + ibdev_path, name, (dev_type ? "dev_id" : "dev_port")); file = fopen(path, "rb"); @@ -222,15 +219,18 @@ struct ethtool_link_settings { mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]) { struct mlx5_priv *priv = dev->data->dev_private; - unsigned int ifindex = - priv->nl_socket_rdma >= 0 ? - mlx5_nl_ifindex(priv->nl_socket_rdma, - priv->sh->ibdev_name, - priv->ibv_port) : 0; + unsigned int ifindex; + assert(priv); + assert(priv->sh); + ifindex = priv->nl_socket_rdma >= 0 ? + mlx5_nl_ifindex(priv->nl_socket_rdma, + priv->sh->ibdev_name, + priv->ibv_port) : 0; if (!ifindex) { if (!priv->representor) - return mlx5_get_master_ifname(dev, ifname); + return mlx5_get_master_ifname(priv->sh->ibdev_path, + ifname); rte_errno = ENXIO; return -rte_errno; } -- 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 223E9A0679 for ; Fri, 5 Apr 2019 15:26:12 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D49C71B4E8; Fri, 5 Apr 2019 15:26:11 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 85A481B4DF for ; Fri, 5 Apr 2019 15:26:10 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from viacheslavo@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Apr 2019 16:26:05 +0300 Received: from pegasus12.mtr.labs.mlnx. (pegasus12.mtr.labs.mlnx [10.210.17.40]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x35DQ5CS008993; Fri, 5 Apr 2019 16:26:05 +0300 From: Viacheslav Ovsiienko To: dev@dpdk.org Cc: shahafs@mellanox.com Date: Fri, 5 Apr 2019 13:25:55 +0000 Message-Id: <1554470755-14183-1-git-send-email-viacheslavo@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1554300882-23990-1-git-send-email-viacheslavo@mellanox.com> References: <1554300882-23990-1-git-send-email-viacheslavo@mellanox.com> Subject: [dpdk-dev] [PATCH v2 1/1] net/mlx5: fix device probing for old kernel drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Content-Type: text/plain; charset="UTF-8" Message-ID: <20190405132555.VEwMiY2_2MvrliaqvVWFerXYfT08ISfGeZe01kS7g6Q@z> Retrieving network interface index via Netlink fails in case of old ib_core kernel driver installed - mlx5_nl_ifindex() routine fails due to RDMA_NLDEV_ATTR_NDEV_INDEX attribute is not supported by the old driver. The patch allowing to retrieve the network interface index and name via Netlink [1]. So, the problem depends on ib_core module version - 4.16 supports getting ifindex via Netlink, 4.15 does not. This error was ignored in previous versions of MLX5 PMD probing routine. For single device ifindex was retrieved via sysfs and link control was not lost, so problem just was not noticed. In order to support MLX5 PMD functioning over old kernel driver this patch adds ifindex retrieving via sysfs into probing routine. It is worth to note this method works for master/standalone device only. [1] https://www.spinics.net/lists/linux-rdma/msg62948.html Linux tree: 5b2cc79d (Leon Romanovsky 2018-03-27 20:40:49 +0300 270) Fixes: ad74bc619504 ("net/mlx5: support multiport IB device during probing") Signed-off-by: Viacheslav Ovsiienko --- v2: comments and log message update v1: http://patches.dpdk.org/patch/52192/ drivers/net/mlx5/mlx5.c | 36 ++++++++++++++++++++++++++++++++---- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_ethdev.c | 28 ++++++++++++++-------------- 3 files changed, 47 insertions(+), 18 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 09f4a21..b7e6234 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1888,12 +1888,40 @@ struct mlx5_dev_spawn_data { list[ns].ifindex = mlx5_nl_ifindex (nl_rdma, list[ns].ibv_dev->name, 1); if (!list[ns].ifindex) { + char ifname[IF_NAMESIZE]; + /* - * No network interface index found for the - * specified device, it means there it is not - * a representor/master. + * Netlink failed, it may happen with old + * ib_core kernel driver (before 4.16). + * We can assume there is old driver because + * here we are processing single ports IB + * devices. Let's try sysfs to retrieve + * the ifindex. The method works for + * master device only. */ - continue; + if (nd > 1) { + /* + * Multiple devices found, assume + * representors, can not distinguish + * master/representor and retrieve + * ifindex via sysfs. + */ + continue; + } + ret = mlx5_get_master_ifname + (ibv_match[i]->ibdev_path, &ifname); + if (!ret) + list[ns].ifindex = + if_nametoindex(ifname); + if (!list[ns].ifindex) { + /* + * No network interface index found + * for the specified device, it means + * there it is neither representor + * nor master. + */ + continue; + } } ret = -1; if (nl_route >= 0) diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index ef05d9f..c9b2251 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -364,6 +364,7 @@ struct mlx5_priv { /* mlx5_ethdev.c */ int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]); +int mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]); unsigned int mlx5_ifindex(const struct rte_eth_dev *dev); int mlx5_ifreq(const struct rte_eth_dev *dev, int req, struct ifreq *ifr); int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu); diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 9ae9ddd..1e6fe19 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -127,21 +127,18 @@ struct ethtool_link_settings { * @return * 0 on success, a negative errno value otherwise and rte_errno is set. */ -static int -mlx5_get_master_ifname(const struct rte_eth_dev *dev, - char (*ifname)[IF_NAMESIZE]) +int +mlx5_get_master_ifname(const char *ibdev_path, char (*ifname)[IF_NAMESIZE]) { - struct mlx5_priv *priv = dev->data->dev_private; DIR *dir; struct dirent *dent; unsigned int dev_type = 0; unsigned int dev_port_prev = ~0u; char match[IF_NAMESIZE] = ""; - assert(priv); - assert(priv->sh); + assert(ibdev_path); { - MKSTR(path, "%s/device/net", priv->sh->ibdev_path); + MKSTR(path, "%s/device/net", ibdev_path); dir = opendir(path); if (dir == NULL) { @@ -161,7 +158,7 @@ struct ethtool_link_settings { continue; MKSTR(path, "%s/device/net/%s/%s", - priv->sh->ibdev_path, name, + ibdev_path, name, (dev_type ? "dev_id" : "dev_port")); file = fopen(path, "rb"); @@ -222,15 +219,18 @@ struct ethtool_link_settings { mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]) { struct mlx5_priv *priv = dev->data->dev_private; - unsigned int ifindex = - priv->nl_socket_rdma >= 0 ? - mlx5_nl_ifindex(priv->nl_socket_rdma, - priv->sh->ibdev_name, - priv->ibv_port) : 0; + unsigned int ifindex; + assert(priv); + assert(priv->sh); + ifindex = priv->nl_socket_rdma >= 0 ? + mlx5_nl_ifindex(priv->nl_socket_rdma, + priv->sh->ibdev_name, + priv->ibv_port) : 0; if (!ifindex) { if (!priv->representor) - return mlx5_get_master_ifname(dev, ifname); + return mlx5_get_master_ifname(priv->sh->ibdev_path, + ifname); rte_errno = ENXIO; return -rte_errno; } -- 1.8.3.1