From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by dpdk.org (Postfix) with ESMTP id F2FAB1BF67 for ; Wed, 4 Jul 2018 19:28:01 +0200 (CEST) Received: by mail-wm0-f66.google.com with SMTP id w16-v6so7092160wmc.2 for ; Wed, 04 Jul 2018 10:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EM3d5ieHBtd5+GEdF7A4rgOgY3EDx79w9cdZs81cSjg=; b=N4FHoykg6jluw82u/08YKMDht6GuAxnRcor7lNwWXo/kr6848y6/9AujX7jF3dPX+C +AVwAmgYy43zJcjlEPdrWonxdNbTnJVcJTaf+5ek391fvvVKhLToHORhTHXsI2OjCK/F iemjt1hAMsraJ17OhQHDMg5OYgv3/i+8WlN7YzQKKonA28LGS2tUjkmtL/+lEoqgkuG4 SAsLDHZ2FjOPLk5U/I6OCf0DNw9ntDBttvxRYgiJX2bTw197W9X1wjLsSl38lhomEq3h G1BUkT/SlttIVkqDjSVKNLbHSB6yCn+AS+H6qsVjK3Ix/qmHEL1PLExPtvZTA6T4t12F 0Z3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EM3d5ieHBtd5+GEdF7A4rgOgY3EDx79w9cdZs81cSjg=; b=cLFjoXyUUAWwg0j489gq9tZfu2sOVSBbQAnK/CbAvMNfWVoYpkXLYE3pTev2cyEkPn pnriKClwYyv2Mcai6UjBB24Sr4Zi8wCAQXzSE5b13aY0nJ/M+ogA4c0/MPrknzSOorbM Ud+AMeoX50SvYmNEV0ToDhOT/Q39VdTWv5HbS1rwtifdju+dIi7KogaYt7d9/O+Jhf2o 7WDgWB2G1oN8oKaw9aYSktwSsh/JK6zA+wgmS6mlg/Y7aL4Wyj5CvPofkubLcOVUKN7/ iigRXR5pMk+qhcC0twRQJI5dTeIOUE0yQv31xf2EiqN5okek0S4RzAHks6+IY5TXCZ2J 6Ejg== X-Gm-Message-State: APt69E0nslX5lIpJpbttB646C/xS6d2YQEgQEvmdr33MkHpEIgYSa8iI yp0al2G+cQe7Hnn5EV5MlW/Qlw== X-Google-Smtp-Source: AAOMgpdyxyGbiZxlskaG6TvGTBtXMNR6axqytosFZWuktjN8ZzZvYzTPJywBxFnWtE2Xb4tThuNx4w== X-Received: by 2002:a1c:8a04:: with SMTP id m4-v6mr2088134wmd.137.1530725281217; Wed, 04 Jul 2018 10:28:01 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id 127-v6sm5938960wmd.18.2018.07.04.10.28.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Jul 2018 10:28:00 -0700 (PDT) Date: Wed, 4 Jul 2018 19:27:44 +0200 From: Adrien Mazarguil To: Shahaf Shuler Cc: dev@dpdk.org, Xueming Li Message-ID: <20180704172322.22571-7-adrien.mazarguil@6wind.com> References: <20180614083047.10812-1-adrien.mazarguil@6wind.com> <20180704172322.22571-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180704172322.22571-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v3 06/10] net/mlx5: add port representor awareness X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2018 17:28:02 -0000 The current PCI probing method is not aware of Verbs port representors, which appear as standard Verbs devices bound to the same PCI address and cannot be distinguished. Problem is that more often than not, the wrong Verbs device is used, resulting in unexpected traffic. This patch makes the driver discard representors to only use the master device. If unable to identify it (e.g. kernel drivers not recent enough), either: - There is only one matching device which isn't identified as a representor, in that case use it. - Otherwise log an error and do not probe the device. (Patch based on prior work from Yuanhan Liu) Signed-off-by: Adrien Mazarguil Reviewed-by: Xueming Li Cc: Xueming Li -- v3 changes: - Replaced all heuristics (including mlx5_cmp_ibv_name()) with Netlink queries to associate IB devices with network interfaces. - Reworded commit log. v2 changes: - Fixed digit detection in mlx5_cmp_ibv_name() so that "foo1" and "foo10" are compared on the integer conversion of "1" against "10" instead of "" and "0". --- drivers/net/mlx5/Makefile | 30 ++++ drivers/net/mlx5/mlx5.c | 109 +++++++++++++-- drivers/net/mlx5/mlx5.h | 16 ++- drivers/net/mlx5/mlx5_nl.c | 297 ++++++++++++++++++++++++++++++++++++++-- 4 files changed, 428 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index 955861a41..745752e23 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -152,6 +152,36 @@ mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh infiniband/verbs.h \ enum IBV_FLOW_SPEC_ACTION_COUNT \ $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_CMD_GET \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_CMD_GET \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_CMD_PORT_GET \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_CMD_PORT_GET \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_ATTR_DEV_INDEX \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_ATTR_DEV_INDEX \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_ATTR_DEV_NAME \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_ATTR_DEV_NAME \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_ATTR_PORT_INDEX \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_ATTR_PORT_INDEX \ + $(AUTOCONF_OUTPUT) + $Q sh -- '$<' '$@' \ + HAVE_RDMA_NLDEV_ATTR_NDEV_INDEX \ + linux/rdma_netlink.h \ + enum RDMA_NLDEV_ATTR_NDEV_INDEX \ + $(AUTOCONF_OUTPUT) # Create mlx5_autoconf.h or update it in case it differs from the new one. diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 1054bf6d0..d06ba9886 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -13,6 +13,7 @@ #include #include #include +#include #include /* Verbs header. */ @@ -274,8 +275,10 @@ mlx5_dev_close(struct rte_eth_dev *dev) mlx5_socket_uninit(dev); if (priv->config.vf) mlx5_nl_mac_addr_flush(dev); - if (priv->nl_socket >= 0) - close(priv->nl_socket); + if (priv->nl_socket_route >= 0) + close(priv->nl_socket_route); + if (priv->nl_socket_rdma >= 0) + close(priv->nl_socket_rdma); ret = mlx5_hrxq_ibv_verify(dev); if (ret) DRV_LOG(WARNING, "port %u some hash Rx queue still remain", @@ -876,6 +879,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->device_attr = attr; priv->pd = pd; priv->mtu = ETHER_MTU; + /* Some internal functions rely on Netlink sockets, open them now. */ + priv->nl_socket_rdma = mlx5_nl_init(0, NETLINK_RDMA); + priv->nl_socket_route = mlx5_nl_init(RTMGRP_LINK, NETLINK_ROUTE); + priv->nl_sn = 0; err = mlx5_args(&config, dpdk_dev->devargs); if (err) { err = rte_errno; @@ -1010,14 +1017,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, eth_dev->dev_ops = &mlx5_dev_ops; /* Register MAC address. */ claim_zero(mlx5_mac_addr_add(eth_dev, &mac, 0, 0)); - priv->nl_socket = -1; - priv->nl_sn = 0; - if (vf && config.vf_nl_en) { - priv->nl_socket = mlx5_nl_init(RTMGRP_LINK); - if (priv->nl_socket < 0) - priv->nl_socket = -1; + if (vf && config.vf_nl_en) mlx5_nl_mac_addr_sync(eth_dev); - } TAILQ_INIT(&priv->flows); TAILQ_INIT(&priv->ctrl_flows); /* Hint libmlx5 to use PMD allocator for data plane resources */ @@ -1078,8 +1079,13 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, rte_rwlock_write_unlock(&mlx5_shared_data->mem_event_rwlock); return eth_dev; error: - if (priv) + if (priv) { + if (priv->nl_socket_route >= 0) + close(priv->nl_socket_route); + if (priv->nl_socket_rdma >= 0) + close(priv->nl_socket_rdma); rte_free(priv); + } if (pd) claim_zero(mlx5_glue->dealloc_pd(pd)); if (eth_dev) @@ -1110,6 +1116,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, { struct ibv_device **ibv_list; struct rte_eth_dev *eth_dev = NULL; + unsigned int n = 0; int vf; int ret; @@ -1121,6 +1128,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, DRV_LOG(ERR, "cannot list devices, is ib_uverbs loaded?"); return -rte_errno; } + + struct ibv_device *ibv_match[ret + 1]; + while (ret-- > 0) { struct rte_pci_addr pci_addr; @@ -1132,10 +1142,81 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, pci_dev->addr.devid != pci_addr.devid || pci_dev->addr.function != pci_addr.function) continue; - DRV_LOG(INFO, "PCI information matches, using device \"%s\"", + DRV_LOG(INFO, "PCI information matches for device \"%s\"", ibv_list[ret]->name); + ibv_match[n++] = ibv_list[ret]; + } + ibv_match[n] = NULL; + + unsigned int ifindex[n]; + struct mlx5_switch_info info[n]; + int nl_route = n ? mlx5_nl_init(0, NETLINK_ROUTE) : -1; + int nl_rdma = n ? mlx5_nl_init(0, NETLINK_RDMA) : -1; + unsigned int i; + + /* + * The existence of several matching entries (n > 1) means port + * representors have been instantiated. No existing Verbs call nor + * /sys entries can tell them apart, this can only be done through + * Netlink calls assuming kernel drivers are recent enough to + * support them. + * + * In the event of identification failure through Netlink, either: + * + * 1. No device matches (n == 0), complain and bail out. + * 2. A single IB device matches (n == 1) and is not a representor, + * assume no switch support. + * 3. Otherwise no safe assumptions can be made; complain louder and + * bail out. + */ + for (i = 0; i != n; ++i) { + if (nl_rdma < 0) + ifindex[i] = 0; + else + ifindex[i] = mlx5_nl_ifindex(nl_rdma, + ibv_match[i]->name); + if (nl_route < 0 || + !ifindex[i] || + mlx5_nl_switch_info(nl_route, ifindex[i], &info[i])) { + ifindex[i] = 0; + memset(&info[i], 0, sizeof(info[i])); + continue; + } + } + if (nl_rdma >= 0) + close(nl_rdma); + if (nl_route >= 0) + close(nl_route); + /* Look for master device. */ + for (i = 0; i != n; ++i) { + if (!info[i].master) + continue; + /* Make it the first entry. */ + if (i == 0) + break; + ibv_match[n] = ibv_match[0]; + ibv_match[0] = ibv_match[i]; + ibv_match[n] = NULL; break; } + if (n && i == n) { + if (n == 1 && !info[0].representor) { + /* Case #2. */ + DRV_LOG(INFO, "no switch support detected"); + } else if (n == 1) { + /* Case #3. */ + DRV_LOG(ERR, + "device looks like a port representor, this is" + " not supported yet"); + n = 0; + } else { + /* Case #3. */ + DRV_LOG(ERR, + "unable to tell which of the matching devices" + " is the master (lack of kernel support?)"); + n = 0; + } + } switch (pci_dev->id.device_id) { case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF: @@ -1146,10 +1227,10 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, default: vf = 0; } - if (ret >= 0) - eth_dev = mlx5_dev_spawn(&pci_dev->device, ibv_list[ret], vf); + if (n) + eth_dev = mlx5_dev_spawn(&pci_dev->device, ibv_match[0], vf); mlx5_glue->free_device_list(ibv_list); - if (!ret) { + if (!n) { DRV_LOG(WARNING, "no Verbs device matches PCI device " PCI_PRI_FMT "," " are kernel drivers loaded?", diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index f55ff4a21..704046270 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -53,6 +53,14 @@ enum { PCI_DEVICE_ID_MELLANOX_CONNECTX5BF = 0xa2d2, }; +/** Switch information returned by mlx5_nl_switch_info(). */ +struct mlx5_switch_info { + uint32_t master:1; /**< Master device. */ + uint32_t representor:1; /**< Representor device. */ + int32_t port_name; /**< Representor port name. */ + uint64_t switch_id; /**< Switch identifier. */ +}; + LIST_HEAD(mlx5_dev_list, priv); /* Shared memory between primary and secondary processes. */ @@ -195,7 +203,8 @@ struct priv { struct mlx5_dev_config config; /* Device configuration. */ struct mlx5_verbs_alloc_ctx verbs_alloc_ctx; /* Context for Verbs allocator. */ - int nl_socket; /* Netlink socket. */ + int nl_socket_rdma; /* Netlink socket (NETLINK_RDMA). */ + int nl_socket_route; /* Netlink socket (NETLINK_ROUTE). */ uint32_t nl_sn; /* Netlink message sequence number. */ }; @@ -342,7 +351,7 @@ int mlx5_socket_connect(struct rte_eth_dev *priv); /* mlx5_nl.c */ -int mlx5_nl_init(uint32_t nlgroups); +int mlx5_nl_init(uint32_t nlgroups, int protocol); int mlx5_nl_mac_addr_add(struct rte_eth_dev *dev, struct ether_addr *mac, uint32_t index); int mlx5_nl_mac_addr_remove(struct rte_eth_dev *dev, struct ether_addr *mac, @@ -351,5 +360,8 @@ void mlx5_nl_mac_addr_sync(struct rte_eth_dev *dev); void mlx5_nl_mac_addr_flush(struct rte_eth_dev *dev); int mlx5_nl_promisc(struct rte_eth_dev *dev, int enable); int mlx5_nl_allmulti(struct rte_eth_dev *dev, int enable); +unsigned int mlx5_nl_ifindex(int nl, const char *name); +int mlx5_nl_switch_info(int nl, unsigned int ifindex, + struct mlx5_switch_info *info); #endif /* RTE_PMD_MLX5_H_ */ diff --git a/drivers/net/mlx5/mlx5_nl.c b/drivers/net/mlx5/mlx5_nl.c index dca85835a..58ef2f4f0 100644 --- a/drivers/net/mlx5/mlx5_nl.c +++ b/drivers/net/mlx5/mlx5_nl.c @@ -3,10 +3,21 @@ * Copyright 2018 Mellanox Technologies, Ltd */ +#include +#include #include #include +#include +#include +#include +#include +#include +#include +#include #include +#include + #include "mlx5.h" #include "mlx5_utils.h" @@ -27,6 +38,29 @@ ((struct rtattr *)(((char *)(r)) + NLMSG_ALIGN(sizeof(struct ndmsg)))) #endif +/* + * The following definitions are normally found in rdma/rdma_netlink.h, + * however they are so recent that most systems do not expose them yet. + */ +#ifndef HAVE_RDMA_NLDEV_CMD_GET +#define RDMA_NLDEV_CMD_GET 1 +#endif +#ifndef HAVE_RDMA_NLDEV_CMD_PORT_GET +#define RDMA_NLDEV_CMD_PORT_GET 5 +#endif +#ifndef HAVE_RDMA_NLDEV_ATTR_DEV_INDEX +#define RDMA_NLDEV_ATTR_DEV_INDEX 1 +#endif +#ifndef HAVE_RDMA_NLDEV_ATTR_DEV_NAME +#define RDMA_NLDEV_ATTR_DEV_NAME 2 +#endif +#ifndef HAVE_RDMA_NLDEV_ATTR_PORT_INDEX +#define RDMA_NLDEV_ATTR_PORT_INDEX 3 +#endif +#ifndef HAVE_RDMA_NLDEV_ATTR_NDEV_INDEX +#define RDMA_NLDEV_ATTR_NDEV_INDEX 50 +#endif + /* Add/remove MAC address through Netlink */ struct mlx5_nl_mac_addr { struct ether_addr (*mac)[]; @@ -34,18 +68,27 @@ struct mlx5_nl_mac_addr { int mac_n; /**< Number of addresses in the array. */ }; +/** Data structure used by mlx5_nl_ifindex_cb(). */ +struct mlx5_nl_ifindex_data { + const char *name; /**< IB device name (in). */ + uint32_t ibindex; /**< IB device index (out). */ + uint32_t ifindex; /**< Network interface index (out). */ +}; + /** * Opens a Netlink socket. * * @param nl_groups * Netlink group value (e.g. RTMGRP_LINK). + * @param protocol + * Netlink protocol (e.g. NETLINK_ROUTE, NETLINK_RDMA). * * @return * A file descriptor on success, a negative errno value otherwise and * rte_errno is set. */ int -mlx5_nl_init(uint32_t nl_groups) +mlx5_nl_init(uint32_t nl_groups, int protocol) { int fd; int sndbuf_size = MLX5_SEND_BUF_SIZE; @@ -56,7 +99,7 @@ mlx5_nl_init(uint32_t nl_groups) }; int ret; - fd = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_ROUTE); + fd = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, protocol); if (fd == -1) { rte_errno = errno; return -rte_errno; @@ -334,9 +377,9 @@ mlx5_nl_mac_addr_list(struct rte_eth_dev *dev, struct ether_addr (*mac)[], int ret; uint32_t sn = priv->nl_sn++; - if (priv->nl_socket == -1) + if (priv->nl_socket_route == -1) return 0; - fd = priv->nl_socket; + fd = priv->nl_socket_route; ret = mlx5_nl_request(fd, &req.hdr, sn, &req.ifm, sizeof(struct ifinfomsg)); if (ret < 0) @@ -398,9 +441,9 @@ mlx5_nl_mac_addr_modify(struct rte_eth_dev *dev, struct ether_addr *mac, int ret; uint32_t sn = priv->nl_sn++; - if (priv->nl_socket == -1) + if (priv->nl_socket_route == -1) return 0; - fd = priv->nl_socket; + fd = priv->nl_socket_route; memcpy(RTA_DATA(&req.rta), mac, ETHER_ADDR_LEN); req.hdr.nlmsg_len = NLMSG_ALIGN(req.hdr.nlmsg_len) + RTA_ALIGN(req.rta.rta_len); @@ -569,9 +612,9 @@ mlx5_nl_device_flags(struct rte_eth_dev *dev, uint32_t flags, int enable) int ret; assert(!(flags & ~(IFF_PROMISC | IFF_ALLMULTI))); - if (priv->nl_socket < 0) + if (priv->nl_socket_route < 0) return 0; - fd = priv->nl_socket; + fd = priv->nl_socket_route; ret = mlx5_nl_send(fd, &req.hdr, priv->nl_sn++); if (ret < 0) return ret; @@ -625,3 +668,241 @@ mlx5_nl_allmulti(struct rte_eth_dev *dev, int enable) strerror(rte_errno)); return ret; } + +/** + * Process network interface information from Netlink message. + * + * @param nh + * Pointer to Netlink message header. + * @param arg + * Opaque data pointer for this callback. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_nl_ifindex_cb(struct nlmsghdr *nh, void *arg) +{ + struct mlx5_nl_ifindex_data *data = arg; + size_t off = NLMSG_HDRLEN; + uint32_t ibindex = 0; + uint32_t ifindex = 0; + int found = 0; + + if (nh->nlmsg_type != + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_GET) && + nh->nlmsg_type != + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_PORT_GET)) + goto error; + while (off < nh->nlmsg_len) { + struct nlattr *na = (void *)((uintptr_t)nh + off); + void *payload = (void *)((uintptr_t)na + NLA_HDRLEN); + + if (na->nla_len > nh->nlmsg_len - off) + goto error; + switch (na->nla_type) { + case RDMA_NLDEV_ATTR_DEV_INDEX: + ibindex = *(uint32_t *)payload; + break; + case RDMA_NLDEV_ATTR_DEV_NAME: + if (!strcmp(payload, data->name)) + found = 1; + break; + case RDMA_NLDEV_ATTR_NDEV_INDEX: + ifindex = *(uint32_t *)payload; + break; + default: + break; + } + off += NLA_ALIGN(na->nla_len); + } + if (found) { + data->ibindex = ibindex; + data->ifindex = ifindex; + } + return 0; +error: + rte_errno = EINVAL; + return -rte_errno; +} + +/** + * Get index of network interface associated with some IB device. + * + * This is the only somewhat safe method to avoid resorting to heuristics + * when faced with port representors. Unfortunately it requires at least + * Linux 4.17. + * + * @param nl + * Netlink socket of the RDMA kind (NETLINK_RDMA). + * @param[in] name + * IB device name. + * + * @return + * A valid (nonzero) interface index on success, 0 otherwise and rte_errno + * is set. + */ +unsigned int +mlx5_nl_ifindex(int nl, const char *name) +{ + static const uint32_t pindex = 1; + uint32_t seq = random(); + struct mlx5_nl_ifindex_data data = { + .name = name, + .ibindex = 0, /* Determined during first pass. */ + .ifindex = 0, /* Determined during second pass. */ + }; + union { + struct nlmsghdr nh; + uint8_t buf[NLMSG_HDRLEN + + NLA_HDRLEN + NLA_ALIGN(sizeof(data.ibindex)) + + NLA_HDRLEN + NLA_ALIGN(sizeof(pindex))]; + } req = { + .nh = { + .nlmsg_len = NLMSG_LENGTH(0), + .nlmsg_type = RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, + RDMA_NLDEV_CMD_GET), + .nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP, + }, + }; + struct nlattr *na; + int ret; + + ret = mlx5_nl_send(nl, &req.nh, seq); + if (ret < 0) + return 0; + ret = mlx5_nl_recv(nl, seq, mlx5_nl_ifindex_cb, &data); + if (ret < 0) + return 0; + if (!data.ibindex) + goto error; + ++seq; + req.nh.nlmsg_type = RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, + RDMA_NLDEV_CMD_PORT_GET); + req.nh.nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK; + req.nh.nlmsg_len = NLMSG_LENGTH(sizeof(req.buf) - NLMSG_HDRLEN); + na = (void *)((uintptr_t)req.buf + NLMSG_HDRLEN); + na->nla_len = NLA_HDRLEN + sizeof(data.ibindex); + na->nla_type = RDMA_NLDEV_ATTR_DEV_INDEX; + memcpy((void *)((uintptr_t)na + NLA_HDRLEN), + &data.ibindex, sizeof(data.ibindex)); + na = (void *)((uintptr_t)na + NLA_ALIGN(na->nla_len)); + na->nla_len = NLA_HDRLEN + sizeof(pindex); + na->nla_type = RDMA_NLDEV_ATTR_PORT_INDEX; + memcpy((void *)((uintptr_t)na + NLA_HDRLEN), + &pindex, sizeof(pindex)); + ret = mlx5_nl_send(nl, &req.nh, seq); + if (ret < 0) + return 0; + ret = mlx5_nl_recv(nl, seq, mlx5_nl_ifindex_cb, &data); + if (ret < 0) + return 0; + if (!data.ifindex) + goto error; + return data.ifindex; +error: + rte_errno = ENODEV; + return 0; +} + +/** + * Process switch information from Netlink message. + * + * @param nh + * Pointer to Netlink message header. + * @param arg + * Opaque data pointer for this callback. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +static int +mlx5_nl_switch_info_cb(struct nlmsghdr *nh, void *arg) +{ + struct mlx5_switch_info info = { + .master = 0, + .representor = 0, + .port_name = 0, + .switch_id = 0, + }; + size_t off = NLMSG_LENGTH(sizeof(struct ifinfomsg)); + bool port_name_set = false; + bool switch_id_set = false; + + if (nh->nlmsg_type != RTM_NEWLINK) + goto error; + while (off < nh->nlmsg_len) { + struct rtattr *ra = (void *)((uintptr_t)nh + off); + void *payload = RTA_DATA(ra); + char *end; + unsigned int i; + + if (ra->rta_len > nh->nlmsg_len - off) + goto error; + switch (ra->rta_type) { + case IFLA_PHYS_PORT_NAME: + errno = 0; + info.port_name = strtol(payload, &end, 0); + if (errno || + (size_t)(end - (char *)payload) != strlen(payload)) + goto error; + port_name_set = true; + break; + case IFLA_PHYS_SWITCH_ID: + info.switch_id = 0; + for (i = 0; i < RTA_PAYLOAD(ra); ++i) { + info.switch_id <<= 8; + info.switch_id |= ((uint8_t *)payload)[i]; + } + switch_id_set = true; + break; + } + off += RTA_ALIGN(ra->rta_len); + } + info.master = switch_id_set && !port_name_set; + info.representor = switch_id_set && port_name_set; + memcpy(arg, &info, sizeof(info)); + return 0; +error: + rte_errno = EINVAL; + return -rte_errno; +} + +/** + * Get switch information associated with network interface. + * + * @param nl + * Netlink socket of the ROUTE kind (NETLINK_ROUTE). + * @param ifindex + * Network interface index. + * @param[out] info + * Switch information object, populated in case of success. + * + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_nl_switch_info(int nl, unsigned int ifindex, struct mlx5_switch_info *info) +{ + uint32_t seq = random(); + struct { + struct nlmsghdr nh; + struct ifinfomsg info; + } req = { + .nh = { + .nlmsg_len = NLMSG_LENGTH(sizeof(req.info)), + .nlmsg_type = RTM_GETLINK, + .nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK, + }, + .info = { + .ifi_family = AF_UNSPEC, + .ifi_index = ifindex, + }, + }; + int ret; + + ret = mlx5_nl_send(nl, &req.nh, seq); + if (ret >= 0) + ret = mlx5_nl_recv(nl, seq, mlx5_nl_switch_info_cb, info); + return ret; +} -- 2.11.0