From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85BCC45C0B; Tue, 29 Oct 2024 14:44:31 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A8EF642E77; Tue, 29 Oct 2024 14:44:21 +0100 (CET) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2048.outbound.protection.outlook.com [40.107.93.48]) by mails.dpdk.org (Postfix) with ESMTP id 3747642E91 for ; Tue, 29 Oct 2024 14:44:04 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KL8ucFMvQwpvxbcaUNeTorh4qmvFFysoyLFRmQTjawVzSqW+D3lUx5iHGGn817RCoyU43hegEcelQIytzvm884O/pMb5AXqxjl9UMqL1snDe1qCOVlgpHUKYCNAudhB3wJ3/OIG6lpb1NwAl5NqZ3Z6xl0rjZXhZC/bP5V6T9xyt5r//ln1YfaZJHXoN2geYtAb7XB9YFy72VqPzX4KGFM15kpzygEzIa3BtTPXbCPYJoPf3ghccLUioA6pQHEIRDwqxSieY0atvK/SjFguWNsfNCkgEAMhwZPN/X6x4GPWMHLQBu2qaLi13GD6qfRg6JYwovBbNYFcs3a/cLZ3a3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=n2Ed5kzt3zzt+7aWbC8VdS1KlFxGVdj9hz/uVXE9BVA=; b=gndGboTSYKF5E2KoZXn0ZBoiu0oT3dB5z0aO0I1ffl6OUpxLtPuEqYbI1u1u6Rc/jxdTIOvPWhNUjRFAjKm3HfPKk9hVsdIxUdcvi4ObFvMVLNee/bc8rxBzZaHSJJdBqDKlWqfD8HVAu1SXwQkcf5MbmmsfqGKTCQn7jNsgmpQgc7TlvmFNkZkEmd9P+O7Msb/O86LjfXWQ+ZFg94taLc01XRUYTSdOUPTFJGG2Sw419GBcJ0MYOqhEVHMqJ/fVBsSz5NtY5GUx4FBhsczI9osnLAyt74gLAU5ssMrusbFEdTx4f//dJM8cdd+wshwV4b6jy/tW7w0NyOwtxBF2BA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n2Ed5kzt3zzt+7aWbC8VdS1KlFxGVdj9hz/uVXE9BVA=; b=DzGTet7c+KagwfnpApzVZXW0Px19xyIRvaZGF1hgDb4kPnqv3eZHOSJvHBr78da/pLdu2WQCIRwhT+FHFI1uTGsUZoexYhujPY1LYJYHyjNfTy/si4ESt10GGW421+y5p+M9t7uHVdlXRHTNalY4dVbk0u6ThF9Kn5pFpxQGF1GwSmIYMKuRabMm6WOHG3WNNuNJ+Mmz/dlgVJplHI6ulPvx4dVtbXMN7SvReD26DTTlIwwydDrHWrIiMXwg0y3a3tdkPTL5nWYd9VYc3ULDerPb5RWYl2fVOpiqSKl8QPUkFf1dV0mMv9sktTwhQRejT8fJ+9nq2G5G2BspEruUHg== Received: from BY3PR03CA0025.namprd03.prod.outlook.com (2603:10b6:a03:39a::30) by SN7PR12MB7204.namprd12.prod.outlook.com (2603:10b6:806:2ab::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Tue, 29 Oct 2024 13:43:59 +0000 Received: from SJ1PEPF0000231C.namprd03.prod.outlook.com (2603:10b6:a03:39a:cafe::4) by BY3PR03CA0025.outlook.office365.com (2603:10b6:a03:39a::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.26 via Frontend Transport; Tue, 29 Oct 2024 13:43:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF0000231C.mail.protection.outlook.com (10.167.242.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.16 via Frontend Transport; Tue, 29 Oct 2024 13:43:59 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 29 Oct 2024 06:43:33 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 29 Oct 2024 06:43:31 -0700 From: "Minggang Li(Gavin)" To: , , , , Dariusz Sosnowski , Bing Zhao , Suanming Mou CC: , Subject: [PATCH V3 6/7] mlx5: use RDMA Netlink to update port information Date: Tue, 29 Oct 2024 15:42:55 +0200 Message-ID: <20241029134256.874767-7-gavinl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241029134256.874767-1-gavinl@nvidia.com> References: <20241028091822.860660-8-gavinl@nvidia.com> <20241029134256.874767-1-gavinl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF0000231C:EE_|SN7PR12MB7204:EE_ X-MS-Office365-Filtering-Correlation-Id: 41d0bc4a-182a-4316-e868-08dcf81fbdcf X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?okL87XjtGSDqNFDKasouza58EsWy9yKirn2ZbWAR6sB1Jj1+o9MP+o+/Pb8/?= =?us-ascii?Q?pQ638W3+THG6U8gc/xG+DYovt/B9rZFiP7Jhrh9/bMYdNahW51av0WMgkOEa?= =?us-ascii?Q?Ku5mecNsQVnJReYkmNv/IpvfOqmZgSUuzBJnSutHHp8bz5aQ8Fm2DNP1GNMH?= =?us-ascii?Q?sXbpdVMurzQKTPkK0iYixVp9VqMhUf5TvEAdbSrxoVbXiBSVvx5sGbQ053mR?= =?us-ascii?Q?iv9T8T6UTqPzjp4sxpkYOvzDy96BjV8T1fhnJL/mZ+Snmuf2fsT2LVs88cQD?= =?us-ascii?Q?b7viMsg/wDLGnDkAhGJqtI5pL4v7HCoEAi5uKng4kW9Xtl2TZBdcdAw95MIj?= =?us-ascii?Q?Fi6DMUqyTV2K/mBOHh2jH/isxIokuyxaSeRu1hInCUuI28gMuUF7FYqzhWXW?= =?us-ascii?Q?AEJvPYSLymE6MVcrHTQgu8hY7IDZBB2g5j/7a8R+x3Rga+1YQOsSjH7QSceH?= =?us-ascii?Q?r9G3ctbtRCF02B1eXeE5nTRjA22sLqc7M5fqgRCRQguELJMK5OSecgfsfM0Q?= =?us-ascii?Q?sC2gpixq+TV9HKpRss4RFbIVyGdIUy5fZ1uDSVlTXWBJiTfMQIGG+i5/Gnwt?= =?us-ascii?Q?t3nh9VPemTDw9VKno8wEC/Jn+Q8yV/42SQ1M7cXXEj6ogDHiKmkRlu/cbjVS?= =?us-ascii?Q?2ZtStljUoXxCIFztRR0vWXSJu/5GkVMvVocfTEGIePZOIHcRjmu7zDDD9ZVf?= =?us-ascii?Q?uXS8zyr8gqJW6ybX0nbIOyhkz1dZARmQKS1jdZacL/tpo3MTgghLCQ2zm7Wn?= =?us-ascii?Q?2aUx3M2Z6oKBy/Sm97k2kSSNEHii/WGMXtZTbSbOyTO8R9A2XkhKVrKpK99t?= =?us-ascii?Q?QJlfzfdpmDezguOlrOaqWTMUamyOnzIjLumlkMtBMS7QRO2RqneQ4z7BEVDM?= =?us-ascii?Q?5e2i1vsgZ9xV0+aIM3DCDWISbfr7Hjt7Cla+cXlWCgUyASIknRwCsvTaywaL?= =?us-ascii?Q?qhFwAT+Z82P/s01ZChmaDLAv4LJvkZEeAhZTIQ2A8MronxiulY86hs7gLy5e?= =?us-ascii?Q?yF1s/Z/YHqERYlBy6dJTsAil3aUQ2VEypMtn40rrtEQPQC06oxH8GkELMAfx?= =?us-ascii?Q?Dg0iTot1TA5x/d+iD8zbNwFAbsc6QOvaEnHrPTUWxfD+MAd4+xy1GvkUYqDI?= =?us-ascii?Q?Cne8Z0Dw4FaG/BEDgaisMpoJ1epyxTfr4949ohtGa3PBhgi3UM08dgto2QAc?= =?us-ascii?Q?s15cNjFHZX9t1JumFtACAzS4pUzBm8GOJyCEqkQdnnoc+v9dfYHBxlbpWNu9?= =?us-ascii?Q?4n5S/c3g3iPPzBcjl1bdFKx9ddd8x6IFJz/zaktTEVZS+QXR5km5GcPmkVDL?= =?us-ascii?Q?wX3kBRqrT1YeeGWa6UIt73o0MmCKID+QwW2IVfqgirJiF6BiDtg3cbhNdxb8?= =?us-ascii?Q?xkuCBpnS5P5QrEQ3SXw+vz5312eUeVBGFH9906GNwP96QIuxlg=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2024 13:43:59.0711 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 41d0bc4a-182a-4316-e868-08dcf81fbdcf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF0000231C.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7204 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Previously, port information, such as adding and deleting, is updated via route netlink. And the events used are link up/down, not the exact event for port adding or deleting, which does not performance well. To improve the performance, use RDMA monitor events to track port adding and deleting events and update corresponding port information. Signed-off-by: Minggang Li(Gavin) Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 6 ++ drivers/common/mlx5/linux/mlx5_nl.c | 74 ++++++++++++++++++----- drivers/common/mlx5/linux/mlx5_nl.h | 28 +++++++++ drivers/common/mlx5/version.map | 2 + drivers/net/mlx5/linux/mlx5_ethdev_os.c | 79 +++++++++++++++++++++++++ drivers/net/mlx5/linux/mlx5_os.c | 20 +++++++ drivers/net/mlx5/mlx5.h | 2 + 7 files changed, 195 insertions(+), 16 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 981401a9f2..1a9ec1bd62 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -1443,6 +1443,12 @@ for an additional list of options shared with other mlx5 drivers. By default, the PMD will set this value to 0. + .. note:: + + There is a race condition in probing port if probe_opt_en is set to 1. + Port probe may fail with wrong ifindex in cache while the interrupt + thread is updating the cache. Please try again if port probe failed. + - ``lacp_by_user`` parameter [int] A nonzero value enables the control of LACP traffic by the user application. diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c index e03db4f918..ce1c2a8e75 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.c +++ b/drivers/common/mlx5/linux/mlx5_nl.c @@ -101,6 +101,7 @@ #ifndef HAVE_RDMA_NL_GROUP_NOTIFY #define RDMA_NL_GROUP_NOTIFY 4 #endif +#define RDMA_NL_GROUP_NOTIFICATION (1 << (RDMA_NL_GROUP_NOTIFY - 1)) /* These are normally found in linux/if_link.h. */ #ifndef HAVE_IFLA_NUM_VF @@ -176,22 +177,6 @@ struct mlx5_nl_mac_addr { int mac_n; /**< Number of addresses in the array. */ }; -#define MLX5_NL_CMD_GET_IB_NAME (1 << 0) -#define MLX5_NL_CMD_GET_IB_INDEX (1 << 1) -#define MLX5_NL_CMD_GET_NET_INDEX (1 << 2) -#define MLX5_NL_CMD_GET_PORT_INDEX (1 << 3) -#define MLX5_NL_CMD_GET_PORT_STATE (1 << 4) - -/** Data structure used by mlx5_nl_cmdget_cb(). */ -struct mlx5_nl_port_info { - const char *name; /**< IB device name (in). */ - uint32_t flags; /**< found attribute flags (out). */ - uint32_t ibindex; /**< IB device index (out). */ - uint32_t ifindex; /**< Network interface index (out). */ - uint32_t portnum; /**< IB device max port number (out). */ - uint16_t state; /**< IB device port state (out). */ -}; - RTE_ATOMIC(uint32_t) atomic_sn; /* Generate Netlink sequence number. */ @@ -2110,3 +2095,60 @@ mlx5_nl_devlink_esw_multiport_get(int nlsk_fd, int family_id, const char *pci_ad *enable ? "en" : "dis", pci_addr); return ret; } + +int +mlx5_nl_rdma_monitor_init(void) +{ + return mlx5_nl_init(NETLINK_RDMA, RDMA_NL_GROUP_NOTIFICATION); +} + +void +mlx5_nl_rdma_monitor_info_get(struct nlmsghdr *hdr, struct mlx5_nl_port_info *data) +{ + size_t off = NLMSG_HDRLEN; + uint8_t event_type = 0; + + if (hdr->nlmsg_type != RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_MONITOR)) + goto error; + + while (off < hdr->nlmsg_len) { + struct nlattr *na = (void *)((uintptr_t)hdr + off); + void *payload = (void *)((uintptr_t)na + NLA_HDRLEN); + + if (na->nla_len > hdr->nlmsg_len - off) + goto error; + switch (na->nla_type) { + case RDMA_NLDEV_ATTR_EVENT_TYPE: + event_type = *(uint8_t *)payload; + if (event_type == RDMA_NETDEV_ATTACH_EVENT) { + data->flags |= MLX5_NL_CMD_GET_EVENT_TYPE; + data->event_type = MLX5_NL_RDMA_NETDEV_ATTACH_EVENT; + } else if (event_type == RDMA_NETDEV_DETACH_EVENT) { + data->flags |= MLX5_NL_CMD_GET_EVENT_TYPE; + data->event_type = MLX5_NL_RDMA_NETDEV_DETACH_EVENT; + } + break; + case RDMA_NLDEV_ATTR_DEV_INDEX: + data->ibindex = *(uint32_t *)payload; + data->flags |= MLX5_NL_CMD_GET_IB_INDEX; + break; + case RDMA_NLDEV_ATTR_PORT_INDEX: + data->portnum = *(uint32_t *)payload; + data->flags |= MLX5_NL_CMD_GET_PORT_INDEX; + break; + case RDMA_NLDEV_ATTR_NDEV_INDEX: + data->ifindex = *(uint32_t *)payload; + data->flags |= MLX5_NL_CMD_GET_NET_INDEX; + break; + default: + DRV_LOG(DEBUG, "Unknown attribute[%d] found", na->nla_type); + break; + } + off += NLA_ALIGN(na->nla_len); + } + + return; + +error: + rte_errno = EINVAL; +} diff --git a/drivers/common/mlx5/linux/mlx5_nl.h b/drivers/common/mlx5/linux/mlx5_nl.h index 396ffc98ce..e32080fa63 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.h +++ b/drivers/common/mlx5/linux/mlx5_nl.h @@ -32,6 +32,27 @@ struct mlx5_nl_vlan_vmwa_context { struct mlx5_nl_vlan_dev vlan_dev[4096]; }; +#define MLX5_NL_CMD_GET_IB_NAME (1 << 0) +#define MLX5_NL_CMD_GET_IB_INDEX (1 << 1) +#define MLX5_NL_CMD_GET_NET_INDEX (1 << 2) +#define MLX5_NL_CMD_GET_PORT_INDEX (1 << 3) +#define MLX5_NL_CMD_GET_PORT_STATE (1 << 4) +#define MLX5_NL_CMD_GET_EVENT_TYPE (1 << 5) + +/** Data structure used by mlx5_nl_cmdget_cb(). */ +struct mlx5_nl_port_info { + const char *name; /**< IB device name (in). */ + uint32_t flags; /**< found attribute flags (out). */ + uint32_t ibindex; /**< IB device index (out). */ + uint32_t ifindex; /**< Network interface index (out). */ + uint32_t portnum; /**< IB device max port number (out). */ + uint16_t state; /**< IB device port state (out). */ + uint8_t event_type; /**< IB RDMA event type (out). */ +}; + +#define MLX5_NL_RDMA_NETDEV_ATTACH_EVENT (1) +#define MLX5_NL_RDMA_NETDEV_DETACH_EVENT (2) + __rte_internal int mlx5_nl_init(int protocol, int groups); __rte_internal @@ -89,4 +110,11 @@ __rte_internal int mlx5_nl_devlink_esw_multiport_get(int nlsk_fd, int family_id, const char *pci_addr, int *enable); +__rte_internal +int mlx5_nl_rdma_monitor_init(void); +__rte_internal +void mlx5_nl_rdma_monitor_info_get(struct nlmsghdr *hdr, struct mlx5_nl_port_info *data); +__rte_internal +int mlx5_nl_rdma_monitor_cap_get(int nl, uint8_t *cap); + #endif /* RTE_PMD_MLX5_NL_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index a2f72ef46a..5230576006 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -146,6 +146,8 @@ INTERNAL { mlx5_nl_vf_mac_addr_modify; # WINDOWS_NO_EXPORT mlx5_nl_vlan_vmwa_create; # WINDOWS_NO_EXPORT mlx5_nl_vlan_vmwa_delete; # WINDOWS_NO_EXPORT + mlx5_nl_rdma_monitor_init; # WINDOWS_NO_EXPORT + mlx5_nl_rdma_monitor_info_get; # WINDOWS_NO_EXPORT mlx5_os_umem_dereg; mlx5_os_umem_reg; diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index 88d3c57c6e..5156d96b3a 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -894,6 +894,85 @@ mlx5_dev_interrupt_handler_devx(void *cb_arg) #endif /* HAVE_IBV_DEVX_ASYNC */ } +static void +mlx5_dev_interrupt_ib_cb(struct nlmsghdr *hdr, void *cb_arg) +{ + mlx5_nl_rdma_monitor_info_get(hdr, (struct mlx5_nl_port_info *)cb_arg); +} + +void +mlx5_dev_interrupt_handler_ib(void *arg) +{ + struct mlx5_dev_ctx_shared *sh = arg; + struct mlx5_nl_port_info data = { + .flags = 0, + .name = "", + .ifindex = 0, + .ibindex = 0, + .portnum = 0, + }; + int nlsk_fd = rte_intr_fd_get(sh->intr_handle_ib); + struct mlx5_dev_info *dev_info; + uint32_t i; + + dev_info = &sh->cdev->dev_info; + DRV_LOG(DEBUG, "IB device %s received RDMA monitor netlink event", dev_info->ibname); + if (dev_info->port_num <= 1 || dev_info->port_info == NULL) + return; + + if (nlsk_fd < 0) + return; + + if (mlx5_nl_read_events(nlsk_fd, mlx5_dev_interrupt_ib_cb, &data) < 0) + DRV_LOG(ERR, "Failed to process Netlink events: %s", + rte_strerror(rte_errno)); + + if (!(data.flags & MLX5_NL_CMD_GET_EVENT_TYPE) || + !(data.flags & MLX5_NL_CMD_GET_PORT_INDEX) || + !(data.flags & MLX5_NL_CMD_GET_IB_INDEX)) + return; + + if (data.ibindex != dev_info->ibindex) + return; + + if (data.event_type != MLX5_NL_RDMA_NETDEV_ATTACH_EVENT && + data.event_type != MLX5_NL_RDMA_NETDEV_DETACH_EVENT) + return; + + if (data.event_type == MLX5_NL_RDMA_NETDEV_ATTACH_EVENT && + !(data.flags & MLX5_NL_CMD_GET_NET_INDEX)) + return; + + DRV_LOG(DEBUG, "Event info: type %d, ibindex %d, ifindex %d, portnum %d,", + data.event_type, data.ibindex, data.ifindex, data.portnum); + + /* Changes found in number of SF/VF ports. All information is likely unreliable. */ + if (data.portnum > dev_info->port_num) { + DRV_LOG(ERR, "Port[%d] exceeds maximum[%d]", data.portnum, dev_info->port_num); + goto flush_all; + } + if (data.event_type == MLX5_NL_RDMA_NETDEV_ATTACH_EVENT) { + if (!dev_info->port_info[data.portnum].ifindex) { + dev_info->port_info[data.portnum].ifindex = data.ifindex; + dev_info->port_info[data.portnum].valid = 1; + } else { + DRV_LOG(WARNING, "Duplicate RDMA event for port[%d] ifindex[%d]", + data.portnum, data.ifindex); + if (data.ifindex != dev_info->port_info[data.portnum].ifindex) + goto flush_all; + } + } else if (data.event_type == MLX5_NL_RDMA_NETDEV_DETACH_EVENT) { + memset(dev_info->port_info + data.portnum, 0, sizeof(struct mlx5_port_nl_info)); + } + return; + +flush_all: + for (i = 1; i <= dev_info->port_num; i++) { + dev_info->port_info[i].ifindex = 0; + dev_info->port_info[i].valid = 0; + } +} + /** * DPDK callback to bring the link DOWN. * diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 4537ca0466..16b275c71e 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -3025,6 +3025,21 @@ mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh) DRV_LOG(ERR, "Failed to allocate intr_handle."); return; } + if (sh->cdev->config.probe_opt && sh->cdev->dev_info.port_num > 1) { + nlsk_fd = mlx5_nl_rdma_monitor_init(); + if (nlsk_fd < 0) { + DRV_LOG(ERR, "Failed to create a socket for RDMA Netlink events: %s", + rte_strerror(rte_errno)); + return; + } + sh->intr_handle_ib = mlx5_os_interrupt_handler_create + (RTE_INTR_INSTANCE_F_SHARED, true, + nlsk_fd, mlx5_dev_interrupt_handler_ib, sh); + if (sh->intr_handle_ib == NULL) { + DRV_LOG(ERR, "Fail to allocate intr_handle"); + return; + } + } nlsk_fd = mlx5_nl_init(NETLINK_ROUTE, RTMGRP_LINK); if (nlsk_fd < 0) { DRV_LOG(ERR, "Failed to create a socket for Netlink events: %s", @@ -3086,6 +3101,11 @@ mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh) if (sh->devx_comp) mlx5_glue->devx_destroy_cmd_comp(sh->devx_comp); #endif + fd = rte_intr_fd_get(sh->intr_handle_ib); + mlx5_os_interrupt_handler_destroy(sh->intr_handle_ib, + mlx5_dev_interrupt_handler_ib, sh); + if (fd >= 0) + close(fd); } /** diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 503366580b..adc21c272b 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1574,6 +1574,7 @@ struct mlx5_dev_ctx_shared { struct rte_intr_handle *intr_handle; /* Interrupt handler for device. */ struct rte_intr_handle *intr_handle_devx; /* DEVX interrupt handler. */ struct rte_intr_handle *intr_handle_nl; /* Netlink interrupt handler. */ + struct rte_intr_handle *intr_handle_ib; /* Interrupt handler for IB device. */ void *devx_comp; /* DEVX async comp obj. */ struct mlx5_devx_obj *tis[16]; /* TIS object. */ struct mlx5_devx_obj *td; /* Transport domain. */ @@ -2274,6 +2275,7 @@ int mlx5_dev_set_flow_ctrl(struct rte_eth_dev *dev, void mlx5_dev_interrupt_handler(void *arg); void mlx5_dev_interrupt_handler_devx(void *arg); void mlx5_dev_interrupt_handler_nl(void *arg); +void mlx5_dev_interrupt_handler_ib(void *arg); int mlx5_set_link_down(struct rte_eth_dev *dev); int mlx5_set_link_up(struct rte_eth_dev *dev); int mlx5_is_removed(struct rte_eth_dev *dev); -- 2.34.1