From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id AFA8AA0503;
Fri, 6 May 2022 05:57:21 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 4038C42828;
Fri, 6 May 2022 05:57:11 +0200 (CEST)
Received: from NAM10-BN7-obe.outbound.protection.outlook.com
(mail-bn7nam10on2089.outbound.protection.outlook.com [40.107.92.89])
by mails.dpdk.org (Postfix) with ESMTP id 49D9142824
for <dev@dpdk.org>; Fri, 6 May 2022 05:57:09 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=Yc2wrZbtyyz1xN6jV4mTTteKDcU2Bsj7FN47PYs18WeeOFg+4hdGx2DBzcNhG1VEv/iLjb00l4CBQCihWuDzSIJMjsWzu9vv0968tUd2fXhcylfytQZtnaYYRxcxArTNQLB68Jm4RYVQKbXnrtbFlWcwfJkXM2/BTsfN9tsITOZ6JPfsNv2DR9hYNi48uKswiOA0JdnONY03iVAzyDL9mufOI6c2SyRwkTPvCYMBxEW1il9zaWJhhfNw3QGg/bjhFkiFzbgBU7vF0u5j1zYuGEB14nbZfMoqh8+SlgmKJh53bycfd57v4yPAGgKmcEEe9Xe+Np7A0l9Tprh0/T6NUg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=iKdEXYINEt2kBDlr2hwDdbmKFMUc+IBdawVCQf2l54k=;
b=TSRneZaY0e9tP3WkOUC3sD+a1V7V3T+/WC3K5pljAHSDpNufcks3Nj8xdWQIiI7cTdkYrkRZRveCN7zoDVtsGRVuvi0dxosPkgVks0aJeaqXg2RBTgz7C5rZyfY3JhqAViSPQRRtljH+CyGgOn4GJYB3JxbMZU/N89V84N43IyGHTuhAWtOjjBkYQTkrdafa1AvYjmCKc+q/NBHyp22igoyXDTyHvJU/MFGVLpSSTHbQD5NqD7h0uF/fLBrdC9CppYb9GPqoZF0IiuVb4xW/WpI2qADOjK1XNOzIVBvx5YWeJjbr9d7KodaSp2Yz5gN7cJUC5Vtxr+pN8gdfutQpWA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
12.22.5.234) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass
(p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none
(message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
s=selector2;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=iKdEXYINEt2kBDlr2hwDdbmKFMUc+IBdawVCQf2l54k=;
b=KMOG+gik0XY83PRC8E+ovDrpMKko78DgLwLR0Z+qYIE34u2EEyhaRfyarhaPWqBBF6LQvtJPl+1dieu84xHxz4WDjjo+0Iwv40zsvOdYUAQ/9PtLHZzVoQnKeV7puu4nca4VC1W/lTa8fc+x4oxSAtMDsySl0jqySgWaPDwCN7rH3SUioKhO6zH0q3jtE1m746mNJOup/Gj7Z2PNPitwsbx2djGuEg9I4mto/eMGYPqo9g+QnbgNCyT9md/vprCjeQ8vbDfsbYKiCNEzbjNdXJKBYokX8Iaw606MWrqCnBtevxuroQJnB/gWsTn8zstG5Ufucgq0uwh3kUKLP0B+DA==
Received: from BN0PR04CA0045.namprd04.prod.outlook.com (2603:10b6:408:e8::20)
by CY5PR12MB6274.namprd12.prod.outlook.com (2603:10b6:930:21::14)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.18; Fri, 6 May
2022 03:57:07 +0000
Received: from BN8NAM11FT014.eop-nam11.prod.protection.outlook.com
(2603:10b6:408:e8:cafe::c2) by BN0PR04CA0045.outlook.office365.com
(2603:10b6:408:e8::20) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13 via Frontend
Transport; Fri, 6 May 2022 03:57:07 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234)
smtp.mailfrom=nvidia.com; dkim=none (message not signed)
header.d=none;dmarc=pass action=none header.from=nvidia.com;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
12.22.5.234 as permitted sender) receiver=protection.outlook.com;
client-ip=12.22.5.234; helo=mail.nvidia.com;
Received: from mail.nvidia.com (12.22.5.234) by
BN8NAM11FT014.mail.protection.outlook.com (10.13.177.142) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id
15.20.5227.15 via Frontend Transport; Fri, 6 May 2022 03:57:07 +0000
Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com
(10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.32;
Fri, 6 May 2022 03:57:06 +0000
Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com
(10.129.68.8) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Thu, 5 May 2022
20:57:04 -0700
From: Spike Du <spiked@nvidia.com>
To: <matan@nvidia.com>, <viacheslavo@nvidia.com>, <orika@nvidia.com>,
<thomas@monjalon.net>
CC: <dev@dpdk.org>, <rasland@nvidia.com>
Subject: [RFC v1 2/7] common/mlx5: share interrupt management
Date: Fri, 6 May 2022 06:56:40 +0300
Message-ID: <20220506035645.4101714-3-spiked@nvidia.com>
X-Mailer: git-send-email 2.27.0
In-Reply-To: <20220506035645.4101714-1-spiked@nvidia.com>
References: <20220401032232.1267376-2-spiked@nvidia.com>
<20220506035645.4101714-1-spiked@nvidia.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Originating-IP: [10.126.231.35]
X-ClientProxiedBy: rnnvmail202.nvidia.com (10.129.68.7) To
rnnvmail201.nvidia.com (10.129.68.8)
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 10fc9c1f-75b9-4ed3-963a-08da2f147d58
X-MS-TrafficTypeDiagnostic: CY5PR12MB6274:EE_
X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
X-Microsoft-Antispam-PRVS: <CY5PR12MB627463FA9114AD7F800FC775A8C59@CY5PR12MB6274.namprd12.prod.outlook.com>
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: im6takUG5AYr93MyRs1VSyGJBTBCf4eRgOAurBQpc3Nf/9SMw6CF52crk0PPtVgjI67VMrSfS7UnyCeyG+DLf9T+jN7w7kOPtzWdsF+cL7GtBaCsFklahjyjKXp+BmqfJbeMn5qyf6socVOAetgh3q3uQ12ULeqrsNRaPisfW3TSqoNUn9GhvEwSIwLXo0+yorC/1zmbUZXQWPkexK0fIbHct7rUY8C2QDNMU9XklzP9yYpGwzCSVyQNTmE7OwF749181dcAxSHMzgNTRkPlHDWTGb4nfA6RYkyZVISovFBgIVLAb8tC1bYWCq3z9cKCknwuWLbpvL9lxivtxm1tz8JehqHlaDatPjxajsVTl7WRmKWqEOP0mkzBaJJ0U2YMcpuEjb/ef2QD22Bb29gZa4cOzpzNbjhwuwEZqVVym9cngb/90sH/OadxlsF8q8TW69jFFSy75iBMbU2sKPZRvAxybVaguvy5BwoYfX3jq3zU1+8ED+knU/u4A3Y9typaInXxusCi6zbtUNHP4Fxj+c7GBA1u+XEOxxIg/ILcd2zTO91+PubDmhJv2ZFWM4x2/U94e5ZwKMXf+smXJlTATJ/RqeCz5xN6t1Br2LXj83gKQzX/1ocbqZhl+vPErNcaUPjDM45qNrTFL8yJq24eBmXCbl4FS6CofBAMw01pNogA4vAXMGmTyJNHqhgeJu8OgwK69YwXYTCKffbUtZCxoA==
X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE;
SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(70586007)(70206006)(40460700003)(110136005)(508600001)(55016003)(4326008)(8676002)(6286002)(30864003)(82310400005)(2906002)(26005)(6666004)(54906003)(7696005)(316002)(83380400001)(36756003)(107886003)(336012)(86362001)(1076003)(47076005)(2616005)(186003)(16526019)(36860700001)(356005)(5660300002)(81166007)(426003)(8936002)(36900700001);
DIR:OUT; SFP:1101;
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 03:57:07.2811 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 10fc9c1f-75b9-4ed3-963a-08da2f147d58
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234];
Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT014.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6274
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
There are many duplicate code of creating and initializing rte_intr_handle.
Add a new mlx5_os API to do this, replace all PMD related code with this
API.
Signed-off-by: Spike Du <spiked@nvidia.com>
---
drivers/common/mlx5/linux/mlx5_common_os.c | 131 ++++++++++++++++++++++++++
drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++
drivers/common/mlx5/version.map | 3 +-
drivers/common/mlx5/windows/mlx5_common_os.h | 24 +++++
drivers/net/mlx5/linux/mlx5_ethdev_os.c | 71 --------------
drivers/net/mlx5/linux/mlx5_os.c | 132 ++++++---------------------
drivers/net/mlx5/linux/mlx5_socket.c | 53 ++---------
drivers/net/mlx5/mlx5.h | 2 -
drivers/net/mlx5/mlx5_txpp.c | 28 ++----
drivers/net/mlx5/windows/mlx5_ethdev_os.c | 22 -----
drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 52 ++---------
11 files changed, 217 insertions(+), 312 deletions(-)
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c
index d40cfd5..f10a981 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.c
+++ b/drivers/common/mlx5/linux/mlx5_common_os.c
@@ -11,6 +11,7 @@
#endif
#include <dirent.h>
#include <net/if.h>
+#include <fcntl.h>
#include <rte_errno.h>
#include <rte_string_fns.h>
@@ -964,3 +965,133 @@
claim_zero(mlx5_glue->dereg_mr(pmd_mr->obj));
memset(pmd_mr, 0, sizeof(*pmd_mr));
}
+
+/**
+ * Rte_intr_handle create and init helper.
+ *
+ * @param[in] mode
+ * interrupt instance can be shared between primary and secondary
+ * processes or not.
+ * @param[in] set_fd_nonblock
+ * Whether to set fd to O_NONBLOCK.
+ * @param[in] fd
+ * Fd to set in created intr_handle.
+ * @param[in] cb
+ * Callback to register for intr_handle.
+ * @param[in] cb_arg
+ * Callback argument for cb.
+ *
+ * @return
+ * - Interrupt handle on success.
+ * - NULL on failure, with rte_errno set.
+ */
+struct rte_intr_handle *
+mlx5_os_interrupt_handler_create(int mode, bool set_fd_nonblock, int fd,
+ rte_intr_callback_fn cb, void *cb_arg)
+{
+ struct rte_intr_handle *tmp_intr_handle;
+ int ret, flags;
+
+ tmp_intr_handle = rte_intr_instance_alloc(mode);
+ if (!tmp_intr_handle) {
+ rte_errno = ENOMEM;
+ goto err;
+ }
+ if (set_fd_nonblock) {
+ flags = fcntl(fd, F_GETFL);
+ ret = fcntl(fd, F_SETFL, flags | O_NONBLOCK);
+ if (ret) {
+ rte_errno = errno;
+ goto err;
+ }
+ }
+ ret = rte_intr_fd_set(tmp_intr_handle, fd);
+ if (ret)
+ goto err;
+ ret = rte_intr_type_set(tmp_intr_handle, RTE_INTR_HANDLE_EXT);
+ if (ret)
+ goto err;
+ ret = rte_intr_callback_register(tmp_intr_handle, cb, cb_arg);
+ if (ret) {
+ rte_errno = -ret;
+ goto err;
+ }
+ return tmp_intr_handle;
+err:
+ if (tmp_intr_handle)
+ rte_intr_instance_free(tmp_intr_handle);
+ return NULL;
+}
+
+/* Safe unregistration for interrupt callback. */
+static void
+mlx5_intr_callback_unregister(const struct rte_intr_handle *handle,
+ rte_intr_callback_fn cb_fn, void *cb_arg)
+{
+ uint64_t twait = 0;
+ uint64_t start = 0;
+
+ do {
+ int ret;
+
+ ret = rte_intr_callback_unregister(handle, cb_fn, cb_arg);
+ if (ret >= 0)
+ return;
+ if (ret != -EAGAIN) {
+ DRV_LOG(INFO, "failed to unregister interrupt"
+ " handler (error: %d)", ret);
+ MLX5_ASSERT(false);
+ return;
+ }
+ if (twait) {
+ struct timespec onems;
+
+ /* Wait one millisecond and try again. */
+ onems.tv_sec = 0;
+ onems.tv_nsec = NS_PER_S / MS_PER_S;
+ nanosleep(&onems, 0);
+ /* Check whether one second elapsed. */
+ if ((rte_get_timer_cycles() - start) <= twait)
+ continue;
+ } else {
+ /*
+ * We get the amount of timer ticks for one second.
+ * If this amount elapsed it means we spent one
+ * second in waiting. This branch is executed once
+ * on first iteration.
+ */
+ twait = rte_get_timer_hz();
+ MLX5_ASSERT(twait);
+ }
+ /*
+ * Timeout elapsed, show message (once a second) and retry.
+ * We have no other acceptable option here, if we ignore
+ * the unregistering return code the handler will not
+ * be unregistered, fd will be closed and we may get the
+ * crush. Hanging and messaging in the loop seems not to be
+ * the worst choice.
+ */
+ DRV_LOG(INFO, "Retrying to unregister interrupt handler");
+ start = rte_get_timer_cycles();
+ } while (true);
+}
+
+/**
+ * Rte_intr_handle destroy helper.
+ *
+ * @param[in] intr_handle
+ * Rte_intr_handle to destroy.
+ * @param[in] cb
+ * Callback which is registered to intr_handle.
+ * @param[in] cb_arg
+ * Callback argument for cb.
+ *
+ */
+void
+mlx5_os_interrupt_handler_destroy(struct rte_intr_handle *intr_handle,
+ rte_intr_callback_fn cb, void *cb_arg)
+{
+ if (rte_intr_fd_get(intr_handle) >= 0)
+ mlx5_intr_callback_unregister(intr_handle, cb, cb_arg);
+ rte_intr_instance_free(intr_handle);
+}
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 27f1192..479bb3c 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -15,6 +15,7 @@
#include <rte_log.h>
#include <rte_kvargs.h>
#include <rte_devargs.h>
+#include <rte_interrupts.h>
#include "mlx5_autoconf.h"
#include "mlx5_glue.h"
@@ -299,4 +300,14 @@
int
mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len);
+__rte_internal
+struct rte_intr_handle *
+mlx5_os_interrupt_handler_create(int mode, bool set_fd_nonblock, int fd,
+ rte_intr_callback_fn cb, void *cb_arg);
+
+__rte_internal
+void
+mlx5_os_interrupt_handler_destroy(struct rte_intr_handle *intr_handle,
+ rte_intr_callback_fn cb, void *cb_arg);
+
#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index a23a30a..2900544 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -152,6 +152,7 @@ INTERNAL {
mlx5_mp_req_mempool_reg;
mlx5_mr_mempool2mr_bh;
mlx5_mr_mempool_populate_cache;
-
+ mlx5_os_interrupt_handler_create; # WINDOWS_NO_EXPORT
+ mlx5_os_interrupt_handler_destroy; # WINDOWS_NO_EXPORT
local: *;
};
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index ee7973f..e9e9108 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -9,6 +9,7 @@
#include <sys/types.h>
#include <rte_errno.h>
+#include <rte_interrupts.h>
#include "mlx5_autoconf.h"
#include "mlx5_glue.h"
@@ -253,4 +254,27 @@
__rte_internal
int mlx5_os_umem_dereg(void *pumem);
+static inline struct rte_intr_handle *
+mlx5_os_interrupt_handler_create(int mode, bool set_fd_nonblock, int fd,
+ rte_intr_callback_fn cb, void *cb_arg)
+{
+ (void)mode;
+ (void)set_fd_nonblock;
+ (void)fd;
+ (void)cb;
+ (void)cb_arg;
+ rte_errno = ENOTSUP;
+ return NULL;
+}
+
+static inline void
+mlx5_os_interrupt_handler_destroy(struct rte_intr_handle *intr_handle,
+ rte_intr_callback_fn cb, void *cb_arg)
+{
+ (void)intr_handle;
+ (void)cb;
+ (void)cb_arg;
+}
+
+
#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index 8fe73f1..a276b2b 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -881,77 +881,6 @@ struct ethtool_link_settings {
}
}
-/*
- * Unregister callback handler safely. The handler may be active
- * while we are trying to unregister it, in this case code -EAGAIN
- * is returned by rte_intr_callback_unregister(). This routine checks
- * the return code and tries to unregister handler again.
- *
- * @param handle
- * interrupt handle
- * @param cb_fn
- * pointer to callback routine
- * @cb_arg
- * opaque callback parameter
- */
-void
-mlx5_intr_callback_unregister(const struct rte_intr_handle *handle,
- rte_intr_callback_fn cb_fn, void *cb_arg)
-{
- /*
- * Try to reduce timeout management overhead by not calling
- * the timer related routines on the first iteration. If the
- * unregistering succeeds on first call there will be no
- * timer calls at all.
- */
- uint64_t twait = 0;
- uint64_t start = 0;
-
- do {
- int ret;
-
- ret = rte_intr_callback_unregister(handle, cb_fn, cb_arg);
- if (ret >= 0)
- return;
- if (ret != -EAGAIN) {
- DRV_LOG(INFO, "failed to unregister interrupt"
- " handler (error: %d)", ret);
- MLX5_ASSERT(false);
- return;
- }
- if (twait) {
- struct timespec onems;
-
- /* Wait one millisecond and try again. */
- onems.tv_sec = 0;
- onems.tv_nsec = NS_PER_S / MS_PER_S;
- nanosleep(&onems, 0);
- /* Check whether one second elapsed. */
- if ((rte_get_timer_cycles() - start) <= twait)
- continue;
- } else {
- /*
- * We get the amount of timer ticks for one second.
- * If this amount elapsed it means we spent one
- * second in waiting. This branch is executed once
- * on first iteration.
- */
- twait = rte_get_timer_hz();
- MLX5_ASSERT(twait);
- }
- /*
- * Timeout elapsed, show message (once a second) and retry.
- * We have no other acceptable option here, if we ignore
- * the unregistering return code the handler will not
- * be unregistered, fd will be closed and we may get the
- * crush. Hanging and messaging in the loop seems not to be
- * the worst choice.
- */
- DRV_LOG(INFO, "Retrying to unregister interrupt handler");
- start = rte_get_timer_cycles();
- } while (true);
-}
-
/**
* Handle DEVX interrupts from the NIC.
* This function is probably called from the DPDK host thread.
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a821153..0741028 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2494,40 +2494,6 @@
mlx5_pmd_socket_uninit();
}
-static int
-mlx5_os_dev_shared_handler_install_lsc(struct mlx5_dev_ctx_shared *sh)
-{
- int nlsk_fd, flags, ret;
-
- nlsk_fd = mlx5_nl_init(NETLINK_ROUTE, RTMGRP_LINK);
- if (nlsk_fd < 0) {
- DRV_LOG(ERR, "Failed to create a socket for Netlink events: %s",
- rte_strerror(rte_errno));
- return -1;
- }
- flags = fcntl(nlsk_fd, F_GETFL);
- ret = fcntl(nlsk_fd, F_SETFL, flags | O_NONBLOCK);
- if (ret != 0) {
- DRV_LOG(ERR, "Failed to make Netlink event socket non-blocking: %s",
- strerror(errno));
- rte_errno = errno;
- goto error;
- }
- rte_intr_type_set(sh->intr_handle_nl, RTE_INTR_HANDLE_EXT);
- rte_intr_fd_set(sh->intr_handle_nl, nlsk_fd);
- if (rte_intr_callback_register(sh->intr_handle_nl,
- mlx5_dev_interrupt_handler_nl,
- sh) != 0) {
- DRV_LOG(ERR, "Failed to register Netlink events interrupt");
- rte_intr_fd_set(sh->intr_handle_nl, -1);
- goto error;
- }
- return 0;
-error:
- close(nlsk_fd);
- return -1;
-}
-
/**
* Install shared asynchronous device events handler.
* This function is implemented to support event sharing
@@ -2539,76 +2505,47 @@
void
mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
{
- int ret;
- int flags;
struct ibv_context *ctx = sh->cdev->ctx;
+ int nlsk_fd;
- sh->intr_handle = rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
- if (sh->intr_handle == NULL) {
- DRV_LOG(ERR, "Fail to allocate intr_handle");
- rte_errno = ENOMEM;
+ sh->intr_handle = mlx5_os_interrupt_handler_create
+ (RTE_INTR_INSTANCE_F_SHARED, true,
+ ctx->async_fd, mlx5_dev_interrupt_handler, sh);
+ if (!sh->intr_handle) {
+ DRV_LOG(ERR, "Failed to allocate intr_handle.");
return;
}
- rte_intr_fd_set(sh->intr_handle, -1);
-
- flags = fcntl(ctx->async_fd, F_GETFL);
- ret = fcntl(ctx->async_fd, F_SETFL, flags | O_NONBLOCK);
- if (ret) {
- DRV_LOG(INFO, "failed to change file descriptor async event"
- " queue");
- } else {
- rte_intr_fd_set(sh->intr_handle, ctx->async_fd);
- rte_intr_type_set(sh->intr_handle, RTE_INTR_HANDLE_EXT);
- if (rte_intr_callback_register(sh->intr_handle,
- mlx5_dev_interrupt_handler, sh)) {
- DRV_LOG(INFO, "Fail to install the shared interrupt.");
- rte_intr_fd_set(sh->intr_handle, -1);
- }
+ nlsk_fd = mlx5_nl_init(NETLINK_ROUTE, RTMGRP_LINK);
+ if (nlsk_fd < 0) {
+ DRV_LOG(ERR, "Failed to create a socket for Netlink events: %s",
+ rte_strerror(rte_errno));
+ return;
}
- sh->intr_handle_nl = rte_intr_instance_alloc
- (RTE_INTR_INSTANCE_F_SHARED);
+ sh->intr_handle_nl = mlx5_os_interrupt_handler_create
+ (RTE_INTR_INSTANCE_F_SHARED, true,
+ nlsk_fd, mlx5_dev_interrupt_handler_nl, sh);
if (sh->intr_handle_nl == NULL) {
DRV_LOG(ERR, "Fail to allocate intr_handle");
- rte_errno = ENOMEM;
return;
}
- rte_intr_fd_set(sh->intr_handle_nl, -1);
- if (mlx5_os_dev_shared_handler_install_lsc(sh) < 0) {
- DRV_LOG(INFO, "Fail to install the shared Netlink event handler.");
- rte_intr_fd_set(sh->intr_handle_nl, -1);
- }
if (sh->cdev->config.devx) {
#ifdef HAVE_IBV_DEVX_ASYNC
- sh->intr_handle_devx =
- rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
- if (!sh->intr_handle_devx) {
- DRV_LOG(ERR, "Fail to allocate intr_handle");
- rte_errno = ENOMEM;
- return;
- }
- rte_intr_fd_set(sh->intr_handle_devx, -1);
+ struct mlx5dv_devx_cmd_comp *devx_comp;
+
sh->devx_comp = (void *)mlx5_glue->devx_create_cmd_comp(ctx);
- struct mlx5dv_devx_cmd_comp *devx_comp = sh->devx_comp;
+ devx_comp = sh->devx_comp;
if (!devx_comp) {
DRV_LOG(INFO, "failed to allocate devx_comp.");
return;
}
- flags = fcntl(devx_comp->fd, F_GETFL);
- ret = fcntl(devx_comp->fd, F_SETFL, flags | O_NONBLOCK);
- if (ret) {
- DRV_LOG(INFO, "failed to change file descriptor"
- " devx comp");
+ sh->intr_handle_devx = mlx5_os_interrupt_handler_create
+ (RTE_INTR_INSTANCE_F_SHARED, true,
+ devx_comp->fd,
+ mlx5_dev_interrupt_handler_devx, sh);
+ if (!sh->intr_handle_devx) {
+ DRV_LOG(ERR, "Failed to allocate intr_handle.");
return;
}
- rte_intr_fd_set(sh->intr_handle_devx, devx_comp->fd);
- rte_intr_type_set(sh->intr_handle_devx,
- RTE_INTR_HANDLE_EXT);
- if (rte_intr_callback_register(sh->intr_handle_devx,
- mlx5_dev_interrupt_handler_devx, sh)) {
- DRV_LOG(INFO, "Fail to install the devx shared"
- " interrupt.");
- rte_intr_fd_set(sh->intr_handle_devx, -1);
- }
#endif /* HAVE_IBV_DEVX_ASYNC */
}
}
@@ -2624,24 +2561,13 @@
void
mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
{
- int nlsk_fd;
-
- if (rte_intr_fd_get(sh->intr_handle) >= 0)
- mlx5_intr_callback_unregister(sh->intr_handle,
- mlx5_dev_interrupt_handler, sh);
- rte_intr_instance_free(sh->intr_handle);
- nlsk_fd = rte_intr_fd_get(sh->intr_handle_nl);
- if (nlsk_fd >= 0) {
- mlx5_intr_callback_unregister
- (sh->intr_handle_nl, mlx5_dev_interrupt_handler_nl, sh);
- close(nlsk_fd);
- }
- rte_intr_instance_free(sh->intr_handle_nl);
+ mlx5_os_interrupt_handler_destroy(sh->intr_handle,
+ mlx5_dev_interrupt_handler, sh);
+ mlx5_os_interrupt_handler_destroy(sh->intr_handle_nl,
+ mlx5_dev_interrupt_handler_nl, sh);
#ifdef HAVE_IBV_DEVX_ASYNC
- if (rte_intr_fd_get(sh->intr_handle_devx) >= 0)
- rte_intr_callback_unregister(sh->intr_handle_devx,
- mlx5_dev_interrupt_handler_devx, sh);
- rte_intr_instance_free(sh->intr_handle_devx);
+ mlx5_os_interrupt_handler_destroy(sh->intr_handle_devx,
+ mlx5_dev_interrupt_handler_devx, sh);
if (sh->devx_comp)
mlx5_glue->devx_destroy_cmd_comp(sh->devx_comp);
#endif
diff --git a/drivers/net/mlx5/linux/mlx5_socket.c b/drivers/net/mlx5/linux/mlx5_socket.c
index 4882e5f..0e01aff 100644
--- a/drivers/net/mlx5/linux/mlx5_socket.c
+++ b/drivers/net/mlx5/linux/mlx5_socket.c
@@ -134,51 +134,6 @@
}
/**
- * Install interrupt handler.
- *
- * @param dev
- * Pointer to Ethernet device.
- * @return
- * 0 on success, a negative errno value otherwise.
- */
-static int
-mlx5_pmd_interrupt_handler_install(void)
-{
- MLX5_ASSERT(server_socket != -1);
-
- server_intr_handle =
- rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_PRIVATE);
- if (server_intr_handle == NULL) {
- DRV_LOG(ERR, "Fail to allocate intr_handle");
- return -ENOMEM;
- }
- if (rte_intr_fd_set(server_intr_handle, server_socket))
- return -rte_errno;
-
- if (rte_intr_type_set(server_intr_handle, RTE_INTR_HANDLE_EXT))
- return -rte_errno;
-
- return rte_intr_callback_register(server_intr_handle,
- mlx5_pmd_socket_handle, NULL);
-}
-
-/**
- * Uninstall interrupt handler.
- */
-static void
-mlx5_pmd_interrupt_handler_uninstall(void)
-{
- if (server_socket != -1) {
- mlx5_intr_callback_unregister(server_intr_handle,
- mlx5_pmd_socket_handle,
- NULL);
- }
- rte_intr_fd_set(server_intr_handle, 0);
- rte_intr_type_set(server_intr_handle, RTE_INTR_HANDLE_UNKNOWN);
- rte_intr_instance_free(server_intr_handle);
-}
-
-/**
* Initialise the socket to communicate with external tools.
*
* @return
@@ -224,7 +179,10 @@
strerror(errno));
goto remove;
}
- if (mlx5_pmd_interrupt_handler_install()) {
+ server_intr_handle = mlx5_os_interrupt_handler_create
+ (RTE_INTR_INSTANCE_F_PRIVATE, false,
+ server_socket, mlx5_pmd_socket_handle, NULL);
+ if (server_intr_handle == NULL) {
DRV_LOG(WARNING, "cannot register interrupt handler for mlx5 socket: %s",
strerror(errno));
goto remove;
@@ -248,7 +206,8 @@
{
if (server_socket == -1)
return;
- mlx5_pmd_interrupt_handler_uninstall();
+ mlx5_os_interrupt_handler_destroy(server_intr_handle,
+ mlx5_pmd_socket_handle, NULL);
claim_zero(close(server_socket));
server_socket = -1;
MKSTR(path, MLX5_SOCKET_PATH, getpid());
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index f3e6682..4821ff0 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1678,8 +1678,6 @@ int mlx5_sysfs_switch_info(unsigned int ifindex,
struct mlx5_switch_info *info);
void mlx5_translate_port_name(const char *port_name_in,
struct mlx5_switch_info *port_info_out);
-void mlx5_intr_callback_unregister(const struct rte_intr_handle *handle,
- rte_intr_callback_fn cb_fn, void *cb_arg);
int mlx5_sysfs_bond_info(unsigned int pf_ifindex, unsigned int *ifindex,
char *ifname);
int mlx5_get_module_info(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index fe74317..f853a67 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -741,11 +741,8 @@
static void
mlx5_txpp_stop_service(struct mlx5_dev_ctx_shared *sh)
{
- if (!rte_intr_fd_get(sh->txpp.intr_handle))
- return;
- mlx5_intr_callback_unregister(sh->txpp.intr_handle,
- mlx5_txpp_interrupt_handler, sh);
- rte_intr_instance_free(sh->txpp.intr_handle);
+ mlx5_os_interrupt_handler_destroy(sh->txpp.intr_handle,
+ mlx5_txpp_interrupt_handler, sh);
}
/* Attach interrupt handler and fires first request to Rearm Queue. */
@@ -769,23 +766,12 @@
rte_errno = errno;
return -rte_errno;
}
- sh->txpp.intr_handle =
- rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
- if (sh->txpp.intr_handle == NULL) {
- DRV_LOG(ERR, "Fail to allocate intr_handle");
- return -ENOMEM;
- }
fd = mlx5_os_get_devx_channel_fd(sh->txpp.echan);
- if (rte_intr_fd_set(sh->txpp.intr_handle, fd))
- return -rte_errno;
-
- if (rte_intr_type_set(sh->txpp.intr_handle, RTE_INTR_HANDLE_EXT))
- return -rte_errno;
-
- if (rte_intr_callback_register(sh->txpp.intr_handle,
- mlx5_txpp_interrupt_handler, sh)) {
- rte_intr_fd_set(sh->txpp.intr_handle, 0);
- DRV_LOG(ERR, "Failed to register CQE interrupt %d.", rte_errno);
+ sh->txpp.intr_handle = mlx5_os_interrupt_handler_create
+ (RTE_INTR_INSTANCE_F_SHARED, false,
+ fd, mlx5_txpp_interrupt_handler, sh);
+ if (!sh->txpp.intr_handle) {
+ DRV_LOG(ERR, "Fail to allocate intr_handle");
return -rte_errno;
}
/* Subscribe CQ event to the event channel controlled by the driver. */
diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index f975265..88d8213 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -140,28 +140,6 @@
return 0;
}
-/*
- * Unregister callback handler safely. The handler may be active
- * while we are trying to unregister it, in this case code -EAGAIN
- * is returned by rte_intr_callback_unregister(). This routine checks
- * the return code and tries to unregister handler again.
- *
- * @param handle
- * interrupt handle
- * @param cb_fn
- * pointer to callback routine
- * @cb_arg
- * opaque callback parameter
- */
-void
-mlx5_intr_callback_unregister(const struct rte_intr_handle *handle,
- rte_intr_callback_fn cb_fn, void *cb_arg)
-{
- RTE_SET_USED(handle);
- RTE_SET_USED(cb_fn);
- RTE_SET_USED(cb_arg);
-}
-
/**
* DPDK callback to get flow control status.
*
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
index 3416797..2ca48f5 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
@@ -59,26 +59,10 @@
mlx5_vdpa_virtq_unset(struct mlx5_vdpa_virtq *virtq)
{
unsigned int i;
- int retries = MLX5_VDPA_INTR_RETRIES;
- int ret = -EAGAIN;
-
- if (rte_intr_fd_get(virtq->intr_handle) != -1) {
- while (retries-- && ret == -EAGAIN) {
- ret = rte_intr_callback_unregister(virtq->intr_handle,
- mlx5_vdpa_virtq_handler,
- virtq);
- if (ret == -EAGAIN) {
- DRV_LOG(DEBUG, "Try again to unregister fd %d "
- "of virtq %d interrupt, retries = %d.",
- rte_intr_fd_get(virtq->intr_handle),
- (int)virtq->index, retries);
-
- usleep(MLX5_VDPA_INTR_RETRIES_USEC);
- }
- }
- rte_intr_fd_set(virtq->intr_handle, -1);
- }
- rte_intr_instance_free(virtq->intr_handle);
+ int ret;
+
+ mlx5_os_interrupt_handler_destroy(virtq->intr_handle,
+ mlx5_vdpa_virtq_handler, virtq);
if (virtq->virtq) {
ret = mlx5_vdpa_virtq_stop(virtq->priv, virtq->index);
if (ret)
@@ -342,35 +326,13 @@
virtq->priv = priv;
rte_write32(virtq->index, priv->virtq_db_addr);
/* Setup doorbell mapping. */
- virtq->intr_handle =
- rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
+ virtq->intr_handle = mlx5_os_interrupt_handler_create(
+ RTE_INTR_INSTANCE_F_SHARED, false,
+ vq.kickfd, mlx5_vdpa_virtq_handler, virtq);
if (virtq->intr_handle == NULL) {
DRV_LOG(ERR, "Fail to allocate intr_handle");
goto error;
}
-
- if (rte_intr_fd_set(virtq->intr_handle, vq.kickfd))
- goto error;
-
- if (rte_intr_fd_get(virtq->intr_handle) == -1) {
- DRV_LOG(WARNING, "Virtq %d kickfd is invalid.", index);
- } else {
- if (rte_intr_type_set(virtq->intr_handle, RTE_INTR_HANDLE_EXT))
- goto error;
-
- if (rte_intr_callback_register(virtq->intr_handle,
- mlx5_vdpa_virtq_handler,
- virtq)) {
- rte_intr_fd_set(virtq->intr_handle, -1);
- DRV_LOG(ERR, "Failed to register virtq %d interrupt.",
- index);
- goto error;
- } else {
- DRV_LOG(DEBUG, "Register fd %d interrupt for virtq %d.",
- rte_intr_fd_get(virtq->intr_handle),
- index);
- }
- }
/* Subscribe virtq error event. */
virtq->version++;
cookie = ((uint64_t)virtq->version << 32) + index;
--
1.8.3.1