From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B4FF346121; Fri, 24 Jan 2025 12:44:13 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F09ED42799; Fri, 24 Jan 2025 12:44:06 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2066.outbound.protection.outlook.com [40.107.244.66]) by mails.dpdk.org (Postfix) with ESMTP id 268E0410FA for ; Fri, 24 Jan 2025 12:44:05 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=W25Yn2KU28Gah1iAEIGDG/3zTtIsWIL0kvCXO/JFkUYestSQ8NtCzA1nMpjbWLebTfxgVMV6cm6adIeLCJL7VKkuIMDZvAapqUIBAgH/mOJihF4X3fI1l6eOjONMY/1Xps4YHhux264rIbh3Zi61UbPVMC2lj1vV++6NKrZ4iS8IQQX5bbMY6VUqeCv6c45zjNLw24/TxU9EJ3R9Ia7SRlDxojrGtzujuW1GIBjJPI84YJ7oSCcgl8KbZW/5LSG7KT5IhLDkdF9iVjbTLalnxvFw4WJRBBo4/ixKKHwAXz4upSz6lmNoyzfP7OtCHnK7C/jxc4ABJiqC/j8UWbIn/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yoDkJze73R3dBszgNtWqmw2cgOBfR55WuA4W0UjcRXQ=; b=wfOK1y/fPu0loGHSsr7PYE39CSPDW8+QfPPxj1l/75caEhYxcMqsbJiXM7RKCGyK03lHr/hfPNx/Y6AuXb4KIw4k6JDlJX8+iJkfaE+U5NykxIIuwjyhE5lf7gwjcWn8tf8S9pngzD2B+B2NQZoqIylmAa9H2/FB/iW6G7cJvOxNG+zole6j5S04wrrsaWEoT9S+iIxdhd775q62+P394KHYu7A2V6DIXuZXW8qfND1c9+mRESIoRs0xMQWgUZSnIjqKlTCTE/gTQBMbA8EmGYa5KCe6F4UtqIZ5iRbafShK1hZ9f+Bv238TZMocG/AufXGvwnR2EWo+2XfwW91Zcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yoDkJze73R3dBszgNtWqmw2cgOBfR55WuA4W0UjcRXQ=; b=pM8pxHWqAyy8HiWAZxCC6XBaLrCa4ECFHSvLKnTL2I4+zeuulHZsjxNsmprWm0IJxig4SHkKhtxyP8bWbAXw/CX7Rp0fRmYb69o6OEhWVv8DwQN/VAQ/yYzSNCk5J9ki6dbZjkPc/iUlOUpvin/CowoU/zxAu4EC2zvwYiXcNxF5i75wV/haEYy8kq8jXDdIZM98sg45KjccJOrQNs/twkFXogcVlfJcmHoQOfUMj6pMRh5uVR3HKRnkDSKlTJ6Z75gpP8X9nGnsS0icIHNgryeVh1n6NCN0iiVqYDg1UeZ6ReBTKIm87h69GMOHwN3q2/anUHvhsl9sxyBNtPg0Gg== Received: from MN2PR19CA0070.namprd19.prod.outlook.com (2603:10b6:208:19b::47) by LV8PR12MB9417.namprd12.prod.outlook.com (2603:10b6:408:204::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8377.17; Fri, 24 Jan 2025 11:43:59 +0000 Received: from BN2PEPF000044A2.namprd02.prod.outlook.com (2603:10b6:208:19b:cafe::bf) by MN2PR19CA0070.outlook.office365.com (2603:10b6:208:19b::47) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8293.15 via Frontend Transport; Fri, 24 Jan 2025 11:43:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044A2.mail.protection.outlook.com (10.167.243.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8377.8 via Frontend Transport; Fri, 24 Jan 2025 11:43:58 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 24 Jan 2025 03:43:51 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 24 Jan 2025 03:43:50 -0800 Received: from nvidia.com (10.127.8.9) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4 via Frontend Transport; Fri, 24 Jan 2025 03:43:48 -0800 From: Shani Peretz To: CC: , , Shani Peretz , Dariusz Sosnowski , "Viacheslav Ovsiienko" , Bing Zhao , Ori Kam , Suanming Mou , Matan Azrad , Anatoly Burakov Subject: [PATCH v2 3/3] net/mlx5: add port and queue level counters of hairpin drops Date: Fri, 24 Jan 2025 13:43:35 +0200 Message-ID: <20250124114336.16610-4-shperetz@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250124114336.16610-1-shperetz@nvidia.com> References: <20241222153819.62519-1-shperetz@nvidia.com> <20250124114336.16610-1-shperetz@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A2:EE_|LV8PR12MB9417:EE_ X-MS-Office365-Filtering-Correlation-Id: 40285c88-85e1-437d-d65e-08dd3c6c6432 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?TjhjEFyuBJuoKAWIDlybkhPyB/VmCllTf63wgenCI/0ciB9pzKe53fvMIGDm?= =?us-ascii?Q?9rKxNqLt1h7AhEhsHYU0OPhELSPtsEtlba+arLBEMDeJ9mtiZs3OxCxyPHQA?= =?us-ascii?Q?vez5PdZfgKdjr7130spn16QCECugwsQlZC95TP0+PyGhBOw+OzrT6PhUl032?= =?us-ascii?Q?mY3s/JVgndKUgle3OmwvRBr/82AmRsJKv+4qVtfkCyn7qah+zUT4uwJPmXbl?= =?us-ascii?Q?r72pfVr+Kjw9Xm/5BQg9QxBXoSF3amt+t5+hJAPT4aOuG39VmoWh62lXTAWY?= =?us-ascii?Q?JKvkk9fMtAk1JJbSRsapDwQQAXr/gbyeSCRQbWqoMVTB1UKLdtl6yUCMKZP5?= =?us-ascii?Q?C6mwUxIgNb8BPUPCkZ9ObonMknVP+/nUZ6lnoJJ73+s7840eaXFPZ5gqe90d?= =?us-ascii?Q?LT+5uWMhtdDvhd+YZm/ubFQ6HLQ2NAjAbNgwdSp4yBx2cYxqREgpkMaFbyNE?= =?us-ascii?Q?xwAGaJaMFEi6H9CJ0CqGNbpPmsnMAjzf7Mwbp8n6oFWCh0R5NZ7kdz55GPpn?= =?us-ascii?Q?cS2DFuZcy3r/BEATmroEJxgUMgj8q4rvdjFTtvJQkMTw9BW0GHgEHMchnl42?= =?us-ascii?Q?sQA9YGc8BCSLy5hXoeVvvO7HjLGDpILkKO4eDn5qLsfxQY4MEtehPCMhARWs?= =?us-ascii?Q?n7tLtpXQ+GPDRX/eHmYTtScNiFTk1yUJ12DdO0gEVa1JkQa7UwmGE/ROJYoi?= =?us-ascii?Q?rncPSXkPRr6MRKXm2Q90clzqgfmDl852vhc4MNOIbu4t05lP9x7H4buDEHBn?= =?us-ascii?Q?rvhyV7xQdp/2enFfhpAdlcIwVcFaOAgNXTwJTWE/c3gyXfD1UFJGgMK+p7qS?= =?us-ascii?Q?hqQIBm1bAiZ4RD+x+ilEq4chM8aL4YUBr1sRNsL/3j0pMTnrlpf+hkYKlxMw?= =?us-ascii?Q?wIYVvfB2QWLwQIOMFuWQEcudCHIOF7JOcTEI3yut9RjEhTXempQp8OCujRyy?= =?us-ascii?Q?3loRhjsU92eO0QW23eS7SbhYxgAdPtNYqiJDkPODsLB9fcS5SlZf9vulJ/c4?= =?us-ascii?Q?/MWynHWSsHitzUeVQz7gcuds7cfND+k4q7g6PG7CTGPGHjeARr+9xtphMflA?= =?us-ascii?Q?am/Eq5boq0VWDMmbaKsEoOZXV4M+3HjksL9tkqpjkw0wRUKNALrraul6K2So?= =?us-ascii?Q?FEYXYfVwxpyzgFoPDviLtqmEuTKatYomjt++P+FT11kco76Xy8iizKi/PFi3?= =?us-ascii?Q?g/L8WHE+42Hw7IB9br8pe4fVWzHx7bAEGnr+Nj19df8E0H11UtRMdoT396MA?= =?us-ascii?Q?/xM1CnTd4c6S1FLYmM1ABPJaPx3UKSt9dK+HtyFGg+BZZJDDcbwfIQDOTeVi?= =?us-ascii?Q?ncBJPk+gkPIedU4YdxTCXAhQFV5tiYV4csw4+cjRdBfiTSL7OeZ/PaAZxqLb?= =?us-ascii?Q?92sLr+NEqAL78LvN22wAMHdSgWT9wpNSts8bD52qQ42eUCDXgYA6iYWX5gKJ?= =?us-ascii?Q?p0a58ANXywBiuSp10fVfAFE/XMlhdWn2W9+kGlvAsmn1f3I6BljGtVYVeAvt?= =?us-ascii?Q?gVhwwXttZK79q4Y=3D?= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2025 11:43:58.9478 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40285c88-85e1-437d-d65e-08dd3c6c6432 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A2.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9417 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Currently the mlx5 pmd expose global hairpin counter (dev_internal_queue_oob) that tracks packets dropped in the device's hairpin rx queues. To offer more detailed insights, we are making the following counters available: - hairpin_out_of_buffer (port-level) counts drops from all the hairpin queues - hairpin_out_of_buffer_rxq* (queue-level) counts drops of a specific queue Both the port-level and per-queue counters can be enabled, disabled, and queried. Signed-off-by: Shani Peretz --- drivers/common/mlx5/mlx5_devx_cmds.c | 9 +- drivers/common/mlx5/mlx5_devx_cmds.h | 2 +- drivers/common/mlx5/mlx5_prm.h | 3 + drivers/net/mlx5/linux/mlx5_ethdev_os.c | 5 + drivers/net/mlx5/linux/mlx5_os.c | 21 +- drivers/net/mlx5/mlx5.c | 288 ++++++++++++++++++++++- drivers/net/mlx5/mlx5.h | 28 ++- drivers/net/mlx5/mlx5_devx.c | 75 ++---- drivers/net/mlx5/mlx5_rx.h | 20 ++ drivers/net/mlx5/mlx5_rxq.c | 36 +++ drivers/net/mlx5/mlx5_stats.c | 292 +++++++++++++++++++++++- drivers/net/mlx5/windows/mlx5_os.c | 3 +- 12 files changed, 691 insertions(+), 91 deletions(-) diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index a75f011750..f7ba4a87ca 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1590,6 +1590,7 @@ mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq, } ret = mlx5_glue->devx_obj_modify(rq->obj, in, sizeof(in), out, sizeof(out)); + if (ret) { DRV_LOG(ERR, "Failed to modify RQ using DevX"); rte_errno = errno; @@ -3091,14 +3092,17 @@ mlx5_devx_cmd_wq_query(void *wq, uint32_t *counter_set_id) * * @param[in] ctx * Context returned from mlx5 open_device() glue function. + * @param[out] syndrome + * Get syndrome of devx command response. * * @return * Pointer to counter object on success, a NULL value otherwise and * rte_errno is set. */ struct mlx5_devx_obj * -mlx5_devx_cmd_queue_counter_alloc(void *ctx) +mlx5_devx_cmd_queue_counter_alloc(void *ctx, int *syndrome) { + int status; struct mlx5_devx_obj *dcs = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*dcs), 0, SOCKET_ID_ANY); uint32_t in[MLX5_ST_SZ_DW(alloc_q_counter_in)] = {0}; @@ -3113,6 +3117,9 @@ mlx5_devx_cmd_queue_counter_alloc(void *ctx) sizeof(out)); if (!dcs->obj) { DEVX_DRV_LOG(DEBUG, out, "create q counter set", NULL, 0); + status = MLX5_GET(alloc_q_counter_out, out, status); + if (status && syndrome) + *syndrome = MLX5_GET(alloc_q_counter_out, out, syndrome); mlx5_free(dcs); return NULL; } diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index f523bf8529..38548b4c9f 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -848,7 +848,7 @@ __rte_internal int mlx5_devx_cmd_wq_query(void *wq, uint32_t *counter_set_id); __rte_internal -struct mlx5_devx_obj *mlx5_devx_cmd_queue_counter_alloc(void *ctx); +struct mlx5_devx_obj *mlx5_devx_cmd_queue_counter_alloc(void *ctx, int *syndrome); __rte_internal int mlx5_devx_cmd_queue_counter_query(struct mlx5_devx_obj *dcs, int clear, uint32_t *out_of_buffers); diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 0ecddaa36b..d0cb0131f6 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -275,6 +275,9 @@ #define MLX5_ERROR_CQE_SYNDROME_OFFSET 52 #endif +/* Firmware error code for allocating the maximum number of queue counters */ +#define MLX5_Q_COUNTERS_LIMIT_REACHED 0x587239 + /* The completion mode offset in the WQE control segment line 2. */ #define MLX5_COMP_MODE_OFFSET 2 diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c index 5d64984022..63277fc4ed 100644 --- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c +++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c @@ -1425,6 +1425,11 @@ static const struct mlx5_counter_ctrl mlx5_counters_init[] = { .dpdk_name = "hairpin_out_of_buffer", .ctr_name = "hairpin_out_of_buffer", .dev = 1, + .ctrl = { + .enable = mlx5_enable_port_level_hairpin_counter, + .disable = mlx5_disable_port_level_hairpin_counter, + .enabled = 0, + } }, { .dpdk_name = "dev_internal_queue_oob", diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 69a80b9ddc..0c63ab228e 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -973,7 +973,7 @@ mlx5_queue_counter_id_prepare(struct rte_eth_dev *dev) struct mlx5_priv *priv = dev->data->dev_private; void *ctx = priv->sh->cdev->ctx; - priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx); + priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx, NULL); if (!priv->q_counters) { struct ibv_cq *cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0); struct ibv_wq *wq; @@ -981,7 +981,6 @@ mlx5_queue_counter_id_prepare(struct rte_eth_dev *dev) DRV_LOG(DEBUG, "Port %d queue counter object cannot be created " "by DevX - fall-back to use the kernel driver global " "queue counter.", dev->data->port_id); - priv->q_counters_allocation_failure = 1; /* Create WQ by kernel and query its queue counter ID. */ if (cq) { @@ -3052,23 +3051,11 @@ mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name, if (priv->sh) { if (priv->q_counters != NULL && strcmp(ctr_name, "out_of_buffer") == 0) { - if (rte_eal_process_type() == RTE_PROC_SECONDARY) { - DRV_LOG(WARNING, "DevX out_of_buffer counter is not supported in the secondary process"); - rte_errno = ENOTSUP; - return 1; - } - return mlx5_devx_cmd_queue_counter_query - (priv->q_counters, 0, (uint32_t *)stat); + return mlx5_read_queue_counter(priv->q_counters, ctr_name, stat); } - if (priv->q_counters_hairpin != NULL && + if (priv->q_counter_hairpin != NULL && strcmp(ctr_name, "hairpin_out_of_buffer") == 0) { - if (rte_eal_process_type() == RTE_PROC_SECONDARY) { - DRV_LOG(WARNING, "DevX out_of_buffer counter is not supported in the secondary process"); - rte_errno = ENOTSUP; - return 1; - } - return mlx5_devx_cmd_queue_counter_query - (priv->q_counters_hairpin, 0, (uint32_t *)stat); + return mlx5_read_queue_counter(priv->q_counter_hairpin, ctr_name, stat); } MKSTR(path, "%s/ports/%d/hw_counters/%s", priv->sh->ibdev_path, diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 6e4473e2f4..3a9e642fa6 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2373,6 +2373,7 @@ mlx5_dev_close(struct rte_eth_dev *dev) priv->ptype_rss_groups = NULL; } #endif + mlx5_q_counters_destroy(dev); if (priv->rxq_privs != NULL) { /* XXX race condition if mlx5_rx_burst() is still running. */ rte_delay_us_sleep(1000); @@ -2393,14 +2394,6 @@ mlx5_dev_close(struct rte_eth_dev *dev) mlx5_proc_priv_uninit(dev); if (priv->drop_queue.hrxq) mlx5_drop_action_destroy(dev); - if (priv->q_counters) { - mlx5_devx_cmd_destroy(priv->q_counters); - priv->q_counters = NULL; - } - if (priv->q_counters_hairpin) { - mlx5_devx_cmd_destroy(priv->q_counters_hairpin); - priv->q_counters_hairpin = NULL; - } mlx5_mprq_free_mp(dev); mlx5_os_free_shared_dr(priv); #ifdef HAVE_MLX5_HWS_SUPPORT @@ -2520,6 +2513,11 @@ const struct eth_dev_ops mlx5_dev_ops = { .xstats_get = mlx5_xstats_get, .xstats_reset = mlx5_xstats_reset, .xstats_get_names = mlx5_xstats_get_names, + + .xstats_enable = mlx5_xstats_enable, + .xstats_disable = mlx5_xstats_disable, + .xstats_query_state = mlx5_xstats_query_state, + .fw_version_get = mlx5_fw_version_get, .dev_infos_get = mlx5_dev_infos_get, .representor_info_get = mlx5_representor_info_get, @@ -3393,6 +3391,280 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) return port_id; } +static int +mlx5_hairpin_queue_counter_supported(struct mlx5_priv *priv) +{ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { + DRV_LOG(WARNING, + "DevX counter is not supported in the secondary process"); + return -ENOTSUP; + } + + if (priv->obj_ops.rxq_obj_modify_counter_set_id == NULL) { + DRV_LOG(WARNING, + "DevX counter is not supported in this device"); + return -ENOTSUP; + } + return 0; +} + +/** + * Disables the port-level hairpin counter. + * + * This function iterates over each RXQ, detaches it from the global + * counter if it's a hairpin counter, and then destroys the global counter object if + * it exists. + * + * @param dev + * Pointer to Ethernet device. + * @param id + * The counter ID to disable (not used in this implementation). + * + * @return + * 0 on success, error code otherwise. + */ +int +mlx5_disable_port_level_hairpin_counter(struct rte_eth_dev *dev, uint64_t id __rte_unused) +{ + struct mlx5_priv *priv = dev->data->dev_private; + unsigned int num_rxqs = priv->rxqs_n; + unsigned int i; + + if (priv->q_counter_hairpin == NULL) + return 0; + + /* Detach each RXQ from the global hairpin counter */ + for (i = 0; i < num_rxqs; ++i) { + struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, i); + + if (rxq == NULL || rxq->ctrl->obj->rq == NULL || !rxq->ctrl->is_hairpin) + continue; + + if (priv->obj_ops.rxq_obj_modify_counter_set_id(rxq, 0) != 0) + DRV_LOG(ERR, "Port %u failed to modify rq object %s", + priv->dev_data->port_id, strerror(rte_errno)); + } + + mlx5_devx_cmd_destroy(priv->q_counter_hairpin); + priv->q_counter_hairpin = NULL; + + /* Reset oob stats. */ + mlx5_reset_xstats_by_name(priv, "hairpin_out_of_buffer"); + return 0; +} + +/** + * Enables the port-level hairpin counter. + * + * This function iterates over each RXQ, allocate a q counter and attach it to each + * hairpin queue. + * + * @param dev + * Pointer to Ethernet device. + * @param id + * The counter ID to disable (not used in this implementation). + * + * @return + * 0 on success, error code otherwise. + */ +int +mlx5_enable_port_level_hairpin_counter(struct rte_eth_dev *dev, uint64_t id __rte_unused) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_priv *rxq; + unsigned int i; + + int ret = mlx5_hairpin_queue_counter_supported(priv); + if (ret) { + DRV_LOG(DEBUG, "Hairpin out of buffer counter " + "is not available on this NIC."); + return ret; + } + + /* check if counter is enable per queue - if yes - fail to enable per port */ + if (priv->num_of_hairpin_q_counter_enabled != 0) { + DRV_LOG(WARNING, "Hairpin out of buffer counter is enabled per queue."); + return -EINVAL; + } + + /* Alloc global hairpin queue counter. */ + priv->q_counter_hairpin = mlx5_devx_cmd_queue_counter_alloc(priv->sh->cdev->ctx, NULL); + if (!priv->q_counter_hairpin) { + if (ret == MLX5_Q_COUNTERS_LIMIT_REACHED) { + DRV_LOG(WARNING, "Maximum number of queue counters reached. " + "Unable to create counter object for Port %d using DevX.", + priv->dev_data->port_id); + return -ENOSPC; + } + DRV_LOG(WARNING, "Port %d global hairpin queue counter object cannot be created " + "by DevX.", priv->dev_data->port_id); + return -ENOMEM; + } + + /* go over each queue and attach to global counter */ + for (i = 0; (i != priv->rxqs_n); ++i) { + rxq = mlx5_rxq_get(dev, i); + + if (rxq == NULL || rxq->ctrl->obj->rq == NULL || !rxq->ctrl->is_hairpin) + continue; + + ret = priv->obj_ops.rxq_obj_modify_counter_set_id(rxq, priv->q_counter_hairpin->id); + if (ret) { + DRV_LOG(ERR, "failed to modify rq object for port %u" + "%s", priv->dev_data->port_id, strerror(rte_errno)); + return ret; + } + } + + /* Reset oob stats. */ + mlx5_reset_xstats_by_name(priv, "hairpin_out_of_buffer"); + return 0; +} + +/** + * Creates a queue counter for hairpin Rx queue. + * + * @param dev + * Pointer to Ethernet device structure. + * @param id + * Index of the RX queue to disable the hairpin queue counter for. + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_enable_per_queue_hairpin_counter(struct rte_eth_dev *dev, uint64_t id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_priv *rxq; + struct mlx5_rxq_data *rxq_data; + + int ret = mlx5_hairpin_queue_counter_supported(priv); + if (ret) { + DRV_LOG(DEBUG, "Hairpin out of buffer counter " + "is not available on this NIC."); + return ret; + } + + /* check if we have port level counter enabled. if yes, don't set the queue level counter */ + if (priv->q_counter_hairpin) { + DRV_LOG(WARNING, "Hairpin out of buffer counter is enabled per port."); + return -EINVAL; + } + + rxq = mlx5_rxq_get(dev, id); + if (rxq == NULL || rxq->ctrl->obj->rq == NULL || !rxq->ctrl->is_hairpin) + return -EINVAL; + + if (rxq->q_counter != NULL) + return 0; + + /* Alloc hairpin queue counter. */ + rxq->q_counter = mlx5_devx_cmd_queue_counter_alloc(priv->sh->cdev->ctx, NULL); + if (rxq->q_counter == NULL) { + if (ret == MLX5_Q_COUNTERS_LIMIT_REACHED) { + DRV_LOG(WARNING, "Maximum number of queue counters reached. " + "Unable to create counter object for Port %d, Queue %d " + "using DevX. The counter from this queue will not increment.", + priv->dev_data->port_id, rxq->idx); + return -ENOSPC; + } + DRV_LOG(WARNING, "Port %d queue %d counter object cannot be created " + "by DevX. Counter from this queue will not increment.", + priv->dev_data->port_id, rxq->idx); + return -ENOMEM; + } + + ret = priv->obj_ops.rxq_obj_modify_counter_set_id(rxq, rxq->q_counter->id); + if (ret) { + DRV_LOG(ERR, "failed to modify rq object for port %u" + "%s", priv->dev_data->port_id, strerror(rte_errno)); + return ret; + } + + rxq_data = mlx5_rxq_data_get(dev, id); + if (rxq_data != NULL) + rxq_data->stats.oobs.ctrl.enabled = 1; + + priv->num_of_hairpin_q_counter_enabled++; + return 0; +} + +/** + * Disables the hairpin queue counter for a specified RX queue. + * + * @param dev + * Pointer to Ethernet device structure. + * @param id + * Index of the RX queue to disable the hairpin queue counter for. + * @return + * 0 on success, a negative errno value otherwise and rte_errno is set. + */ +int +mlx5_disable_per_queue_hairpin_counter(struct rte_eth_dev *dev, uint64_t id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_priv *rxq; + struct mlx5_rxq_data *rxq_data; + int ret = 0; + + rxq = mlx5_rxq_get(dev, id); + rxq_data = mlx5_rxq_data_get(dev, id); + + if (rxq == NULL || rxq->ctrl->obj->rq == NULL || !rxq->ctrl->is_hairpin) + return 0; + + if (rxq->q_counter != NULL) { + /* Modify rxq. */ + ret = priv->obj_ops.rxq_obj_modify_counter_set_id(rxq, 0); + if (ret) + DRV_LOG(ERR, "Port %u failed to modify rq object " + " %s", priv->dev_data->port_id, strerror(rte_errno)); + + mlx5_devx_cmd_destroy(rxq->q_counter); + rxq->q_counter = NULL; + } + + /* Reset queue oob stats. */ + if (rxq_data != NULL) { + rxq_data->stats.oobs.count = 0; + rxq_data->stats_reset.oobs.count = 0; + rxq_data->stats.oobs.ctrl.enabled = 0; + } + + priv->num_of_hairpin_q_counter_enabled--; + return 0; +} + +/** + * Read statistics per queue by a named counter. + * + * @param[in] q_counter + * Pointer to the queue's counter object. + * @param[in] ctr_name + * Pointer to the name of the statistic counter to read + * @param[out] stat + * Pointer to read statistic value. + * @return + * 0 on success and stat is valid, 1 if failed to read the value + * rte_errno is set. + * + */ +int +mlx5_read_queue_counter(struct mlx5_devx_obj *q_counter, const char *ctr_name, + uint64_t *stat) +{ + if (rte_eal_process_type() == RTE_PROC_SECONDARY) { + DRV_LOG(WARNING, + "DevX %s counter is not supported in the secondary process", ctr_name); + return -ENOTSUP; + } + + if (q_counter == NULL) + return -EINVAL; + + return mlx5_devx_cmd_queue_counter_query(q_counter, 0, (uint32_t *)stat); +} + /** * Callback to remove a device. * diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index cc8a03ccd6..df9a387433 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -261,17 +261,31 @@ struct mlx5_local_data { extern struct mlx5_shared_data *mlx5_shared_data; +int mlx5_xstats_enable(struct rte_eth_dev *dev, uint64_t id); +int mlx5_xstats_disable(struct rte_eth_dev *dev, uint64_t id); +int mlx5_xstats_query_state(struct rte_eth_dev *dev, uint64_t id); + /* Dev ops structs */ extern const struct eth_dev_ops mlx5_dev_ops; extern const struct eth_dev_ops mlx5_dev_sec_ops; extern const struct eth_dev_ops mlx5_dev_ops_isolate; +typedef int (*mlx5_enable_counter_t)(struct rte_eth_dev *dev, uint64_t id); +typedef int (*mlx5_disable_counter_t)(struct rte_eth_dev *dev, uint64_t id); + +struct mlx5_stat_counter_ctrl { + mlx5_enable_counter_t enable; + mlx5_disable_counter_t disable; + uint32_t enabled; +}; + struct mlx5_counter_ctrl { /* Name of the counter. */ char dpdk_name[RTE_ETH_XSTATS_NAME_SIZE]; /* Name of the counter on the device table. */ char ctr_name[RTE_ETH_XSTATS_NAME_SIZE]; uint32_t dev:1; /**< Nonzero for dev counters. */ + struct mlx5_stat_counter_ctrl ctrl; }; struct mlx5_xstats_ctrl { @@ -1783,6 +1797,7 @@ struct mlx5_priv; /* HW objects operations structure. */ struct mlx5_obj_ops { int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_priv *rxq, int on); + int (*rxq_obj_modify_counter_set_id)(struct mlx5_rxq_priv *rxq, uint32_t counter_set_id); int (*rxq_obj_new)(struct mlx5_rxq_priv *rxq); int (*rxq_event_get)(struct mlx5_rxq_obj *rxq_obj); int (*rxq_obj_modify)(struct mlx5_rxq_priv *rxq, uint8_t type); @@ -2044,12 +2059,12 @@ struct mlx5_priv { LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */ rte_spinlock_t shared_act_sl; /* Shared actions spinlock. */ uint32_t rss_shared_actions; /* RSS shared actions. */ - /* If true, indicates that we failed to allocate a q counter in the past. */ - bool q_counters_allocation_failure; + /**< Total number of hairpin queues attach to q counters. */ + uint64_t num_of_hairpin_q_counter_enabled; struct mlx5_devx_obj *q_counters; /* DevX queue counter object. */ uint32_t counter_set_id; /* Queue counter ID to set in DevX objects. */ /* DevX queue counter object for all hairpin queues of the port. */ - struct mlx5_devx_obj *q_counters_hairpin; + struct mlx5_devx_obj *q_counter_hairpin; uint32_t lag_affinity_idx; /* LAG mode queue 0 affinity starting. */ rte_spinlock_t flex_item_sl; /* Flex item list spinlock. */ struct mlx5_flex_item flex_item[MLX5_PORT_FLEX_ITEM_NUM]; @@ -2224,6 +2239,10 @@ bool mlx5_is_sf_repr(struct rte_eth_dev *dev); void mlx5_age_event_prepare(struct mlx5_dev_ctx_shared *sh); int mlx5_lwm_setup(struct mlx5_priv *priv); void mlx5_lwm_unset(struct mlx5_dev_ctx_shared *sh); +int mlx5_enable_port_level_hairpin_counter(struct rte_eth_dev *dev, uint64_t id); +int mlx5_disable_port_level_hairpin_counter(struct rte_eth_dev *dev, uint64_t id); +int mlx5_enable_per_queue_hairpin_counter(struct rte_eth_dev *dev, uint64_t id); +int mlx5_disable_per_queue_hairpin_counter(struct rte_eth_dev *dev, uint64_t id); /* Macro to iterate over all valid ports for mlx5 driver. */ #define MLX5_ETH_FOREACH_DEV(port_id, dev) \ @@ -2257,6 +2276,7 @@ int mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh); struct mlx5_physical_device * mlx5_get_locked_physical_device(struct mlx5_priv *priv); void mlx5_unlock_physical_device(void); +int mlx5_read_queue_counter(struct mlx5_devx_obj *q_counter, const char *ctr_name, uint64_t *stat); /* mlx5_ethdev.c */ @@ -2364,6 +2384,8 @@ int mlx5_xstats_reset(struct rte_eth_dev *dev); int mlx5_xstats_get_names(struct rte_eth_dev *dev __rte_unused, struct rte_eth_xstat_name *xstats_names, unsigned int n); +void mlx5_reset_xstats_by_name(struct mlx5_priv *priv, const char *ctr_name); +void mlx5_reset_xstats_rq(struct rte_eth_dev *dev); /* mlx5_vlan.c */ diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 8ebe784000..a12891a983 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -91,6 +91,30 @@ mlx5_rxq_obj_modify_rq_vlan_strip(struct mlx5_rxq_priv *rxq, int on) return mlx5_devx_cmd_modify_rq(rxq->devx_rq.rq, &rq_attr); } +/** + * Modify the q counter of a given RQ + * + * @param rxq + * Rx queue. + * @param counter_set_id + * Q counter id to set + * + * @return + * 0 on success, non-0 otherwise + */ +static int +mlx5_rxq_obj_modify_counter(struct mlx5_rxq_priv *rxq, uint32_t counter_set_id) +{ + struct mlx5_devx_modify_rq_attr rq_attr; + + memset(&rq_attr, 0, sizeof(rq_attr)); + rq_attr.rq_state = MLX5_RQC_STATE_RDY; + rq_attr.state = MLX5_RQC_STATE_RDY; + rq_attr.counter_set_id = counter_set_id; + rq_attr.modify_bitmask = MLX5_MODIFY_RQ_IN_MODIFY_BITMASK_RQ_COUNTER_SET_ID; + return mlx5_devx_cmd_modify_rq(rxq->ctrl->obj->rq, &rq_attr); +} + /** * Modify RQ using DevX API. * @@ -496,55 +520,6 @@ mlx5_rxq_create_devx_cq_resources(struct mlx5_rxq_priv *rxq) return 0; } -/** - * Create a global queue counter for all the port hairpin queues. - * - * @param priv - * Device private data. - * - * @return - * The counter_set_id of the queue counter object, 0 otherwise. - */ -static uint32_t -mlx5_set_hairpin_queue_counter_obj(struct mlx5_priv *priv) -{ - if (priv->q_counters_hairpin != NULL) - return priv->q_counters_hairpin->id; - - /* Queue counter allocation failed in the past - don't try again. */ - if (priv->q_counters_allocation_failure != 0) - return 0; - - if (priv->pci_dev == NULL) { - DRV_LOG(DEBUG, "Hairpin out of buffer counter is " - "only supported on PCI device."); - priv->q_counters_allocation_failure = 1; - return 0; - } - - switch (priv->pci_dev->id.device_id) { - /* Counting out of buffer drops on hairpin queues is supported only on CX7 and up. */ - case PCI_DEVICE_ID_MELLANOX_CONNECTX7: - case PCI_DEVICE_ID_MELLANOX_CONNECTXVF: - case PCI_DEVICE_ID_MELLANOX_BLUEFIELD3: - case PCI_DEVICE_ID_MELLANOX_BLUEFIELDVF: - - priv->q_counters_hairpin = mlx5_devx_cmd_queue_counter_alloc(priv->sh->cdev->ctx); - if (priv->q_counters_hairpin == NULL) { - /* Failed to allocate */ - DRV_LOG(DEBUG, "Some of the statistics of port %d " - "will not be available.", priv->dev_data->port_id); - priv->q_counters_allocation_failure = 1; - return 0; - } - return priv->q_counters_hairpin->id; - default: - DRV_LOG(DEBUG, "Hairpin out of buffer counter " - "is not available on this NIC."); - priv->q_counters_allocation_failure = 1; - return 0; - } -} /** * Create the Rx hairpin queue object. @@ -592,7 +567,6 @@ mlx5_rxq_obj_hairpin_new(struct mlx5_rxq_priv *rxq) unlocked_attr.wq_attr.log_hairpin_data_sz - MLX5_HAIRPIN_QUEUE_STRIDE; - unlocked_attr.counter_set_id = mlx5_set_hairpin_queue_counter_obj(priv); rxq_ctrl->rxq.delay_drop = priv->config.hp_delay_drop; unlocked_attr.delay_drop_en = priv->config.hp_delay_drop; @@ -1710,6 +1684,7 @@ mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj) struct mlx5_obj_ops devx_obj_ops = { .rxq_obj_modify_vlan_strip = mlx5_rxq_obj_modify_rq_vlan_strip, + .rxq_obj_modify_counter_set_id = mlx5_rxq_obj_modify_counter, .rxq_obj_new = mlx5_rxq_devx_obj_new, .rxq_event_get = mlx5_rx_devx_get_event, .rxq_obj_modify = mlx5_devx_modify_rq, diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 1a6f174c40..f80a2e3227 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -30,6 +30,12 @@ /* First entry must be NULL for comparison. */ #define mlx5_mr_btree_len(bt) ((bt)->len - 1) +struct mlx5_rxq_stat { + int id; + uint64_t count; + struct mlx5_stat_counter_ctrl ctrl; +}; + struct mlx5_rxq_stats { #ifdef MLX5_PMD_SOFT_COUNTERS uint64_t ipackets; /**< Total of successfully received packets. */ @@ -37,6 +43,18 @@ struct mlx5_rxq_stats { #endif uint64_t idropped; /**< Total of packets dropped when RX ring full. */ uint64_t rx_nombuf; /**< Total of RX mbuf allocation failures. */ + struct mlx5_rxq_stat oobs; /**< Total of hairpin queue out of buffers. */ +}; + +/* store statistics names and its offset in stats structure */ +struct mlx5_xstats_name_off { + char name[RTE_ETH_XSTATS_NAME_SIZE]; + unsigned int offset; +}; + +struct mlx5_rq_stats { + /** Total number of hairpin queue packets received that are dropped. */ + uint64_t q_oobs[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; /* Compressed CQE context. */ @@ -183,6 +201,7 @@ struct mlx5_rxq_priv { uint32_t lwm:16; uint32_t lwm_event_pending:1; uint32_t lwm_devx_subscribed:1; + struct mlx5_devx_obj *q_counter; /* DevX hairpin queue counter object. */ }; /* mlx5_rxq.c */ @@ -208,6 +227,7 @@ void mlx5_rx_intr_vec_disable(struct rte_eth_dev *dev); int mlx5_rx_intr_enable(struct rte_eth_dev *dev, uint16_t rx_queue_id); int mlx5_rx_intr_disable(struct rte_eth_dev *dev, uint16_t rx_queue_id); int mlx5_rxq_obj_verify(struct rte_eth_dev *dev); +void mlx5_q_counters_destroy(struct rte_eth_dev *dev); struct mlx5_rxq_ctrl *mlx5_rxq_new(struct rte_eth_dev *dev, uint16_t idx, uint16_t desc, unsigned int socket, const struct rte_eth_rxconf *conf, diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 126b1970e6..a5971b5cdd 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1314,6 +1314,42 @@ mlx5_rxq_obj_verify(struct rte_eth_dev *dev) return ret; } +/** + * Destroy all queue counters. + * + * @param dev + * Pointer to Ethernet device. + */ +void +mlx5_q_counters_destroy(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + unsigned int i; + + /* Destroy port q counter */ + if (priv->q_counters) { + mlx5_devx_cmd_destroy(priv->q_counters); + priv->q_counters = NULL; + } + + /* Destroy port global hairpin q counter */ + if (priv->q_counter_hairpin) { + mlx5_devx_cmd_destroy(priv->q_counter_hairpin); + priv->q_counter_hairpin = NULL; + } + + /* Destroy per hairpin queue counter */ + for (i = 0; i != priv->rxqs_n; ++i) { + struct mlx5_rxq_priv *rxq = mlx5_rxq_get(dev, i); + + if (rxq == NULL || rxq->q_counter == NULL) + continue; + + mlx5_devx_cmd_destroy(rxq->q_counter); + rxq->q_counter = NULL; + } +} + /** * Callback function to initialize mbufs for Multi-Packet RQ. */ diff --git a/drivers/net/mlx5/mlx5_stats.c b/drivers/net/mlx5/mlx5_stats.c index f4ac58e2f9..ebc6f9fb53 100644 --- a/drivers/net/mlx5/mlx5_stats.c +++ b/drivers/net/mlx5/mlx5_stats.c @@ -20,6 +20,107 @@ #include "mlx5_tx.h" #include "mlx5_malloc.h" + +static const struct mlx5_xstats_name_off mlx5_rxq_stats_strings[] = { + {"out_of_buffer", offsetof(struct mlx5_rq_stats, q_oobs)}, +}; + +#define NB_RXQ_STATS RTE_DIM(mlx5_rxq_stats_strings) + +/** + * Retrieve extended device statistics + * for Rx queues. It appends the specific statistics + * before the parts filled by preceding modules (eth stats, etc.) + * + * @param dev + * Pointer to Ethernet device. + * @param[out] stats + * Pointer to an array to store the retrieved statistics. + * @return + * Number of extended stats is filled, + * negative on error and rte_errno is set. + */ +static int +mlx5_rq_xstats_get(struct rte_eth_dev *dev, + struct rte_eth_xstat *stats) +{ + uint16_t n_stats_rq = RTE_MIN(dev->data->nb_rx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS); + int cnt_used_entries = 0; + + for (unsigned int idx = 0; idx < n_stats_rq; idx++) { + struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, idx); + struct mlx5_rxq_priv *rxq_priv = mlx5_rxq_get(dev, idx); + + if (rxq_data == NULL) + continue; + + struct mlx5_rxq_stat *rxq_stat = &rxq_data->stats.oobs; + if (rxq_stat == NULL) + continue; + + /* Handle initial stats setup - Flag uninitialized stat */ + rxq_stat->id = -1; + + /* Handle hairpin statistics */ + if (rxq_priv && rxq_priv->ctrl->is_hairpin) { + if (stats) { + mlx5_read_queue_counter(rxq_priv->q_counter, "hairpin_out_of_buffer", + &rxq_stat->count); + + stats[cnt_used_entries].id = cnt_used_entries; + stats[cnt_used_entries].value = rxq_stat->count - + rxq_data->stats_reset.oobs.count; + } + rxq_stat->ctrl.enable = mlx5_enable_per_queue_hairpin_counter; + rxq_stat->ctrl.disable = mlx5_disable_per_queue_hairpin_counter; + rxq_stat->id = cnt_used_entries; + cnt_used_entries++; + } + } + return cnt_used_entries; +} + +/** + * Retrieve names of extended device statistics + * for Rx queues. It appends the specific stats names + * before the parts filled by preceding modules (eth stats, etc.) + * + * @param dev + * Pointer to Ethernet device structure. + * @param[out] xstats_names + * Buffer to insert names into. + * + * @return + * Number of xstats names. + */ +static int +mlx5_rq_xstats_get_names(struct rte_eth_dev *dev __rte_unused, + struct rte_eth_xstat_name *xstats_names) +{ + struct mlx5_rxq_priv *rxq; + unsigned int i; + int cnt_used_entries = 0; + + uint16_t n_stats_rq = RTE_MIN(dev->data->nb_rx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS); + + for (i = 0; (i != n_stats_rq); ++i) { + rxq = mlx5_rxq_get(dev, i); + + if (rxq == NULL) + continue; + + if (rxq->ctrl->is_hairpin) { + if (xstats_names) + snprintf(xstats_names[cnt_used_entries].name, + sizeof(xstats_names[0].name), + "hairpin_%s_rxq%u", + mlx5_rxq_stats_strings[0].name, i); + cnt_used_entries++; + } + } + return cnt_used_entries; +} + /** * DPDK callback to get extended device statistics. * @@ -46,6 +147,7 @@ mlx5_xstats_get(struct rte_eth_dev *dev, struct rte_eth_xstat *stats, uint16_t stats_n_2nd = 0; uint16_t mlx5_stats_n = xstats_ctrl->mlx5_stats_n; bool bond_master = (priv->master && priv->pf_bond >= 0); + int n_used = mlx5_rq_xstats_get(dev, stats); if (n >= mlx5_stats_n && stats) { int ret; @@ -69,27 +171,27 @@ mlx5_xstats_get(struct rte_eth_dev *dev, struct rte_eth_xstat *stats, if (ret < 0) return ret; for (i = 0; i != mlx5_stats_n; i++) { - stats[i].id = i; + stats[i + n_used].id = i + n_used; if (xstats_ctrl->info[i].dev) { uint64_t wrap_n; uint64_t hw_stat = xstats_ctrl->hw_stats[i]; - stats[i].value = (counters[i] - + stats[i + n_used].value = (counters[i] - xstats_ctrl->base[i]) & (uint64_t)UINT32_MAX; wrap_n = hw_stat >> 32; - if (stats[i].value < + if (stats[i + n_used].value < (hw_stat & (uint64_t)UINT32_MAX)) wrap_n++; - stats[i].value |= (wrap_n) << 32; - xstats_ctrl->hw_stats[i] = stats[i].value; + stats[i + n_used].value |= (wrap_n) << 32; + xstats_ctrl->hw_stats[i] = stats[i + n_used].value; } else { - stats[i].value = + stats[i + n_used].value = (counters[i] - xstats_ctrl->base[i]); } } } - mlx5_stats_n = mlx5_txpp_xstats_get(dev, stats, n, mlx5_stats_n); + mlx5_stats_n = mlx5_txpp_xstats_get(dev, stats, n, mlx5_stats_n + n_used); return mlx5_stats_n; } @@ -273,11 +375,58 @@ mlx5_xstats_reset(struct rte_eth_dev *dev) xstats_ctrl->base[i] = counters[i]; xstats_ctrl->hw_stats[i] = 0; } + mlx5_reset_xstats_rq(dev); mlx5_txpp_xstats_reset(dev); mlx5_free(counters); return 0; } +void +mlx5_reset_xstats_by_name(struct mlx5_priv *priv, const char *ctr_name) +{ + struct mlx5_xstats_ctrl *xstats_ctrl = &priv->xstats_ctrl; + unsigned int mlx5_xstats_n = xstats_ctrl->mlx5_stats_n; + unsigned int i; + + for (i = 0; i != mlx5_xstats_n; ++i) { + if (strcmp(xstats_ctrl->info[i].ctr_name, ctr_name) == 0) { + xstats_ctrl->base[i] = 0; + xstats_ctrl->hw_stats[i] = 0; + xstats_ctrl->xstats[i] = 0; + return; + } + } +} + +/** + * Clear device extended statistics for each Rx queue. + * + * @param dev + * Pointer to Ethernet device structure. + */ +void +mlx5_reset_xstats_rq(struct rte_eth_dev *dev) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_rxq_priv *rxq; + struct mlx5_rxq_data *rxq_data; + unsigned int i; + + for (i = 0; (i != priv->rxqs_n); ++i) { + rxq = mlx5_rxq_get(dev, i); + rxq_data = mlx5_rxq_data_get(dev, i); + + if (rxq == NULL || rxq_data == NULL || rxq->q_counter == NULL) + continue; + if (rxq->ctrl->is_hairpin) + mlx5_read_queue_counter(rxq->q_counter, + "hairpin_out_of_buffer", &rxq_data->stats_reset.oobs.count); + else + mlx5_read_queue_counter(rxq->q_counter, + "out_of_buffer", &rxq_data->stats_reset.oobs.count); + } +} + /** * DPDK callback to retrieve names of extended device statistics * @@ -299,15 +448,140 @@ mlx5_xstats_get_names(struct rte_eth_dev *dev, struct mlx5_priv *priv = dev->data->dev_private; struct mlx5_xstats_ctrl *xstats_ctrl = &priv->xstats_ctrl; unsigned int mlx5_xstats_n = xstats_ctrl->mlx5_stats_n; + unsigned int n_used = mlx5_rq_xstats_get_names(dev, xstats_names); if (n >= mlx5_xstats_n && xstats_names) { for (i = 0; i != mlx5_xstats_n; ++i) { - strlcpy(xstats_names[i].name, + rte_strscpy(xstats_names[i + n_used].name, xstats_ctrl->info[i].dpdk_name, RTE_ETH_XSTATS_NAME_SIZE); + xstats_names[i + n_used].name[RTE_ETH_XSTATS_NAME_SIZE - 1] = 0; } } mlx5_xstats_n = mlx5_txpp_xstats_get_names(dev, xstats_names, - n, mlx5_xstats_n); + n, mlx5_xstats_n + n_used); return mlx5_xstats_n; } + +static struct mlx5_stat_counter_ctrl* +mlx5_rxq_get_counter_by_id(struct rte_eth_dev *dev, uint64_t id, uint64_t *rq_id) +{ + uint16_t n_stats_rq = RTE_MIN(dev->data->nb_rx_queues, RTE_ETHDEV_QUEUE_STAT_CNTRS); + + for (int i = 0; (i != n_stats_rq); i++) { + struct mlx5_rxq_data *rxq_data = mlx5_rxq_data_get(dev, i); + if (rxq_data == NULL || rxq_data->stats.oobs.id == -1) + continue; + + if ((uint64_t)rxq_data->stats.oobs.id == id) { + *rq_id = rxq_data->idx; + return &rxq_data->stats.oobs.ctrl; + } + } + + return NULL; +} + +/** + * Callback to enable an xstat counter of the given id. + * + * @param dev + * Pointer to Ethernet device structure. + * @param id + * The ID of the counter to enable + * + * @return + * 1 xstat is enabled, 0 if xstat is disabled, + * -ENOTSUP if enabling/disabling is not implemented and -EINVAL if xstat id is invalid. + */ +int +mlx5_xstats_enable(struct rte_eth_dev *dev, uint64_t id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_xstats_ctrl *xstats_ctrl = &priv->xstats_ctrl; + struct mlx5_stat_counter_ctrl *counter_ctrl = NULL; + uint16_t n_stats_rq = mlx5_rq_xstats_get(dev, NULL); + + if (id < n_stats_rq) + counter_ctrl = mlx5_rxq_get_counter_by_id(dev, id, &id); + else + counter_ctrl = &xstats_ctrl->info[id - n_stats_rq].ctrl; + + if (counter_ctrl == NULL) + return -EINVAL; + + if (counter_ctrl->enable == NULL) + return -ENOTSUP; + + counter_ctrl->enabled = counter_ctrl->enable(dev, id) == 0 ? 1 : 0; + return counter_ctrl->enabled; +} + +/** + * Callback to disable an xstat counter of the given id. + * + * @param dev + * Pointer to Ethernet device structure. + * @param id + * The ID of the counter to enable + * + * @return + * 1 if xstat is disabled, 0 xstat is enabled, + * -ENOTSUP if enabling/disabling is not implemented and -EINVAL if xstat id is invalid. + */ +int +mlx5_xstats_disable(struct rte_eth_dev *dev, uint64_t id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_xstats_ctrl *xstats_ctrl = &priv->xstats_ctrl; + struct mlx5_stat_counter_ctrl *counter_ctrl = NULL; + + uint16_t n_stats_rq = mlx5_rq_xstats_get(dev, NULL); + if (id < n_stats_rq) + counter_ctrl = mlx5_rxq_get_counter_by_id(dev, id, &id); + else + counter_ctrl = &xstats_ctrl->info[id - n_stats_rq].ctrl; + + if (counter_ctrl == NULL) + return -EINVAL; + + if (counter_ctrl->disable == NULL) + return -ENOTSUP; + + counter_ctrl->enabled = counter_ctrl->disable(dev, id) == 0 ? 0 : 1; + return counter_ctrl->enabled; +} + +/** + * Query the state of the xstat counter. + * + * @param dev + * Pointer to Ethernet device structure. + * @param id + * The ID of the counter to enable + * + * @return + * 1 if xstat is disabled, 0 xstat is enabled, + * -ENOTSUP if enabling/disabling is not implemented and -EINVAL if xstat id is invalid. + */ +int +mlx5_xstats_query_state(struct rte_eth_dev *dev, uint64_t id) +{ + struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_xstats_ctrl *xstats_ctrl = &priv->xstats_ctrl; + struct mlx5_stat_counter_ctrl *counter_ctrl = NULL; + + uint16_t n_stats_rq = mlx5_rq_xstats_get(dev, NULL); + if (id < n_stats_rq) + counter_ctrl = mlx5_rxq_get_counter_by_id(dev, id, &id); + else + counter_ctrl = &xstats_ctrl->info[id - n_stats_rq].ctrl; + + if (counter_ctrl == NULL) + return -EINVAL; + + if (counter_ctrl->disable == NULL) + return -ENOTSUP; + + return counter_ctrl->enabled; +} diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c index 268598f209..d583730066 100644 --- a/drivers/net/mlx5/windows/mlx5_os.c +++ b/drivers/net/mlx5/windows/mlx5_os.c @@ -78,12 +78,11 @@ mlx5_queue_counter_id_prepare(struct rte_eth_dev *dev) struct mlx5_priv *priv = dev->data->dev_private; void *ctx = priv->sh->cdev->ctx; - priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx); + priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx, NULL); if (!priv->q_counters) { DRV_LOG(ERR, "Port %d queue counter object cannot be created " "by DevX - imissed counter will be unavailable", dev->data->port_id); - priv->q_counters_allocation_failure = 1; return; } priv->counter_set_id = priv->q_counters->id; -- 2.34.1