From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0639944158; Tue, 4 Jun 2024 23:12:21 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D19A14021D; Tue, 4 Jun 2024 23:12:20 +0200 (CEST) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2057.outbound.protection.outlook.com [40.107.96.57]) by mails.dpdk.org (Postfix) with ESMTP id 8CB5F40156 for ; Tue, 4 Jun 2024 23:12:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S2h5ILkkOCcY9f3r01ga3KW2RO5vgd0dwVEuziYcTZN0JU69ZC+Adfae59j97QlCyJAqULmGGKp2RleNCrkgDVjnk9pOJp3To+ZjozvkL/Liq3Kw8S9TeDGQD2MNOJ9ffbPOfb4hwlfFN+uy+OUycX40E8WmZEAqq9tHKbPIey88NQwiDSTsF0D1MrZlsdk4YCxwZzHeWPG2Vx1U2V5pbMwTg+417WfTwyKfPM75vX2GdFgEt/FK3cHMKdZY253AbKevxhNUUVQlSZYDk8HDfuUK5+ofpv0yMl8JQim7kjoMBnmp8mFQv9sM1bph6QzKvi32Q/M/PdYMQs0nCDz9Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HtlEq2RB/lByRrcVql5C3UPQkQxWJzHWOlP/s0WIE3c=; b=SP9Cw9ditEhMRAKAlUQhAWwXNLMwcWtp1Lr7XjxygU9bubwkfLfUshYjiZ3tDJkhcMwQ1ugKv5JIxD2x5idWXf7N9yZ/uiiDQNbro//ZM/IWC27O0b3OeuXJLp81xsmypPZjJbJdBQ/V+g0l/yQZGIp1t8PTsMZJdPwZX4Dum+kSoYzMkdRz0JdeHOx28yeaLJXcimR870pVPNbQ/CJBxu1UBRBHv+xYNoTLHsyGyIxuBgSqDSb/iU6vOREiprE1YBKXhzFNX6H/zO0I1Aubog7lYuUySxfKqnIvmw970ylPZPofGe0nWBQVYlXm8zgJBd3TIs8isR2oT2MdX7OJ2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HtlEq2RB/lByRrcVql5C3UPQkQxWJzHWOlP/s0WIE3c=; b=EEFGfuDQMDnWlaM1zJkKEsIlVFbxDD5m0vZurVF1s4juIrRXt315hgsGYprmz9eBxHB8nOEL3ut/35EeFuWXz2WN6WRaamvthFgaVGy0Ddosq4x8P+0Ea7R7EErweqVsTxUdBir/0JD831mXZUbIw6dSRHsPIXxaC5p/+NlbEIKkYYn2ek2gL22jDrV1Ir0tal9davQ4oPIMoT/6kA/+bHjYQvidrpfq5SiuySYdhg0lhF3Xcuj89w6jSHj15jtpveGPdk83+o6MbLgwMRoHpMqA7SA5jVin2Dm6lt6abZKd3YHWZheuKTD8pEUMny8Hn7uonZEDDB3sXTeBPrgHsA== Received: from CH2PR18CA0002.namprd18.prod.outlook.com (2603:10b6:610:4f::12) by DM4PR12MB6159.namprd12.prod.outlook.com (2603:10b6:8:a8::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Tue, 4 Jun 2024 21:12:14 +0000 Received: from CH1PEPF0000AD7A.namprd04.prod.outlook.com (2603:10b6:610:4f:cafe::f6) by CH2PR18CA0002.outlook.office365.com (2603:10b6:610:4f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.28 via Frontend Transport; Tue, 4 Jun 2024 21:12:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CH1PEPF0000AD7A.mail.protection.outlook.com (10.167.244.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.15 via Frontend Transport; Tue, 4 Jun 2024 21:12:14 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 4 Jun 2024 14:11:58 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 4 Jun 2024 14:11:55 -0700 From: Kiran Vedere To: CC: , , , Subject: [PATCH] net/mlx5: add queue hardware object context dump Date: Wed, 5 Jun 2024 00:11:36 +0300 Message-ID: <20240604211136.1616519-1-kiranv@nvidia.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD7A:EE_|DM4PR12MB6159:EE_ X-MS-Office365-Filtering-Correlation-Id: b1f26908-5d28-43f5-0c2b-08dc84db01e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230031|1800799015|82310400017|36860700004|376005; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?jNQaSVpkzcsHY5k+ivqoQkSiv4XI5ugBB0V82RJ1KwH0OoE+zHjPKsZf5f9+?= =?us-ascii?Q?hmSA0wOOT/CrempIyZGhqCpX9u3N/p/sCPKgHt7zCYvwDJFK87TFdsIgXtBO?= =?us-ascii?Q?LWqs+VBU0YBdVb3A144bKVUTxe8GeFUhfYl4aa42to20x6FKs8ytmh/wNdkO?= =?us-ascii?Q?PmgYZ3fwuU5hN6CG4h18arS+vhYsBk7icOP0trJMGoNXxvjcohtKNgKBuASN?= =?us-ascii?Q?YP/5m9DW/Ga5cY9d1X0NRS1B/3Vn/1lKcPpvCKAh7t81Dx95vFqQzkTgoJkz?= =?us-ascii?Q?A35fV0RSpYhHTzopB9XlO1Fb0k350ta2hThspUUDNvupQSR2m2B9inKH8ruK?= =?us-ascii?Q?RzKLzgKZC3oT0kKmOl9UC9H67x94/XUejoF+NWvVuTawCZbFUHvGRO8+4RyA?= =?us-ascii?Q?S+Yaz2d8lxUu5NnYI0I2k/+hTt9WZSgWy+dvXxaEXAZIxp8WLEi6aVSq07fB?= =?us-ascii?Q?OFPD34rBFMUenlXgUrQRqEvbPg8bK1BNe3EOQ8L6DmEw/TOev58VVeWhvA05?= =?us-ascii?Q?KC+DiOhyFsztDI5oi8sYOCwvivQM03+heo2XrsV15Ps1zEEN9FFSV/P67zb+?= =?us-ascii?Q?sSD0N5An94DLIxATEv3aJno5zUrJafzT+nGdnKwCnkdbUIkplYf4vne3g80d?= =?us-ascii?Q?Y74HxEsce+po/nX9P9Acgb0iNZRbuwEadF4qeHmf9oE+XF1C8VsuGyS29wjI?= =?us-ascii?Q?iPJIlgAttmo0g1YXxcOvBBRxL96O+chl2PIy3AVd42isGUuCUUsXTjGP/GqD?= =?us-ascii?Q?hiB4XPXuNB+axPrrz5IuFOPGiWGQQZEAoMXeNztoYVQZoms+DlomGOkoPG8I?= =?us-ascii?Q?xeW7Ikz0LFxb/gVcb+B85e7ptMUEtjO2EtFBxxisy3MJUKdJzGPXhNlML+59?= =?us-ascii?Q?/yNS/HTh10oZwf6+HoeDHLh04ksVi2SxOJ0Xp1MhjDTV8LY2l3vohpQauuM0?= =?us-ascii?Q?lewHijPRR3k/yUFoP9NKP92PuMlmIhqc2/hG7a8SqGLzpSkK6Nsk6td/Yzxo?= =?us-ascii?Q?Wf1/n02+EEQple88LmwB292Ictzm71UnDVG4UtaD/Dew9YtV6iMDEshHuyZ9?= =?us-ascii?Q?+/YEj3KfbrdTnG30ZHzLJJENxKnKZzn+1fIs9HkdZp6vYYAL+4nRrfkT5s2i?= =?us-ascii?Q?NB27gnPvHXR6dHtRw2LefG+Z7g7D6hy+tMWz6foagoj8m2f+x+vcrS7P37tt?= =?us-ascii?Q?I1XVoufKIokehkHls/C5/FGJhtaBEgDEJ3djlDknPzV3qjQAnkN6BmkyElbw?= =?us-ascii?Q?uMYF9dvBeHHXuoJDrp5Z70rsH6Wi5310meeKOEoTctbEsQwn2YqJKiAzn6Fy?= =?us-ascii?Q?CRCLZQk9TRF+gSAfpjHhuZOwLyPSSPw0JckpYF5mMvgKEB/0RIvyn2MYEtaw?= =?us-ascii?Q?Olc+dwYgZUvb3hbBKP6YGwMAlG7f?= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2024 21:12:14.2697 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b1f26908-5d28-43f5-0c2b-08dc84db01e5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD7A.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6159 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add debug capability to mlx5 PMD to dump SQ/RQ/CQ HW object context for a given port/queue. The context dump can provide some real-time information on cause of certain Tx/Rx Failures. Signed-off-by: Kiran Vedere --- .mailmap | 1 + doc/guides/nics/mlx5.rst | 11 ++ doc/guides/rel_notes/release_24_07.rst | 5 + drivers/common/mlx5/mlx5_devx_cmds.c | 87 +++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 9 ++ drivers/common/mlx5/mlx5_prm.h | 30 +++++ drivers/common/mlx5/version.map | 3 + drivers/net/mlx5/mlx5_rx.c | 82 ++++++++++++ drivers/net/mlx5/mlx5_rxtx.c | 24 ++++ drivers/net/mlx5/mlx5_rxtx.h | 2 + drivers/net/mlx5/mlx5_testpmd.c | 174 +++++++++++++++++++++++++ drivers/net/mlx5/mlx5_tx.c | 85 ++++++++++++ drivers/net/mlx5/rte_pmd_mlx5.h | 36 +++++ drivers/net/mlx5/version.map | 2 + 14 files changed, 551 insertions(+) diff --git a/.mailmap b/.mailmap index 87fa24714e..a5ee56fe0c 100644 --- a/.mailmap +++ b/.mailmap @@ -762,6 +762,7 @@ Kiran KN Kiran Kumar K Kiran Kumar Kiran Patil +Kiran Vedere Kirill Rybalchenko Kishore Padmanabha Klaus Degner diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index b5928d40b2..40e36de7e8 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -2601,3 +2601,14 @@ Destroy GENEVE TLV parser for specific port:: This command doesn't destroy the global list, For releasing options, ``flush`` command should be used. + +Dump RQ/SQ/CQ HW Context for debug purposes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Dump RQ/CQ HW Context for a given port/queue to a file: + + testpmd> mlx5 port (port_id) queue (queue_id) dump rq_context (file_name) + +Dump SQ/CQ HW Context for a given port/queue to a file: + + testpmd> mlx5 port (port_id) queue (queue_id) dump sq_context (file_name) diff --git a/doc/guides/rel_notes/release_24_07.rst b/doc/guides/rel_notes/release_24_07.rst index ffbe9ce051..c18a0be469 100644 --- a/doc/guides/rel_notes/release_24_07.rst +++ b/doc/guides/rel_notes/release_24_07.rst @@ -81,6 +81,11 @@ New Features * Added SSE/NEON vector datapath. +* **Add Queue Hardware object context dump.** + + * Added debug capability to mlx5 PMD to dump SQ/RQ/CQ HW object context for + a given port/queue. The context dump could provide useful information when + debugging certain Rx/Tx Failures. Removed Items ------------- diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c index 9952733c90..8d8af700f2 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.c +++ b/drivers/common/mlx5/mlx5_devx_cmds.c @@ -1594,6 +1594,35 @@ mlx5_devx_cmd_modify_rq(struct mlx5_devx_obj *rq, return ret; } +/* + * Query RQ using DevX API. + * + * @param[in] rq_obj + * RQ Devx Object + * @param[out] out + * RQ Query Output + * @param[in] outlen + * RQ Query Output Length + * + * @return + * 0 if Query successful, else non-zero return value from devx_obj_query API + */ +int +mlx5_devx_cmd_query_rq(struct mlx5_devx_obj *rq_obj, void *out, size_t outlen) +{ + uint32_t in[MLX5_ST_SZ_DW(query_rq_in)] = {0}; + int rc; + + MLX5_SET(query_rq_in, in, opcode, MLX5_CMD_OP_QUERY_RQ); + MLX5_SET(query_rq_in, in, rqn, rq_obj->id); + rc = mlx5_glue->devx_obj_query(rq_obj->obj, in, sizeof(in), out, outlen); + if (rc || MLX5_FW_STATUS(out)) { + DEVX_DRV_LOG(ERR, out, "RQ query", "rq_id", rq_obj->id); + return MLX5_DEVX_ERR_RC(rc); + } + return 0; +} + /** * Create RMP using DevX API. * @@ -2001,6 +2030,35 @@ mlx5_devx_cmd_modify_sq(struct mlx5_devx_obj *sq, return ret; } +/* + * Query SQ using DevX API. + * + * @param[in] sq_obj + * SQ Devx Object + * @param[out] out + * SQ Query Output + * @param[in] outlen + * SQ Query Output Length + * + * @return + * 0 if Query successful, else non-zero return value from devx_obj_query API + */ +int +mlx5_devx_cmd_query_sq(struct mlx5_devx_obj *sq_obj, void *out, size_t outlen) +{ + uint32_t in[MLX5_ST_SZ_DW(query_sq_in)] = {0}; + int rc; + + MLX5_SET(query_sq_in, in, opcode, MLX5_CMD_OP_QUERY_SQ); + MLX5_SET(query_sq_in, in, sqn, sq_obj->id); + rc = mlx5_glue->devx_obj_query(sq_obj->obj, in, sizeof(in), out, outlen); + if (rc || MLX5_FW_STATUS(out)) { + DEVX_DRV_LOG(ERR, out, "SQ query", "sq_id", sq_obj->id); + return MLX5_DEVX_ERR_RC(rc); + } + return 0; +} + /** * Create TIS using DevX API. * @@ -2202,6 +2260,35 @@ mlx5_devx_cmd_create_cq(void *ctx, struct mlx5_devx_cq_attr *attr) return cq_obj; } +/* + * Query CQ using DevX API. + * + * @param[in] cq_obj + * CQ Devx Object + * @param[out] out + * CQ Query Output + * @param[in] outlen + * CQ Query Output Length + * + * @return + * 0 if Query successful, else non-zero return value from devx_obj_query API + */ +int +mlx5_devx_cmd_query_cq(struct mlx5_devx_obj *cq_obj, void *out, size_t outlen) +{ + uint32_t in[MLX5_ST_SZ_DW(query_cq_in)] = {0}; + int rc; + + MLX5_SET(query_cq_in, in, opcode, MLX5_CMD_OP_QUERY_CQ); + MLX5_SET(query_cq_in, in, cqn, cq_obj->id); + rc = mlx5_glue->devx_obj_query(cq_obj->obj, in, sizeof(in), out, outlen); + if (rc || MLX5_FW_STATUS(out)) { + DEVX_DRV_LOG(ERR, out, "CQ query", "cq_id", cq_obj->id); + return MLX5_DEVX_ERR_RC(rc); + } + return 0; +} + /** * Create VIRTQ using DevX API. * diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h index c79f8dc48d..540f002088 100644 --- a/drivers/common/mlx5/mlx5_devx_cmds.h +++ b/drivers/common/mlx5/mlx5_devx_cmds.h @@ -735,6 +735,15 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_sq(void *ctx, __rte_internal int mlx5_devx_cmd_modify_sq(struct mlx5_devx_obj *sq, struct mlx5_devx_modify_sq_attr *sq_attr); +__rte_internal +int mlx5_devx_cmd_query_sq(struct mlx5_devx_obj *sq, void *out, size_t outlen); + +__rte_internal +int mlx5_devx_cmd_query_cq(struct mlx5_devx_obj *cq, void *out, size_t outlen); + +__rte_internal +int mlx5_devx_cmd_query_rq(struct mlx5_devx_obj *rq, void *out, size_t outlen); + __rte_internal struct mlx5_devx_obj *mlx5_devx_cmd_create_tis(void *ctx, struct mlx5_devx_tis_attr *tis_attr); diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h index 178a18a978..359f02f17c 100644 --- a/drivers/common/mlx5/mlx5_prm.h +++ b/drivers/common/mlx5/mlx5_prm.h @@ -1215,6 +1215,7 @@ enum { MLX5_CMD_OP_QUERY_HCA_CAP = 0x100, MLX5_CMD_OP_CREATE_MKEY = 0x200, MLX5_CMD_OP_CREATE_CQ = 0x400, + MLX5_CMD_OP_QUERY_CQ = 0x402, MLX5_CMD_OP_CREATE_QP = 0x500, MLX5_CMD_OP_RST2INIT_QP = 0x502, MLX5_CMD_OP_INIT2RTR_QP = 0x503, @@ -1240,6 +1241,7 @@ enum { MLX5_CMD_OP_MODIFY_TIR = 0x901, MLX5_CMD_OP_CREATE_SQ = 0X904, MLX5_CMD_OP_MODIFY_SQ = 0X905, + MLX5_CMD_OP_QUERY_SQ = 0x907, MLX5_CMD_OP_CREATE_RQ = 0x908, MLX5_CMD_OP_MODIFY_RQ = 0x909, MLX5_CMD_OP_QUERY_RQ = 0x90b, @@ -3140,6 +3142,14 @@ struct mlx5_ifc_sqc_bits { struct mlx5_ifc_wq_bits wq; }; +struct mlx5_ifc_query_sq_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_40[0xc0]; + struct mlx5_ifc_sqc_bits sq_context; +}; + struct mlx5_ifc_query_sq_in_bits { u8 opcode[0x10]; u8 reserved_at_10[0x10]; @@ -3361,6 +3371,26 @@ struct mlx5_ifc_create_cq_in_bits { u8 pas[]; }; +struct mlx5_ifc_query_cq_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + u8 syndrome[0x20]; + u8 reserved_at_40[0x40]; + struct mlx5_ifc_cqc_bits cq_context; + u8 reserved_at_280[0x600]; + u8 pas[][0x40]; +}; + +struct mlx5_ifc_query_cq_in_bits { + u8 opcode[0x10]; + u8 reserved_at_10[0x10]; + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + u8 reserved_at_40[0x8]; + u8 cqn[0x18]; + u8 reserved_at_60[0x20]; +}; + enum { MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT = 0x000b, MLX5_GENERAL_OBJ_TYPE_DEK = 0x000c, diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index 589a450145..a2f72ef46a 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -51,6 +51,9 @@ INTERNAL { mlx5_devx_cmd_modify_rq; mlx5_devx_cmd_modify_rqt; mlx5_devx_cmd_modify_sq; + mlx5_devx_cmd_query_sq; + mlx5_devx_cmd_query_rq; + mlx5_devx_cmd_query_cq; mlx5_devx_cmd_modify_tir; mlx5_devx_cmd_modify_virtq; mlx5_devx_cmd_qp_query_tis_td; diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c index cc087348a4..f241809e08 100644 --- a/drivers/net/mlx5/mlx5_rx.c +++ b/drivers/net/mlx5/mlx5_rx.c @@ -1586,3 +1586,85 @@ int rte_pmd_mlx5_host_shaper_config(int port_id, uint8_t rate, return mlxreg_host_shaper_config(dev, priv->sh->lwm_triggered, priv->sh->host_shaper_rate); } + +/** + * Dump RQ/CQ Context to a file. + * + * @param[in] port_id + * Port ID + * @param[in] queue_id + * Queue ID + * @param[in] filename + * Name of file to dump the Rx Queue Context + * + * @return + * 0 for Success, non-zero value depending on failure type + */ +int rte_pmd_mlx5_rxq_dump_contexts(uint16_t port_id, uint16_t queue_id, const char *filename) +{ + struct rte_eth_dev *dev; + struct mlx5_rxq_priv *rxq; + struct mlx5_rxq_ctrl *rxq_ctrl; + struct mlx5_rxq_obj *rxq_obj; + struct mlx5_devx_rq *rq; + struct mlx5_devx_cq *cq; + struct mlx5_devx_obj *rq_devx_obj; + struct mlx5_devx_obj *cq_devx_obj; + + uint32_t rq_out[MLX5_ST_SZ_DW(query_rq_out)] = {0}; + uint32_t cq_out[MLX5_ST_SZ_DW(query_cq_out)] = {0}; + + int ret; + FILE *fd; + MKSTR(path, "./%s", filename); + + if (!rte_eth_dev_is_valid_port(port_id)) + return -ENODEV; + + if (rte_eth_rx_queue_is_valid(port_id, queue_id)) + return -EINVAL; + + fd = fopen(path, "w"); + if (!fd) { + rte_errno = errno; + return -EIO; + } + + dev = &rte_eth_devices[port_id]; + rxq = mlx5_rxq_ref(dev, queue_id); + rxq_ctrl = rxq->ctrl; + rxq_obj = rxq_ctrl->obj; + rq = &rxq->devx_rq; + cq = &rxq_obj->cq_obj; + rq_devx_obj = rq->rq; + cq_devx_obj = cq->cq; + + do { + ret = mlx5_devx_cmd_query_rq(rq_devx_obj, rq_out, sizeof(rq_out)); + if (ret) + break; + + /* Dump rq query output to file */ + MKSTR(rq_headline, "RQ DevX ID = %u Port = %u Queue index = %u ", + rq_devx_obj->id, port_id, queue_id); + mlx5_dump_to_file(fd, NULL, rq_headline, 0); + mlx5_dump_to_file(fd, "Query RQ Dump:", + (const void *)((uintptr_t)rq_out), + sizeof(rq_out)); + + ret = mlx5_devx_cmd_query_cq(cq_devx_obj, cq_out, sizeof(cq_out)); + if (ret) + break; + + /* Dump cq query output to file */ + MKSTR(cq_headline, "CQ DevX ID = %u Port = %u Queue index = %u ", + cq_devx_obj->id, port_id, queue_id); + mlx5_dump_to_file(fd, NULL, cq_headline, 0); + mlx5_dump_to_file(fd, "Query CQ Dump:", + (const void *)((uintptr_t)cq_out), + sizeof(cq_out)); + } while (false); + + fclose(fd); + return ret; +} diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index d3d4470acd..eadadcdffb 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -353,6 +353,30 @@ mlx5_dump_debug_information(const char *fname, const char *hex_title, fclose(fd); } +/** + * Dump information to a logfile + * + * @param fd + * File descriptor to logfile. File descriptor open/close is managed by caller. + * @param title + * If not NULL this string is printed as a header to the output + * and the output will be in hexadecimal view. + * @param buf + * This is the buffer address to print out. + * @param len + * The number of bytes to dump out. + */ +void +mlx5_dump_to_file(FILE *fd, const char *title, + const void *buf, unsigned int len) +{ + if (title) + rte_hexdump(fd, title, buf, len); + else + fprintf(fd, "%s", (const char *)buf); + fprintf(fd, "\n\n\n"); +} + /** * Modify a Verbs/DevX queue state. * This must be called from the primary process. diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index b109d50758..3fa9245769 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -38,6 +38,8 @@ void mlx5_set_cksum_table(void); void mlx5_set_swp_types_table(void); void mlx5_dump_debug_information(const char *path, const char *title, const void *buf, unsigned int len); +void mlx5_dump_to_file(FILE *fd, const char *title, + const void *buf, unsigned int len); int mlx5_queue_state_modify_primary(struct rte_eth_dev *dev, const struct mlx5_mp_arg_queue_state_modify *sm); int mlx5_queue_state_modify(struct rte_eth_dev *dev, diff --git a/drivers/net/mlx5/mlx5_testpmd.c b/drivers/net/mlx5/mlx5_testpmd.c index 5bc4dd0551..1bb5a89559 100644 --- a/drivers/net/mlx5/mlx5_testpmd.c +++ b/drivers/net/mlx5/mlx5_testpmd.c @@ -1201,6 +1201,170 @@ cmdline_parse_inst_t mlx5_cmd_destroy_tlv_options = { } }; +/* Dump SQ Context for a given port/queue*/ +struct mlx5_cmd_dump_sq_context_options { + cmdline_fixed_string_t mlx5; + cmdline_fixed_string_t port; + portid_t port_id; + cmdline_fixed_string_t queue; + queueid_t queue_id; + cmdline_fixed_string_t dump; + cmdline_fixed_string_t sq_context; + cmdline_fixed_string_t file_name; +}; + +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_mlx5 = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, mlx5, + "mlx5"); +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_port = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, port, + "port"); +cmdline_parse_token_num_t mlx5_cmd_dump_sq_context_options_port_id = + TOKEN_NUM_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, port_id, + RTE_UINT16); +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_queue = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, queue, + "queue"); +cmdline_parse_token_num_t mlx5_cmd_dump_sq_context_options_queue_id = + TOKEN_NUM_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, queue_id, + RTE_UINT16); +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_dump = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, dump, + "dump"); +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_sq_context = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, sq_context, + "sq_context"); +cmdline_parse_token_string_t mlx5_cmd_dump_sq_context_options_file_name = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_sq_context_options, file_name, + NULL); + +static void +mlx5_cmd_dump_sq_context_options_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct mlx5_cmd_dump_sq_context_options *res = parsed_result; + int ret; + + ret = rte_pmd_mlx5_txq_dump_contexts(res->port_id, res->queue_id, res->file_name); + + switch (ret) { + case 0: + break; + case -EINVAL: + fprintf(stderr, "invalid queue index (%u), out of range\n", + res->queue_id); + break; + case -ENODEV: + fprintf(stderr, "invalid port_id %u\n", res->port_id); + break; + case -EIO: + fprintf(stderr, "File Access Error (%s)\n", strerror(rte_errno)); + break; + default: + fprintf(stderr, "Unable to dump SQ/CQ HW Context (%s)\n", strerror(rte_errno)); + } +} + +cmdline_parse_inst_t mlx5_cmd_dump_sq_context_options = { + .f = mlx5_cmd_dump_sq_context_options_parsed, + .data = NULL, + .help_str = "mlx5 port queue dump sq_context ", + .tokens = { + (void *)&mlx5_cmd_dump_sq_context_options_mlx5, + (void *)&mlx5_cmd_dump_sq_context_options_port, + (void *)&mlx5_cmd_dump_sq_context_options_port_id, + (void *)&mlx5_cmd_dump_sq_context_options_queue, + (void *)&mlx5_cmd_dump_sq_context_options_queue_id, + (void *)&mlx5_cmd_dump_sq_context_options_dump, + (void *)&mlx5_cmd_dump_sq_context_options_sq_context, + (void *)&mlx5_cmd_dump_sq_context_options_file_name, + NULL, + } +}; + +/* Dump RQ Context for a given port/queue*/ +struct mlx5_cmd_dump_rq_context_options { + cmdline_fixed_string_t mlx5; + cmdline_fixed_string_t port; + portid_t port_id; + cmdline_fixed_string_t queue; + queueid_t queue_id; + cmdline_fixed_string_t dump; + cmdline_fixed_string_t rq_context; + cmdline_fixed_string_t file_name; +}; + +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_mlx5 = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, mlx5, + "mlx5"); +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_port = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, port, + "port"); +cmdline_parse_token_num_t mlx5_cmd_dump_rq_context_options_port_id = + TOKEN_NUM_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, port_id, + RTE_UINT16); +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_queue = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, queue, + "queue"); +cmdline_parse_token_num_t mlx5_cmd_dump_rq_context_options_queue_id = + TOKEN_NUM_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, queue_id, + RTE_UINT16); +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_dump = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, dump, + "dump"); +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_rq_context = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, rq_context, + "rq_context"); +cmdline_parse_token_string_t mlx5_cmd_dump_rq_context_options_file_name = + TOKEN_STRING_INITIALIZER(struct mlx5_cmd_dump_rq_context_options, file_name, + NULL); + +static void +mlx5_cmd_dump_rq_context_options_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct mlx5_cmd_dump_rq_context_options *res = parsed_result; + int ret; + + ret = rte_pmd_mlx5_rxq_dump_contexts(res->port_id, res->queue_id, res->file_name); + + switch (ret) { + case 0: + break; + case -EINVAL: + fprintf(stderr, "invalid queue index (%u), out of range\n", + res->queue_id); + break; + case -ENODEV: + fprintf(stderr, "invalid port_id %u\n", res->port_id); + break; + case -EIO: + fprintf(stderr, "File Access Error (%s)\n", strerror(rte_errno)); + break; + default: + fprintf(stderr, "Unable to dump RQ/CQ HW Context (%s)\n", strerror(rte_errno)); + } +} + +cmdline_parse_inst_t mlx5_cmd_dump_rq_context_options = { + .f = mlx5_cmd_dump_rq_context_options_parsed, + .data = NULL, + .help_str = "mlx5 port queue dump rq_context ", + .tokens = { + (void *)&mlx5_cmd_dump_rq_context_options_mlx5, + (void *)&mlx5_cmd_dump_rq_context_options_port, + (void *)&mlx5_cmd_dump_rq_context_options_port_id, + (void *)&mlx5_cmd_dump_rq_context_options_queue, + (void *)&mlx5_cmd_dump_rq_context_options_queue_id, + (void *)&mlx5_cmd_dump_rq_context_options_dump, + (void *)&mlx5_cmd_dump_rq_context_options_rq_context, + (void *)&mlx5_cmd_dump_rq_context_options_file_name, + NULL, + } +}; + static struct testpmd_driver_commands mlx5_driver_cmds = { .commands = { { @@ -1266,6 +1430,16 @@ static struct testpmd_driver_commands mlx5_driver_cmds = { .help = "mlx5 port (port_id) destroy tlv_options\n" " Destroy GENEVE TLV parser\n\n", }, + { + .ctx = &mlx5_cmd_dump_sq_context_options, + .help = "mlx5 port (port_id) queue (queue_id) dump sq_context (file_name)\n" + " Dump mlx5 SQ Context\n\n", + }, + { + .ctx = &mlx5_cmd_dump_rq_context_options, + .help = "mlx5 port (port_id) queue (queue_id) dump rq_context (file_name)\n" + " Dump mlx5 RQ Context\n\n", + }, { .ctx = NULL, }, diff --git a/drivers/net/mlx5/mlx5_tx.c b/drivers/net/mlx5/mlx5_tx.c index 1fe9521dfc..04f80bb9bd 100644 --- a/drivers/net/mlx5/mlx5_tx.c +++ b/drivers/net/mlx5/mlx5_tx.c @@ -761,3 +761,88 @@ mlx5_tx_burst_mode_get(struct rte_eth_dev *dev, } return -EINVAL; } + +/** + * Dump SQ/CQ Context to a file. + * + * @param[in] port_id + * Port ID + * @param[in] queue_id + * Queue ID + * @param[in] filename + * Name of file to dump the Tx Queue Context + * + * @return + * 0 for success, non-zero value depending on failure. + * + */ +int rte_pmd_mlx5_txq_dump_contexts(uint16_t port_id, uint16_t queue_id, const char *filename) +{ + struct rte_eth_dev *dev; + struct mlx5_priv *priv; + struct mlx5_txq_data *txq_data; + struct mlx5_txq_ctrl *txq_ctrl; + struct mlx5_txq_obj *txq_obj; + struct mlx5_devx_sq *sq; + struct mlx5_devx_cq *cq; + struct mlx5_devx_obj *sq_devx_obj; + struct mlx5_devx_obj *cq_devx_obj; + + uint32_t sq_out[MLX5_ST_SZ_DW(query_sq_out)] = {0}; + uint32_t cq_out[MLX5_ST_SZ_DW(query_cq_out)] = {0}; + + int ret; + FILE *fd; + MKSTR(path, "./%s", filename); + + if (!rte_eth_dev_is_valid_port(port_id)) + return -ENODEV; + + if (rte_eth_tx_queue_is_valid(port_id, queue_id)) + return -EINVAL; + + fd = fopen(path, "w"); + if (!fd) { + rte_errno = errno; + return -EIO; + } + + dev = &rte_eth_devices[port_id]; + priv = dev->data->dev_private; + txq_data = (*priv->txqs)[queue_id]; + txq_ctrl = container_of(txq_data, struct mlx5_txq_ctrl, txq); + txq_obj = txq_ctrl->obj; + sq = &txq_obj->sq_obj; + cq = &txq_obj->cq_obj; + sq_devx_obj = sq->sq; + cq_devx_obj = cq->cq; + + do { + ret = mlx5_devx_cmd_query_sq(sq_devx_obj, sq_out, sizeof(sq_out)); + if (ret) + break; + + /* Dump sq query output to file */ + MKSTR(sq_headline, "SQ DevX ID = %u Port = %u Queue index = %u ", + sq_devx_obj->id, port_id, queue_id); + mlx5_dump_to_file(fd, NULL, sq_headline, 0); + mlx5_dump_to_file(fd, "Query SQ Dump:", + (const void *)((uintptr_t)sq_out), + sizeof(sq_out)); + + ret = mlx5_devx_cmd_query_cq(cq_devx_obj, cq_out, sizeof(cq_out)); + if (ret) + break; + + /* Dump cq query output to file */ + MKSTR(cq_headline, "CQ DevX ID = %u Port = %u Queue index = %u ", + cq_devx_obj->id, port_id, queue_id); + mlx5_dump_to_file(fd, NULL, cq_headline, 0); + mlx5_dump_to_file(fd, "Query CQ Dump:", + (const void *)((uintptr_t)cq_out), + sizeof(cq_out)); + } while (false); + + fclose(fd); + return ret; +} diff --git a/drivers/net/mlx5/rte_pmd_mlx5.h b/drivers/net/mlx5/rte_pmd_mlx5.h index 004be0eea1..c823280717 100644 --- a/drivers/net/mlx5/rte_pmd_mlx5.h +++ b/drivers/net/mlx5/rte_pmd_mlx5.h @@ -331,6 +331,42 @@ __rte_experimental int rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle); +/** + * Dump Rx Queue Context for a given port/queue + * + * @param[in] port_id + * Port ID + * @param[in] queue_id + * Queue ID + * @param[in] filename + * Name of file to dump the Rx Queue Context + * + * @return + * 0 for success, non-zero value depending on failure type + */ + +__rte_experimental +int +rte_pmd_mlx5_rxq_dump_contexts(uint16_t port_id, uint16_t queue_id, const char *filename); + +/** + * Dump Tx Queue Context for a given port/queue + * + * @param[in] port_id + * Port ID + * @param[in] queue_id + * Queue ID + * @param[in] filename + * Name of file to dump the Tx Queue Context + * + * @return + * 0 for success, non-zero value depending on failure type + */ + +__rte_experimental +int +rte_pmd_mlx5_txq_dump_contexts(uint16_t port_id, uint16_t queue_id, const char *filename); + #ifdef __cplusplus } #endif diff --git a/drivers/net/mlx5/version.map b/drivers/net/mlx5/version.map index 8fb0e07303..f76b7ec18f 100644 --- a/drivers/net/mlx5/version.map +++ b/drivers/net/mlx5/version.map @@ -20,4 +20,6 @@ EXPERIMENTAL { # added in 24.03 rte_pmd_mlx5_create_geneve_tlv_parser; rte_pmd_mlx5_destroy_geneve_tlv_parser; + rte_pmd_mlx5_txq_dump_contexts; + rte_pmd_mlx5_rxq_dump_contexts; }; -- 2.34.1