From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7082145B36; Mon, 14 Oct 2024 10:05:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0A1D440669; Mon, 14 Oct 2024 10:05:50 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2045.outbound.protection.outlook.com [40.107.92.45]) by mails.dpdk.org (Postfix) with ESMTP id 2B97740151; Mon, 14 Oct 2024 10:05:48 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KPTSM/Ecr9GfPYk4NCE0AoUXQRGGaUR2hXeM5LTBO+bpJOfE6U89ifEc1M6sSv8QFOJWtYymiokNrnUlN/fgDIHeP6XB7kOD/hhdTzJXLowshBJLBeMkhlY1Fn7/rAP98KLDTwNvy0myuSqrmAftsPpMUEMKHNDvA6bkfN+CnCpDER2xh2gcFDYGeRJbUYr/XNw4KDEg3xd9tv+TxaR+YwYJjwOXesfwFJnglDlvZQ9LAj3xTyeQ12pHZgvBsov4YeNHtPzgSQS9rBg5W232eEdC8iucMOLW1mtmFaEf4C667PeJmnvcNa5vMCa0+Bp5WEcvVKtDeQm44GNT0iAbgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uHVb2e6sKrJRNh9WqNz66R4hg2LsfsXmWiz3XP4b0nA=; b=lEaHnTjl9M+kB8tARmA7+sf1UF0s1yWVkv6DzLPJeBxogue3e5lOuyjd7X6TgVRneyr6ZtFXs7GWyq6zLcdupF6IboSq/v08zlYukjjPpPW7jOEeTyMoT2eMqOYEhO1Q1RhY9jP7F+qr5NcFFMoi/n4d9EY9p4OKIFEo3nHdtVloEmd121mMWHV1md06AQo6IsHb32vnQbbIi2Cuh5DSTBN7+vBm+QEibDflE+621qF7gnYNtTxuIU6S1dwG4wmKxjxgruBEOUKFkxDv30BQdkCefA7nQOhv9P82i1sImGURTzo5UX+e0GsXlKRLbdYV3Z+idEp9390tSNRMY3Q6EQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uHVb2e6sKrJRNh9WqNz66R4hg2LsfsXmWiz3XP4b0nA=; b=E4C1p8jBpJVIAWUDrAx6Qys0AykcvRfO/bJRFET1aoqXqlQnW4TaXY1rf0oYQNbidFvKAqKUIKMzUdVVnhQruWgiJI811FGO6NNftp1IJZsNiO8dqB0qaGeCEIgvEq/w19AToGTIfbSkpfiyhbR+zEDPaz+if6QPic8Sq8omQr86lBRd4UXyLYaScW0KrGnSSCqBwWr5fQXzOLHj2xaxV4uvW5g2NkCIQbWPgBcXj47wl81qtLliHby8JPIponfT8JT++BwpLIl676Ko7PYIQwcQfRilUSj0EZSHgs+LRjVK9NqPLF50ZO1Xt8DTfNbYJvBuQDS56HziHuIDmrVBUg== Received: from BYAPR08CA0015.namprd08.prod.outlook.com (2603:10b6:a03:100::28) by CY8PR12MB7755.namprd12.prod.outlook.com (2603:10b6:930:87::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.25; Mon, 14 Oct 2024 08:05:43 +0000 Received: from SJ5PEPF000001F0.namprd05.prod.outlook.com (2603:10b6:a03:100:cafe::7) by BYAPR08CA0015.outlook.office365.com (2603:10b6:a03:100::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26 via Frontend Transport; Mon, 14 Oct 2024 08:05:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SJ5PEPF000001F0.mail.protection.outlook.com (10.167.242.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.13 via Frontend Transport; Mon, 14 Oct 2024 08:05:42 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 14 Oct 2024 01:05:31 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 14 Oct 2024 01:05:28 -0700 From: Viacheslav Ovsiienko To: CC: , , , , Tim Martin , Subject: [PATCH v2 3/4] net/mlx5: fix Tx tracing to use single clock source Date: Mon, 14 Oct 2024 11:04:33 +0300 Message-ID: <20241014080434.1211629-4-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241014080434.1211629-1-viacheslavo@nvidia.com> References: <20241009114028.973284-1-viacheslavo@nvidia.com> <20241014080434.1211629-1-viacheslavo@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001F0:EE_|CY8PR12MB7755:EE_ X-MS-Office365-Filtering-Correlation-Id: b726d15a-0ce7-49f5-3c10-08dcec27000f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NDj/4RfR2lywACyqH4oM79NirCavMfe82gGO6mbR2P24KpJimfXM1NttTHAa?= =?us-ascii?Q?3iGEB/vUafCL/nWzyeIMJDqIDib7oldXrUDr0gFFs1tESg1SuXuPiYRNKUxQ?= =?us-ascii?Q?Swo/l7iPbYXLkp2Z9cmMIQsVKlYjxOXPCexAZP/lEYBjfDk8beHO1GrWfzPV?= =?us-ascii?Q?cY8cQMMAXz2gbx5hDnxeB4tr9awY3lj4WzRNzBGk8LVKJ4bDPdd6SAcodcmS?= =?us-ascii?Q?UY3mMeds0cENnX+YKsuX8OWVwpaVy+Xu/9rT6nnX7rmWTIXjixk8dtFSae9D?= =?us-ascii?Q?K8BM7z9TAnMILUlh8GBZ9GPTmeM29FRo88cF9dBViMjz1IUXOMXz1jYfv6LY?= =?us-ascii?Q?9ar09sD6YjHa7eoWN4iPen1R+/VZWUqnuDFsNkkJKnINLTypusu0LcWkAlho?= =?us-ascii?Q?B7XQgafgxJVb8RscKVqS9blRmlMsLbz7SuAv5kTd/ThU1ApPcuiJqlFWJFLz?= =?us-ascii?Q?x9b43CFKcsBY8yJNabqC+P+zM55eX0OokbGBBRBnQjcjbc3Ae/zPzXT87KOS?= =?us-ascii?Q?1FAX3fIOJlP1/tcvvqApmlxT+3yO73aYgu49d6e/S5Nl/GDde7DkbF8Q1tJR?= =?us-ascii?Q?OLanGNzQXzTmnFrU1NXPV25yPHsgONsd7b4vWo6ui527OyRCLQJTVL1xHVsa?= =?us-ascii?Q?4EDIAEH7ovYCkQl1lkYz50LUcVvkqI+GkcL5LHMlmu+P2xTrECPaxBzvvgdq?= =?us-ascii?Q?E7fZHJTnEp4BKa7DfBKQjIFBPJbC9DAz+YVjwIkufShbZWquDVoU6qJ+1IuX?= =?us-ascii?Q?RpO9rs475yT7H3LfH3lNEGNmvN2LgGOukjxX6+yWS449nAh+PGH2rpF4r/Ys?= =?us-ascii?Q?ubjjBLaZavHcD2PkQUpLFv7laRkp3nYx3285EJEv7HjpbsuhR1ajuloevSpI?= =?us-ascii?Q?BQcPCrLa75T/uUeygDp+vgMuAfFtDFXy8PdA4tUO62V3x753pDSjzFWTZUxS?= =?us-ascii?Q?VZjGJWPDy136YKq4lWkuXhOsy16GgnZDda3sDaDMgNa26iQygTyyJ4pX4XG1?= =?us-ascii?Q?r570cM0Kf44GEZAtYXjbibi4kJi4svwVph53z/aKEWcD/hdEhd7w6YtuHqs1?= =?us-ascii?Q?eK8vm3rsjOk7JMMwvm0oSNy6Jg/B1wAuj/76sNJMPxDe9CF+K8DJqfGSnBiK?= =?us-ascii?Q?ZoMYk6glGmtqCoXTeqFTAHEaNa2XSZJL4JpTLM4JyD3Zr1jHFp2ia9UdJafT?= =?us-ascii?Q?6E3QbV2T4ayF/WZS5N1k0tVxkKb4c2tF1pPv++Q6epCp4i6osn/JPaSx2aTX?= =?us-ascii?Q?VFvfBVRNQbed2UdRs/h5DtvTsi9RDPPGa7Oe7Sv7tzOmofUjmjWTr6l0e2GQ?= =?us-ascii?Q?JqpVJ+SKs4GLjv0DVI9GyPUSl8JdDmT3FAdj2F/8uwP5BLA9bkFO8J/1Be9V?= =?us-ascii?Q?E2qGl4yMYkRmy/1g1WOxRn5ICGme?= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Oct 2024 08:05:42.8489 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b726d15a-0ce7-49f5-3c10-08dcec27000f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001F0.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7755 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Tim Martin The prior commit introduced tracing for mlx5, but there is a mixture of two unrelated clocks used: the TSC for host work submission timestamps and the NIC HW clock for CQE completion times. It is necessary to have timestamps from a single common clock, and the NIC HW clock is the better choice since it can be used with externally synchronized clocks. This patch adds the NIC HW clock as an additional logged parameter for trace_tx_entry, trace_tx_exit, and trace_tx_wqe. The included trace analysis python script is also updated to use the new clock when it is available. Fixes: a1e910f5b8d4 ("net/mlx5: introduce tracepoints") Fixes: 9725191a7e14 ("net/mlx5: add Tx datapath trace analyzing script") Cc: stable@dpdk.org Signed-off-by: Tim Martin Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_trace.h | 9 ++++++--- drivers/net/mlx5/mlx5_tx.h | 21 +++++++++++++++++---- drivers/net/mlx5/tools/mlx5_trace.py | 12 +++++++++--- 3 files changed, 32 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx5/mlx5_trace.h b/drivers/net/mlx5/mlx5_trace.h index a8f0b372c8..4fc3584acc 100644 --- a/drivers/net/mlx5/mlx5_trace.h +++ b/drivers/net/mlx5/mlx5_trace.h @@ -22,21 +22,24 @@ extern "C" { /* TX burst subroutines trace points. */ RTE_TRACE_POINT_FP( rte_pmd_mlx5_trace_tx_entry, - RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t queue_id), + RTE_TRACE_POINT_ARGS(uint64_t real_time, uint16_t port_id, uint16_t queue_id), + rte_trace_point_emit_u64(real_time); rte_trace_point_emit_u16(port_id); rte_trace_point_emit_u16(queue_id); ) RTE_TRACE_POINT_FP( rte_pmd_mlx5_trace_tx_exit, - RTE_TRACE_POINT_ARGS(uint16_t nb_sent, uint16_t nb_req), + RTE_TRACE_POINT_ARGS(uint64_t real_time, uint16_t nb_sent, uint16_t nb_req), + rte_trace_point_emit_u64(real_time); rte_trace_point_emit_u16(nb_sent); rte_trace_point_emit_u16(nb_req); ) RTE_TRACE_POINT_FP( rte_pmd_mlx5_trace_tx_wqe, - RTE_TRACE_POINT_ARGS(uint32_t opcode), + RTE_TRACE_POINT_ARGS(uint64_t real_time, uint32_t opcode), + rte_trace_point_emit_u64(real_time); rte_trace_point_emit_u32(opcode); ) diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index 587e6a9f7d..55568c41b1 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -404,6 +404,14 @@ static __rte_always_inline uint64_t mlx5_read_pcibar_clock(struct rte_eth_dev *d return 0; } +static __rte_always_inline uint64_t mlx5_read_pcibar_clock_from_txq(struct mlx5_txq_data *txq) +{ + struct mlx5_txq_ctrl *txq_ctrl = container_of(txq, struct mlx5_txq_ctrl, txq); + struct rte_eth_dev *dev = ETH_DEV(txq_ctrl->priv); + + return mlx5_read_pcibar_clock(dev); +} + /** * Set Software Parser flags and offsets in Ethernet Segment of WQE. * Flags must be preliminary initialized to zero. @@ -841,6 +849,7 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, unsigned int olx) { struct mlx5_wqe_cseg *__rte_restrict cs = &wqe->cseg; + uint64_t real_time; /* For legacy MPW replace the EMPW by TSO with modifier. */ if (MLX5_TXOFF_CONFIG(MPW) && opcode == MLX5_OPCODE_ENHANCED_MPSW) @@ -854,9 +863,12 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); - if (__rte_trace_point_fp_is_enabled()) - rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); - rte_pmd_mlx5_trace_tx_wqe((txq->wqe_ci << 8) | opcode); + if (__rte_trace_point_fp_is_enabled()) { + real_time = mlx5_read_pcibar_clock_from_txq(txq); + if (!loc->pkts_sent) + rte_pmd_mlx5_trace_tx_entry(real_time, txq->port_id, txq->idx); + rte_pmd_mlx5_trace_tx_wqe(real_time, (txq->wqe_ci << 8) | opcode); + } } /** @@ -3818,7 +3830,8 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, __mlx5_tx_free_mbuf(txq, pkts, loc.mbuf_free, olx); /* Trace productive bursts only. */ if (__rte_trace_point_fp_is_enabled() && loc.pkts_sent) - rte_pmd_mlx5_trace_tx_exit(loc.pkts_sent, pkts_n); + rte_pmd_mlx5_trace_tx_exit(mlx5_read_pcibar_clock_from_txq(txq), + loc.pkts_sent, pkts_n); return loc.pkts_sent; } diff --git a/drivers/net/mlx5/tools/mlx5_trace.py b/drivers/net/mlx5/tools/mlx5_trace.py index 67461520a9..5eb634a490 100755 --- a/drivers/net/mlx5/tools/mlx5_trace.py +++ b/drivers/net/mlx5/tools/mlx5_trace.py @@ -174,7 +174,9 @@ def do_tx_entry(msg, trace): return # allocate the new burst and append to the queue burst = MlxBurst() - burst.call_ts = msg.default_clock_snapshot.ns_from_origin + burst.call_ts = event["real_time"] + if burst.call_ts == 0: + burst.call_ts = msg.default_clock_snapshot.ns_from_origin trace.tx_blst[cpu_id] = burst pq_id = event["port_id"] << 16 | event["queue_id"] queue = trace.tx_qlst.get(pq_id) @@ -194,7 +196,9 @@ def do_tx_exit(msg, trace): burst = trace.tx_blst.get(cpu_id) if burst is None: return - burst.done_ts = msg.default_clock_snapshot.ns_from_origin + burst.done_ts = event["real_time"] + if burst.done_ts == 0: + burst.done_ts = msg.default_clock_snapshot.ns_from_origin burst.req = event["nb_req"] burst.done = event["nb_sent"] trace.tx_blst.pop(cpu_id) @@ -210,7 +214,9 @@ def do_tx_wqe(msg, trace): wqe = MlxWqe() wqe.wait_ts = trace.tx_wlst.get(cpu_id) if wqe.wait_ts is None: - wqe.wait_ts = msg.default_clock_snapshot.ns_from_origin + wqe.wait_ts = event["real_time"] + if wqe.wait_ts == 0: + wqe.wait_ts = msg.default_clock_snapshot.ns_from_origin wqe.opcode = event["opcode"] burst.wqes.append(wqe) -- 2.34.1