From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 52D0146C54 for ; Wed, 30 Jul 2025 17:01:59 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D47A40E78; Wed, 30 Jul 2025 17:01:59 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2076.outbound.protection.outlook.com [40.107.92.76]) by mails.dpdk.org (Postfix) with ESMTP id DADBB40E78 for ; Wed, 30 Jul 2025 17:01:57 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ifE3aqhILGbFTkYfKqKHqoQqcmL25mBfTo9RPRJdav/iX8eCvAjlhR42xKwfXA7Dt6LKa2B1oiR/0CjiBQZZZ+cqR3wxYcsoeFN7oPSklhljGTH5qOUQlaSPIo9+FE4ydV95WoEvbVX2eR+ygt5Z4K9XAd/bN1ShM0vtorH/B9dixAffResDVymBK3OWj+RYJiqcqTLWIirIvsNpypCtche0RcBJT0IVnP4qC8VeleSXftxdo1bDifAVvFphFM2KL650qmojMjGQSSzP7H8G2nYwQ/TtELwDvvY+qgDFDH4q8rEyYhcp5Ra49GKwgCAyV+wqLzTzDRQsfRF2cpHgzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WzQU8naiC/1QApnhUHgXLXZkLS96w0KgP17lVhiOpkU=; b=mOAYpVM9p3Me4xKWmvD6ZqPRq/HwwdOUZIUV9zNRwF/O93DDQybRVLVdPPABfWdh7aA5nmjfUZDxsV4k7wz9icbMfN19ZTQFjHJUTYxzcNsX0yPBDGIoewhAhJ2zKSkOhbebBC62VsCfrwNJjLzmsopFd3ilS1r+B1HzhR9BK/Nr0FWLS+45p0lUj2FzZBKvMz8tUTFt8n47Sx1kQX3G52JsuTSfaH7Ql5HrKICdhN3CymMmJ3OWIZPp92sBywizZUbbn/Spg2ROoHmJqMscqbgjQLQQtD6R9VBaaOA9NopxGjhuiHhuwOlLXPdpp0vtr8xXSGCYXPrW+rs4ki5qNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=networkplumber.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WzQU8naiC/1QApnhUHgXLXZkLS96w0KgP17lVhiOpkU=; b=WF9GjKYxAe9yEjZxVJhmigBMJf0XujRoyyI4TN51sCUXylgOfbx7Oxyuv/NbmO7cObfv9vxNE+0AeWH3rGhjxQM+G5yhMirW4FQJNcbLeE2J471Hm5XhiE8qdK3Y3nBv1mNE+5QCEKibu3OJQC0fl2EtNXve4Qs/rnoKACuZgRUY88k5+eB464daJxiN0bYnLo/Md9wv/kQ8E4xkgPuMJN/oaGY7v3D1AWYlbfM8S//rptnJp6snWWiT/HDwts4sdLdNR32OEEOQdD/sM3s9slR1xHbRQIhcEJ3WtIt0pStQbq1FHKtkDBbBmZtAoODiGFf0nUaq9t1SeFAEIprkXA== Received: from BN8PR03CA0017.namprd03.prod.outlook.com (2603:10b6:408:94::30) by MN0PR12MB6003.namprd12.prod.outlook.com (2603:10b6:208:37f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8964.26; Wed, 30 Jul 2025 15:01:54 +0000 Received: from BL6PEPF0002256E.namprd02.prod.outlook.com (2603:10b6:408:94:cafe::fd) by BN8PR03CA0017.outlook.office365.com (2603:10b6:408:94::30) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8989.13 via Frontend Transport; Wed, 30 Jul 2025 15:01:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL6PEPF0002256E.mail.protection.outlook.com (10.167.249.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8989.10 via Frontend Transport; Wed, 30 Jul 2025 15:01:53 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Wed, 30 Jul 2025 08:01:32 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Wed, 30 Jul 2025 08:01:21 -0700 From: Xueming Li To: Stephen Hemminger CC: Xueming Li , Thiyagarajan P , Vipin Varghese , dpdk stable Subject: patch 'latencystats: fix receive sample race' has been queued to stable release 23.11.5 Date: Wed, 30 Jul 2025 22:56:26 +0800 Message-ID: <20250730145633.245984-18-xuemingl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250730145633.245984-1-xuemingl@nvidia.com> References: <20250626120145.27369-1-xuemingl@nvidia.com> <20250730145633.245984-1-xuemingl@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0002256E:EE_|MN0PR12MB6003:EE_ X-MS-Office365-Filtering-Correlation-Id: 65aa19c2-9a50-4cee-57c4-08ddcf7a0557 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|36860700013|376014|82310400026|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?eUu+GG8AMIdjFiyY8+AHaXsI+DPxQildGJ7NxZgLALla23OsEM0TqVEmEmot?= =?us-ascii?Q?yJeKPNi3RcdW7tvXibnRSv3QAM0fNeCwYpa7Pv+qpC6nT4P0X/q3WGJqBnOP?= =?us-ascii?Q?FfbWMfBwcoJxej5uvbG/zaM3p9NmX8o2EiQ4m9FLd56pAb/nwMXp+iCTa5jY?= =?us-ascii?Q?lxIL6qe4GAS2RD2RsQmI4cTzfnTumbeizz2YDU/1lq3I7CNibg0woWykyBp9?= =?us-ascii?Q?ecxO98e1tbkuLdcNh59tdLwfn+aTVJws/GZi4TVsqGweQD6iCHw9FLPUMqjl?= =?us-ascii?Q?HwbLLVG7Vo5dC2FxBeajn0hMx3wuLh+fk1rljVqOlDDqW/MwjbWOE5fX7z73?= =?us-ascii?Q?0qvkzPkq6h6S+TJnVso9CLGWFlyyk8ecpmkoubJqeVfDX0Or0/4rHItw57zS?= =?us-ascii?Q?IEosJjYCmRs80lLD/sg6kLIMPTl1YWFoMp4K4EQSUOzf3zjQA2P2TRYA9ohh?= =?us-ascii?Q?q3wh/cX/HMOCKbQikY0w+PQyTbn6xyUuwyFTFKZ82tJWX0fjZYdxAYRJLBNz?= =?us-ascii?Q?nOXq0JOksloJRmYd1qmQ95fYhav4ANNurf8MgeJJS+EmLWgkKroOo1bVx5gP?= =?us-ascii?Q?fMB4blecqJQ+akqeQg/oA0AYDaSU6JR7TbieoDdcmHs2NgqDF+OmeSMHhicn?= =?us-ascii?Q?KCFOPJLzLvtdtnvT01qRJ2Gh6lmBWDbrBSQAnXX9mkGbndMGva6FuYxPH82L?= =?us-ascii?Q?ivtkx4y8WBayviQoAXm2W7InWUGQCe3kxPv7hY3uvalBbzUbuSTER4eU7m31?= =?us-ascii?Q?LXMjd0tbxgI7jXbiujo9icxHTYy//byO1FhsxWnvWWDFRYKWh6hV/DwddNib?= =?us-ascii?Q?rWU1QzE+/aTJHuYFoOHFq5YvNvLucbPo23fX11tkkNsYJYcE9nyVzUiqfRBM?= =?us-ascii?Q?nKcXw8wNh9pFhEqHD9abdux0sVigSq9mWZzqKKcpw0KQWP0NjCmOEEHyE9Z1?= =?us-ascii?Q?EQ6Cbua13Z1RSR/MhtDdcDrZOhyIU+xx7bYJ/zO7kVycBcdiuW5YZrhLCgqB?= =?us-ascii?Q?62IAgWtDzX0jTtfaRfyIFP8TCZXdmqYWKJIE8azMiWq7tqWXz5AmvAYEPFDC?= =?us-ascii?Q?2bSut0TcjVxKzx16rbvdMpPcdOYzlQNai8yiM3JU018fA417FQuf1f9XL9ju?= =?us-ascii?Q?5YCU0zO1i2MZDKLzfT8xHcHpXbljsTS+psfX2VAcnk6ydBzYJARurWMByGoQ?= =?us-ascii?Q?QNqgz7MjmlGYA9PTbvewQJqPjqiRicSoL/iy5YAb+F0fFMs6cV8dMNoP/p49?= =?us-ascii?Q?JXnBa6M3iV8VGkDcV8AK+wv5MMCW9X6gjQQ9AOVKmxlnpI9AqUHgLz2vky5U?= =?us-ascii?Q?51nGr7/D/5B6cIsPI5hcs0mZghLfOXEVmtEtFwK8oUsdFr3oFhCKD4Vb1CYh?= =?us-ascii?Q?tCriNhndUZTPui6mv+mYfDqGrxoMsaEcN8HYQek2yCcJ6Qu8FaAJzQpFBCAc?= =?us-ascii?Q?lYT2QRlryiByW912OzYtE1O9+K58eV8eshkOcwQ9/gx77wJOXJ3Sp0C3btbo?= =?us-ascii?Q?gZBb5QKkfp4xRcWePLwrNsCqQv5eYgvPz6MQ?= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026)(7053199007); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2025 15:01:53.6960 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 65aa19c2-9a50-4cee-57c4-08ddcf7a0557 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0002256E.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6003 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 23.11.5 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 08/10/25. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://git.dpdk.org/dpdk-stable/log/?h=23.11-staging This queued commit can be viewed at: https://git.dpdk.org/dpdk-stable/commit/?h=23.11-staging&id=b5b33cbcaa6a0df137f259d43062d18aef367223 Thanks. Xueming Li --- >From b5b33cbcaa6a0df137f259d43062d18aef367223 Mon Sep 17 00:00:00 2001 From: Stephen Hemminger Date: Tue, 17 Jun 2025 08:00:16 -0700 Subject: [PATCH] latencystats: fix receive sample race Cc: Xueming Li [ upstream commit 4e12258d782e2511bf0d781faff23fada54ee7d8 ] The receive callback was not safe with multiple queues. If one receive queue callback decides to take a sample it needs to add that sample and do atomic update to the previous TSC sample value. Add a new lock for that. Optimize the check for when to take sample so that it only needs to lock when likely to need a sample. Also, add code to handle TSC wraparound in comparison. Perhaps this should move to rte_cycles.h? Bugzilla ID: 1723 Fixes: 5cd3cac9ed22 ("latency: added new library for latency stats") Cc: stable@dpdk.org Signed-off-by: Stephen Hemminger Tested-by: Thiyagarajan P Reviewed-by: Vipin Varghese --- .mailmap | 1 + lib/latencystats/rte_latencystats.c | 55 ++++++++++++++++++----------- 2 files changed, 36 insertions(+), 20 deletions(-) diff --git a/.mailmap b/.mailmap index 86920fea08..e64da3ab08 100644 --- a/.mailmap +++ b/.mailmap @@ -1460,6 +1460,7 @@ Thibaut Collet Thierry Herbelot Thierry Martin Thinh Tran +Thiyagarajan P Thomas Faivre Thomas F Herbert Thomas Graf diff --git a/lib/latencystats/rte_latencystats.c b/lib/latencystats/rte_latencystats.c index e47eac2cf8..a68eb3dcb6 100644 --- a/lib/latencystats/rte_latencystats.c +++ b/lib/latencystats/rte_latencystats.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "rte_latencystats.h" @@ -38,11 +39,20 @@ timestamp_dynfield(struct rte_mbuf *mbuf) timestamp_dynfield_offset, rte_mbuf_timestamp_t *); } +/* Compare two 64 bit timer counter but deal with wraparound correctly. */ +static inline bool tsc_after(uint64_t t0, uint64_t t1) +{ + return (int64_t)(t1 - t0) < 0; +} + +#define tsc_before(a, b) tsc_after(b, a) + static const char *MZ_RTE_LATENCY_STATS = "rte_latencystats"; static int latency_stats_index; + +static rte_spinlock_t sample_lock = RTE_SPINLOCK_INITIALIZER; static uint64_t samp_intvl; -static uint64_t timer_tsc; -static uint64_t prev_tsc; +static RTE_ATOMIC(uint64_t) next_tsc; struct rte_latency_stats { float min_latency; /**< Minimum latency in nano seconds */ @@ -124,25 +134,29 @@ add_time_stamps(uint16_t pid __rte_unused, void *user_cb __rte_unused) { unsigned int i; - uint64_t diff_tsc, now; - - /* - * For every sample interval, - * time stamp is marked on one received packet. - */ - now = rte_rdtsc(); - for (i = 0; i < nb_pkts; i++) { - diff_tsc = now - prev_tsc; - timer_tsc += diff_tsc; - - if ((pkts[i]->ol_flags & timestamp_dynflag) == 0 - && (timer_tsc >= samp_intvl)) { - *timestamp_dynfield(pkts[i]) = now; - pkts[i]->ol_flags |= timestamp_dynflag; - timer_tsc = 0; + uint64_t now = rte_rdtsc(); + + /* Check without locking */ + if (likely(tsc_before(now, rte_atomic_load_explicit(&next_tsc, + rte_memory_order_relaxed)))) + return nb_pkts; + + /* Try and get sample, skip if sample is being done by other core. */ + if (likely(rte_spinlock_trylock(&sample_lock))) { + for (i = 0; i < nb_pkts; i++) { + struct rte_mbuf *m = pkts[i]; + + /* skip if already timestamped */ + if (unlikely(m->ol_flags & timestamp_dynflag)) + continue; + + m->ol_flags |= timestamp_dynflag; + *timestamp_dynfield(m) = now; + rte_atomic_store_explicit(&next_tsc, now + samp_intvl, + rte_memory_order_relaxed); + break; } - prev_tsc = now; - now = rte_rdtsc(); + rte_spinlock_unlock(&sample_lock); } return nb_pkts; @@ -235,6 +249,7 @@ rte_latencystats_init(uint64_t app_samp_intvl, glob_stats = mz->addr; rte_spinlock_init(&glob_stats->lock); samp_intvl = app_samp_intvl * latencystat_cycles_per_ns(); + next_tsc = rte_rdtsc(); /** Register latency stats with stats library */ for (i = 0; i < NUM_LATENCY_STATS; i++) -- 2.34.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2025-07-30 22:50:04.173146104 +0800 +++ 0017-latencystats-fix-receive-sample-race.patch 2025-07-30 22:50:03.036757880 +0800 @@ -1 +1 @@ -From 4e12258d782e2511bf0d781faff23fada54ee7d8 Mon Sep 17 00:00:00 2001 +From b5b33cbcaa6a0df137f259d43062d18aef367223 Mon Sep 17 00:00:00 2001 @@ -4,0 +5,3 @@ +Cc: Xueming Li + +[ upstream commit 4e12258d782e2511bf0d781faff23fada54ee7d8 ] @@ -30 +33 @@ -index 8e898e303e..1ea4f9446d 100644 +index 86920fea08..e64da3ab08 100644 @@ -33 +36 @@ -@@ -1553,6 +1553,7 @@ Thibaut Collet +@@ -1460,6 +1460,7 @@ Thibaut Collet @@ -42 +45 @@ -index 6873a44a92..72a58d78d1 100644 +index e47eac2cf8..a68eb3dcb6 100644 @@ -45 +48 @@ -@@ -22,6 +22,7 @@ +@@ -12,6 +12,7 @@ @@ -47,2 +50,2 @@ - #include - #include + #include + #include @@ -53 +56 @@ -@@ -45,11 +46,20 @@ timestamp_dynfield(struct rte_mbuf *mbuf) +@@ -38,11 +39,20 @@ timestamp_dynfield(struct rte_mbuf *mbuf) @@ -74,3 +77,3 @@ - #define LATENCY_AVG_SCALE 4 - #define LATENCY_JITTER_SCALE 16 -@@ -147,25 +157,29 @@ add_time_stamps(uint16_t pid __rte_unused, + struct rte_latency_stats { + float min_latency; /**< Minimum latency in nano seconds */ +@@ -124,25 +134,29 @@ add_time_stamps(uint16_t pid __rte_unused, @@ -124 +127 @@ -@@ -270,6 +284,7 @@ rte_latencystats_init(uint64_t app_samp_intvl, +@@ -235,6 +249,7 @@ rte_latencystats_init(uint64_t app_samp_intvl, @@ -127 +130 @@ - samp_intvl = (uint64_t)(app_samp_intvl * cycles_per_ns); + samp_intvl = app_samp_intvl * latencystat_cycles_per_ns();