From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4048D469D9 for ; Tue, 17 Jun 2025 17:03:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 30E064111B; Tue, 17 Jun 2025 17:03:02 +0200 (CEST) Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by mails.dpdk.org (Postfix) with ESMTP id 38CE740EE4 for ; Tue, 17 Jun 2025 17:03:00 +0200 (CEST) Received: by mail-qv1-f46.google.com with SMTP id 6a1803df08f44-6fae04a3795so61725126d6.3 for ; Tue, 17 Jun 2025 08:03:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1750172579; x=1750777379; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ytBze7v0nvqs/hU5+ElOBmyhxS3NBjPnERpOK1I0qdU=; b=OBAiVVa8Go+ifLAJ1Kg/4D8XufoysOJi3Uj7MfOPgHfWO19F2LxCqva7ZyOFTcd5yd Qr0UIe8h5Lyx9blYyC8hGoBp1O0OJf40mEcP/n5udDCcTYsvrjt3El8sP0QQvRHKe17h qV3tIW8E9bN4eFltWRKm9j+rdIpG8hc8Sy8p3B9EvhTkwY4UgT9ysQakZfwfeLwV1wN5 6XQsSLlprhKKIiZJnXAT4nMkSBNRX9vPcs9WAE/A1B8317DPy8ouwIz3dHrtam0mP6H3 97+/BAyAlk2fktfsWB0ls09vWw5Ri6zbOB3t7qvNPSAAwPbVSDUigeYxp97R3KWwWSNz 1dDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750172579; x=1750777379; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ytBze7v0nvqs/hU5+ElOBmyhxS3NBjPnERpOK1I0qdU=; b=IKc42N/A4D9+x6e9W7lyFauN7SWOT5riMCA1yalw80n0IXVNGY1ARcF22T0GNTB/JK npDhzyeLzfu90ATBA0zLfyVWeCZsCLA6m1/6wO0BfSIlZIZoMT2Ql1QiAYt65uuUUZGE eggetyDcb2IhwVzeGEZ+BwWxsYDj/npILpILULf45l3Ih9A8AsiidSVP1/o20EU75ldz JjMEEYo4tAicTA7W1tnqRC5nmktV7ky/i+VRJK1VKbriqBE32uEtZicetESJ+SGzThP+ WNYR23WVW63f6f5U03kQUH8EIQfr6Wnmlf0FvpL+3Su6BWWpul9i0D9WdnJJvS5YCx+L ifGw== X-Forwarded-Encrypted: i=1; AJvYcCVilM31hZCZA2s2zRxQmSdnNlCNaO3iNb7LP8PaFJtLF5r2vNIviqI9FW/yjnqIh8GKByZykBw=@dpdk.org X-Gm-Message-State: AOJu0Yx9GB2gY6XxzfhEwJdBhPuHv53VKgoI1CD1ZOazDZYjPD/0hrZg Gi5FaT5EXBLJhZvuD8Z0ksp7FN8Qt7uSTfffcGxdziRtGdYvYo4tVQ6b9qHf4zlDZQdYoS/m7IC tk9IK X-Gm-Gg: ASbGncv2Eqf5pwYgUfy9uaGEc2ka0sAz3xcM62GLEqFH46IgPbwtqHhGE5GPANGHiSO KzAHEmhau9clVz1ourMPQ16iZNy7rImkUaKdApiRDIg9FPQCA2rXoHi/sYr0IkWMWzxl4IzN3ma DL0piGN1DADux2/TRhwz/ExyhwDZZVlgnTCK68Jz66VZ7TBFn8Lk1YH+yILbDTJWS1Mu2+xNiF2 lgL/NTn0Ybv4RL/iuOEvISt44E4U/KYxUVQkQtABOLijM9LRW0jExjvFIqi48v9hrMX/2upKz/L z5feJW7QWxCq9GdUpcb6CtKPnmHxNeD5OxtaluZGb/WA5ctdqfMUC5jgblLaNPga1t1DxRSCsmp H3BIrCY8Re8EoJJN/xROPPFd370jG2jbu//xi X-Google-Smtp-Source: AGHT+IHD4HY/j1d9ir9FpNE9cybhMWzV6mr8hCT/Oe4c0+PDfCyjRyykwy1fzaidrW2sLhTFPnKwSA== X-Received: by 2002:a05:6214:d07:b0:6e8:ebc6:fd5f with SMTP id 6a1803df08f44-6fb47758efamr216456416d6.20.1750172577158; Tue, 17 Jun 2025 08:02:57 -0700 (PDT) Received: from hermes.lan (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6fb5a9dbf8csm13051046d6.106.2025.06.17.08.02.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jun 2025 08:02:56 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , stable@dpdk.org Subject: [PATCH v3 1/2] latencystats: fix receive sample MP issues Date: Tue, 17 Jun 2025 08:00:16 -0700 Message-ID: <20250617150252.814215-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250617150252.814215-1-stephen@networkplumber.org> References: <20250613003547.39239-1-stephen@networkplumber.org> <20250617150252.814215-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org The receive callback was not safe with multiple queues. If one receive queue callback decides to take a sample it needs to add that sample and do atomic update to the previous TSC sample value. Add a new lock for that. Optimize the check for when to take sample so that it only needs to lock when likely to need a sample. Also, add code to handle TSC wraparound in comparison. Perhaps this should move to rte_cycles.h? Bugzilla ID: 1723 Signed-off-by: Stephen Hemminger Fixes: 5cd3cac9ed22 ("latency: added new library for latency stats") Cc: stable@dpdk.org --- lib/latencystats/rte_latencystats.c | 55 ++++++++++++++++++----------- 1 file changed, 35 insertions(+), 20 deletions(-) diff --git a/lib/latencystats/rte_latencystats.c b/lib/latencystats/rte_latencystats.c index 6873a44a92..72a58d78d1 100644 --- a/lib/latencystats/rte_latencystats.c +++ b/lib/latencystats/rte_latencystats.c @@ -22,6 +22,7 @@ #include #include #include +#include #include "rte_latencystats.h" @@ -45,11 +46,20 @@ timestamp_dynfield(struct rte_mbuf *mbuf) timestamp_dynfield_offset, rte_mbuf_timestamp_t *); } +/* Compare two 64 bit timer counter but deal with wraparound correctly. */ +static inline bool tsc_after(uint64_t t0, uint64_t t1) +{ + return (int64_t)(t1 - t0) < 0; +} + +#define tsc_before(a, b) tsc_after(b, a) + static const char *MZ_RTE_LATENCY_STATS = "rte_latencystats"; static int latency_stats_index; + +static rte_spinlock_t sample_lock = RTE_SPINLOCK_INITIALIZER; static uint64_t samp_intvl; -static uint64_t timer_tsc; -static uint64_t prev_tsc; +static RTE_ATOMIC(uint64_t) next_tsc; #define LATENCY_AVG_SCALE 4 #define LATENCY_JITTER_SCALE 16 @@ -147,25 +157,29 @@ add_time_stamps(uint16_t pid __rte_unused, void *user_cb __rte_unused) { unsigned int i; - uint64_t diff_tsc, now; - - /* - * For every sample interval, - * time stamp is marked on one received packet. - */ - now = rte_rdtsc(); - for (i = 0; i < nb_pkts; i++) { - diff_tsc = now - prev_tsc; - timer_tsc += diff_tsc; - - if ((pkts[i]->ol_flags & timestamp_dynflag) == 0 - && (timer_tsc >= samp_intvl)) { - *timestamp_dynfield(pkts[i]) = now; - pkts[i]->ol_flags |= timestamp_dynflag; - timer_tsc = 0; + uint64_t now = rte_rdtsc(); + + /* Check without locking */ + if (likely(tsc_before(now, rte_atomic_load_explicit(&next_tsc, + rte_memory_order_relaxed)))) + return nb_pkts; + + /* Try and get sample, skip if sample is being done by other core. */ + if (likely(rte_spinlock_trylock(&sample_lock))) { + for (i = 0; i < nb_pkts; i++) { + struct rte_mbuf *m = pkts[i]; + + /* skip if already timestamped */ + if (unlikely(m->ol_flags & timestamp_dynflag)) + continue; + + m->ol_flags |= timestamp_dynflag; + *timestamp_dynfield(m) = now; + rte_atomic_store_explicit(&next_tsc, now + samp_intvl, + rte_memory_order_relaxed); + break; } - prev_tsc = now; - now = rte_rdtsc(); + rte_spinlock_unlock(&sample_lock); } return nb_pkts; @@ -270,6 +284,7 @@ rte_latencystats_init(uint64_t app_samp_intvl, glob_stats = mz->addr; rte_spinlock_init(&glob_stats->lock); samp_intvl = (uint64_t)(app_samp_intvl * cycles_per_ns); + next_tsc = rte_rdtsc(); /** Register latency stats with stats library */ for (i = 0; i < NUM_LATENCY_STATS; i++) -- 2.47.2