From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD17B46900 for ; Mon, 16 Jun 2025 18:07:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AF29740677; Mon, 16 Jun 2025 18:07:30 +0200 (CEST) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by mails.dpdk.org (Postfix) with ESMTP id 402F240668 for ; Mon, 16 Jun 2025 18:07:29 +0200 (CEST) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-7390d21bb1cso3654953b3a.2 for ; Mon, 16 Jun 2025 09:07:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1750090048; x=1750694848; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9uNTSUdMT39Bx9qV+/2J8A+V80kOw2d+sDqfCWDt/tg=; b=iAjwZQw+o+q6F/yptI+r7Smd8nJo53c+zjm0xtTH0fhQE7gpLBDeRTtcJztk2cHwXq NPizCowh9mh1AemyyL+6cRR37e3x8tF3kaiJCjlEKYlvV0cSd/CqSWpY0j+KmoQ1iTdR T/fNqfYS8oUTVmUuKWyMTXmZk96fDBpPdY9nk9QZ+VrP+tDEoeKYR8jXDC/Kxcs9MlQt WyZw4UZ+cTSV+UHBsFhfojpP6hRP+PV3s8VQPHtWEgcMTTsKAD9kJ+WplOR9LE6cSK+K UUW34yRNipx9nMlzhxGh7RGpHuvh9khMXfPwGtrNVIDNl/EFdmwAUnssRhPh8vevHl71 V0pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750090048; x=1750694848; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9uNTSUdMT39Bx9qV+/2J8A+V80kOw2d+sDqfCWDt/tg=; b=Y4X4xWEAYpxdwYf3Rmy53uwTZc44DagBRON5JyGvEczMLr0p1TIIGc0gAS7N/lpe29 tfQ3JR+PToLb7mC6HjuSbF/9svHelWw3A2GNoK1uWR/I6Xyb8GITe/WQYqOErl94WhJd Bk86gKxDvDp3i/wDQTsKH4VYvjlj01J4yLG6q8AVYl7yrn2bi4caAGAhW9g70H0EODr/ /pJUhJSf6bgWmMBSEb1BYaT+DWNJHjKeD9CJhsbEK435IdfFCNW5FYWNCIM826mkj9b5 r4G8YvecBoqFM6lc5Wwwj1cLCp7qSPo4GUuFIyXoPUIIE28H+ARbNdTJVwVn9Tk2UW/q JRWQ== X-Forwarded-Encrypted: i=1; AJvYcCW0XUI3ZQ8jh2O2CEPh3OsV1L4SKlHXlKJUgOqeg5j1xEikDfOb/mbpVs9FSZ5x6a1qsAWA1KA=@dpdk.org X-Gm-Message-State: AOJu0Yw29Sm0r1rCwJIbRRlAdBAkOp5AQPAdY0tvqj4WkLU/Ru7MIJK0 h04wOFdwRWhSUjJc64Vg0/nvrqoEAL0oDLrN8jdl3aLK4Lx47PXotHfjfHyOmJy/r6U= X-Gm-Gg: ASbGncsjpUoVrjAfLMmkQW/VuKJL+LqxX2J5RlYfnMz8a2JNR6Ha/3q4yWHqVpO8xaj 4y4L8eQ8NAwSYqQA63S7Yxv6fwwzjeDU6/uV7j4gG6sFabzAz6hErSJvJNu4EtrUNrEsKA6Ir4w rSr2lRLABKyiVNkAhRdEKzoRWoDg1XqzviWHGPn9q8u1fIWkqmMBfLq98n1Q80lROvNZOL6DYbY rj5CShjZenpsQ09eVQviAffZQ+b516nIJ/+Tm5RmMs+pkGNj9BNDGF/zDne3H/pgjsdCshrLtk2 IYlt4AY3x6SBAfEHykFfSDYx2p1Hp3AFmNfy5nyia9Okl8W4xx1O814rBf8nIADc603MZdkLUJV H5EqqN6n5J0H/fd8DqE/jMYIcEGRyqsG6uwBl X-Google-Smtp-Source: AGHT+IEXRpVaWkBDuP3JnYHzW+LJKXpQXI5mImG1ZmSA/mIicIhEfW1g0O1uZ5DpnGfN8fwHoVrXHQ== X-Received: by 2002:a05:6a00:399d:b0:736:33fd:f57d with SMTP id d2e1a72fcca58-7489cfc360dmr10087573b3a.17.1750090048256; Mon, 16 Jun 2025 09:07:28 -0700 (PDT) Received: from hermes.lan (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-748900d252esm6992623b3a.163.2025.06.16.09.07.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Jun 2025 09:07:27 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , stable@dpdk.org Subject: [PATCH v2 1/2] latencystats: fix receive sample MP issues Date: Mon, 16 Jun 2025 09:04:32 -0700 Message-ID: <20250616160718.49938-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616160718.49938-1-stephen@networkplumber.org> References: <20250613003547.39239-1-stephen@networkplumber.org> <20250616160718.49938-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org The receive callback was not safe with multiple queues. If one receive queue callback decides to take a sample it needs to add that sample and do atomic update to the previous TSC sample value. Add a new lock for that. Optimize the check for when to take sample so that it only needs to lock when likely to need a sample. Also, add code to handle TSC wraparound in comparison. Perhaps this should move to rte_cycles.h? Signed-off-by: Stephen Hemminger Fixes: 5cd3cac9ed22 ("latency: added new library for latency stats") Cc: stable@dpdk.org --- lib/latencystats/rte_latencystats.c | 51 ++++++++++++++++++----------- 1 file changed, 32 insertions(+), 19 deletions(-) diff --git a/lib/latencystats/rte_latencystats.c b/lib/latencystats/rte_latencystats.c index 6873a44a92..2b994656fb 100644 --- a/lib/latencystats/rte_latencystats.c +++ b/lib/latencystats/rte_latencystats.c @@ -22,6 +22,7 @@ #include #include #include +#include #include "rte_latencystats.h" @@ -45,11 +46,20 @@ timestamp_dynfield(struct rte_mbuf *mbuf) timestamp_dynfield_offset, rte_mbuf_timestamp_t *); } +/* Compare two 64 bit timer counter but deal with wraparound correctly. */ +static inline bool tsc_after(uint64_t t0, uint64_t t1) +{ + return (int64_t)(t1 - t0) < 0; +} + +#define tsc_before(a, b) tsc_after(b, a) + static const char *MZ_RTE_LATENCY_STATS = "rte_latencystats"; static int latency_stats_index; + +static rte_spinlock_t sample_lock = RTE_SPINLOCK_INITIALIZER; static uint64_t samp_intvl; -static uint64_t timer_tsc; -static uint64_t prev_tsc; +static RTE_ATOMIC(uint64_t) next_tsc; #define LATENCY_AVG_SCALE 4 #define LATENCY_JITTER_SCALE 16 @@ -147,25 +157,27 @@ add_time_stamps(uint16_t pid __rte_unused, void *user_cb __rte_unused) { unsigned int i; - uint64_t diff_tsc, now; + uint64_t now = rte_rdtsc(); - /* - * For every sample interval, - * time stamp is marked on one received packet. - */ - now = rte_rdtsc(); - for (i = 0; i < nb_pkts; i++) { - diff_tsc = now - prev_tsc; - timer_tsc += diff_tsc; - - if ((pkts[i]->ol_flags & timestamp_dynflag) == 0 - && (timer_tsc >= samp_intvl)) { - *timestamp_dynfield(pkts[i]) = now; - pkts[i]->ol_flags |= timestamp_dynflag; - timer_tsc = 0; + /* Check without locking */ + if (likely(tsc_before(now, rte_atomic_load_explicit(&next_tsc, rte_memory_order_relaxed)))) + return nb_pkts; + + /* Try and get sample, skip if sample is being done by other core. */ + if (likely(rte_spinlock_trylock(&sample_lock))) { + for (i = 0; i < nb_pkts; i++) { + struct rte_mbuf *m = pkts[i]; + + /* skip if already timestamped */ + if (unlikely(m->ol_flags & timestamp_dynflag)) + continue; + + m->ol_flags |= timestamp_dynflag; + *timestamp_dynfield(m) = now; + rte_atomic_store_explicit(&next_tsc, now + samp_intvl, rte_memory_order_relaxed); + break; } - prev_tsc = now; - now = rte_rdtsc(); + rte_spinlock_unlock(&sample_lock); } return nb_pkts; @@ -270,6 +282,7 @@ rte_latencystats_init(uint64_t app_samp_intvl, glob_stats = mz->addr; rte_spinlock_init(&glob_stats->lock); samp_intvl = (uint64_t)(app_samp_intvl * cycles_per_ns); + next_tsc = rte_rdtsc(); /** Register latency stats with stats library */ for (i = 0; i < NUM_LATENCY_STATS; i++) -- 2.47.2