patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Kevin Traynor <ktraynor@redhat.com>
To: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
Cc: Gavin Hu <gavin.hu@arm.com>, dpdk stable <stable@dpdk.org>
Subject: [dpdk-stable] patch 'timer: fix race condition' has been queued to LTS release 18.11.1
Date: Fri,  4 Jan 2019 13:24:06 +0000	[thread overview]
Message-ID: <20190104132455.15170-24-ktraynor@redhat.com> (raw)
In-Reply-To: <20190104132455.15170-1-ktraynor@redhat.com>

Hi,

FYI, your patch has been queued to LTS release 18.11.1

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 01/11/19. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Thanks.

Kevin Traynor

---
>From 8e49368645c700b04341096cf9c93d0916b314d9 Mon Sep 17 00:00:00 2001
From: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
Date: Wed, 19 Dec 2018 10:09:34 -0600
Subject: [PATCH] timer: fix race condition

[ upstream commit 7079e29f7f28661b712620f46e6a8514eb0a708a ]

rte_timer_manage() adds expired timers to a "run list", and walks the
list, transitioning each timer from the PENDING to the RUNNING state.
If another lcore resets or stops the timer at precisely this
moment, the timer state would instead be set to CONFIG by that other
lcore, which would cause timer_manage() to skip over it. This is
expected behavior.

However, if a timer expires quickly enough, there exists the
following race condition that causes the timer_manage() routine to
misinterpret a timer in CONFIG state, resulting in lost timers:

- Thread A:
  - starts a timer with rte_timer_reset()
  - the timer is moved to CONFIG state
  - the spinlock associated with the appropriate skiplist is acquired
  - timer is inserted into the skiplist
  - the spinlock is released
- Thread B:
  - executes rte_timer_manage()
  - find above timer as expired, add it to run list
  - walk run list, see above timer still in CONFIG state, unlink it from
    run list and continue on
- Thread A:
  - move timer to PENDING state
  - return from rte_timer_reset()
  - timer is now in PENDING state, but not actually linked into a
    pending list or a run list and will never get processed further
    by rte_timer_manage()

This commit fixes this race condition by only releasing the spinlock
after the timer state has been transitioned from CONFIG to PENDING,
which prevents rte_timer_manage() from seeing an incorrect state.

Fixes: 9b15ba895b9f ("timer: use a skip list")

Signed-off-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
---
 lib/librte_timer/rte_timer.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c
index 590488c7e..30c7b0ab4 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -242,22 +242,15 @@ timer_get_prev_entries_for_node(struct rte_timer *tim, unsigned tim_lcore,
 }
 
-/*
- * add in list, lock if needed
+/* call with lock held as necessary
+ * add in list
  * timer must be in config state
  * timer must not be in a list
  */
 static void
-timer_add(struct rte_timer *tim, unsigned tim_lcore, int local_is_locked)
+timer_add(struct rte_timer *tim, unsigned int tim_lcore)
 {
-	unsigned lcore_id = rte_lcore_id();
 	unsigned lvl;
 	struct rte_timer *prev[MAX_SKIPLIST_DEPTH+1];
 
-	/* if timer needs to be scheduled on another core, we need to
-	 * lock the list; if it is on local core, we need to lock if
-	 * we are not called from rte_timer_manage() */
-	if (tim_lcore != lcore_id || !local_is_locked)
-		rte_spinlock_lock(&priv_timer[tim_lcore].list_lock);
-
 	/* find where exactly this element goes in the list of elements
 	 * for each depth. */
@@ -283,7 +276,4 @@ timer_add(struct rte_timer *tim, unsigned tim_lcore, int local_is_locked)
 	priv_timer[tim_lcore].pending_head.expire = priv_timer[tim_lcore].\
 			pending_head.sl_next[0]->expire;
-
-	if (tim_lcore != lcore_id || !local_is_locked)
-		rte_spinlock_unlock(&priv_timer[tim_lcore].list_lock);
 }
 
@@ -380,6 +370,13 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 	tim->arg = arg;
 
+	/* if timer needs to be scheduled on another core, we need to
+	 * lock the destination list; if it is on local core, we need to lock if
+	 * we are not called from rte_timer_manage()
+	 */
+	if (tim_lcore != lcore_id || !local_is_locked)
+		rte_spinlock_lock(&priv_timer[tim_lcore].list_lock);
+
 	__TIMER_STAT_ADD(pending, 1);
-	timer_add(tim, tim_lcore, local_is_locked);
+	timer_add(tim, tim_lcore);
 
 	/* update state: as we are in CONFIG state, only us can modify
@@ -390,4 +387,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 	tim->status.u32 = status.u32;
 
+	if (tim_lcore != lcore_id || !local_is_locked)
+		rte_spinlock_unlock(&priv_timer[tim_lcore].list_lock);
+
 	return 0;
 }
-- 
2.19.0

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2019-01-04 13:23:08.027948323 +0000
+++ 0024-timer-fix-race-condition.patch	2019-01-04 13:23:07.000000000 +0000
@@ -1,8 +1,10 @@
-From 7079e29f7f28661b712620f46e6a8514eb0a708a Mon Sep 17 00:00:00 2001
+From 8e49368645c700b04341096cf9c93d0916b314d9 Mon Sep 17 00:00:00 2001
 From: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
 Date: Wed, 19 Dec 2018 10:09:34 -0600
 Subject: [PATCH] timer: fix race condition
 
+[ upstream commit 7079e29f7f28661b712620f46e6a8514eb0a708a ]
+
 rte_timer_manage() adds expired timers to a "run list", and walks the
 list, transitioning each timer from the PENDING to the RUNNING state.
 If another lcore resets or stops the timer at precisely this
@@ -37,7 +39,6 @@
 which prevents rte_timer_manage() from seeing an incorrect state.
 
 Fixes: 9b15ba895b9f ("timer: use a skip list")
-Cc: stable@dpdk.org
 
 Signed-off-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
 Reviewed-by: Gavin Hu <gavin.hu@arm.com>

  parent reply	other threads:[~2019-01-04 13:26 UTC|newest]

Thread overview: 82+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-04 13:23 [dpdk-stable] patch 'config: enable C11 memory model for armv8 with meson' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'mk: do not install meson.build in usertools' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'log: add missing experimental tag' " Kevin Traynor
2019-01-10  9:52   ` David Marchand
2019-01-10 10:28     ` Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'bus/vmbus: fix race in subchannel creation' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'net/netvsc: enable SR-IOV' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'net/netvsc: disable multi-queue on older servers' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'bus/dpaa: do nothing if bus not present' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'doc: fix garbage text in generated HTML guides' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'eal: clean up unused files on initialization' " Kevin Traynor
2019-01-08 16:53   ` Burakov, Anatoly
2019-01-08 18:09     ` Kevin Traynor
2019-01-10 11:38       ` Burakov, Anatoly
2019-01-04 13:23 ` [dpdk-stable] patch 'gro: fix overflow of payload length calculation' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'eventdev: fix error log in eth Rx adapter' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'eventdev: remove redundant timer adapter function prototypes' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'app/eventdev: detect deadlock for timer event producer' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'eventdev: fix xstats documentation typo' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'eventdev: fix eth Tx adapter queue count checks' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'compressdev: fix structure comment' " Kevin Traynor
2019-01-04 13:23 ` [dpdk-stable] patch 'bb/turbo_sw: fix dynamic linking' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'crypto/qat: fix block size error handling' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'crypto/qat: fix message for CCM when setting unused counter' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'crypto/qat: fix message for NULL algo " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'common/qat: remove check of valid firmware response' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'compress/qat: fix return on building request error' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'compress/qat: fix dequeue error counter' " Kevin Traynor
2019-01-04 13:24 ` Kevin Traynor [this message]
2019-01-04 13:24 ` [dpdk-stable] patch 'ip_frag: fix IPv6 when MTU sizes not aligned to 8 bytes' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'eal: fix missing newline in a log' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'eal: fix detection of duplicate option register' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'eal: fix leak on multi-process request error' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'memzone: fix unlock on initialization failure' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'malloc: fix finding maximum contiguous IOVA size' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'malloc: notify primary process about hotplug in secondary' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'malloc: fix duplicate mem event notification' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'malloc: make alignment requirements more stringent' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'mem: fix segment fd API error code for external segment' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'mem: check for memfd support in segment fd API' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'doc: remove note on memory mode limitation in multi-process' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'test/mem: add external mem autotest to meson' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'test/fbarray: add " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'eal: close multi-process socket during cleanup' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'hash: fix return of bulk lookup' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'hash: fix out-of-bound write while freeing key slot' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'devtools: fix return of forbidden addition checks' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'malloc: fix deadlock when reading stats' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/i40e: clear VF reset flags after reset' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/i40e: fix statistics inconsistency' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/netvsc: fix transmit descriptor pool cleanup' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/netvsc: fix probe when VF not found' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'vhost: fix race condition when adding fd in the fdset' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ifc: store only registered device instance' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: add reset reason in Rx error' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: skip packet with wrong request id' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: destroy queues if start failed' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: do not reconfigure queues on reset' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: add supported RSS offloads types' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: fix invalid reference to variable in union' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: fix cleanup for out of order packets' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/ena: update completion queue after cleanup' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/cxgbe: fix overlapping regions in TID table' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/cxgbe: skip parsing match items with no spec' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/i40e: fix config name in comment' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/mlx5: fix Multi-Packet RQ mempool free' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net: fix underflow for checksum of invalid IPv4 packets' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/tap: add buffer overflow checks before checksum' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/vhost: fix double free of MAC address' " Kevin Traynor
2019-01-07  0:04   ` Hideyuki Yamashita
2019-01-07 10:23     ` Kevin Traynor
2019-01-09  7:39       ` Hideyuki Yamashita
2019-01-09 11:04         ` Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'vhost: enforce avail index and desc read ordering' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'vhost: enforce desc flags and content " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/af_packet: fix setting MTU decrements sockaddr twice' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/tap: fix possible uninitialized variable access' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/avf/base: fix comment referencing internal data' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'net/sfc: pass HW Tx queue index on creation' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'telemetry: fix using ports of different types' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'sched: fix memory leak on init failure' " Kevin Traynor
2019-01-04 13:24 ` [dpdk-stable] patch 'app/testpmd: expand RED queue thresholds to 64 bits' " Kevin Traynor

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190104132455.15170-24-ktraynor@redhat.com \
    --to=ktraynor@redhat.com \
    --cc=erik.g.carrillo@intel.com \
    --cc=gavin.hu@arm.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).