From: Artemy Kovalyov <artemyko@nvidia.com>
To: <dev@dpdk.org>
Cc: Thomas Monjalon <thomas@monjalon.net>, Ophir Munk <ophirmu@nvidia.com>
Subject: [PATCH v4 0/2] fix memory initialization deadlock
Date: Fri, 8 Sep 2023 16:17:34 +0300 [thread overview]
Message-ID: <20230908131737.1714750-1-artemyko@nvidia.com> (raw)
In-Reply-To: <20230830103303.2428995-1-artemyko@nvidia.com>
The issue arose due to the change in the DPDK read-write lock
implementation. That change added a new flag, RTE_RWLOCK_WAIT, designed
to prevent new read locks while a write lock is in the queue. However,
this change has led to a scenario where a recursive read lock, where a
lock is acquired twice by the same execution thread, can initiate a
sequence of events resulting in a deadlock:
Process 1 takes the first read lock.
Process 2 attempts to take a write lock, triggering RTE_RWLOCK_WAIT due
to the presence of a read lock. This makes process 2 enter a wait loop
until the read lock is released.
Process 1 tries to take a second read lock. However, since a write lock
is waiting (due to RTE_RWLOCK_WAIT), it also enters a wait loop until
the write lock is acquired and then released.
Both processes end up in a blocked state, unable to proceed, resulting
in a deadlock scenario.
Following these changes, the RW-lock no longer supports
recursion, implying that a single thread shouldn't obtain a read lock if
it already possesses one. The problem arises during initialization: the
rte_eal_init() function acquires the memory_hotplug_lock, and later on,
there are sequences of calls leading to rte_memseg_list_walk() which acquires
it again without releasing it. This scenario introduces the risk of a
potential deadlock when concurrent write locks are applied to the same
memory_hotplug_lock. To address this we resolved the issue by replacing
rte_memseg_list_walk() with rte_memseg_list_walk_thread_unsafe().
Implementing a lock annotation for rte_memseg_list_walk() to
proactively identify bugs similar to this one during compile time.
Artemy Kovalyov (2):
eal: fix memory initialization deadlock
eal: annotate rte_memseg_list_walk()
lib/eal/common/eal_common_dynmem.c | 5 ++++-
lib/eal/common/eal_memalloc.h | 3 ++-
lib/eal/common/eal_private.h | 3 ++-
lib/eal/include/generic/rte_rwlock.h | 4 ++++
lib/eal/include/rte_lock_annotations.h | 5 +++++
lib/eal/include/rte_memory.h | 4 +++-
lib/eal/linux/eal_memalloc.c | 7 +++++--
7 files changed, 25 insertions(+), 6 deletions(-)
--
1.8.3.1
next prev parent reply other threads:[~2023-09-08 13:17 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-30 10:33 [PATCH] eal: " Artemy Kovalyov
2023-08-30 19:13 ` Dmitry Kozlyuk
2023-09-04 8:24 ` [PATCH v2] " Artemy Kovalyov
2023-09-05 7:05 ` Dmitry Kozlyuk
2023-09-05 9:05 ` Artemy Kovalyov
2023-09-05 10:15 ` David Marchand
2023-09-06 9:52 ` [PATCH v3] " Artemy Kovalyov
2023-09-06 12:52 ` David Marchand
2023-09-08 13:17 ` Artemy Kovalyov [this message]
2023-09-08 13:17 ` [PATCH v4 1/2] " Artemy Kovalyov
2023-09-08 13:17 ` [PATCH v4 2/2] eal: annotate rte_memseg_list_walk() Artemy Kovalyov
2023-10-06 10:12 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230908131737.1714750-1-artemyko@nvidia.com \
--to=artemyko@nvidia.com \
--cc=dev@dpdk.org \
--cc=ophirmu@nvidia.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).