* [PATCH] vhost: avoid sleeping under mutex
@ 2023-03-22 17:05 David Marchand
2023-05-11 8:10 ` Xia, Chenbo
2023-05-15 11:18 ` [PATCH v2] " David Marchand
0 siblings, 2 replies; 5+ messages in thread
From: David Marchand @ 2023-03-22 17:05 UTC (permalink / raw)
To: dev; +Cc: stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu
Covscan reported:
2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
"pthread_mutex_lock" acquires lock "vhost_user.mutex".
23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
"vhost_user_reconnect_init" might sleep while holding lock
"vhost_user.mutex".
# 953| vsocket->reconnect =
!(flags & RTE_VHOST_USER_NO_RECONNECT);
# 954| if (vsocket->reconnect && reconn_tid == 0) {
# 955|-> if (vhost_user_reconnect_init() != 0)
# 956| goto out_mutex;
# 957| }
The reason for this warning is that vhost_user_reconnect_init() creates a
ctrl thread and calls nanosleep waiting for this thread to be ready,
while vhost_user.mutex is taken.
Move the call to vhost_user_reconnect_init() out of this mutex.
While at it, a pthread_t value should be considered opaque.
Instead of relying reconn_tid == 0, use an internal flag in
vhost_user_reconnect_init().
Coverity issue: 373686
Bugzilla ID: 981
Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
lib/vhost/socket.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
index 669c322e12..21002848e6 100644
--- a/lib/vhost/socket.c
+++ b/lib/vhost/socket.c
@@ -498,8 +498,12 @@ vhost_user_client_reconnect(void *arg __rte_unused)
static int
vhost_user_reconnect_init(void)
{
+ static bool reconn_init_done;
int ret;
+ if (reconn_init_done)
+ return 0;
+
ret = pthread_mutex_init(&reconn_list.mutex, NULL);
if (ret < 0) {
VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize mutex\n", __func__);
@@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
VHOST_LOG_CONFIG("thread", ERR,
"%s: failed to destroy reconnect mutex\n",
__func__);
+ } else {
+ reconn_init_done = true;
}
return ret;
@@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
if (!path)
return -1;
+ if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
+ (flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
+ vhost_user_reconnect_init() != 0)
+ return -1;
+
pthread_mutex_lock(&vhost_user.mutex);
if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
@@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
}
if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
- vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
- if (vsocket->reconnect && reconn_tid == 0) {
- if (vhost_user_reconnect_init() != 0)
- goto out_mutex;
- }
+ vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) == 0;
} else {
vsocket->is_server = true;
}
--
2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: [PATCH] vhost: avoid sleeping under mutex
2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
@ 2023-05-11 8:10 ` Xia, Chenbo
2023-05-15 11:22 ` David Marchand
2023-05-15 11:18 ` [PATCH v2] " David Marchand
1 sibling, 1 reply; 5+ messages in thread
From: Xia, Chenbo @ 2023-05-11 8:10 UTC (permalink / raw)
To: David Marchand, dev; +Cc: stable, Maxime Coquelin, Yuanhan Liu
Hi David,
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Thursday, March 23, 2023 1:05 AM
> To: dev@dpdk.org
> Cc: stable@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Xia,
> Chenbo <chenbo.xia@intel.com>; Yuanhan Liu <yliu@fridaylinux.org>
> Subject: [PATCH] vhost: avoid sleeping under mutex
>
> Covscan reported:
>
> 2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> "pthread_mutex_lock" acquires lock "vhost_user.mutex".
> 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> "vhost_user_reconnect_init" might sleep while holding lock
> "vhost_user.mutex".
> # 953| vsocket->reconnect =
> !(flags & RTE_VHOST_USER_NO_RECONNECT);
> # 954| if (vsocket->reconnect && reconn_tid == 0) {
> # 955|-> if (vhost_user_reconnect_init() != 0)
> # 956| goto out_mutex;
> # 957| }
>
> The reason for this warning is that vhost_user_reconnect_init() creates a
> ctrl thread and calls nanosleep waiting for this thread to be ready,
> while vhost_user.mutex is taken.
>
> Move the call to vhost_user_reconnect_init() out of this mutex.
>
> While at it, a pthread_t value should be considered opaque.
> Instead of relying reconn_tid == 0, use an internal flag in
> vhost_user_reconnect_init().
Should we make reconn_tid not a global variable anymore then?
Maybe a static pthread_t in vhost_user_reconnect_init?
Thanks,
Chenbo
>
> Coverity issue: 373686
> Bugzilla ID: 981
> Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> lib/vhost/socket.c | 17 ++++++++++++-----
> 1 file changed, 12 insertions(+), 5 deletions(-)
>
> diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
> index 669c322e12..21002848e6 100644
> --- a/lib/vhost/socket.c
> +++ b/lib/vhost/socket.c
> @@ -498,8 +498,12 @@ vhost_user_client_reconnect(void *arg __rte_unused)
> static int
> vhost_user_reconnect_init(void)
> {
> + static bool reconn_init_done;
> int ret;
>
> + if (reconn_init_done)
> + return 0;
> +
> ret = pthread_mutex_init(&reconn_list.mutex, NULL);
> if (ret < 0) {
> VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize
> mutex\n", __func__);
> @@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
> VHOST_LOG_CONFIG("thread", ERR,
> "%s: failed to destroy reconnect mutex\n",
> __func__);
> + } else {
> + reconn_init_done = true;
> }
>
> return ret;
> @@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
> if (!path)
> return -1;
>
> + if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
> + (flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
> + vhost_user_reconnect_init() != 0)
> + return -1;
> +
> pthread_mutex_lock(&vhost_user.mutex);
>
> if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
> @@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
> }
>
> if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
> - vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
> - if (vsocket->reconnect && reconn_tid == 0) {
> - if (vhost_user_reconnect_init() != 0)
> - goto out_mutex;
> - }
> + vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) ==
> 0;
> } else {
> vsocket->is_server = true;
> }
> --
> 2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v2] vhost: avoid sleeping under mutex
2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
2023-05-11 8:10 ` Xia, Chenbo
@ 2023-05-15 11:18 ` David Marchand
2023-05-15 15:51 ` Stephen Hemminger
1 sibling, 1 reply; 5+ messages in thread
From: David Marchand @ 2023-05-15 11:18 UTC (permalink / raw)
To: dev; +Cc: stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu
Covscan reported:
2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
"pthread_mutex_lock" acquires lock "vhost_user.mutex".
23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
"vhost_user_reconnect_init" might sleep while holding lock
"vhost_user.mutex".
# 953| vsocket->reconnect =
!(flags & RTE_VHOST_USER_NO_RECONNECT);
# 954| if (vsocket->reconnect && reconn_tid == 0) {
# 955|-> if (vhost_user_reconnect_init() != 0)
# 956| goto out_mutex;
# 957| }
The reason for this warning is that vhost_user_reconnect_init() creates a
ctrl thread and calls nanosleep waiting for this thread to be ready,
while vhost_user.mutex is taken.
Move the call to vhost_user_reconnect_init() out of this mutex.
While at it, a pthread_t value should be considered opaque.
Instead of relying reconn_tid == 0, use an internal flag in
vhost_user_reconnect_init().
Coverity issue: 373686
Bugzilla ID: 981
Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changes since v1:
- moved reconn_tid in vhost_user_reconnect_init as this
variable is not used anywhere else,
---
lib/vhost/socket.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
index 669c322e12..00a912c59e 100644
--- a/lib/vhost/socket.c
+++ b/lib/vhost/socket.c
@@ -427,7 +427,6 @@ struct vhost_user_reconnect_list {
};
static struct vhost_user_reconnect_list reconn_list;
-static pthread_t reconn_tid;
static int
vhost_user_connect_nonblock(char *path, int fd, struct sockaddr *un, size_t sz)
@@ -498,8 +497,13 @@ vhost_user_client_reconnect(void *arg __rte_unused)
static int
vhost_user_reconnect_init(void)
{
+ static bool reconn_init_done;
+ static pthread_t reconn_tid;
int ret;
+ if (reconn_init_done)
+ return 0;
+
ret = pthread_mutex_init(&reconn_list.mutex, NULL);
if (ret < 0) {
VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize mutex\n", __func__);
@@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
VHOST_LOG_CONFIG("thread", ERR,
"%s: failed to destroy reconnect mutex\n",
__func__);
+ } else {
+ reconn_init_done = true;
}
return ret;
@@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
if (!path)
return -1;
+ if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
+ (flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
+ vhost_user_reconnect_init() != 0)
+ return -1;
+
pthread_mutex_lock(&vhost_user.mutex);
if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
@@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
}
if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
- vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
- if (vsocket->reconnect && reconn_tid == 0) {
- if (vhost_user_reconnect_init() != 0)
- goto out_mutex;
- }
+ vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) == 0;
} else {
vsocket->is_server = true;
}
--
2.40.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] vhost: avoid sleeping under mutex
2023-05-11 8:10 ` Xia, Chenbo
@ 2023-05-15 11:22 ` David Marchand
0 siblings, 0 replies; 5+ messages in thread
From: David Marchand @ 2023-05-15 11:22 UTC (permalink / raw)
To: Xia, Chenbo; +Cc: dev, stable, Maxime Coquelin, Yuanhan Liu
On Thu, May 11, 2023 at 10:10 AM Xia, Chenbo <chenbo.xia@intel.com> wrote:
> > Covscan reported:
> >
> > 2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> > "pthread_mutex_lock" acquires lock "vhost_user.mutex".
> > 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> > "vhost_user_reconnect_init" might sleep while holding lock
> > "vhost_user.mutex".
> > # 953| vsocket->reconnect =
> > !(flags & RTE_VHOST_USER_NO_RECONNECT);
> > # 954| if (vsocket->reconnect && reconn_tid == 0) {
> > # 955|-> if (vhost_user_reconnect_init() != 0)
> > # 956| goto out_mutex;
> > # 957| }
> >
> > The reason for this warning is that vhost_user_reconnect_init() creates a
> > ctrl thread and calls nanosleep waiting for this thread to be ready,
> > while vhost_user.mutex is taken.
> >
> > Move the call to vhost_user_reconnect_init() out of this mutex.
> >
> > While at it, a pthread_t value should be considered opaque.
> > Instead of relying reconn_tid == 0, use an internal flag in
> > vhost_user_reconnect_init().
>
> Should we make reconn_tid not a global variable anymore then?
> Maybe a static pthread_t in vhost_user_reconnect_init?
Yep, I just sent a v2.
Thanks Chenbo.
--
David Marchand
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v2] vhost: avoid sleeping under mutex
2023-05-15 11:18 ` [PATCH v2] " David Marchand
@ 2023-05-15 15:51 ` Stephen Hemminger
0 siblings, 0 replies; 5+ messages in thread
From: Stephen Hemminger @ 2023-05-15 15:51 UTC (permalink / raw)
To: David Marchand; +Cc: dev, stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu
On Mon, 15 May 2023 13:18:44 +0200
David Marchand <david.marchand@redhat.com> wrote:
> Covscan reported:
>
> 2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> "pthread_mutex_lock" acquires lock "vhost_user.mutex".
> 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> "vhost_user_reconnect_init" might sleep while holding lock
> "vhost_user.mutex".
> # 953| vsocket->reconnect =
> !(flags & RTE_VHOST_USER_NO_RECONNECT);
> # 954| if (vsocket->reconnect && reconn_tid == 0) {
> # 955|-> if (vhost_user_reconnect_init() != 0)
> # 956| goto out_mutex;
> # 957| }
>
> The reason for this warning is that vhost_user_reconnect_init() creates a
> ctrl thread and calls nanosleep waiting for this thread to be ready,
> while vhost_user.mutex is taken.
>
> Move the call to vhost_user_reconnect_init() out of this mutex.
>
> While at it, a pthread_t value should be considered opaque.
> Instead of relying reconn_tid == 0, use an internal flag in
> vhost_user_reconnect_init().
>
> Coverity issue: 373686
> Bugzilla ID: 981
> Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Changes since v1:
> - moved reconn_tid in vhost_user_reconnect_init as this
> variable is not used anywhere else,
>
> ---
> lib/vhost/socket.c | 19 +++++++++++++------
> 1 file changed, 13 insertions(+), 6 deletions(-)
>
> diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
> index 669c322e12..00a912c59e 100644
> --- a/lib/vhost/socket.c
> +++ b/lib/vhost/socket.c
> @@ -427,7 +427,6 @@ struct vhost_user_reconnect_list {
> };
>
> static struct vhost_user_reconnect_list reconn_list;
> -static pthread_t reconn_tid;
>
> static int
> vhost_user_connect_nonblock(char *path, int fd, struct sockaddr *un, size_t sz)
> @@ -498,8 +497,13 @@ vhost_user_client_reconnect(void *arg __rte_unused)
> static int
> vhost_user_reconnect_init(void)
> {
> + static bool reconn_init_done;
> + static pthread_t reconn_tid;
This would not be thread safe.
Perhaps use a pthread condition variable instead of spinning here.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-05-15 15:51 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
2023-05-11 8:10 ` Xia, Chenbo
2023-05-15 11:22 ` David Marchand
2023-05-15 11:18 ` [PATCH v2] " David Marchand
2023-05-15 15:51 ` Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).