patches for DPDK stable branches
 help / color / mirror / Atom feed
* [PATCH] vhost: avoid sleeping under mutex
@ 2023-03-22 17:05 David Marchand
  2023-05-11  8:10 ` Xia, Chenbo
  2023-05-15 11:18 ` [PATCH v2] " David Marchand
  0 siblings, 2 replies; 5+ messages in thread
From: David Marchand @ 2023-03-22 17:05 UTC (permalink / raw)
  To: dev; +Cc: stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu

Covscan reported:

 2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
	"pthread_mutex_lock" acquires lock "vhost_user.mutex".
23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
	"vhost_user_reconnect_init" might sleep while holding lock
	"vhost_user.mutex".
 #   953|   		vsocket->reconnect =
	!(flags & RTE_VHOST_USER_NO_RECONNECT);
 #   954|   		if (vsocket->reconnect && reconn_tid == 0) {
 #   955|-> 			if (vhost_user_reconnect_init() != 0)
 #   956|   				goto out_mutex;
 #   957|   		}

The reason for this warning is that vhost_user_reconnect_init() creates a
ctrl thread and calls nanosleep waiting for this thread to be ready,
while vhost_user.mutex is taken.

Move the call to vhost_user_reconnect_init() out of this mutex.

While at it, a pthread_t value should be considered opaque.
Instead of relying reconn_tid == 0, use an internal flag in
vhost_user_reconnect_init().

Coverity issue: 373686
Bugzilla ID: 981
Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 lib/vhost/socket.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
index 669c322e12..21002848e6 100644
--- a/lib/vhost/socket.c
+++ b/lib/vhost/socket.c
@@ -498,8 +498,12 @@ vhost_user_client_reconnect(void *arg __rte_unused)
 static int
 vhost_user_reconnect_init(void)
 {
+	static bool reconn_init_done;
 	int ret;
 
+	if (reconn_init_done)
+		return 0;
+
 	ret = pthread_mutex_init(&reconn_list.mutex, NULL);
 	if (ret < 0) {
 		VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize mutex\n", __func__);
@@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
 			VHOST_LOG_CONFIG("thread", ERR,
 				"%s: failed to destroy reconnect mutex\n",
 				__func__);
+	} else {
+		reconn_init_done = true;
 	}
 
 	return ret;
@@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	if (!path)
 		return -1;
 
+	if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
+			(flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
+			vhost_user_reconnect_init() != 0)
+		return -1;
+
 	pthread_mutex_lock(&vhost_user.mutex);
 
 	if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
@@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	}
 
 	if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
-		vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
-		if (vsocket->reconnect && reconn_tid == 0) {
-			if (vhost_user_reconnect_init() != 0)
-				goto out_mutex;
-		}
+		vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) == 0;
 	} else {
 		vsocket->is_server = true;
 	}
-- 
2.39.2


^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: [PATCH] vhost: avoid sleeping under mutex
  2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
@ 2023-05-11  8:10 ` Xia, Chenbo
  2023-05-15 11:22   ` David Marchand
  2023-05-15 11:18 ` [PATCH v2] " David Marchand
  1 sibling, 1 reply; 5+ messages in thread
From: Xia, Chenbo @ 2023-05-11  8:10 UTC (permalink / raw)
  To: David Marchand, dev; +Cc: stable, Maxime Coquelin, Yuanhan Liu

Hi David,

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Thursday, March 23, 2023 1:05 AM
> To: dev@dpdk.org
> Cc: stable@dpdk.org; Maxime Coquelin <maxime.coquelin@redhat.com>; Xia,
> Chenbo <chenbo.xia@intel.com>; Yuanhan Liu <yliu@fridaylinux.org>
> Subject: [PATCH] vhost: avoid sleeping under mutex
> 
> Covscan reported:
> 
>  2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> 	"pthread_mutex_lock" acquires lock "vhost_user.mutex".
> 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> 	"vhost_user_reconnect_init" might sleep while holding lock
> 	"vhost_user.mutex".
>  #   953|   		vsocket->reconnect =
> 	!(flags & RTE_VHOST_USER_NO_RECONNECT);
>  #   954|   		if (vsocket->reconnect && reconn_tid == 0) {
>  #   955|-> 			if (vhost_user_reconnect_init() != 0)
>  #   956|   				goto out_mutex;
>  #   957|   		}
> 
> The reason for this warning is that vhost_user_reconnect_init() creates a
> ctrl thread and calls nanosleep waiting for this thread to be ready,
> while vhost_user.mutex is taken.
> 
> Move the call to vhost_user_reconnect_init() out of this mutex.
> 
> While at it, a pthread_t value should be considered opaque.
> Instead of relying reconn_tid == 0, use an internal flag in
> vhost_user_reconnect_init().

Should we make reconn_tid not a global variable anymore then?
Maybe a static pthread_t in vhost_user_reconnect_init?

Thanks,
Chenbo

> 
> Coverity issue: 373686
> Bugzilla ID: 981
> Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
>  lib/vhost/socket.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
> index 669c322e12..21002848e6 100644
> --- a/lib/vhost/socket.c
> +++ b/lib/vhost/socket.c
> @@ -498,8 +498,12 @@ vhost_user_client_reconnect(void *arg __rte_unused)
>  static int
>  vhost_user_reconnect_init(void)
>  {
> +	static bool reconn_init_done;
>  	int ret;
> 
> +	if (reconn_init_done)
> +		return 0;
> +
>  	ret = pthread_mutex_init(&reconn_list.mutex, NULL);
>  	if (ret < 0) {
>  		VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize
> mutex\n", __func__);
> @@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
>  			VHOST_LOG_CONFIG("thread", ERR,
>  				"%s: failed to destroy reconnect mutex\n",
>  				__func__);
> +	} else {
> +		reconn_init_done = true;
>  	}
> 
>  	return ret;
> @@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
>  	if (!path)
>  		return -1;
> 
> +	if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
> +			(flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
> +			vhost_user_reconnect_init() != 0)
> +		return -1;
> +
>  	pthread_mutex_lock(&vhost_user.mutex);
> 
>  	if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
> @@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t
> flags)
>  	}
> 
>  	if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
> -		vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
> -		if (vsocket->reconnect && reconn_tid == 0) {
> -			if (vhost_user_reconnect_init() != 0)
> -				goto out_mutex;
> -		}
> +		vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) ==
> 0;
>  	} else {
>  		vsocket->is_server = true;
>  	}
> --
> 2.39.2


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v2] vhost: avoid sleeping under mutex
  2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
  2023-05-11  8:10 ` Xia, Chenbo
@ 2023-05-15 11:18 ` David Marchand
  2023-05-15 15:51   ` Stephen Hemminger
  1 sibling, 1 reply; 5+ messages in thread
From: David Marchand @ 2023-05-15 11:18 UTC (permalink / raw)
  To: dev; +Cc: stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu

Covscan reported:

 2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
	"pthread_mutex_lock" acquires lock "vhost_user.mutex".
23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
	"vhost_user_reconnect_init" might sleep while holding lock
	"vhost_user.mutex".
 #   953|   		vsocket->reconnect =
	!(flags & RTE_VHOST_USER_NO_RECONNECT);
 #   954|   		if (vsocket->reconnect && reconn_tid == 0) {
 #   955|-> 			if (vhost_user_reconnect_init() != 0)
 #   956|   				goto out_mutex;
 #   957|   		}

The reason for this warning is that vhost_user_reconnect_init() creates a
ctrl thread and calls nanosleep waiting for this thread to be ready,
while vhost_user.mutex is taken.

Move the call to vhost_user_reconnect_init() out of this mutex.

While at it, a pthread_t value should be considered opaque.
Instead of relying reconn_tid == 0, use an internal flag in
vhost_user_reconnect_init().

Coverity issue: 373686
Bugzilla ID: 981
Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changes since v1:
- moved reconn_tid in vhost_user_reconnect_init as this
  variable is not used anywhere else,

---
 lib/vhost/socket.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
index 669c322e12..00a912c59e 100644
--- a/lib/vhost/socket.c
+++ b/lib/vhost/socket.c
@@ -427,7 +427,6 @@ struct vhost_user_reconnect_list {
 };
 
 static struct vhost_user_reconnect_list reconn_list;
-static pthread_t reconn_tid;
 
 static int
 vhost_user_connect_nonblock(char *path, int fd, struct sockaddr *un, size_t sz)
@@ -498,8 +497,13 @@ vhost_user_client_reconnect(void *arg __rte_unused)
 static int
 vhost_user_reconnect_init(void)
 {
+	static bool reconn_init_done;
+	static pthread_t reconn_tid;
 	int ret;
 
+	if (reconn_init_done)
+		return 0;
+
 	ret = pthread_mutex_init(&reconn_list.mutex, NULL);
 	if (ret < 0) {
 		VHOST_LOG_CONFIG("thread", ERR, "%s: failed to initialize mutex\n", __func__);
@@ -515,6 +519,8 @@ vhost_user_reconnect_init(void)
 			VHOST_LOG_CONFIG("thread", ERR,
 				"%s: failed to destroy reconnect mutex\n",
 				__func__);
+	} else {
+		reconn_init_done = true;
 	}
 
 	return ret;
@@ -866,6 +872,11 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	if (!path)
 		return -1;
 
+	if ((flags & RTE_VHOST_USER_CLIENT) != 0 &&
+			(flags & RTE_VHOST_USER_NO_RECONNECT) == 0 &&
+			vhost_user_reconnect_init() != 0)
+		return -1;
+
 	pthread_mutex_lock(&vhost_user.mutex);
 
 	if (vhost_user.vsocket_cnt == MAX_VHOST_SOCKET) {
@@ -961,11 +972,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	}
 
 	if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
-		vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
-		if (vsocket->reconnect && reconn_tid == 0) {
-			if (vhost_user_reconnect_init() != 0)
-				goto out_mutex;
-		}
+		vsocket->reconnect = (flags & RTE_VHOST_USER_NO_RECONNECT) == 0;
 	} else {
 		vsocket->is_server = true;
 	}
-- 
2.40.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] vhost: avoid sleeping under mutex
  2023-05-11  8:10 ` Xia, Chenbo
@ 2023-05-15 11:22   ` David Marchand
  0 siblings, 0 replies; 5+ messages in thread
From: David Marchand @ 2023-05-15 11:22 UTC (permalink / raw)
  To: Xia, Chenbo; +Cc: dev, stable, Maxime Coquelin, Yuanhan Liu

On Thu, May 11, 2023 at 10:10 AM Xia, Chenbo <chenbo.xia@intel.com> wrote:
> > Covscan reported:
> >
> >  2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> >       "pthread_mutex_lock" acquires lock "vhost_user.mutex".
> > 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> >       "vhost_user_reconnect_init" might sleep while holding lock
> >       "vhost_user.mutex".
> >  #   953|             vsocket->reconnect =
> >       !(flags & RTE_VHOST_USER_NO_RECONNECT);
> >  #   954|             if (vsocket->reconnect && reconn_tid == 0) {
> >  #   955|->                   if (vhost_user_reconnect_init() != 0)
> >  #   956|                             goto out_mutex;
> >  #   957|             }
> >
> > The reason for this warning is that vhost_user_reconnect_init() creates a
> > ctrl thread and calls nanosleep waiting for this thread to be ready,
> > while vhost_user.mutex is taken.
> >
> > Move the call to vhost_user_reconnect_init() out of this mutex.
> >
> > While at it, a pthread_t value should be considered opaque.
> > Instead of relying reconn_tid == 0, use an internal flag in
> > vhost_user_reconnect_init().
>
> Should we make reconn_tid not a global variable anymore then?
> Maybe a static pthread_t in vhost_user_reconnect_init?

Yep, I just sent a v2.
Thanks Chenbo.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v2] vhost: avoid sleeping under mutex
  2023-05-15 11:18 ` [PATCH v2] " David Marchand
@ 2023-05-15 15:51   ` Stephen Hemminger
  0 siblings, 0 replies; 5+ messages in thread
From: Stephen Hemminger @ 2023-05-15 15:51 UTC (permalink / raw)
  To: David Marchand; +Cc: dev, stable, Maxime Coquelin, Chenbo Xia, Yuanhan Liu

On Mon, 15 May 2023 13:18:44 +0200
David Marchand <david.marchand@redhat.com> wrote:

> Covscan reported:
> 
>  2. dpdk-21.11/lib/vhost/socket.c:852: lock_acquire: Calling function
> 	"pthread_mutex_lock" acquires lock "vhost_user.mutex".
> 23. dpdk-21.11/lib/vhost/socket.c:955: sleep: Call to
> 	"vhost_user_reconnect_init" might sleep while holding lock
> 	"vhost_user.mutex".
>  #   953|   		vsocket->reconnect =
> 	!(flags & RTE_VHOST_USER_NO_RECONNECT);
>  #   954|   		if (vsocket->reconnect && reconn_tid == 0) {
>  #   955|-> 			if (vhost_user_reconnect_init() != 0)
>  #   956|   				goto out_mutex;
>  #   957|   		}
> 
> The reason for this warning is that vhost_user_reconnect_init() creates a
> ctrl thread and calls nanosleep waiting for this thread to be ready,
> while vhost_user.mutex is taken.
> 
> Move the call to vhost_user_reconnect_init() out of this mutex.
> 
> While at it, a pthread_t value should be considered opaque.
> Instead of relying reconn_tid == 0, use an internal flag in
> vhost_user_reconnect_init().
> 
> Coverity issue: 373686
> Bugzilla ID: 981
> Fixes: e623e0c6d8a5 ("vhost: add reconnect ability")
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Changes since v1:
> - moved reconn_tid in vhost_user_reconnect_init as this
>   variable is not used anywhere else,
> 
> ---
>  lib/vhost/socket.c | 19 +++++++++++++------
>  1 file changed, 13 insertions(+), 6 deletions(-)
> 
> diff --git a/lib/vhost/socket.c b/lib/vhost/socket.c
> index 669c322e12..00a912c59e 100644
> --- a/lib/vhost/socket.c
> +++ b/lib/vhost/socket.c
> @@ -427,7 +427,6 @@ struct vhost_user_reconnect_list {
>  };
>  
>  static struct vhost_user_reconnect_list reconn_list;
> -static pthread_t reconn_tid;
>  
>  static int
>  vhost_user_connect_nonblock(char *path, int fd, struct sockaddr *un, size_t sz)
> @@ -498,8 +497,13 @@ vhost_user_client_reconnect(void *arg __rte_unused)
>  static int
>  vhost_user_reconnect_init(void)
>  {
> +	static bool reconn_init_done;
> +	static pthread_t reconn_tid;

This would not be thread safe.
Perhaps use a pthread condition variable instead of spinning here.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2023-05-15 15:51 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-22 17:05 [PATCH] vhost: avoid sleeping under mutex David Marchand
2023-05-11  8:10 ` Xia, Chenbo
2023-05-15 11:22   ` David Marchand
2023-05-15 11:18 ` [PATCH v2] " David Marchand
2023-05-15 15:51   ` Stephen Hemminger

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).