[dpdk-stable] [PATCH v2 v16.11 LTS] vhost-user: fix deadlock in case of NUMA realloc

[dpdk-stable] [PATCH v2 v16.11 LTS] vhost-user: fix deadlock in case of NUMA realloc

[Maxime Coquelin]

Virtqueue's access lock was recently introduced to protect
the device against async changes.

One problem with the v16.11 backport is that in case of NUMA
reallocation, the device gets stuck because the old access_lock
gets unlocked instead of its reallocated copy. On the next
vhost-user message received, the thread keeps spinning on the
lock, as it will never be unlocked.

Fixes: ce3b23dc9296 ("vhost: protect active rings from async ring changes")

Cc: stable@dpdk.org

Tested-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
V2: remove debug changes squashed by mistake...

 lib/librte_vhost/vhost_user.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 80348dbf6..94a48a4b0 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -327,9 +327,11 @@ qva_to_vva(struct virtio_net *dev, uint64_t qva)
  * This function then converts these to our address space.
  */
 static int
-vhost_user_set_vring_addr(struct virtio_net *dev, struct vhost_vring_addr *addr)
+vhost_user_set_vring_addr(struct virtio_net **pdev,
+						  struct vhost_vring_addr *addr)
 {
 	struct vhost_virtqueue *vq;
+	struct virtio_net *dev = *pdev;
 
 	if (dev->mem == NULL)
 		return -1;
@@ -348,6 +350,8 @@ vhost_user_set_vring_addr(struct virtio_net *dev, struct vhost_vring_addr *addr)
 	}
 
 	dev = numa_realloc(dev, addr->index);
+	*pdev = dev;
+
 	vq = dev->virtqueue[addr->index];
 
 	vq->avail = (struct vring_avail *)(uintptr_t)qva_to_vva(dev,
@@ -1092,7 +1096,7 @@ vhost_user_msg_handler(int vid, int fd)
 		vhost_user_set_vring_num(dev, &msg.payload.state);
 		break;
 	case VHOST_USER_SET_VRING_ADDR:
-		vhost_user_set_vring_addr(dev, &msg.payload.addr);
+		vhost_user_set_vring_addr(&dev, &msg.payload.addr);
 		break;
 	case VHOST_USER_SET_VRING_BASE:
 		vhost_user_set_vring_base(dev, &msg.payload.state);
-- 
2.14.3

Re: [dpdk-stable] [PATCH v2 v16.11 LTS] vhost-user: fix deadlock in case of NUMA realloc

[Kevin Traynor]

On 03/28/2018 08:51 PM, Maxime Coquelin wrote:
> Virtqueue's access lock was recently introduced to protect
> the device against async changes.
> 
> One problem with the v16.11 backport is that in case of NUMA
> reallocation, the device gets stuck because the old access_lock
> gets unlocked instead of its reallocated copy. On the next
> vhost-user message received, the thread keeps spinning on the
> lock, as it will never be unlocked.
> 

Reviewed-by: Kevin Traynor <ktraynor@redhat.com>

> Fixes: ce3b23dc9296 ("vhost: protect active rings from async ring changes")
> 
> Cc: stable@dpdk.org
> 
> Tested-by: Kevin Traynor <ktraynor@redhat.com>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> V2: remove debug changes squashed by mistake...
> 
>  lib/librte_vhost/vhost_user.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 80348dbf6..94a48a4b0 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -327,9 +327,11 @@ qva_to_vva(struct virtio_net *dev, uint64_t qva)
>   * This function then converts these to our address space.
>   */
>  static int
> -vhost_user_set_vring_addr(struct virtio_net *dev, struct vhost_vring_addr *addr)
> +vhost_user_set_vring_addr(struct virtio_net **pdev,
> +						  struct vhost_vring_addr *addr)
>  {
>  	struct vhost_virtqueue *vq;
> +	struct virtio_net *dev = *pdev;
>  
>  	if (dev->mem == NULL)
>  		return -1;
> @@ -348,6 +350,8 @@ vhost_user_set_vring_addr(struct virtio_net *dev, struct vhost_vring_addr *addr)
>  	}
>  
>  	dev = numa_realloc(dev, addr->index);
> +	*pdev = dev;
> +
>  	vq = dev->virtqueue[addr->index];
>  
>  	vq->avail = (struct vring_avail *)(uintptr_t)qva_to_vva(dev,
> @@ -1092,7 +1096,7 @@ vhost_user_msg_handler(int vid, int fd)
>  		vhost_user_set_vring_num(dev, &msg.payload.state);
>  		break;
>  	case VHOST_USER_SET_VRING_ADDR:
> -		vhost_user_set_vring_addr(dev, &msg.payload.addr);
> +		vhost_user_set_vring_addr(&dev, &msg.payload.addr);
>  		break;
>  	case VHOST_USER_SET_VRING_BASE:
>  		vhost_user_set_vring_base(dev, &msg.payload.state);
> 

Re: [dpdk-stable] [PATCH v2 v16.11 LTS] vhost-user: fix deadlock in case of NUMA realloc

[Luca Boccassi]

On Wed, 2018-04-04 at 13:57 +0100, Kevin Traynor wrote:
> On 03/28/2018 08:51 PM, Maxime Coquelin wrote:
> > Virtqueue's access lock was recently introduced to protect
> > the device against async changes.
> > 
> > One problem with the v16.11 backport is that in case of NUMA
> > reallocation, the device gets stuck because the old access_lock
> > gets unlocked instead of its reallocated copy. On the next
> > vhost-user message received, the thread keeps spinning on the
> > lock, as it will never be unlocked.
> > 
> 
> Reviewed-by: Kevin Traynor <ktraynor@redhat.com>

Thanks, applied&pushed to dpdk-stable/16.11

-- 
Kind regards,
Luca Boccassi