Re: [dpdk-dev] [PATCH v6 1/7] vhost: fix missing memory table NUMA realloc

["Xia, Chenbo" <chenbo.xia@intel.com>]

> -----Original Message-----
> From: Maxime Coquelin <maxime.coquelin@redhat.com>
> Sent: Friday, June 18, 2021 10:04 PM
> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo <chenbo.xia@intel.com>
> Cc: Maxime Coquelin <maxime.coquelin@redhat.com>; stable@dpdk.org
> Subject: [PATCH v6 1/7] vhost: fix missing memory table NUMA realloc
> 
> When the guest allocates virtqueues on a different NUMA node
> than the one the Vhost metadata are allocated, both the Vhost
> device struct and the virtqueues struct are reallocated.
> 
> However, reallocating the Vhost memory table was missing, which
> likely causes at least one cross-NUMA accesses for every burst
> of packets.
> 
> This patch reallocates this table on the same NUMA node as the
> other metadata.
> 
> Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
> Cc: stable@dpdk.org
> 
> Reported-by: David Marchand <david.marchand@redhat.com>
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>  lib/vhost/vhost_user.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> index 8f0eba6412..b5a84f3dcd 100644
> --- a/lib/vhost/vhost_user.c
> +++ b/lib/vhost/vhost_user.c
> @@ -473,8 +473,8 @@ vhost_user_set_vring_num(struct virtio_net **pdev,
>  }
> 
>  /*
> - * Reallocate virtio_dev and vhost_virtqueue data structure to make them on
> the
> - * same numa node as the memory of vring descriptor.
> + * Reallocate virtio_dev, vhost_virtqueue and related data structures to
> + * make them on the same numa node as the memory of vring descriptor.
>   */
>  #ifdef RTE_LIBRTE_VHOST_NUMA
>  static struct virtio_net*
> @@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index)
>  		goto out;
>  	}
>  	if (oldnode != newnode) {
> +		struct rte_vhost_memory *old_mem;
> +		ssize_t mem_size;
> +
>  		VHOST_LOG_CONFIG(INFO,
>  			"reallocate dev from %d to %d node\n",
>  			oldnode, newnode);
> @@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index)
> 
>  		memcpy(dev, old_dev, sizeof(*dev));
>  		rte_free(old_dev);
> +
> +		mem_size = sizeof(struct rte_vhost_memory) +
> +			sizeof(struct rte_vhost_mem_region) * dev->mem->nregions;
> +		old_mem = dev->mem;
> +		dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode);
> +		if (!dev->mem) {
> +			dev->mem = old_mem;
> +			goto out;
> +		}
> +
> +		memcpy(dev->mem, old_mem, mem_size);
> +		rte_free(old_mem);
>  	}
> 
>  out:
> --
> 2.31.1

Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>