Re: [dpdk-dev] [PATCH v2] vhost: avoid populate guest memory

[Maxime Coquelin <maxime.coquelin@redhat.com>]

On 03/28/2018 08:56 AM, Jianfeng Tan wrote:
> It's not necessary to populate guest memory from vhost side unless
> zerocopy is enabled or users want better performance.
> 
> Update the doc for guest memory requirement clarification.
> 
> Cc: maxime.coquelin@redhat.com
> 
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> ---
>   doc/guides/prog_guide/vhost_lib.rst | 21 +++++++++++++++++++++
>   doc/guides/sample_app_ug/vhost.rst  |  9 ---------
>   lib/librte_vhost/vhost_user.c       |  4 +++-
>   3 files changed, 24 insertions(+), 10 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..f474736 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -214,6 +214,27 @@ the vhost device from the data plane.
>   
>   When the socket connection is closed, vhost will destroy the device.
>   
> +Guest memory requirement
> +------------------------
> +
> +* Memory pre-allocation
> +
> +  For non-zerocopy, guest memory pre-allocation is not a must. This can help
> +  save of memory. If users really want the guest memory to be pre-allocated
> +  (e.g., for performance reason), we can add option ``-mem-prealloc`` when
> +  starting QEMU. Or, we can lock all memory at vhost side which will force
> +  memory to be allocated when mmap at vhost side; option --mlockall in
> +  ovs-dpdk is an example in hand.
> +
> +  For zerocopy, we force the VM memory to be pre-allocated at vhost lib when
> +  mapping the guest memory; and also we need to lock the memory to prevent
> +  pages being swapped out to disk.
> +
> +* Memory sharing
> +
> +  Make sure ``share=on`` QEMU option is given. vhost-user will not work with
> +  a QEMU version without shared memory mapping.
> +

nice write-up!

>   Vhost supported vSwitch reference
>   ---------------------------------
>   
> diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..da161a9 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -175,15 +175,6 @@ Common Issues
>     The command above indicates how many hugepages are free to support QEMU's
>     allocation request.
>   
> -* vhost-user will not work with QEMU without the ``-mem-prealloc`` option
> -
> -  The current implementation works properly only when the guest memory is
> -  pre-allocated.
> -
> -* vhost-user will not work with a QEMU version without shared memory mapping:
> -
> -  Make sure ``share=on`` QEMU option is given.
> -
>   * Failed to build DPDK in VM
>   
>     Make sure "-cpu host" QEMU option is given.
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 90ed211..9bd0391 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -644,6 +644,7 @@ vhost_user_set_mem_table(struct virtio_net *dev, struct VhostUserMsg *pmsg)
>   	uint64_t mmap_offset;
>   	uint64_t alignment;
>   	uint32_t i;
> +	int populate;
>   	int fd;
>   
>   	if (dev->mem && !vhost_memory_changed(&memory, dev->mem)) {
> @@ -714,8 +715,9 @@ vhost_user_set_mem_table(struct virtio_net *dev, struct VhostUserMsg *pmsg)
>   		}
>   		mmap_size = RTE_ALIGN_CEIL(mmap_size, alignment);
>   
> +		populate = (dev->dequeue_zero_copy) ? MAP_POPULATE : 0;
>   		mmap_addr = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
> -				 MAP_SHARED | MAP_POPULATE, fd, 0);
> +				 MAP_SHARED | populate, fd, 0);
>   
>   		if (mmap_addr == MAP_FAILED) {
>   			RTE_LOG(ERR, VHOST_CONFIG,
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime