From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 4A2D91E20 for ; Thu, 22 Feb 2018 13:32:33 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 87FD380401C1; Thu, 22 Feb 2018 12:32:32 +0000 (UTC) Received: from [10.36.112.33] (ovpn-112-33.ams2.redhat.com [10.36.112.33]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AD00D213AEE2; Thu, 22 Feb 2018 12:32:31 +0000 (UTC) To: "Tan, Jianfeng" , "dev@dpdk.org" Cc: "yliu@fridaylinux.org" References: <1518580892-32656-1-git-send-email-jianfeng.tan@intel.com> <1518580892-32656-5-git-send-email-jianfeng.tan@intel.com> <161752fc-9dd8-f952-eb00-ff1cb6e5fbdd@redhat.com> <87337fa8-66fb-f9ec-45c2-78042bb47759@redhat.com> From: Maxime Coquelin Message-ID: <3e2002d7-a22e-8fa5-f917-fa85428aff47@redhat.com> Date: Thu, 22 Feb 2018 13:32:30 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 22 Feb 2018 12:32:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 22 Feb 2018 12:32:32 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH 4/4] vhost: avoid populate guest memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Feb 2018 12:32:33 -0000 On 02/22/2018 09:40 AM, Tan, Jianfeng wrote: > > >> -----Original Message----- >> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >> Sent: Thursday, February 22, 2018 4:26 PM >> To: Tan, Jianfeng; dev@dpdk.org >> Cc: yliu@fridaylinux.org >> Subject: Re: [PATCH 4/4] vhost: avoid populate guest memory >> >> >> >> On 02/22/2018 03:42 AM, Tan, Jianfeng wrote: >>> Hi Maxime, >>> >>>> -----Original Message----- >>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>> Sent: Tuesday, February 20, 2018 4:45 AM >>>> To: Tan, Jianfeng; dev@dpdk.org >>>> Cc: yliu@fridaylinux.org >>>> Subject: Re: [PATCH 4/4] vhost: avoid populate guest memory >>>> >>>> Hi Jianfeng, >>>> >>>> On 02/14/2018 05:01 AM, Jianfeng Tan wrote: >>>>> It's not necessary to polulate guest memory from vhost side. >>>>> >>>>> Cc: maxime.coquelin@redhat.com >>>>> Cc: yliu@fridaylinux.org >>>>> >>>>> Signed-off-by: Jianfeng Tan >>>>> --- >>>>> lib/librte_vhost/vhost_user.c | 4 +++- >>>>> 1 file changed, 3 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >>>>> index 90ed211..9bd0391 100644 >>>>> --- a/lib/librte_vhost/vhost_user.c >>>>> +++ b/lib/librte_vhost/vhost_user.c >>>>> @@ -644,6 +644,7 @@ vhost_user_set_mem_table(struct virtio_net >> *dev, >>>> struct VhostUserMsg *pmsg) >>>>> uint64_t mmap_offset; >>>>> uint64_t alignment; >>>>> uint32_t i; >>>>> + int populate; >>>>> int fd; >>>>> >>>>> if (dev->mem && !vhost_memory_changed(&memory, dev->mem)) >>>> { >>>>> @@ -714,8 +715,9 @@ vhost_user_set_mem_table(struct virtio_net >> *dev, >>>> struct VhostUserMsg *pmsg) >>>>> } >>>>> mmap_size = RTE_ALIGN_CEIL(mmap_size, alignment); >>>>> >>>>> + populate = (dev->dequeue_zero_copy) ? MAP_POPULATE : >>>> 0; >>>>> mmap_addr = mmap(NULL, mmap_size, PROT_READ | >>>> PROT_WRITE, >>>>> - MAP_SHARED | MAP_POPULATE, fd, 0); >>>>> + MAP_SHARED | populate, fd, 0); >>>>> >>>>> if (mmap_addr == MAP_FAILED) { >>>>> RTE_LOG(ERR, VHOST_CONFIG, >>>>> >>>> >>>> Wouldn't not populating all the guest memory have a bad impact on 0% >>>> acceptable loss use-cases? >>> >>> Yes, it could affect such use case; but we can address that by warming up >> the system a little bit, can't we? >> >> I'm not sure this is a good idea to ask the real user to warm-up the >> system. >> >> Also, even with benchmarking, the loss happens when the queues are full, >> so it is likely that it happens with buffers not used before, even if >> system has been warmed-up. > > OK, warm-up is a bad idea here :-) > > But if a VM is used for such use case, I think we'd better pre-allocate the memory at QEMU side. > >> >>> From a good point of view, it could save the memory for VMs without pre- >> allocating. >> >> What could be done is maybe to have an EAL API for mmaping, with an >> associated EAL parameter to state whether it want populating or not. >> This option would be disabled by default. >> >> Does that sounds reasonable? > > If we look for an application-level configuration, it's not necessary to have such a parameter. Refer to the 3rd patch in this series, if we make all (current/future) memory locked, the mmap() syscall will populate the memory. OK, but in that case it should be documented. I see OVS has also a parameter to request the memory to be locked, but it seems not to be the default, so the user could face a change in the behavior it didn't expect. Thanks, Maxime > Thanks, > Jianfeng >