From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id BC8801B20E; Mon, 8 Oct 2018 18:55:01 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AB374C059B87; Mon, 8 Oct 2018 16:55:00 +0000 (UTC) Received: from [10.36.112.51] (ovpn-112-51.ams2.redhat.com [10.36.112.51]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 08FFB2015F95; Mon, 8 Oct 2018 16:54:50 +0000 (UTC) To: Ilya Maximets , dev@dpdk.org, tiwei.bie@intel.com, zhihong.wang@intel.com, jfreimann@redhat.com, nicknickolaev@gmail.com, bruce.richardson@intel.com, alejandro.lucero@netronome.com Cc: dgilbert@redhat.com, stable@dpdk.org References: <20181004081403.8039-1-maxime.coquelin@redhat.com> <20181004081403.8039-19-maxime.coquelin@redhat.com> <20181005142201eucas1p1359f104e0fc99b03019452350b44f631~avFrXXPTu2477524775eucas1p1A@eucas1p1.samsung.com> <20181008162251eucas1p29fb65b432d9b202d635f37240baa2630~brrCTgLSL0635706357eucas1p2Z@eucas1p2.samsung.com> From: Maxime Coquelin Message-ID: Date: Mon, 8 Oct 2018 18:54:48 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20181008162251eucas1p29fb65b432d9b202d635f37240baa2630~brrCTgLSL0635706357eucas1p2Z@eucas1p2.samsung.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.25 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 08 Oct 2018 16:55:00 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH v3 18/19] vhost: add flag to enable postcopy live-migration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2018 16:55:02 -0000 On 10/08/2018 06:25 PM, Ilya Maximets wrote: > Have you missed this mail while preparing v4? Yes I did... Thanks for the heads-up! Maxime > Best regards, Ilya Maximets. > > On 05.10.2018 17:24, Ilya Maximets wrote: >> I think it'll be nice to have a warning in case zero-copy >> and postcopy enabled at the same time. This warning will >> clarify for users why postcopy is not enabled. >> What do you think? >> >> Maybe we can also move the check from the previous patch to >> rte_vhost_driver_register() like it done for VIRTIO_F_IN_ORDER ? >> We can't change the set of enabled features in runtime anyway. >> >> Best regards, Ilya Maximets. >> >> On 04.10.2018 11:14, Maxime Coquelin wrote: >>> Postcopy live-migration feature require the application to >>> not populate the guest memory. As the vhost library cannot >>> prevent the application to that (e.g. preventing the >>> application to call mlockall()), the feature is disabled by >>> default. >>> >>> The application should only enable the feature if it does not >>> force the guest memory to be populated. >>> >>> In case the user passes the RTE_VHOST_USER_POSTCOPY_SUPPORT >>> flag at registration but the feature was not compiled, >>> registration fails. >>> >>> Signed-off-by: Maxime Coquelin >>> --- >>> doc/guides/prog_guide/vhost_lib.rst | 8 ++++++++ >>> lib/librte_vhost/rte_vhost.h | 1 + >>> lib/librte_vhost/socket.c | 19 +++++++++++++++++-- >>> 3 files changed, 26 insertions(+), 2 deletions(-) >>> >>> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst >>> index 77af4d775..c77df338f 100644 >>> --- a/doc/guides/prog_guide/vhost_lib.rst >>> +++ b/doc/guides/prog_guide/vhost_lib.rst >>> @@ -106,6 +106,14 @@ The following is an overview of some key Vhost API functions: >>> Enabling this flag with these Qemu version results in Qemu being blocked >>> when multiple queue pairs are declared. >>> >>> + - ``RTE_VHOST_USER_POSTCOPY_SUPPORT`` >>> + >>> + Postcopy live-migration support will be enabled when this flag is set. >>> + It is disabled by default. >>> + >>> + Enabling this flag should only be done when the calling application does >>> + not pre-fault the guest shared memory, otherwise migration would fail. >>> + >>> * ``rte_vhost_driver_set_features(path, features)`` >>> >>> This function sets the feature bits the vhost-user driver supports. The >>> diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h >>> index b3cc6990d..b26afbffa 100644 >>> --- a/lib/librte_vhost/rte_vhost.h >>> +++ b/lib/librte_vhost/rte_vhost.h >>> @@ -28,6 +28,7 @@ extern "C" { >>> #define RTE_VHOST_USER_NO_RECONNECT (1ULL << 1) >>> #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY (1ULL << 2) >>> #define RTE_VHOST_USER_IOMMU_SUPPORT (1ULL << 3) >>> +#define RTE_VHOST_USER_POSTCOPY_SUPPORT (1ULL << 4) >>> >>> /** Protocol features. */ >>> #ifndef VHOST_USER_PROTOCOL_F_MQ >>> diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c >>> index c04d3d305..3df303be8 100644 >>> --- a/lib/librte_vhost/socket.c >>> +++ b/lib/librte_vhost/socket.c >>> @@ -51,6 +51,8 @@ struct vhost_user_socket { >>> uint64_t supported_features; >>> uint64_t features; >>> >>> + uint64_t protocol_features; >>> + >>> /* >>> * Device id to identify a specific backend device. >>> * It's set to -1 for the default software implementation. >>> @@ -731,7 +733,7 @@ rte_vhost_driver_get_protocol_features(const char *path, >>> did = vsocket->vdpa_dev_id; >>> vdpa_dev = rte_vdpa_get_device(did); >>> if (!vdpa_dev || !vdpa_dev->ops->get_protocol_features) { >>> - *protocol_features = VHOST_USER_PROTOCOL_FEATURES; >>> + *protocol_features = vsocket->protocol_features; >>> goto unlock_exit; >>> } >>> >>> @@ -744,7 +746,7 @@ rte_vhost_driver_get_protocol_features(const char *path, >>> goto unlock_exit; >>> } >>> >>> - *protocol_features = VHOST_USER_PROTOCOL_FEATURES >>> + *protocol_features = vsocket->protocol_features >>> & vdpa_protocol_features; >>> >>> unlock_exit: >>> @@ -863,6 +865,7 @@ rte_vhost_driver_register(const char *path, uint64_t flags) >>> vsocket->use_builtin_virtio_net = true; >>> vsocket->supported_features = VIRTIO_NET_SUPPORTED_FEATURES; >>> vsocket->features = VIRTIO_NET_SUPPORTED_FEATURES; >>> + vsocket->protocol_features = VHOST_USER_PROTOCOL_FEATURES; >>> >>> /* Dequeue zero copy can't assure descriptors returned in order */ >>> if (vsocket->dequeue_zero_copy) { >>> @@ -875,6 +878,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags) >>> vsocket->features &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM); >>> } >>> >>> + if (!(flags & RTE_VHOST_USER_POSTCOPY_SUPPORT)) { >>> + vsocket->protocol_features &= >>> + ~(1ULL << VHOST_USER_PROTOCOL_F_PAGEFAULT); >>> + } else { >>> +#ifndef RTE_LIBRTE_VHOST_POSTCOPY >>> + RTE_LOG(ERR, VHOST_CONFIG, >>> + "Postcopy requested but not compiled\n"); >>> + ret = -1; >>> + goto out_mutex; >>> +#endif >>> + } >>> + >>> if ((flags & RTE_VHOST_USER_CLIENT) != 0) { >>> vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT); >>> if (vsocket->reconnect && reconn_tid == 0) { >>> >> >>