DPDK patches and discussions
 help / color / mirror / Atom feed
From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: dev@dpdk.org, tiwei.bie@intel.com, zhihong.wang@intel.com,
	jfreimann@redhat.com, nicknickolaev@gmail.com,
	i.maximets@samsung.com, bruce.richardson@intel.com,
	alejandro.lucero@netronome.com
Cc: dgilbert@redhat.com, stable@dpdk.org,
	Maxime Coquelin <maxime.coquelin@redhat.com>
Subject: [dpdk-dev] [PATCH v5 17/19] vhost: restrict postcopy live-migration enablement
Date: Tue,  9 Oct 2018 22:54:24 +0200	[thread overview]
Message-ID: <20181009205426.21219-18-maxime.coquelin@redhat.com> (raw)
In-Reply-To: <20181009205426.21219-1-maxime.coquelin@redhat.com>

Postcopy live-migration feature requires the application to
not populate the guest memory. As the vhost library cannot
prevent the application to that (e.g. preventing the
application to call mlockall()), the feature is disabled by
default.

The application should only enable the feature if it does not
force the guest memory to be populated.

In case the user passes the RTE_VHOST_USER_POSTCOPY_SUPPORT
flag at registration but the feature was not compiled,
registration fails.

For the same reason, postcopy and dequeue zero copy features
are not compatible, so don't advertize postcopy support if
dequeue zero copy is requested.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 doc/guides/prog_guide/vhost_lib.rst |  8 ++++++++
 lib/librte_vhost/rte_vhost.h        |  1 +
 lib/librte_vhost/socket.c           | 30 ++++++++++++++++++++++++++---
 3 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 77af4d775..c77df338f 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -106,6 +106,14 @@ The following is an overview of some key Vhost API functions:
     Enabling this flag with these Qemu version results in Qemu being blocked
     when multiple queue pairs are declared.
 
+  - ``RTE_VHOST_USER_POSTCOPY_SUPPORT``
+
+    Postcopy live-migration support will be enabled when this flag is set.
+    It is disabled by default.
+
+    Enabling this flag should only be done when the calling application does
+    not pre-fault the guest shared memory, otherwise migration would fail.
+
 * ``rte_vhost_driver_set_features(path, features)``
 
   This function sets the feature bits the vhost-user driver supports. The
diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h
index 9292c89c5..d280ac420 100644
--- a/lib/librte_vhost/rte_vhost.h
+++ b/lib/librte_vhost/rte_vhost.h
@@ -28,6 +28,7 @@ extern "C" {
 #define RTE_VHOST_USER_NO_RECONNECT	(1ULL << 1)
 #define RTE_VHOST_USER_DEQUEUE_ZERO_COPY	(1ULL << 2)
 #define RTE_VHOST_USER_IOMMU_SUPPORT	(1ULL << 3)
+#define RTE_VHOST_USER_POSTCOPY_SUPPORT		(1ULL << 4)
 
 /** Protocol features. */
 #ifndef VHOST_USER_PROTOCOL_F_MQ
diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c
index 7cad5593e..4b221a805 100644
--- a/lib/librte_vhost/socket.c
+++ b/lib/librte_vhost/socket.c
@@ -51,6 +51,8 @@ struct vhost_user_socket {
 	uint64_t supported_features;
 	uint64_t features;
 
+	uint64_t protocol_features;
+
 	/*
 	 * Device id to identify a specific backend device.
 	 * It's set to -1 for the default software implementation.
@@ -735,7 +737,7 @@ rte_vhost_driver_get_protocol_features(const char *path,
 	did = vsocket->vdpa_dev_id;
 	vdpa_dev = rte_vdpa_get_device(did);
 	if (!vdpa_dev || !vdpa_dev->ops->get_protocol_features) {
-		*protocol_features = VHOST_USER_PROTOCOL_FEATURES;
+		*protocol_features = vsocket->protocol_features;
 		goto unlock_exit;
 	}
 
@@ -748,7 +750,7 @@ rte_vhost_driver_get_protocol_features(const char *path,
 		goto unlock_exit;
 	}
 
-	*protocol_features = VHOST_USER_PROTOCOL_FEATURES
+	*protocol_features = vsocket->protocol_features
 		& vdpa_protocol_features;
 
 unlock_exit:
@@ -867,11 +869,21 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 	vsocket->use_builtin_virtio_net = true;
 	vsocket->supported_features = VIRTIO_NET_SUPPORTED_FEATURES;
 	vsocket->features           = VIRTIO_NET_SUPPORTED_FEATURES;
+	vsocket->protocol_features  = VHOST_USER_PROTOCOL_FEATURES;
 
-	/* Dequeue zero copy can't assure descriptors returned in order */
+	/*
+	 * Dequeue zero copy can't assure descriptors returned in order.
+	 * Also, it requires that the guest memory is populated, which is
+	 * not compatible with postcopy.
+	 */
 	if (vsocket->dequeue_zero_copy) {
 		vsocket->supported_features &= ~(1ULL << VIRTIO_F_IN_ORDER);
 		vsocket->features &= ~(1ULL << VIRTIO_F_IN_ORDER);
+
+		RTE_LOG(INFO, VHOST_CONFIG,
+			"Dequeue zero copy requested, disabling postcopy support\n");
+		vsocket->protocol_features &=
+			~(1ULL << VHOST_USER_PROTOCOL_F_PAGEFAULT);
 	}
 
 	if (!(flags & RTE_VHOST_USER_IOMMU_SUPPORT)) {
@@ -879,6 +891,18 @@ rte_vhost_driver_register(const char *path, uint64_t flags)
 		vsocket->features &= ~(1ULL << VIRTIO_F_IOMMU_PLATFORM);
 	}
 
+	if (!(flags & RTE_VHOST_USER_POSTCOPY_SUPPORT)) {
+		vsocket->protocol_features &=
+			~(1ULL << VHOST_USER_PROTOCOL_F_PAGEFAULT);
+	} else {
+#ifndef RTE_LIBRTE_VHOST_POSTCOPY
+		RTE_LOG(ERR, VHOST_CONFIG,
+			"Postcopy requested but not compiled\n");
+		ret = -1;
+		goto out_mutex;
+#endif
+	}
+
 	if ((flags & RTE_VHOST_USER_CLIENT) != 0) {
 		vsocket->reconnect = !(flags & RTE_VHOST_USER_NO_RECONNECT);
 		if (vsocket->reconnect && reconn_tid == 0) {
-- 
2.17.1

  parent reply	other threads:[~2018-10-09 20:56 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-09 20:54 [dpdk-dev] [PATCH v5 00/19] vhost: add postcopy live-migration support Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 01/19] vhost: fix messages results handling Maxime Coquelin
2018-10-10  7:25   ` Maxime Coquelin
2018-10-11  9:32     ` Ilya Maximets
2018-10-11  9:36       ` Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 02/19] vhost: fix return code of messages requiring replies Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 03/19] vhost: clarify reply-ack in case a reply was already sent Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 04/19] vhost: fix payload size of reply Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 05/19] vhost: fix error handling when mem table gets updated Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 06/19] vhost: define postcopy protocol flag Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 07/19] vhost: add number of fds to vhost-user messages and use it Maxime Coquelin
2018-10-10 10:06   ` Tiwei Bie
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 08/19] vhost: pass socket fd to message handling callbacks Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 09/19] vhost: enable fds passing when sending vhost-user messages Maxime Coquelin
2018-10-10 10:09   ` Tiwei Bie
2018-10-10 10:26     ` Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 10/19] vhost: add config flag for postcopy feature Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 11/19] vhost: introduce postcopy's advise message Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 12/19] vhost: add support for postcopy's listen message Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 13/19] vhost: register new regions with userfaultfd Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 14/19] vhost: avoid useless VhostUserMemory copy Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 15/19] vhost: send userfault range addresses back to qemu Maxime Coquelin
2018-10-10 10:13   ` Tiwei Bie
2018-10-10 10:25     ` Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 16/19] vhost: add support to postcopy's end request Maxime Coquelin
2018-10-09 20:54 ` Maxime Coquelin [this message]
2018-10-10 10:17   ` [dpdk-dev] [PATCH v5 17/19] vhost: restrict postcopy live-migration enablement Tiwei Bie
2018-10-10 10:23     ` Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 18/19] net/vhost: add parameter to enable postcopy support Maxime Coquelin
2018-10-09 20:54 ` [dpdk-dev] [PATCH v5 19/19] vhost: enable postcopy protocol feature Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181009205426.21219-18-maxime.coquelin@redhat.com \
    --to=maxime.coquelin@redhat.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=dgilbert@redhat.com \
    --cc=i.maximets@samsung.com \
    --cc=jfreimann@redhat.com \
    --cc=nicknickolaev@gmail.com \
    --cc=stable@dpdk.org \
    --cc=tiwei.bie@intel.com \
    --cc=zhihong.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).