DPDK patches and discussions
 help / color / mirror / Atom feed
From: Yuanhan Liu <yuanhan.liu@linux.intel.com>
To: dev@dpdk.org
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>,
	Harris James R <james.r.harris@intel.com>,
	Liu Changpeng <changpeng.liu@intel.com>,
	Yuanhan Liu <yuanhan.liu@linux.intel.com>
Subject: [dpdk-dev] [PATCH v2 01/22] vhost: introduce driver features related APIs
Date: Thu, 23 Mar 2017 15:10:38 +0800	[thread overview]
Message-ID: <1490253059-28112-2-git-send-email-yuanhan.liu@linux.intel.com> (raw)
In-Reply-To: <1490253059-28112-1-git-send-email-yuanhan.liu@linux.intel.com>

Introduce few APIs to set/get/enable/disable driver features.

Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 doc/guides/prog_guide/vhost_lib.rst    | 10 +++-
 lib/librte_vhost/rte_vhost_version.map |  4 ++
 lib/librte_vhost/rte_virtio_net.h      | 51 +++++++++++++++++++
 lib/librte_vhost/socket.c              | 90 ++++++++++++++++++++++++++++++++++
 4 files changed, 153 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index f0862e6..6a4d206 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -53,7 +53,7 @@ vhost library should be able to:
 Vhost API Overview
 ------------------
 
-The following is an overview of the Vhost API functions:
+The following is an overview of some key Vhost API functions:
 
 * ``rte_vhost_driver_register(path, flags)``
 
@@ -110,6 +110,12 @@ The following is an overview of the Vhost API functions:
       of those segments, thus the fewer the segments, the quicker we will get
       the mapping. NOTE: we may speed it by using tree searching in future.
 
+* ``rte_vhost_driver_set_features(path, features)``
+
+  This function sets the feature bits the vhost-user driver supports. The
+  vhost-user driver could be vhost-user net, yet it could be something else,
+  say, vhost-user SCSI.
+
 * ``rte_vhost_driver_session_start()``
 
   This function starts the vhost session loop to handle vhost messages. It
@@ -145,7 +151,7 @@ The following is an overview of the Vhost API functions:
 
   Receives (dequeues) ``count`` packets from guest, and stored them at ``pkts``.
 
-* ``rte_vhost_feature_disable/rte_vhost_feature_enable(feature_mask)``
+* ``rte_vhost_driver_disable/enable_features(path, features))``
 
   This function disables/enables some features. For example, it can be used to
   disable mergeable buffers and TSO features, which both are enabled by
diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map
index 30b4671..ca6259c 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -34,6 +34,10 @@ DPDK_16.07 {
 DPDK_17.05 {
 	global:
 
+	rte_vhost_driver_disable_features;
+	rte_vhost_driver_enable_features;
+	rte_vhost_driver_get_features;
+	rte_vhost_driver_set_features;
 	rte_vhost_get_mtu;
 
 } DPDK_16.07;
diff --git a/lib/librte_vhost/rte_virtio_net.h b/lib/librte_vhost/rte_virtio_net.h
index 56829aa..3daf35c 100644
--- a/lib/librte_vhost/rte_virtio_net.h
+++ b/lib/librte_vhost/rte_virtio_net.h
@@ -94,6 +94,57 @@ struct virtio_net_device_ops {
 /* Unregister vhost driver. This is only meaningful to vhost user. */
 int rte_vhost_driver_unregister(const char *path);
 
+/**
+ * Set the feature bits the vhost-user driver supports.
+ *
+ * @param path
+ *  The vhost-user socket file path
+ * @return
+ *  0 on success, -1 on failure
+ */
+int rte_vhost_driver_set_features(const char *path, uint64_t features);
+
+/**
+ * Enable vhost-user driver features.
+ *
+ * Note that
+ * - the param @features should be a subset of the feature bits provided
+ *   by rte_vhost_driver_set_features().
+ * - it must be invoked before vhost-user negotiation starts.
+ *
+ * @param path
+ *  The vhost-user socket file path
+ * @param features
+ *  Features to enable
+ * @return
+ *  0 on success, -1 on failure
+ */
+int rte_vhost_driver_enable_features(const char *path, uint64_t features);
+
+/**
+ * Disable vhost-user driver features.
+ *
+ * The two notes at rte_vhost_driver_enable_features() also apply here.
+ *
+ * @param path
+ *  The vhost-user socket file path
+ * @param features
+ *  Features to disable
+ * @return
+ *  0 on success, -1 on failure
+ */
+int rte_vhost_driver_disable_features(const char *path, uint64_t features);
+
+/**
+ * Get the final feature bits for feature negotiation.
+ *
+ * @param path
+ *  The vhost-user socket file path
+ * @return
+ *  Feature bits on success, 0 on failure
+ */
+uint64_t rte_vhost_driver_get_features(const char *path);
+
 /* Register callbacks. */
 int rte_vhost_driver_callback_register(struct virtio_net_device_ops const * const);
 /* Start vhost driver session blocking loop. */
diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c
index 6a30a31..bdccd96 100644
--- a/lib/librte_vhost/socket.c
+++ b/lib/librte_vhost/socket.c
@@ -63,6 +63,16 @@ struct vhost_user_socket {
 	bool is_server;
 	bool reconnect;
 	bool dequeue_zero_copy;
+
+	/*
+	 * The "supported_features" indicates the feature bits the
+	 * vhost driver supports. The "features" indicates the feature
+	 * bits after the rte_vhost_driver_features_disable/enable().
+	 * It is also the final feature bits used for vhost-user
+	 * features negotiation.
+	 */
+	uint64_t supported_features;
+	uint64_t features;
 };
 
 struct vhost_user_connection {
@@ -475,6 +485,86 @@ struct vhost_user_reconnect_list {
 	return 0;
 }
 
+static struct vhost_user_socket *
+find_vhost_user_socket(const char *path)
+{
+	int i;
+
+	for (i = 0; i < vhost_user.vsocket_cnt; i++) {
+		struct vhost_user_socket *vsocket = vhost_user.vsockets[i];
+
+		if (!strcmp(vsocket->path, path))
+			return vsocket;
+	}
+
+	return NULL;
+}
+
+int
+rte_vhost_driver_disable_features(const char *path, uint64_t features)
+{
+	struct vhost_user_socket *vsocket;
+
+	pthread_mutex_lock(&vhost_user.mutex);
+	vsocket = find_vhost_user_socket(path);
+	if (vsocket)
+		vsocket->features &= ~features;
+	pthread_mutex_unlock(&vhost_user.mutex);
+
+	return vsocket ? 0 : -1;
+}
+
+int
+rte_vhost_driver_enable_features(const char *path, uint64_t features)
+{
+	struct vhost_user_socket *vsocket;
+
+	pthread_mutex_lock(&vhost_user.mutex);
+	vsocket = find_vhost_user_socket(path);
+	if (vsocket) {
+		if ((vsocket->supported_features & features) != features) {
+			/*
+			 * trying to enable features the driver doesn't
+			 * support.
+			 */
+			pthread_mutex_unlock(&vhost_user.mutex);
+			return -1;
+		}
+		vsocket->features |= features;
+	}
+	pthread_mutex_unlock(&vhost_user.mutex);
+
+	return vsocket ? 0 : -1;
+}
+
+int
+rte_vhost_driver_set_features(const char *path, uint64_t features)
+{
+	struct vhost_user_socket *vsocket;
+
+	pthread_mutex_lock(&vhost_user.mutex);
+	vsocket = find_vhost_user_socket(path);
+	if (vsocket) {
+		vsocket->supported_features = features;
+		vsocket->features = features;
+	}
+	pthread_mutex_unlock(&vhost_user.mutex);
+
+	return vsocket ? 0 : -1;
+}
+
+uint64_t
+rte_vhost_driver_get_features(const char *path)
+{
+	struct vhost_user_socket *vsocket;
+
+	pthread_mutex_lock(&vhost_user.mutex);
+	vsocket = find_vhost_user_socket(path);
+	pthread_mutex_unlock(&vhost_user.mutex);
+
+	return vsocket ? vsocket->features : (uint64_t)-1;
+}
+
 /*
  * Register a new vhost-user socket; here we could act as server
  * (the default case), or client (when RTE_VHOST_USER_CLIENT) flag
-- 
1.9.0

  reply	other threads:[~2017-03-23  7:13 UTC|newest]

Thread overview: 135+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-03  9:51 [dpdk-dev] [PATCH 00/17] vhost: generic vhost API Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 01/17] vhost: introduce driver features related APIs Yuanhan Liu
2017-03-14  9:46   ` Maxime Coquelin
2017-03-14  9:53     ` Maxime Coquelin
2017-03-16  7:08       ` Yuanhan Liu
2017-03-16  9:18         ` Maxime Coquelin
2017-03-17  5:50           ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 02/17] net/vhost: remove feature " Yuanhan Liu
2017-03-14 10:15   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 03/17] vhost: use new APIs to handle features Yuanhan Liu
2017-03-14 10:43   ` Maxime Coquelin
2017-03-16  7:43     ` Yuanhan Liu
2017-03-16  9:39       ` Maxime Coquelin
2017-03-17  5:48         ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 04/17] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-14 10:55   ` Maxime Coquelin
2017-03-16  7:50     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 05/17] vhost: export guest memory regions Yuanhan Liu
2017-03-14 11:00   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 06/17] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-14 11:02   ` Maxime Coquelin
2017-03-16  7:35     ` Yuanhan Liu
2017-03-16  9:22       ` Maxime Coquelin
2017-03-17  5:49         ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 07/17] vhost: export vhost vring info Yuanhan Liu
2017-03-14 12:11   ` Maxime Coquelin
2017-03-16  7:24     ` Yuanhan Liu
2017-03-16  9:20       ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 08/17] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-14 12:24   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 09/17] vhost: turn queue pair to vring Yuanhan Liu
2017-03-14 12:31   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 10/17] vhost: export the number of vrings Yuanhan Liu
2017-03-14 12:33   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 11/17] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-14 12:37   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 12/17] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-14 12:42   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 13/17] vhost: do not include net specific headers Yuanhan Liu
2017-03-14 12:46   ` Maxime Coquelin
2017-03-20  7:32   ` Liu, Changpeng
2017-03-22  6:21     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 14/17] vhost: rename device ops struct Yuanhan Liu
2017-03-14 12:48   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 15/17] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-14 12:50   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 16/17] vhost: rename header file Yuanhan Liu
2017-03-14 12:59   ` Maxime Coquelin
2017-03-20  5:35     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 17/17] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-03-23  7:10 ` [dpdk-dev] [PATCH v2 00/22] vhost: generic vhost API Yuanhan Liu
2017-03-23  7:10   ` Yuanhan Liu [this message]
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 02/22] net/vhost: remove feature related APIs Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 05/22] vhost: export guest memory regions Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 07/22] vhost: export vhost vring info Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 10/22] vhost: export the number of vrings Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 14/22] vhost: rename device ops struct Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 16/22] vhost: add features changed callback Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 19/22] vhost: rename header file Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-03-28 12:45   ` [dpdk-dev] [PATCH v3 00/22] vhost: generic vhost API Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 01/22] vhost: introduce driver features related APIs Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 02/22] net/vhost: remove feature " Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-03-29 14:57       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-29 15:03       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 05/22] vhost: export guest memory regions Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-31  7:45       ` Maxime Coquelin
2017-03-31  8:51         ` Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 07/22] vhost: export vhost vring info Yuanhan Liu
2017-03-31  7:48       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 10/22] vhost: export the number of vrings Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 14/22] vhost: rename device ops struct Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 16/22] vhost: add features changed callback Yuanhan Liu
2017-03-31  7:50       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-03-31  8:05       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-03-31  9:11       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 19/22] vhost: rename header file Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-03-31  9:13       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-03-31  9:26       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-04-01  7:22     ` [dpdk-dev] [PATCH v4 00/22] vhost: generic vhost API Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 01/22] vhost: introduce driver features related APIs Yuanhan Liu
2017-04-05  0:01         ` Thomas Monjalon
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 02/22] net/vhost: remove feature " Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 05/22] vhost: export guest memory regions Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-04-01  8:28         ` Maxime Coquelin
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 07/22] vhost: export vhost vring info Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 10/22] vhost: export the number of vrings Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-04-05  0:17         ` Thomas Monjalon
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 14/22] vhost: rename device ops struct Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 16/22] vhost: add features changed callback Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 19/22] vhost: rename header file Yuanhan Liu
2017-04-05  0:26         ` Thomas Monjalon
2017-04-05  5:16           ` Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-04-01  7:23       ` [dpdk-dev] [PATCH v4 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-04-01  8:44       ` [dpdk-dev] [PATCH v4 00/22] vhost: generic vhost API Yuanhan Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1490253059-28112-2-git-send-email-yuanhan.liu@linux.intel.com \
    --to=yuanhan.liu@linux.intel.com \
    --cc=changpeng.liu@intel.com \
    --cc=dev@dpdk.org \
    --cc=james.r.harris@intel.com \
    --cc=maxime.coquelin@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).