DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH 00/17] vhost: generic vhost API
@ 2017-03-03  9:51 Yuanhan Liu
  2017-03-03  9:51 ` [dpdk-dev] [PATCH 01/17] vhost: introduce driver features related APIs Yuanhan Liu
                   ` (17 more replies)
  0 siblings, 18 replies; 135+ messages in thread
From: Yuanhan Liu @ 2017-03-03  9:51 UTC (permalink / raw)
  To: dev; +Cc: Maxime Coquelin, Harris James R, Liu Changpeng, Yuanhan Liu

This is a first attempt to make DPDK vhost library be generic enough,
so that user could built its own vhost-user drivers on top of it. For
example, SPDK (Storage Performance Development Kit) is trying to enable
vhost-user SCSI.

The basic idea is, let DPDK vhost be a vhost-user agent. It stores all
the info about the virtio device (i.e. vring address, negotiated features,
etc) and let the specific vhost-user driver to fetch them (by the API
provided by DPDK vhost lib). With those info being provided, the vhost-user
driver then could get/put vring entries, thus, it could exchange data
between the guest and host.

The last patch demonstrates how to use these new APIs to implement a
very simple vhost-user net driver, without any fancy features enabled.


API/ABI Changes summary
=======================

- some renames
  * "struct virtio_net_device_ops" ==> "struct vhost_device_ops"
  * "rte_virtio_net.h"  ==> "rte_vhost.h"

- driver related APIs are bond with the socket file
  * rte_vhost_driver_set_features(socket_file, features);
  * rte_vhost_driver_get_features(socket_file, features);
  * rte_vhost_driver_enable_features(socket_file, features)
  * rte_vhost_driver_disable_features(socket_file, features)
  * rte_vhost_driver_callback_register(socket_file, notify_ops);

- new APIs to fetch guest and vring info
  * rte_vhost_get_vhost_memory(int vid, struct rte_vhost_memory **mem);
  * rte_vhost_get_negotiated_features(int vid);
  * rte_vhost_get_vhost_vring(int vid, uint16_t vring_idx,
			      struct rte_vhost_vring *vring);

- new exported structures 
  * struct rte_vhost_vring
  * struct rte_vhost_mem_region
  * struct rte_vhost_memory


Some design choices
===================

While making this patchset, I met quite few design choices and here are
two of them, with the issue and the reason I made such choices provided.
Please let me know if you have any comments (or better ideas).

Export public structures or not
-------------------------------

I made an ABI refactor last time (v16.07): move all the structures
internally and let applications use a "vid" to reference the internal
struct. With that, I hope we could never worry about the annoying ABI
issues.

It works great (and as expected) since then, as far as we only support
virito-net, as far as we can handle all the descs inside vhost lib. It
becomes problematic when a user wants to implement a vhost-user driver
somewhere. For example, it needs do the GPA to VVA translation. Without
any structs exported, some functions like gpa_to_vva() can't be inlined.
Calling it would be costly, especially it's a function we have to invoke
for processing each vring desc.

For that reason, the guest memory regions are exported. With that, the
gpa_to_vva could be inlined.

  
Add helper functions to fetch/update descs or not
-------------------------------------------------

I intended to do it like this way: introduce one function to get @count
of descs from a specific vring and another one to update the used descs.
It's something like
    rte_vhost_vring_get_descs(vid, vring_idx, count, offset, iov, descs);
    rte_vhost_vring_update_used_descs(vid, vring_idx, count, offset, descs);

With that, vhost-user driver programmer's task would be easier, as he/she
doesn't have to parse the descs any more (such as to handle indirect desc).

But judging that virtio 1.1 is just emerged and it proposes a completely
ring layout, and most importantly, the vring desc structure is also changed,
I'd like to hold to introduce such two functions. Otherwise, it's very
likely the two will be invalid when virtio 1.1 is out. Though I think it
may could be addressed with a care design, something like making the IOV
generic enough:

	struct rte_vhost_iov {
		uint64_t	gpa;
		uint64_t	vva;
		uint64_t	len;
	};

Instead, I go with the other way: introduce few APIs to export all the vring
infos (vring size, vring addr, callfd, etc), and let the vhost-user driver
read and update the descs. Those info could be passed to vhost-user driver
by introducing one API for each, but for saving few APIs and reducing few
calls for the programmer, I packed few key fields into a new structure, so
that it can be fetched with one call:
        struct rte_vhost_vring {
                struct vring_desc       *desc;
                struct vring_avail      *avail;
                struct vring_used       *used;
                uint64_t                log_guest_addr;
       
                int                     callfd;
                int                     kickfd;
                uint16_t                size;
        };

When virtio 1.1 comes out, likely a simple change like following would
just work:
        struct rte_vhost_vring {
		union {
			struct {
                		struct vring_desc       *desc;
                		struct vring_avail      *avail;
                		struct vring_used       *used;
                		uint64_t                log_guest_addr;
			};
			struct desc	*desc_1_1;	/* vring addr for virtio 1.1 */
		};
       
                int                     callfd;
                int                     kickfd;
                uint16_t                size;
        };

AFAIK, it's not an ABI breakage. Even if it does, we could introduce a new
API to get the virtio 1.1 ring address.

Those fields are the minimum set I got for a specific vring, with the mind
it would bring the minimum chance to break ABI for future extension. If we
need more info, we could introduce a new API.

OTOH, for getting the best performance, the two functions also have to be
inlined ("vid + vring_idx" combo is replaced with "vring"):
    rte_vhost_vring_get_descs(vring, count, offset, iov, descs);
    rte_vhost_vring_update_used_descs(vring, count, offset, descs);

That said, one way or another, we have to export rte_vhost_vring struct.
For this reason, I didn't rush into introducing the two APIs.


TODOs
=====

This series still got few small items to finish, and they are:
- update release note
- fill API comments
- set protocol features


	--yliu

---
Yuanhan Liu (17):
  vhost: introduce driver features related APIs
  net/vhost: remove feature related APIs
  vhost: use new APIs to handle features
  vhost: make notify ops per vhost driver
  vhost: export guest memory regions
  vhost: introduce API to fetch negotiated features
  vhost: export vhost vring info
  vhost: export API to translate gpa to vva
  vhost: turn queue pair to vring
  vhost: export the number of vrings
  vhost: move the device ready check at proper place
  vhost: drop the Rx and Tx queue macro
  vhost: do not include net specific headers
  vhost: rename device ops struct
  vhost: rename virtio-net to vhost
  vhost: rename header file
  examples/vhost: demonstrate the new generic vhost APIs

 doc/guides/rel_notes/deprecation.rst        |   9 -
 drivers/net/vhost/rte_eth_vhost.c           |  51 ++--
 drivers/net/vhost/rte_eth_vhost.h           |  32 +--
 drivers/net/vhost/rte_pmd_vhost_version.map |   3 -
 examples/tep_termination/main.c             |  11 +-
 examples/tep_termination/main.h             |   2 +
 examples/tep_termination/vxlan_setup.c      |   2 +-
 examples/vhost/Makefile                     |   2 +-
 examples/vhost/main.c                       |  88 ++++--
 examples/vhost/main.h                       |  33 ++-
 examples/vhost/virtio_net.c                 | 405 ++++++++++++++++++++++++++++
 lib/librte_vhost/Makefile                   |   4 +-
 lib/librte_vhost/rte_vhost.h                | 259 ++++++++++++++++++
 lib/librte_vhost/rte_vhost_version.map      |  18 +-
 lib/librte_vhost/rte_virtio_net.h           | 193 -------------
 lib/librte_vhost/socket.c                   | 143 ++++++++++
 lib/librte_vhost/vhost.c                    | 209 +++++++-------
 lib/librte_vhost/vhost.h                    |  82 +++---
 lib/librte_vhost/vhost_user.c               |  91 +++----
 lib/librte_vhost/vhost_user.h               |   2 +-
 lib/librte_vhost/virtio_net.c               |  35 +--
 21 files changed, 1140 insertions(+), 534 deletions(-)
 create mode 100644 examples/vhost/virtio_net.c
 create mode 100644 lib/librte_vhost/rte_vhost.h
 delete mode 100644 lib/librte_vhost/rte_virtio_net.h

-- 
1.9.0

^ permalink raw reply	[flat|nested] 135+ messages in thread

end of thread, other threads:[~2017-04-05  5:19 UTC | newest]

Thread overview: 135+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-03  9:51 [dpdk-dev] [PATCH 00/17] vhost: generic vhost API Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 01/17] vhost: introduce driver features related APIs Yuanhan Liu
2017-03-14  9:46   ` Maxime Coquelin
2017-03-14  9:53     ` Maxime Coquelin
2017-03-16  7:08       ` Yuanhan Liu
2017-03-16  9:18         ` Maxime Coquelin
2017-03-17  5:50           ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 02/17] net/vhost: remove feature " Yuanhan Liu
2017-03-14 10:15   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 03/17] vhost: use new APIs to handle features Yuanhan Liu
2017-03-14 10:43   ` Maxime Coquelin
2017-03-16  7:43     ` Yuanhan Liu
2017-03-16  9:39       ` Maxime Coquelin
2017-03-17  5:48         ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 04/17] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-14 10:55   ` Maxime Coquelin
2017-03-16  7:50     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 05/17] vhost: export guest memory regions Yuanhan Liu
2017-03-14 11:00   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 06/17] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-14 11:02   ` Maxime Coquelin
2017-03-16  7:35     ` Yuanhan Liu
2017-03-16  9:22       ` Maxime Coquelin
2017-03-17  5:49         ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 07/17] vhost: export vhost vring info Yuanhan Liu
2017-03-14 12:11   ` Maxime Coquelin
2017-03-16  7:24     ` Yuanhan Liu
2017-03-16  9:20       ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 08/17] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-14 12:24   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 09/17] vhost: turn queue pair to vring Yuanhan Liu
2017-03-14 12:31   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 10/17] vhost: export the number of vrings Yuanhan Liu
2017-03-14 12:33   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 11/17] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-14 12:37   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 12/17] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-14 12:42   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 13/17] vhost: do not include net specific headers Yuanhan Liu
2017-03-14 12:46   ` Maxime Coquelin
2017-03-20  7:32   ` Liu, Changpeng
2017-03-22  6:21     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 14/17] vhost: rename device ops struct Yuanhan Liu
2017-03-14 12:48   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 15/17] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-14 12:50   ` Maxime Coquelin
2017-03-03  9:51 ` [dpdk-dev] [PATCH 16/17] vhost: rename header file Yuanhan Liu
2017-03-14 12:59   ` Maxime Coquelin
2017-03-20  5:35     ` Yuanhan Liu
2017-03-03  9:51 ` [dpdk-dev] [PATCH 17/17] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-03-23  7:10 ` [dpdk-dev] [PATCH v2 00/22] vhost: generic vhost API Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 01/22] vhost: introduce driver features related APIs Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 02/22] net/vhost: remove feature " Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 05/22] vhost: export guest memory regions Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 07/22] vhost: export vhost vring info Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 10/22] vhost: export the number of vrings Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 14/22] vhost: rename device ops struct Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 16/22] vhost: add features changed callback Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 19/22] vhost: rename header file Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-03-23  7:10   ` [dpdk-dev] [PATCH v2 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-03-28 12:45   ` [dpdk-dev] [PATCH v3 00/22] vhost: generic vhost API Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 01/22] vhost: introduce driver features related APIs Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 02/22] net/vhost: remove feature " Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-03-29 14:57       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-03-29 15:03       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 05/22] vhost: export guest memory regions Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-03-31  7:45       ` Maxime Coquelin
2017-03-31  8:51         ` Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 07/22] vhost: export vhost vring info Yuanhan Liu
2017-03-31  7:48       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 10/22] vhost: export the number of vrings Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 14/22] vhost: rename device ops struct Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 16/22] vhost: add features changed callback Yuanhan Liu
2017-03-31  7:50       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-03-31  8:05       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-03-31  9:11       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 19/22] vhost: rename header file Yuanhan Liu
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-03-31  9:13       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-03-31  9:26       ` Maxime Coquelin
2017-03-28 12:45     ` [dpdk-dev] [PATCH v3 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-04-01  7:22     ` [dpdk-dev] [PATCH v4 00/22] vhost: generic vhost API Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 01/22] vhost: introduce driver features related APIs Yuanhan Liu
2017-04-05  0:01         ` Thomas Monjalon
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 02/22] net/vhost: remove feature " Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 03/22] vhost: use new APIs to handle features Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 04/22] vhost: make notify ops per vhost driver Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 05/22] vhost: export guest memory regions Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 06/22] vhost: introduce API to fetch negotiated features Yuanhan Liu
2017-04-01  8:28         ` Maxime Coquelin
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 07/22] vhost: export vhost vring info Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 08/22] vhost: export API to translate gpa to vva Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 09/22] vhost: turn queue pair to vring Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 10/22] vhost: export the number of vrings Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 11/22] vhost: move the device ready check at proper place Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 12/22] vhost: drop the Rx and Tx queue macro Yuanhan Liu
2017-04-05  0:17         ` Thomas Monjalon
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 13/22] vhost: do not include net specific headers Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 14/22] vhost: rename device ops struct Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 15/22] vhost: rename virtio-net to vhost Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 16/22] vhost: add features changed callback Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 17/22] vhost: export APIs for live migration support Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 18/22] vhost: introduce API to start a specific driver Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 19/22] vhost: rename header file Yuanhan Liu
2017-04-05  0:26         ` Thomas Monjalon
2017-04-05  5:16           ` Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 20/22] vhost: workaround the build dependency on mbuf header Yuanhan Liu
2017-04-01  7:22       ` [dpdk-dev] [PATCH v4 21/22] vhost: do not destroy device on repeat mem table message Yuanhan Liu
2017-04-01  7:23       ` [dpdk-dev] [PATCH v4 22/22] examples/vhost: demonstrate the new generic vhost APIs Yuanhan Liu
2017-04-01  8:44       ` [dpdk-dev] [PATCH v4 00/22] vhost: generic vhost API Yuanhan Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).