From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 358601B659 for ; Tue, 3 Apr 2018 10:27:36 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A65608182D06; Tue, 3 Apr 2018 08:27:35 +0000 (UTC) Received: from [10.36.112.46] (ovpn-112-46.ams2.redhat.com [10.36.112.46]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 799E0202698A; Tue, 3 Apr 2018 08:27:33 +0000 (UTC) To: Zhihong Wang , dev@dpdk.org Cc: jianfeng.tan@intel.com, tiwei.bie@intel.com, yliu@fridaylinux.org, cunming.liang@intel.com, xiao.w.wang@intel.com, dan.daly@intel.com References: <1517614137-62926-1-git-send-email-zhihong.wang@intel.com> <20180402114656.17090-1-zhihong.wang@intel.com> From: Maxime Coquelin Message-ID: Date: Tue, 3 Apr 2018 10:27:31 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180402114656.17090-1-zhihong.wang@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 03 Apr 2018 08:27:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 03 Apr 2018 08:27:35 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH v5 0/5] vhost: support selective datapath X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Apr 2018 08:27:36 -0000 On 04/02/2018 01:46 PM, Zhihong Wang wrote: > This patch set introduces support for selective datapath in DPDK vhost-user > lib. vDPA stands for vhost Data Path Acceleration. The idea is to support > virtio ring compatible devices to serve virtio driver directly to enable > datapath acceleration. > > Design details > ==== > > A vDPA device is a virtio ring compatible backend. It's defined as follows: > > struct rte_vdpa_dev_addr { > enum vdpa_addr_type type; > union { > uint8_t __dummy[64]; > struct rte_pci_addr pci_addr; > }; > }; > > struct rte_vdpa_dev_ops { > /* Get capabilities of this device */ > int (*get_queue_num)(int did, uint32_t *queue_num); > int (*get_features)(int did, uint64_t *features); > int (*get_protocol_features)(int did, uint64_t *protocol_features); > > /* Driver configure/close the device */ > int (*dev_conf)(int vid); > int (*dev_close)(int vid); > > /* Enable/disable this vring */ > int (*set_vring_state)(int vid, int vring, int state); > > /* Set features when changed */ > int (*set_features)(int vid); > > /* Destination operations when migration done */ > int (*migration_done)(int vid); > > /* Get the vfio group fd */ > int (*get_vfio_group_fd)(int vid); > > /* Get the vfio device fd */ > int (*get_vfio_device_fd)(int vid); > > /* Get the notify area info of the queue */ > int (*get_notify_area)(int vid, int qid, > uint64_t *offset, uint64_t *size); > > /* Reserved for future extension */ > void *reserved[5]; > }; > > struct rte_vdpa_device { > struct rte_vdpa_dev_addr addr; > struct rte_vdpa_dev_ops *ops; > } __rte_cache_aligned; > > > Changes to the current vhost-user lib are: > ==== > > 1. Make vhost device capabilities configurable to adopt various devices. > Such capabilities include supported features, protocol features, queue > number. APIs are introduced to configure these capabilities. > > 2. A set of device ops is defined for device specific operations: > > a. get_queue_num: Called to get supported queue number of the device. > > b. get_features: Called to get supported features of the device. > > c. get_protocol_features: Called to get supported protocol features of > the device. > > d. dev_conf: Called to configure the actual device when the virtio > device becomes ready. > > e. dev_close: Called to close the actual device when the virtio device > is stopped. > > f. set_vring_state: Called to change the state of the vring in the > actual device when vring state changes. > > g. set_features: Called to set the negotiated features to device. > > h. migration_done: Called to allow the device to response to RARP > sending. > > i. get_vfio_group_fd: Called to get the VFIO group fd of the device. > > j. get_vfio_device_fd: Called to get the VFIO device fd of the device. > > k. get_notify_area: Called to get the notify area info of the queue. > > 3. To make vhost aware of its datapath type, an device id (did) is added > into the vhost data structure to identify the actual device. APIs are > introduced to configure it. When the default software datapath is used, > did is set to -1. When alternative datapath is used, did should be set > to specify which device to use, and each vhost-user socket can have > only 1 connection in this case. > > Working process: > ==== > > 1. Register vDPA device with address and device ops, and get a did. > > 2. For vhost device creation: > > a. Register vhost-user socket. > > b. Set did of the vhost-user socket. > > c. Register vhost-user callbacks. > > d. Start to wait for connection. > > 3. When connection comes and virtio device data structure is negotiated, > the device will be configured with all needed info. > > --- > Changes in v5: > > 1. Rename the vDPA device ops to follow convention. > > 2. Rename APIs for better description. > > 3. Improve sanity check. > > --- > Changes in v4: > > 1. Remove the "engine" concept in the lib. > > --- > Changes in v3: > > 1. Keep macro names the same as in the spec. > > 2. Export new APIs where they're introduced. > > --- > Changes in v2: > > 1. Ensure negotiated capabilities are supported in vhost-user lib. > > 2. Add APIs for live migration. > > 3. Configure the data path at the right time. > > 4. Add VFIO related vDPA device ops. > > 5. Rebase on dpdk-next-virtio. > > Zhihong Wang (5): > vhost: export vhost feature definitions > vhost: support selective datapath > vhost: add apis for datapath configuration > vhost: adapt vhost lib for selective datapath > vhost: add apis for live migration > > lib/librte_vhost/Makefile | 4 +- > lib/librte_vhost/rte_vdpa.h | 87 +++++++++++++++++ > lib/librte_vhost/rte_vhost.h | 154 ++++++++++++++++++++++++++++++ > lib/librte_vhost/rte_vhost_version.map | 16 ++++ > lib/librte_vhost/socket.c | 166 ++++++++++++++++++++++++++++++++- > lib/librte_vhost/vdpa.c | 115 +++++++++++++++++++++++ > lib/librte_vhost/vhost.c | 105 +++++++++++++++++++++ > lib/librte_vhost/vhost.h | 14 ++- > lib/librte_vhost/vhost_user.c | 70 +++++++++++++- > lib/librte_vhost/vhost_user.h | 7 -- > 10 files changed, 718 insertions(+), 20 deletions(-) > create mode 100644 lib/librte_vhost/rte_vdpa.h > create mode 100644 lib/librte_vhost/vdpa.c > Series applied to dpdk-next-virtio/master. Thanks! Maxime