From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 69A552C18 for ; Tue, 6 Mar 2018 15:24:32 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C1DFA8424C; Tue, 6 Mar 2018 14:24:31 +0000 (UTC) Received: from [10.36.112.61] (ovpn-112-61.ams2.redhat.com [10.36.112.61]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DFFB2144B20; Tue, 6 Mar 2018 14:24:29 +0000 (UTC) To: Tiwei Bie , dev@dpdk.org Cc: jianfeng.tan@intel.com, yliu@fridaylinux.org, zhihong.wang@intel.com, xiao.w.wang@intel.com, cunming.liang@intel.com, dan.daly@intel.com References: <20180306104327.14470-1-tiwei.bie@intel.com> <20180306104327.14470-4-tiwei.bie@intel.com> From: Maxime Coquelin Message-ID: <4a4025f9-6886-8adc-51ed-c8508d81a003@redhat.com> Date: Tue, 6 Mar 2018 15:24:27 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180306104327.14470-4-tiwei.bie@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 06 Mar 2018 14:24:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 06 Mar 2018 14:24:31 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH 3/3] vhost: support VFIO based accelerator X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Mar 2018 14:24:32 -0000 On 03/06/2018 11:43 AM, Tiwei Bie wrote: > This commit adds the VFIO based accelerator support to > vhost. A new API is provided to support asking QEMU to > do further setup to allow notifications and interrupts > being delivered directly between the driver in guest > and the vDPA device in host. > > Signed-off-by: Tiwei Bie > --- > lib/librte_vhost/rte_vhost.h | 28 ++++++ > lib/librte_vhost/rte_vhost_version.map | 1 + > lib/librte_vhost/vhost_user.c | 166 +++++++++++++++++++++++++++++++++ > lib/librte_vhost/vhost_user.h | 9 ++ > 4 files changed, 204 insertions(+) > > diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h > index d5589c543..68842e908 100644 > --- a/lib/librte_vhost/rte_vhost.h > +++ b/lib/librte_vhost/rte_vhost.h > @@ -35,6 +35,7 @@ extern "C" { > #define RTE_VHOST_USER_PROTOCOL_F_REPLY_ACK 3 > #define RTE_VHOST_USER_PROTOCOL_F_NET_MTU 4 > #define RTE_VHOST_USER_PROTOCOL_F_SLAVE_REQ 5 > +#define RTE_VHOST_USER_PROTOCOL_F_VFIO 8 > #define RTE_VHOST_USER_F_PROTOCOL_FEATURES 30 > > /** > @@ -591,6 +592,33 @@ rte_vhost_get_vdpa_eid(int vid); > int __rte_experimental > rte_vhost_get_vdpa_did(int vid); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * Enable or disable the VFIO based accelerator for vhost-user. > + * > + * This function is to ask QEMU to do further setup to better > + * support the vDPA device at vhost user backend. With this > + * setup, the notifications and interrupts will be delivered > + * directly between the driver in guest and the vDPA device > + * in host if platform supports e.g. EPT and Posted interrupt. > + * It's nice to have, and not mandatory. > + * > + * @param vid > + * vhost device ID > + * @param int > + * Enable or disable > + * > + * @return > + * 0: success > + * -ENODEV: no such vhost device > + * -ENOTSUP: device does not support VFIO based accelerator feature > + * -EINVAL: there is no accelerator assigned to this vhost device > + * -EFAULT: failed to talk with QEMU > + */ > +int rte_vhost_vfio_accelerator_ctrl(int vid, int enable); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_vhost/rte_vhost_version.map b/lib/librte_vhost/rte_vhost_version.map > index 36257e51b..ca970170f 100644 > --- a/lib/librte_vhost/rte_vhost_version.map > +++ b/lib/librte_vhost/rte_vhost_version.map > @@ -72,6 +72,7 @@ EXPERIMENTAL { > rte_vhost_set_vring_base; > rte_vhost_get_vdpa_eid; > rte_vhost_get_vdpa_did; > + rte_vhost_vfio_accelerator_ctrl; > rte_vdpa_register_engine; > rte_vdpa_unregister_engine; > rte_vdpa_find_engine_id; > diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c > index e3a1dfbfb..a65598d80 100644 > --- a/lib/librte_vhost/vhost_user.c > +++ b/lib/librte_vhost/vhost_user.c > @@ -35,6 +35,7 @@ > #include > #include > #include > +#include > > #include "iotlb.h" > #include "vhost.h" > @@ -1628,6 +1629,27 @@ vhost_user_msg_handler(int vid, int fd) > return 0; > } > > +static int process_slave_message_reply(struct virtio_net *dev, > + const VhostUserMsg *msg) > +{ > + VhostUserMsg msg_reply; > + > + if ((msg->flags & VHOST_USER_NEED_REPLY) == 0) > + return 0; > + > + if (read_vhost_message(dev->slave_req_fd, &msg_reply) < 0) > + return -1; > + > + if (msg_reply.request.slave != msg->request.slave) { > + RTE_LOG(ERR, VHOST_CONFIG, > + "received unexpected msg type (%u), expected %u\n", > + msg_reply.request.slave, msg->request.slave); > + return -1; > + } > + > + return msg_reply.payload.u64; > +} > + > int > vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) > { > @@ -1653,3 +1675,147 @@ vhost_user_iotlb_miss(struct virtio_net *dev, uint64_t iova, uint8_t perm) > > return 0; > } > + > +static int vhost_user_slave_set_vring_file(struct virtio_net *dev, > + uint32_t request, > + struct vhost_vring_file *file) Why passing the request as an argument? It seems to be called only with the same request ID. > +{ > + int *fdp = NULL; > + size_t fd_num = 0; > + int ret; > + struct VhostUserMsg msg = { > + .request.slave = request, > + .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, > + .payload.u64 = file->index & VHOST_USER_VRING_IDX_MASK, > + .size = sizeof(msg.payload.u64), > + }; > + > + if (file->fd < 0) > + msg.payload.u64 |= VHOST_USER_VRING_NOFD_MASK; > + else { > + fdp = &file->fd; > + fd_num = 1; > + } > + > + ret = send_vhost_message(dev->slave_req_fd, &msg, fdp, fd_num); > + if (ret < 0) { > + RTE_LOG(ERR, VHOST_CONFIG, > + "Failed to send slave message %u (%d)\n", > + request, ret); > + return ret; > + } > + > + return process_slave_message_reply(dev, &msg); Maybe not needed right now, but we'll need a lock to avoid concurrent requests sending and waiting for reply. Thanks, Maxime