From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 88C6011DE for ; Thu, 5 Oct 2017 10:44:32 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D6E8DC047B66; Thu, 5 Oct 2017 08:44:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D6E8DC047B66 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=maxime.coquelin@redhat.com Received: from localhost.localdomain (ovpn-112-45.ams2.redhat.com [10.36.112.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D4256251A; Thu, 5 Oct 2017 08:44:04 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, remy.horton@intel.com, tiwei.bie@intel.com, yliu@fridaylinux.org Cc: mst@redhat.com, jfreiman@redhat.com, vkaplans@redhat.com, jasowang@redhat.com, Maxime Coquelin Date: Thu, 5 Oct 2017 10:36:24 +0200 Message-Id: <20171005083627.27828-17-maxime.coquelin@redhat.com> In-Reply-To: <20171005083627.27828-1-maxime.coquelin@redhat.com> References: <20171005083627.27828-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 05 Oct 2017 08:44:32 +0000 (UTC) Subject: [dpdk-dev] [PATCH v3 16/19] vhost-user: translate ring addresses when IOMMU enabled X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Oct 2017 08:44:32 -0000 When IOMMU is enabled, the ring addresses set by the VHOST_USER_SET_VRING_ADDR requests are guest's IO virtual addresses, whereas Qemu virtual addresses when IOMMU is disabled. When enabled and the required translation is not in the IOTLB cache, an IOTLB miss request is sent, but being called by the vhost-user socket handling thread, the function does not wait for the requested IOTLB update. The function will be called again on the next IOTLB update message reception if matching the vring addresses. Signed-off-by: Maxime Coquelin --- lib/librte_vhost/vhost_user.c | 44 +++++++++++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 10 deletions(-) diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c index 319867c65..90b209764 100644 --- a/lib/librte_vhost/vhost_user.c +++ b/lib/librte_vhost/vhost_user.c @@ -324,10 +324,7 @@ numa_realloc(struct virtio_net *dev, int index __rte_unused) } #endif -/* - * Converts QEMU virtual address to Vhost virtual address. This function is - * used to convert the ring addresses to our address space. - */ +/* Converts QEMU virtual address to Vhost virtual address. */ static uint64_t qva_to_vva(struct virtio_net *dev, uint64_t qva) { @@ -348,6 +345,30 @@ qva_to_vva(struct virtio_net *dev, uint64_t qva) return 0; } + +/* + * Converts ring address to Vhost virtual address. + * If IOMMU is enabled, the ring address is a guest IO virtual address, + * else it is a QEMU virtual address. + */ +static uint64_t +ring_addr_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, + uint64_t ra, uint64_t size) +{ + if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) { + uint64_t vva; + + vva = vhost_user_iotlb_cache_find(vq, ra, + &size, VHOST_ACCESS_RW); + if (!vva) + vhost_user_iotlb_miss(dev, ra, VHOST_ACCESS_RW); + + return vva; + } + + return qva_to_vva(dev, ra); +} + /* * The virtio device sends us the desc, used and avail ring addresses. * This function then converts these to our address space. @@ -380,8 +401,11 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, struct vhost_vring_addr *addr = &vq->ring_addrs; /* The addresses are converted from QEMU virtual to Vhost virtual. */ - vq->desc = (struct vring_desc *)(uintptr_t)qva_to_vva(dev, - addr->desc_user_addr); + if (vq->desc && vq->avail && vq->used) + return dev; + + vq->desc = (struct vring_desc *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->desc_user_addr, sizeof(struct vring_desc)); if (vq->desc == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find desc ring address.\n", @@ -392,8 +416,8 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, dev = numa_realloc(dev, vq_index); vq = dev->virtqueue[vq_index]; - vq->avail = (struct vring_avail *)(uintptr_t)qva_to_vva(dev, - addr->avail_user_addr); + vq->avail = (struct vring_avail *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->avail_user_addr, sizeof(struct vring_avail)); if (vq->avail == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find avail ring address.\n", @@ -401,8 +425,8 @@ static struct virtio_net *translate_ring_addresses(struct virtio_net *dev, return NULL; } - vq->used = (struct vring_used *)(uintptr_t)qva_to_vva(dev, - addr->used_user_addr); + vq->used = (struct vring_used *)(uintptr_t)ring_addr_to_vva(dev, + vq, addr->used_user_addr, sizeof(struct vring_used)); if (vq->used == 0) { RTE_LOG(ERR, VHOST_CONFIG, "(%d) failed to find used ring address.\n", -- 2.13.6