From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6FBCA0613 for ; Mon, 23 Sep 2019 12:28:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9895A1BE9B; Mon, 23 Sep 2019 12:28:41 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 8B7EF1BE98 for ; Mon, 23 Sep 2019 12:28:39 +0200 (CEST) Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B4F2C0546FC for ; Mon, 23 Sep 2019 10:28:38 +0000 (UTC) Received: by mail-wm1-f69.google.com with SMTP id o188so6451491wmo.5 for ; Mon, 23 Sep 2019 03:28:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=ZwTNpHnNyOYTBkrkPwlxdiVF/ZGbC0NHpa6U+8AYyQc=; b=ESf21aCFagnSTwpLn8LCIhTeYzHfEkAoi6uLBqCOT2Ux/ZwoBoPYunv26Fkg7vsJI4 gGgTjhZsIMa11tW5vvfVae127PvzgbnC3agLry2/lBFPF37G5X97VYdwXg+Y8Noi0lxs k/ezmVg1YZn/TIBE4GXjDAn+m3+za07uOa17VnrNcaFLJkQY4RkyqzTUYPmEMXEGbsnS PA4rOwKjymMeFkyDxrFTU7m0HCjRbwQBDp/HpEHbZ1TzplRtG8tKCmg1M2Cvoiv+hXbP Jk0bBRqmTZC+YJxo/1oqjVtUauqCpp70fV8hExgZZ6BV1h+4o+SqE6CEM5sE26jhskj5 chnA== X-Gm-Message-State: APjAAAWNPnI7RfOejGB0+mBGmvJMaEB6dJMSQTmA1Hjf2z5OX5nEvKpv J2b/+7UwSgC2ntuwbK7qQtTLz6YuPhQEwpNTmWwLNSibDYi/MrGX5SnsrwUQ8+vF9Bcpz1H8Kuo Ce6U= X-Received: by 2002:a1c:3182:: with SMTP id x124mr14168719wmx.168.1569234517402; Mon, 23 Sep 2019 03:28:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqyVlTFbcEJohxN7Rg3klzH11+SGOi8uNGTOZSq0xFeCbT2+vAHihCKZKOIE9I6/we0Rx4qkQg== X-Received: by 2002:a1c:3182:: with SMTP id x124mr14168701wmx.168.1569234517182; Mon, 23 Sep 2019 03:28:37 -0700 (PDT) Received: from amorenoz.users.ipa.redhat.com (static.169.4.0.81.ibercom.com. [81.0.4.169]) by smtp.gmail.com with ESMTPSA id w22sm8767534wmc.16.2019.09.23.03.28.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 Sep 2019 03:28:36 -0700 (PDT) To: Tiwei Bie Cc: dev@dpdk.org, zhihong.wang@intel.com, Maxime Coquelin , Pei Zhang References: <20190917144900.14407-1-amorenoz@redhat.com> <20190923072515.GA20810@___> From: Adrian Moreno Autocrypt: addr=amorenoz@redhat.com; prefer-encrypt=mutual; keydata= mQENBF1syNUBCADQ9dk3fDMxOZ/+OQpmbanpodYxEv8IRtDz8PXw8YX7UyGfozOpLjQ8Fftj ZxuubYNbt2QVbSgviFilFdNWu2eTnN/JaGtfhmTOLPVoakkPHZF8lbgImMoch7L0fH8wN2IM KPxQyPNlX+K9FD5brHsV1lfe1TwAxmhcvLW8yNrVq+9eDIDykxc7tH4exIqXgZroahGxMHKy c8Ti2kJka/t6pDfRaY0J+6J7I1nrn6GXXSMNA45EH8+0N/QlcXhP3rfftnoPeVmpjswzvJqY FNjf/Q5VPLx7RX0Qx+y8mMB2JcChV5Bl7D7x5EUbItj6+Sy7QfOgCtPegk9HSrBCNYaLABEB AAG0I0FkcmlhbiBNb3Jlbm8gPGFtb3Jlbm96QHJlZGhhdC5jb20+iQFUBBMBCAA+FiEEogUD gihhmbOPHy26d5C5fbYeFsUFAl1syNUCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQd5C5fbYeFsX7qwgArGHSkX+ILNcujkVzjTG4OtkpJMPFlkn/1PxSEKD0jLuzx14B COzpg/Mqj3Re/QBuOas+ci9bsUA0/2nORtmmEBvzDOJpR5FH1jaGCx8USlY4WM6QqEDNZgTw hsy9KhjFzFjMk+oo3HyItXA+Uq9yrRBTjNBGTXxezMRcMuUZ4MIAfY0IRBglL2BufiuL43jD BvTENNFLoQ/wFV7qkFWSkv+8IjTsxr7M6XUo1QLd29Hn0dvwssN579HL1+BP46i2REpzeBEG L75iVChi+YnIQQNMJ9NYarVabZx4Y1Gn8+7B/1SNArDV+IDgnYgt7E58otoV2Ap310dmtuvE VbxGpbkBDQRdbMjVAQgAqyp9oA7WDu7/Y9T4Ommt69iZx8os7shUIfdgPEy5xrcPn6qGwN1/ HQ4j8nWfBG9uuX1X0RXUZIUEtYTxtED4yaCQMTqDUf9cBAwAA2mYxBfoiNYx8YqxM+sT0/J4 2qmDd+y+20UR4yzHE8AmIbspTzDFIJDAi+jKSR8F355z0sfW7CIMDC4ZWrPsskjEy1YN/U10 r6tRRH1kNyrCSbTG0d9MtcQO58h7DLXuzUhErB+BtG52A04t5cweIJTJC+koV5XPeilzlHnm RFoj0ncruGa9Odns21BNt3cy9wLfK+aUnWuAB1uc6bJGQPiAwjkilz7g7MBRUuIQ2Zt7HGLc SwARAQABiQE8BBgBCAAmFiEEogUDgihhmbOPHy26d5C5fbYeFsUFAl1syNUCGwwFCQHhM4AA CgkQd5C5fbYeFsUlSwf8CH+u/IXaE7WeWxwFkMaORfW8cM4q0xrL3M6yRGuQNW+kMjnrvK9U J9G+L1/5uTRbDQ/4LdoKqize8LjehA+iF6ba4t9Npikh8fLKWgaJfQ/hPhH4C3O5gWPOLTW6 ylGxiuER4CdFwQIoAMMslhFA7G+teeOKBq36E+1+zrybI6Xy1UBSlpDK9j4CtTnMQejjuSQb Qhle+l8VroaUHq869wjAhRHHhqmtJKggI+OvzgQpDIwfHIDypb1BuKydi2W6cVYEALUYyCLS dTBDhzj8zR5tPCsga8J7+TclQzkWOiI2C6ZtiWrMsL/Uym3uXk5nsoc7lSj7yLd/MrBRhYfP JQ== Message-ID: <21f69f49-5921-bf3b-9655-f986d68d7202@redhat.com> Date: Mon, 23 Sep 2019 12:28:35 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 In-Reply-To: <20190923072515.GA20810@___> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH] vhost: translate incoming log address to gpa X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 9/23/19 9:25 AM, Tiwei Bie wrote: > On Tue, Sep 17, 2019 at 04:49:00PM +0200, Adrian Moreno wrote: >> When IOMMU is enabled the incoming log address is in IOVA space. In that >> case, look in IOTLB table and translate the resulting HVA to GPA. >> >> If IOMMU is not enabled, the incoming log address is already a GPA so no >> transformation is needed. >> >> This change makes page logging work when IOVA_VA is selected in the guest. > > Besides the log address of the ring, when IOMMU is enabled, > the addresses in descriptors are also IOVAs and should be > translated to GPAs before doing the dirty page logging. > Thanks Tiwei. You're right. In fact, it's not the only place where IOVAs are assumed to be GPAs, for example in vhost.h:gpa_to_hpa: /* Convert guest physical address to host physical address */ static __rte_always_inline rte_iova_t gpa_to_hpa(struct virtio_net *dev, uint64_t gpa, uint64_t size) { uint32_t i; struct guest_page *page; for (i = 0; i < dev->nr_guest_pages; i++) { page = &dev->guest_pages[i]; if (gpa >= page->guest_phys_addr && gpa + size < page->guest_phys_addr + page->size) { return gpa - page->guest_phys_addr + page->host_phys_addr; } } return 0; } used in ZERO-COPY mode to check if the IOVA range is continuous in host physical memory and called buffer IOVAs. If I'm not mistaken this should also be translated as: IOVA --> HVA (iotlb lookup) HVA --> GPA (mem_region lookup) GPA --> HIOVA (guest_page lookup) Right? Thanks. Adrián >> Further information: https://bugs.dpdk.org/show_bug.cgi?id=337 > > As this is fixing a real bug, we also need a "Fixes: " line and Cc stable. > > Thanks! > Tiwei > >> >> Cc: Maxime Coquelin >> Reported-by: Pei Zhang >> >> Signed-off-by: Adrian Moreno >> --- >> lib/librte_vhost/vhost.c | 1 + >> lib/librte_vhost/vhost_user.c | 78 ++++++++++++++++++++++++++++++++++- >> 2 files changed, 78 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c >> index 981837b5d..e57dda22f 100644 >> --- a/lib/librte_vhost/vhost.c >> +++ b/lib/librte_vhost/vhost.c >> @@ -383,6 +383,7 @@ vring_invalidate(struct virtio_net *dev, struct vhost_virtqueue *vq) >> vq->desc = NULL; >> vq->avail = NULL; >> vq->used = NULL; >> + vq->log_guest_addr = 0; >> >> if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) >> vhost_user_iotlb_wr_unlock(vq); >> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >> index 0b72648a5..35fca00fe 100644 >> --- a/lib/librte_vhost/vhost_user.c >> +++ b/lib/librte_vhost/vhost_user.c >> @@ -570,6 +570,74 @@ ring_addr_to_vva(struct virtio_net *dev, struct vhost_virtqueue *vq, >> return qva_to_vva(dev, ra, size); >> } >> >> +/* >> + * Converts Vhost Virtual Address to Guest Physical Address >> + */ >> +static uint64_t >> +vva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t *len) >> +{ >> + struct rte_vhost_mem_region *r; >> + uint32_t i; >> + >> + if (unlikely(!dev || !dev->mem)) >> + goto out_error; >> + >> + /* Find the region where the address lives. */ >> + for (i = 0; i < dev->mem->nregions; i++) { >> + r = &dev->mem->regions[i]; >> + >> + if (vva >= r->host_user_addr && >> + vva < r->host_user_addr + r->size) { >> + >> + if (unlikely(vva + *len > r->host_user_addr + r->size)) >> + *len = r->guest_user_addr + r->size - vva; >> + >> + return r->guest_phys_addr + vva - r->host_user_addr; >> + } >> + } >> +out_error: >> + *len = 0; >> + >> + return 0; >> +} >> + >> +/* >> + * Converts vring log address to GPA >> + * If IOMMU is enabled, the log address is IOVA >> + * If IOMMU not enabled, the log address is already GPA >> + */ >> +static uint64_t >> +translate_log_addr(struct virtio_net *dev, struct vhost_virtqueue *vq, >> + uint64_t log_addr) >> +{ >> + if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM)) { >> + const uint64_t exp_size = sizeof(struct vring_used) + >> + sizeof(struct vring_used_elem) * vq->size; >> + uint64_t vva, gpa; >> + uint64_t size = exp_size; >> + >> + vva = vhost_user_iotlb_cache_find(vq, log_addr, >> + &size, VHOST_ACCESS_RW); >> + if (size != exp_size) { >> + vhost_user_iotlb_miss(dev, log_addr + size, >> + VHOST_ACCESS_RW); >> + return 0; >> + } >> + >> + gpa = vva_to_gpa(dev, vva, &size); >> + if (size != exp_size) { >> + RTE_LOG(ERR, VHOST_CONFIG, >> + "VQ: Failed to find GPA mapping for log_addr." >> + "log_addr: 0x%0lx vva: 0x%0lx\n", >> + log_addr, vva); >> + return 0; >> + } >> + return gpa; >> + >> + } else >> + return log_addr; >> +} >> + >> static struct virtio_net * >> translate_ring_addresses(struct virtio_net *dev, int vq_index) >> { >> @@ -676,7 +744,15 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index) >> vq->last_avail_idx = vq->used->idx; >> } >> >> - vq->log_guest_addr = addr->log_guest_addr; >> + vq->log_guest_addr = >> + translate_log_addr(dev, vq, addr->log_guest_addr); >> + if (vq->log_guest_addr == 0) { >> + RTE_LOG(DEBUG, VHOST_CONFIG, >> + "(%d) failed to map log_guest_addr .\n", >> + dev->vid); >> + return dev; >> + } >> + >> >> VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n", >> dev->vid, vq->desc); >> -- >> 2.21.0 >>