From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FBFA41D5F; Fri, 24 Feb 2023 16:06:05 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E772F40A87; Fri, 24 Feb 2023 16:06:04 +0100 (CET) Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) by mails.dpdk.org (Postfix) with ESMTP id 24E3B40693 for ; Fri, 24 Feb 2023 16:06:03 +0100 (CET) Received: by mail-ot1-f52.google.com with SMTP id f19-20020a9d5f13000000b00693ce5a2f3eso2540599oti.8 for ; Fri, 24 Feb 2023 07:06:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=AeYDKYuAKc088XphBuAEGZ/hLwZUcKP4wQeymBNJOl4=; b=Bbs2UtPxfwJ6ea8QiGDCk0BAWvptPZLhfvbXzXsK/6NMffeANVD3uw3nMsdcXW1djK m0S+35UuKMBj3nNs2ew62R2KBpWQXjdrJhcGTb53R2O0lG/1Ylac8B7XQ/sF2kpW042p k6cnahwYBeJ/5jofVnx/y1pqqwUvbU/xXivZs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AeYDKYuAKc088XphBuAEGZ/hLwZUcKP4wQeymBNJOl4=; b=v2x+sTU0qPvPOVyxc7FkSsNp0jUeO8zupP1zIAcZOH9wQ0QTu6OeMwvs3bioiAbhAC kfsehMlNzIgZRKIgVXuhCNCJQbdGvZLlK+UDGJ0qZXzuS5uYnBhPKsAoHQ+LxmUq9WLe 3e/zrynnTWnd0+UMexvTia/yUcSwhiwXgGvjZkXcWoBO1TSbJhcTWYADWifCBit79FaN Xl3t0FzRAglvCn8hT+8Euf2a7sTv1/BvvTiMaOQ0yOkj18zPKm6a3FQZZ2kmADpAso+H fRS/Vo1dTwL04mD8z7NPoWo9TwIF0M65IQOlPIdNtL1BHHOAkx/vf/w4XKiH9ou+bhqi cgzA== X-Gm-Message-State: AO0yUKXF8Lvx+N/9xknOB3cEb+c0u1U3wF8ghxpRS16Neka0nuKZKJiX oiYCLDGyOjp+8L5JJOcU5I8DcKGgCtlbWji4FiVqmA== X-Google-Smtp-Source: AK7set+Mz+s2l6AzCzjFnysgmBAoY6rUsHOS7j+1BcU9qF5vD45X9Px6Qn1cxsrLc7xG5Nx/W6QgvIHitVP8+DGWrlY= X-Received: by 2002:a05:6830:1e90:b0:68d:48f0:9bad with SMTP id n16-20020a0568301e9000b0068d48f09badmr1696255otr.7.1677251162138; Fri, 24 Feb 2023 07:06:02 -0800 (PST) MIME-Version: 1.0 References: <20230222214342.1092333-1-mkp@redhat.com> <20230223043551.1108541-1-mkp@redhat.com> In-Reply-To: From: Patrick Robb Date: Fri, 24 Feb 2023 10:05:51 -0500 Message-ID: Subject: Re: [PATCH v2] vhost: fix madvise arguments alignment To: Mike Pattrick Cc: Maxime Coquelin , dev@dpdk.org, david.marchand@redhat.com, chenbo.xia@intel.com Content-Type: multipart/alternative; boundary="000000000000b8098a05f57377cf" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000b8098a05f57377cf Content-Type: text/plain; charset="UTF-8" UNH CI reported an ABI failure for this patch which did not report due to a bug on our end, so I'm manually reporting it now. I see Maxime you already predicted the issue though! *07:58:32* 1 function with some indirect sub-type change:*07:58:32* *07:58:32* [C] 'function int rte_vhost_get_mem_table(int, rte_vhost_memory**)' at vhost.c:922:1 has some indirect sub-type changes:*07:58:32* parameter 2 of type 'rte_vhost_memory**' has sub-type changes:*07:58:32* in pointed to type 'rte_vhost_memory*':*07:58:32* in pointed to type 'struct rte_vhost_memory' at rte_vhost.h:145:1:*07:58:32* type size hasn't changed*07:58:32* 1 data member change:*07:58:32* type of 'rte_vhost_mem_region regions[]' changed:*07:58:32* array element type 'struct rte_vhost_mem_region' changed:*07:58:32* type size changed from 448 to 512 (in bits)*07:58:32* 1 data member insertion:*07:58:32* 'uint64_t alignment', at offset 448 (in bits) at rte_vhost.h:139:1*07:58:32* type size hasn't changed*07:58:32* *07:58:32* Error: ABI issue reported for abidiff --suppr dpdk/devtools/libabigail.abignore --no-added-syms --headers-dir1 reference/include --headers-dir2 build_install/include reference/dump/librte_vhost.dump build_install/dump/librte_vhost.dump*07:58:32* ABIDIFF_ABI_CHANGE, this change requires a review (abidiff flagged this as a potential issue). On Thu, Feb 23, 2023 at 11:57 AM Mike Pattrick wrote: > On Thu, Feb 23, 2023 at 11:12 AM Maxime Coquelin > wrote: > > > > Hi Mike, > > > > Thanks for looking into this issue. > > > > On 2/23/23 05:35, Mike Pattrick wrote: > > > The arguments passed to madvise should be aligned to the alignment of > > > the backing memory. Now we keep track of each regions alignment and use > > > then when setting coredump preferences. To facilitate this, a new > member > > > was added to rte_vhost_mem_region. A new function was added to easily > > > translate memory address back to region alignment. Unneeded calls to > > > madvise were reduced, as the cache removal case should already be > > > covered by the cache insertion case. The previously inline function > > > mem_set_dump was removed from a header file and made not inline. > > > > > > Fixes: 338ad77c9ed3 ("vhost: exclude VM hugepages from coredumps") > > > > > > Signed-off-by: Mike Pattrick > > > --- > > > Since v1: > > > - Corrected a cast for 32bit compiles > > > --- > > > lib/vhost/iotlb.c | 9 +++--- > > > lib/vhost/rte_vhost.h | 1 + > > > lib/vhost/vhost.h | 12 ++------ > > > lib/vhost/vhost_user.c | 63 > +++++++++++++++++++++++++++++++++++------- > > > 4 files changed, 60 insertions(+), 25 deletions(-) > > > > > > diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c > > > index a0b8fd7302..5293507b63 100644 > > > --- a/lib/vhost/iotlb.c > > > +++ b/lib/vhost/iotlb.c > > > @@ -149,7 +149,6 @@ vhost_user_iotlb_cache_remove_all(struct > vhost_virtqueue *vq) > > > rte_rwlock_write_lock(&vq->iotlb_lock); > > > > > > RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { > > > - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, > true); > > > > Hmm, it should have been called with enable=false here since we are > > removing the entry from the IOTLB cache. It should be kept in order to > > "DONTDUMP" pages evicted from the cache. > > Here I was thinking that if we add an entry and then remove a > different entry, they could be in the same page. But on I should have > kept an enable=false in remove_all(). > > And now that I think about it again, I could just check if there are > any active cache entries in the page on every evict/remove, they're > sorted so that should be an easy check. Unless there are any > objections I'll go forward with that. > > > > > > TAILQ_REMOVE(&vq->iotlb_list, node, next); > > > vhost_user_iotlb_pool_put(vq, node); > > > } > > > @@ -171,7 +170,6 @@ vhost_user_iotlb_cache_random_evict(struct > vhost_virtqueue *vq) > > > > > > RTE_TAILQ_FOREACH_SAFE(node, &vq->iotlb_list, next, temp_node) { > > > if (!entry_idx) { > > > - mem_set_dump((void *)(uintptr_t)node->uaddr, > node->size, true); > > > > Same here. > > > > > TAILQ_REMOVE(&vq->iotlb_list, node, next); > > > vhost_user_iotlb_pool_put(vq, node); > > > vq->iotlb_cache_nr--; > > > @@ -224,14 +222,16 @@ vhost_user_iotlb_cache_insert(struct virtio_net > *dev, struct vhost_virtqueue *vq > > > vhost_user_iotlb_pool_put(vq, new_node); > > > goto unlock; > > > } else if (node->iova > new_node->iova) { > > > - mem_set_dump((void *)(uintptr_t)node->uaddr, > node->size, true); > > > + mem_set_dump((void *)(uintptr_t)new_node->uaddr, > new_node->size, true, > > > + hua_to_alignment(dev->mem, (void > *)(uintptr_t)node->uaddr)); > > > TAILQ_INSERT_BEFORE(node, new_node, next); > > > vq->iotlb_cache_nr++; > > > goto unlock; > > > } > > > } > > > > > > - mem_set_dump((void *)(uintptr_t)node->uaddr, node->size, true); > > > + mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node->size, > true, > > > + hua_to_alignment(dev->mem, (void > *)(uintptr_t)new_node->uaddr)); > > > TAILQ_INSERT_TAIL(&vq->iotlb_list, new_node, next); > > > vq->iotlb_cache_nr++; > > > > > > @@ -259,7 +259,6 @@ vhost_user_iotlb_cache_remove(struct > vhost_virtqueue *vq, > > > break; > > > > > > if (iova < node->iova + node->size) { > > > - mem_set_dump((void *)(uintptr_t)node->uaddr, > node->size, true); > > > TAILQ_REMOVE(&vq->iotlb_list, node, next); > > > vhost_user_iotlb_pool_put(vq, node); > > > vq->iotlb_cache_nr--; > > > diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h > > > index a395843fe9..c5c97ea67e 100644 > > > --- a/lib/vhost/rte_vhost.h > > > +++ b/lib/vhost/rte_vhost.h > > > @@ -136,6 +136,7 @@ struct rte_vhost_mem_region { > > > void *mmap_addr; > > > uint64_t mmap_size; > > > int fd; > > > + uint64_t alignment; > > > > This is not possible to do this as it breaks the ABI. > > You have to store the information somewhere else, or simply call > > get_blk_size() in hua_to_alignment() since the fd is not closed. > > > > Sorry about that! You're right, checking the fd per operation should > be easy enough. > > Thanks for the review, > > M > > > > }; > > > > > > /** > > > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h > > > index 5750f0c005..a2467ba509 100644 > > > --- a/lib/vhost/vhost.h > > > +++ b/lib/vhost/vhost.h > > > @@ -1009,14 +1009,6 @@ mbuf_is_consumed(struct rte_mbuf *m) > > > return true; > > > } > > > > > > -static __rte_always_inline void > > > -mem_set_dump(__rte_unused void *ptr, __rte_unused size_t size, > __rte_unused bool enable) > > > -{ > > > -#ifdef MADV_DONTDUMP > > > - if (madvise(ptr, size, enable ? MADV_DODUMP : MADV_DONTDUMP) == > -1) { > > > - rte_log(RTE_LOG_INFO, vhost_config_log_level, > > > - "VHOST_CONFIG: could not set coredump preference > (%s).\n", strerror(errno)); > > > - } > > > -#endif > > > -} > > > +uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *ptr); > > > +void mem_set_dump(void *ptr, size_t size, bool enable, uint64_t > alignment); > > > #endif /* _VHOST_NET_CDEV_H_ */ > > > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c > > > index d702d082dd..6d09597fbe 100644 > > > --- a/lib/vhost/vhost_user.c > > > +++ b/lib/vhost/vhost_user.c > > > @@ -737,6 +737,40 @@ log_addr_to_gpa(struct virtio_net *dev, struct > vhost_virtqueue *vq) > > > return log_gpa; > > > } > > > > > > +uint64_t > > > +hua_to_alignment(struct rte_vhost_memory *mem, void *ptr) > > > +{ > > > + struct rte_vhost_mem_region *r; > > > + uint32_t i; > > > + uintptr_t hua = (uintptr_t)ptr; > > > + > > > + for (i = 0; i < mem->nregions; i++) { > > > + r = &mem->regions[i]; > > > + if (hua >= r->host_user_addr && > > > + hua < r->host_user_addr + r->size) { > > > + return r->alignment; > > > + } > > > + } > > > + > > > + /* If region isn't found, don't align at all */ > > > + return 1; > > > +} > > > + > > > +void > > > +mem_set_dump(void *ptr, size_t size, bool enable, uint64_t pagesz) > > > +{ > > > +#ifdef MADV_DONTDUMP > > > + void *start = RTE_PTR_ALIGN_FLOOR(ptr, pagesz); > > > + uintptr_t end = RTE_ALIGN_CEIL((uintptr_t)ptr + size, pagesz); > > > + size_t len = end - (uintptr_t)start; > > > + > > > + if (madvise(start, len, enable ? MADV_DODUMP : MADV_DONTDUMP) == > -1) { > > > + rte_log(RTE_LOG_INFO, vhost_config_log_level, > > > + "VHOST_CONFIG: could not set coredump preference > (%s).\n", strerror(errno)); > > > + } > > > +#endif > > > +} > > > + > > > static void > > > translate_ring_addresses(struct virtio_net **pdev, struct > vhost_virtqueue **pvq) > > > { > > > @@ -767,6 +801,8 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > + mem_set_dump(vq->desc_packed, len, true, > > > + hua_to_alignment(dev->mem, vq->desc_packed)); > > > numa_realloc(&dev, &vq); > > > *pdev = dev; > > > *pvq = vq; > > > @@ -782,6 +818,8 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > + mem_set_dump(vq->driver_event, len, true, > > > + hua_to_alignment(dev->mem, vq->driver_event)); > > > len = sizeof(struct vring_packed_desc_event); > > > vq->device_event = (struct vring_packed_desc_event *) > > > (uintptr_t)ring_addr_to_vva(dev, > > > @@ -793,9 +831,8 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > - mem_set_dump(vq->desc_packed, len, true); > > > - mem_set_dump(vq->driver_event, len, true); > > > - mem_set_dump(vq->device_event, len, true); > > > + mem_set_dump(vq->device_event, len, true, > > > + hua_to_alignment(dev->mem, vq->device_event)); > > > vq->access_ok = true; > > > return; > > > } > > > @@ -812,6 +849,7 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > + mem_set_dump(vq->desc, len, true, hua_to_alignment(dev->mem, > vq->desc)); > > > numa_realloc(&dev, &vq); > > > *pdev = dev; > > > *pvq = vq; > > > @@ -827,6 +865,7 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > + mem_set_dump(vq->avail, len, true, hua_to_alignment(dev->mem, > vq->avail)); > > > len = sizeof(struct vring_used) + > > > sizeof(struct vring_used_elem) * vq->size; > > > if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX)) > > > @@ -839,6 +878,8 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > return; > > > } > > > > > > + mem_set_dump(vq->used, len, true, hua_to_alignment(dev->mem, > vq->used)); > > > + > > > if (vq->last_used_idx != vq->used->idx) { > > > VHOST_LOG_CONFIG(dev->ifname, WARNING, > > > "last_used_idx (%u) and vq->used->idx (%u) > mismatches;\n", > > > @@ -849,9 +890,6 @@ translate_ring_addresses(struct virtio_net **pdev, > struct vhost_virtqueue **pvq) > > > "some packets maybe resent for Tx and dropped > for Rx\n"); > > > } > > > > > > - mem_set_dump(vq->desc, len, true); > > > - mem_set_dump(vq->avail, len, true); > > > - mem_set_dump(vq->used, len, true); > > > vq->access_ok = true; > > > > > > VHOST_LOG_CONFIG(dev->ifname, DEBUG, "mapped address desc: > %p\n", vq->desc); > > > @@ -1230,7 +1268,8 @@ vhost_user_mmap_region(struct virtio_net *dev, > > > region->mmap_addr = mmap_addr; > > > region->mmap_size = mmap_size; > > > region->host_user_addr = (uint64_t)(uintptr_t)mmap_addr + > mmap_offset; > > > - mem_set_dump(mmap_addr, mmap_size, false); > > > + region->alignment = alignment; > > > + mem_set_dump(mmap_addr, mmap_size, false, alignment); > > > > > > if (dev->async_copy) { > > > if (add_guest_pages(dev, region, alignment) < 0) { > > > @@ -1535,7 +1574,6 @@ inflight_mem_alloc(struct virtio_net *dev, const > char *name, size_t size, int *f > > > return NULL; > > > } > > > > > > - mem_set_dump(ptr, size, false); > > > *fd = mfd; > > > return ptr; > > > } > > > @@ -1566,6 +1604,7 @@ vhost_user_get_inflight_fd(struct virtio_net > **pdev, > > > uint64_t pervq_inflight_size, mmap_size; > > > uint16_t num_queues, queue_size; > > > struct virtio_net *dev = *pdev; > > > + uint64_t alignment; > > > int fd, i, j; > > > int numa_node = SOCKET_ID_ANY; > > > void *addr; > > > @@ -1628,6 +1667,8 @@ vhost_user_get_inflight_fd(struct virtio_net > **pdev, > > > dev->inflight_info->fd = -1; > > > } > > > > > > + alignment = get_blk_size(fd); > > > + mem_set_dump(addr, mmap_size, false, alignment); > > > dev->inflight_info->addr = addr; > > > dev->inflight_info->size = ctx->msg.payload.inflight.mmap_size = > mmap_size; > > > dev->inflight_info->fd = ctx->fds[0] = fd; > > > @@ -1744,10 +1785,10 @@ vhost_user_set_inflight_fd(struct virtio_net > **pdev, > > > dev->inflight_info->fd = -1; > > > } > > > > > > - mem_set_dump(addr, mmap_size, false); > > > dev->inflight_info->fd = fd; > > > dev->inflight_info->addr = addr; > > > dev->inflight_info->size = mmap_size; > > > + mem_set_dump(addr, mmap_size, false, get_blk_size(fd)); > > > > > > for (i = 0; i < num_queues; i++) { > > > vq = dev->virtqueue[i]; > > > @@ -2242,6 +2283,7 @@ vhost_user_set_log_base(struct virtio_net **pdev, > > > struct virtio_net *dev = *pdev; > > > int fd = ctx->fds[0]; > > > uint64_t size, off; > > > + uint64_t alignment; > > > void *addr; > > > uint32_t i; > > > > > > @@ -2280,6 +2322,7 @@ vhost_user_set_log_base(struct virtio_net **pdev, > > > * fail when offset is not page size aligned. > > > */ > > > addr = mmap(0, size + off, PROT_READ | PROT_WRITE, MAP_SHARED, > fd, 0); > > > + alignment = get_blk_size(fd); > > > close(fd); > > > if (addr == MAP_FAILED) { > > > VHOST_LOG_CONFIG(dev->ifname, ERR, "mmap log base > failed!\n"); > > > @@ -2296,7 +2339,7 @@ vhost_user_set_log_base(struct virtio_net **pdev, > > > dev->log_addr = (uint64_t)(uintptr_t)addr; > > > dev->log_base = dev->log_addr + off; > > > dev->log_size = size; > > > - mem_set_dump(addr, size, false); > > > + mem_set_dump(addr, size + off, false, alignment); > > > > > > for (i = 0; i < dev->nr_vring; i++) { > > > struct vhost_virtqueue *vq = dev->virtqueue[i]; > > > > -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu --000000000000b8098a05f57377cf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
UNH CI reported an ABI failure for this patch which d= id not report due to a bug on our end, so I'm manually reporting it now= . I see Maxime you already predicted the issue though!
07:58:32  1 function with s=
ome indirect sub-type change:
07:58:32 =20
07:58:32    [C] 'function int rte_vhost_ge=
t_mem_table(int, rte_vhost_memory**)' at vhost.c:922:1 has some indirec=
t sub-type changes:
07:58:32      parameter 2 of type 'rte_vho=
st_memory**' has sub-type changes:
07:58:32        in pointed to type 'rte_vh=
ost_memory*':
07:58:32          in pointed to type 'stru=
ct rte_vhost_memory' at rte_vhost.h:145:1:
07:58:32            type size hasn't chang=
ed
07:58:32            1 data member change:
07:58:32              type of 'rte_vhost_m=
em_region regions[]' changed:
07:58:32                array element type =
9;struct rte_vhost_mem_region' changed:
07:58:32                  type size changed fr=
om 448 to 512 (in bits)
07:58:32                  1 data member insert=
ion:
07:58:32                    'uint64_t alig=
nment', at offset 448 (in bits) at rte_vhost.h:139:1
07:58:32                type size hasn't c=
hanged
07:58:32 =20
07:58:32  Error: ABI issue reported for abidif=
f --suppr dpdk/devtools/libabigail.abignore --no-added-syms --headers-dir1 =
reference/include --headers-dir2 build_install/include reference/dump/librt=
e_vhost.dump build_install/dump/librte_vhost.dump
07:58:32  ABIDIFF_ABI_CHANGE, this change requ=
ires a review (abidiff flagged this as a potential issue).

On Thu, Feb 23, 2023 at 11:57 AM Mike Pattrick <mkp@redhat.com> wrote:
On Thu, Feb 23, 2023 at 11:12 AM Maxime Coquelin<= br> <maxime.= coquelin@redhat.com> wrote:
>
> Hi Mike,
>
> Thanks for=C2=A0 looking into this issue.
>
> On 2/23/23 05:35, Mike Pattrick wrote:
> > The arguments passed to madvise should be aligned to the alignmen= t of
> > the backing memory. Now we keep track of each regions alignment a= nd use
> > then when setting coredump preferences. To facilitate this, a new= member
> > was added to rte_vhost_mem_region. A new function was added to ea= sily
> > translate memory address back to region alignment. Unneeded calls= to
> > madvise were reduced, as the cache removal case should already be=
> > covered by the cache insertion case. The previously inline functi= on
> > mem_set_dump was removed from a header file and made not inline.<= br> > >
> > Fixes: 338ad77c9ed3 ("vhost: exclude VM hugepages from cored= umps")
> >
> > Signed-off-by: Mike Pattrick <mkp@redhat.com>
> > ---
> > Since v1:
> >=C2=A0 =C2=A0- Corrected a cast for 32bit compiles
> > ---
> >=C2=A0 =C2=A0lib/vhost/iotlb.c=C2=A0 =C2=A0 =C2=A0 |=C2=A0 9 +++--= -
> >=C2=A0 =C2=A0lib/vhost/rte_vhost.h=C2=A0 |=C2=A0 1 +
> >=C2=A0 =C2=A0lib/vhost/vhost.h=C2=A0 =C2=A0 =C2=A0 | 12 ++------ > >=C2=A0 =C2=A0lib/vhost/vhost_user.c | 63 +++++++++++++++++++++++++= ++++++++++-------
> >=C2=A0 =C2=A04 files changed, 60 insertions(+), 25 deletions(-) > >
> > diff --git a/lib/vhost/iotlb.c b/lib/vhost/iotlb.c
> > index a0b8fd7302..5293507b63 100644
> > --- a/lib/vhost/iotlb.c
> > +++ b/lib/vhost/iotlb.c
> > @@ -149,7 +149,6 @@ vhost_user_iotlb_cache_remove_all(struct vhos= t_virtqueue *vq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0rte_rwlock_write_lock(&vq->iotlb= _lock);
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_TAILQ_FOREACH_SAFE(node, &vq-&g= t;iotlb_list, next, temp_node) {
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump((vo= id *)(uintptr_t)node->uaddr, node->size, true);
>
> Hmm, it should have been called with enable=3Dfalse here since we are<= br> > removing the entry from the IOTLB cache. It should be kept in order to=
> "DONTDUMP" pages evicted from the cache.

Here I was thinking that if we add an entry and then remove a
different entry, they could be in the same page. But on I should have
kept an enable=3Dfalse in remove_all().

And now that I think about it again, I could just check if there are
any active cache entries in the page on every evict/remove, they're
sorted so that should be an easy check. Unless there are any
objections I'll go forward with that.

>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0TAILQ_REMOV= E(&vq->iotlb_list, node, next);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vhost_user_= iotlb_pool_put(vq, node);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> > @@ -171,7 +170,6 @@ vhost_user_iotlb_cache_random_evict(struct vh= ost_virtqueue *vq)
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0RTE_TAILQ_FOREACH_SAFE(node, &vq-&g= t;iotlb_list, next, temp_node) {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (!entry_= idx) {
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)node->uaddr, node->size,= true);
>
> Same here.
>
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0TAILQ_REMOVE(&vq->iotlb_list, node, next);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vhost_user_iotlb_pool_put(vq, node);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vq->iotlb_cache_nr--;
> > @@ -224,14 +222,16 @@ vhost_user_iotlb_cache_insert(struct virtio= _net *dev, struct vhost_virtqueue *vq
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vhost_user_iotlb_pool_put(vq, new_node);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0goto unlock;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0} else if (= node->iova > new_node->iova) {
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)node->uaddr, node->size,= true);
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)new_node->uaddr, new_node-&= gt;size, true,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0hua_to_alignment(dev->mem, (voi= d *)(uintptr_t)node->uaddr));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0TAILQ_INSERT_BEFORE(node, new_node, next);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vq->iotlb_cache_nr++;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0goto unlock;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)node->uad= dr, node->size, true);
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)new_node->= ;uaddr, new_node->size, true,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0hua_to_alignment= (dev->mem, (void *)(uintptr_t)new_node->uaddr));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0TAILQ_INSERT_TAIL(&vq->iotlb_lis= t, new_node, next);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0vq->iotlb_cache_nr++;
> >
> > @@ -259,7 +259,6 @@ vhost_user_iotlb_cache_remove(struct vhost_vi= rtqueue *vq,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0break;
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (iova &l= t; node->iova + node->size) {
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0mem_set_dump((void *)(uintptr_t)node->uaddr, node->size,= true);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0TAILQ_REMOVE(&vq->iotlb_list, node, next);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vhost_user_iotlb_pool_put(vq, node);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0vq->iotlb_cache_nr--;
> > diff --git a/lib/vhost/rte_vhost.h b/lib/vhost/rte_vhost.h
> > index a395843fe9..c5c97ea67e 100644
> > --- a/lib/vhost/rte_vhost.h
> > +++ b/lib/vhost/rte_vhost.h
> > @@ -136,6 +136,7 @@ struct rte_vhost_mem_region {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0void=C2=A0 =C2=A0 =C2=A0*mmap_addr;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t mmap_size;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0int fd;
> > +=C2=A0 =C2=A0 =C2=A0uint64_t alignment;
>
> This is not possible to do this as it breaks the ABI.
> You have to store the information somewhere else, or simply call
> get_blk_size() in hua_to_alignment() since the fd is not closed.
>

Sorry about that! You're right, checking the fd per operation should be easy enough.

Thanks for the review,

M

> >=C2=A0 =C2=A0};
> >
> >=C2=A0 =C2=A0/**
> > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> > index 5750f0c005..a2467ba509 100644
> > --- a/lib/vhost/vhost.h
> > +++ b/lib/vhost/vhost.h
> > @@ -1009,14 +1009,6 @@ mbuf_is_consumed(struct rte_mbuf *m)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0return true;
> >=C2=A0 =C2=A0}
> >
> > -static __rte_always_inline void
> > -mem_set_dump(__rte_unused void *ptr, __rte_unused size_t size, _= _rte_unused bool enable)
> > -{
> > -#ifdef MADV_DONTDUMP
> > -=C2=A0 =C2=A0 =C2=A0if (madvise(ptr, size, enable ? MADV_DODUMP = : MADV_DONTDUMP) =3D=3D -1) {
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_log(RTE_LOG_= INFO, vhost_config_log_level,
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0"VHOST_CONFIG: could not set coredump preference (%s).\n&= quot;, strerror(errno));
> > -=C2=A0 =C2=A0 =C2=A0}
> > -#endif
> > -}
> > +uint64_t hua_to_alignment(struct rte_vhost_memory *mem, void *pt= r);
> > +void mem_set_dump(void *ptr, size_t size, bool enable, uint64_t = alignment);
> >=C2=A0 =C2=A0#endif /* _VHOST_NET_CDEV_H_ */
> > diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
> > index d702d082dd..6d09597fbe 100644
> > --- a/lib/vhost/vhost_user.c
> > +++ b/lib/vhost/vhost_user.c
> > @@ -737,6 +737,40 @@ log_addr_to_gpa(struct virtio_net *dev, stru= ct vhost_virtqueue *vq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0return log_gpa;
> >=C2=A0 =C2=A0}
> >
> > +uint64_t
> > +hua_to_alignment(struct rte_vhost_memory *mem, void *ptr)
> > +{
> > +=C2=A0 =C2=A0 =C2=A0struct rte_vhost_mem_region *r;
> > +=C2=A0 =C2=A0 =C2=A0uint32_t i;
> > +=C2=A0 =C2=A0 =C2=A0uintptr_t hua =3D (uintptr_t)ptr;
> > +
> > +=C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < mem->nregions; i++) = {
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r =3D &mem-&= gt;regions[i];
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (hua >=3D = r->host_user_addr &&
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0hua < r->host_user_addr + r->size) {
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0return r->alignment;
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> > +=C2=A0 =C2=A0 =C2=A0}
> > +
> > +=C2=A0 =C2=A0 =C2=A0/* If region isn't found, don't alig= n at all */
> > +=C2=A0 =C2=A0 =C2=A0return 1;
> > +}
> > +
> > +void
> > +mem_set_dump(void *ptr, size_t size, bool enable, uint64_t pages= z)
> > +{
> > +#ifdef MADV_DONTDUMP
> > +=C2=A0 =C2=A0 =C2=A0void *start =3D RTE_PTR_ALIGN_FLOOR(ptr, pag= esz);
> > +=C2=A0 =C2=A0 =C2=A0uintptr_t end =3D RTE_ALIGN_CEIL((uintptr_t)= ptr + size, pagesz);
> > +=C2=A0 =C2=A0 =C2=A0size_t len =3D end - (uintptr_t)start;
> > +
> > +=C2=A0 =C2=A0 =C2=A0if (madvise(start, len, enable ? MADV_DODUMP= : MADV_DONTDUMP) =3D=3D -1) {
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_log(RTE_LOG_= INFO, vhost_config_log_level,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0"VHOST_CONFIG: could not set coredump preference (%s).\n&= quot;, strerror(errno));
> > +=C2=A0 =C2=A0 =C2=A0}
> > +#endif
> > +}
> > +
> >=C2=A0 =C2=A0static void
> >=C2=A0 =C2=A0translate_ring_addresses(struct virtio_net **pdev, st= ruct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0{
> > @@ -767,6 +801,8 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >desc_packed, len, true,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0hua_to_alignment(dev->mem, vq->desc_packed));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0numa_reallo= c(&dev, &vq);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*pdev =3D d= ev;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*pvq =3D vq= ;
> > @@ -782,6 +818,8 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >driver_event, len, true,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0hua_to_alignment(dev->mem, vq->driver_event));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0len =3D siz= eof(struct vring_packed_desc_event);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vq->devi= ce_event =3D (struct vring_packed_desc_event *)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(u= intptr_t)ring_addr_to_vva(dev,
> > @@ -793,9 +831,8 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >desc_packed, len, true);
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >driver_event, len, true);
> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >device_event, len, true);
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0mem_set_dump(vq-= >device_event, len, true,
> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0hua_to_alignment(dev->mem, vq->device_event));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vq->acce= ss_ok =3D true;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> > @@ -812,6 +849,7 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->desc, len, true, hua_to_= alignment(dev->mem, vq->desc));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0numa_realloc(&dev, &vq);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0*pdev =3D dev;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0*pvq =3D vq;
> > @@ -827,6 +865,7 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->avail, len, true, hua_to= _alignment(dev->mem, vq->avail));
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0len =3D sizeof(struct vring_used) +
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sizeof(stru= ct vring_used_elem) * vq->size;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (dev->features & (1ULL <&l= t; VIRTIO_RING_F_EVENT_IDX))
> > @@ -839,6 +878,8 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->used, len, true, hua_to_= alignment(dev->mem, vq->used));
> > +
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (vq->last_used_idx !=3D vq->us= ed->idx) {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0VHOST_LOG_C= ONFIG(dev->ifname, WARNING,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0"last_used_idx (%u) and vq->used->idx (%u) mism= atches;\n",
> > @@ -849,9 +890,6 @@ translate_ring_addresses(struct virtio_net **= pdev, struct vhost_virtqueue **pvq)
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0"some packets maybe resent for Tx and dropped for Rx\= n");
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->desc, len, true);
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->avail, len, true);
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(vq->used, len, true);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0vq->access_ok =3D true;
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0VHOST_LOG_CONFIG(dev->ifname, DEBUG,= "mapped address desc: %p\n", vq->desc);
> > @@ -1230,7 +1268,8 @@ vhost_user_mmap_region(struct virtio_net *d= ev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0region->mmap_addr =3D mmap_addr;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0region->mmap_size =3D mmap_size;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0region->host_user_addr =3D (uint64_t= )(uintptr_t)mmap_addr + mmap_offset;
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(mmap_addr, mmap_size, false); > > +=C2=A0 =C2=A0 =C2=A0region->alignment =3D alignment;
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(mmap_addr, mmap_size, false, al= ignment);
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (dev->async_copy) {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (add_gue= st_pages(dev, region, alignment) < 0) {
> > @@ -1535,7 +1574,6 @@ inflight_mem_alloc(struct virtio_net *dev, = const char *name, size_t size, int *f
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return NULL= ;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(ptr, size, false);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0*fd =3D mfd;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0return ptr;
> >=C2=A0 =C2=A0}
> > @@ -1566,6 +1604,7 @@ vhost_user_get_inflight_fd(struct virtio_ne= t **pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t pervq_inflight_size, mmap_size= ;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0uint16_t num_queues, queue_size;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0struct virtio_net *dev =3D *pdev;
> > +=C2=A0 =C2=A0 =C2=A0uint64_t alignment;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0int fd, i, j;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0int numa_node =3D SOCKET_ID_ANY;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0void *addr;
> > @@ -1628,6 +1667,8 @@ vhost_user_get_inflight_fd(struct virtio_ne= t **pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inf= light_info->fd =3D -1;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > +=C2=A0 =C2=A0 =C2=A0alignment =3D get_blk_size(fd);
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(addr, mmap_size, false, alignme= nt);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->addr =3D addr= ;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->size =3D ctx-= >msg.payload.inflight.mmap_size =3D mmap_size;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->fd =3D ctx-&g= t;fds[0] =3D fd;
> > @@ -1744,10 +1785,10 @@ vhost_user_set_inflight_fd(struct virtio_= net **pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inf= light_info->fd =3D -1;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> >
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(addr, mmap_size, false);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->fd =3D fd; > >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->addr =3D addr= ;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->inflight_info->size =3D mmap= _size;
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(addr, mmap_size, false, get_blk= _size(fd));
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < num_queues; i++) {=
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vq =3D dev-= >virtqueue[i];
> > @@ -2242,6 +2283,7 @@ vhost_user_set_log_base(struct virtio_net *= *pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0struct virtio_net *dev =3D *pdev;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0int fd =3D ctx->fds[0];
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0uint64_t size, off;
> > +=C2=A0 =C2=A0 =C2=A0uint64_t alignment;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0void *addr;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0uint32_t i;
> >
> > @@ -2280,6 +2322,7 @@ vhost_user_set_log_base(struct virtio_net *= *pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 * fail when offset is not page size al= igned.
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 */
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0addr =3D mmap(0, size + off, PROT_READ = | PROT_WRITE, MAP_SHARED, fd, 0);
> > +=C2=A0 =C2=A0 =C2=A0alignment =3D get_blk_size(fd);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0close(fd);
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if (addr =3D=3D MAP_FAILED) {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0VHOST_LOG_C= ONFIG(dev->ifname, ERR, "mmap log base failed!\n");
> > @@ -2296,7 +2339,7 @@ vhost_user_set_log_base(struct virtio_net *= *pdev,
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->log_addr =3D (uint64_t)(uintptr= _t)addr;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->log_base =3D dev->log_addr += off;
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->log_size =3D size;
> > -=C2=A0 =C2=A0 =C2=A0mem_set_dump(addr, size, false);
> > +=C2=A0 =C2=A0 =C2=A0mem_set_dump(addr, size + off, false, alignm= ent);
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0for (i =3D 0; i < dev->nr_vring; = i++) {
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0struct vhos= t_virtqueue *vq =3D dev->virtqueue[i];
>



--

Patrick Rob= b

Technical Service Manager

UNH InterOpera= bility Laboratory

21 Madbury Rd, Suite 100, Durham, NH 0= 3824

www.iol.unh.edu


--000000000000b8098a05f57377cf--