DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation
@ 2021-06-18 13:57 Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc Maxime Coquelin
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 13:57 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia; +Cc: Maxime Coquelin

This patch series first fixes missing reallocations of some
Virtqueue and device metadata.

Then, it improves the numa_realloc function by using
rte_realloc_socket API that takes care of the memcpy &
freeing. The VQs NUMA IDs are also saved in the VQ metadata
and used for every allocations so that all allocations
before NUMA realloc are on the same VQ, later ones are
allocated on the proper one.

Finally inflight feature metada are converted from calloc()
to rte_zmalloc_socket() and their reallocation is handled
in numa_realloc().

Changes in v5:
==============
- Do not reallocate if VS is ready (Chenbo)
- Fix typos & cosmetics (Chenbo)
- Improve numa_realloc() comment (Chenbo)

Changes in v4:
==============
- Check Vhose device numa node to avoid rte_realloc_socket
  to realloc even if already right node/size/align.

Changes in v3:
==============
- Fix copy/paste issues (David)
- Ad new patch to fix multiqueue reallocation

Changes in v2:
==============
- Add missing NUMA realloc in patch 6

Maxime Coquelin (4):
  vhost: fix missing memory table NUMA realloc
  vhost: fix missing guest pages table NUMA realloc
  vhost: fix missing cache logging NUMA realloc
  vhost: fix NUMA reallocation with multiqueue

 lib/vhost/vhost_user.c | 54 ++++++++++++++++++++++++++++++++++++++----
 1 file changed, 49 insertions(+), 5 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc
  2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
@ 2021-06-18 13:57 ` Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 2/4] vhost: fix missing guest pages " Maxime Coquelin
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 13:57 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia; +Cc: Maxime Coquelin, stable

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the Vhost memory table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling")
Cc: stable@dpdk.org

Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 lib/vhost/vhost_user.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
index 8f0eba6412..b5a84f3dcd 100644
--- a/lib/vhost/vhost_user.c
+++ b/lib/vhost/vhost_user.c
@@ -473,8 +473,8 @@ vhost_user_set_vring_num(struct virtio_net **pdev,
 }
 
 /*
- * Reallocate virtio_dev and vhost_virtqueue data structure to make them on the
- * same numa node as the memory of vring descriptor.
+ * Reallocate virtio_dev, vhost_virtqueue and related data structures to
+ * make them on the same numa node as the memory of vring descriptor.
  */
 #ifdef RTE_LIBRTE_VHOST_NUMA
 static struct virtio_net*
@@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index)
 		goto out;
 	}
 	if (oldnode != newnode) {
+		struct rte_vhost_memory *old_mem;
+		ssize_t mem_size;
+
 		VHOST_LOG_CONFIG(INFO,
 			"reallocate dev from %d to %d node\n",
 			oldnode, newnode);
@@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index)
 
 		memcpy(dev, old_dev, sizeof(*dev));
 		rte_free(old_dev);
+
+		mem_size = sizeof(struct rte_vhost_memory) +
+			sizeof(struct rte_vhost_mem_region) * dev->mem->nregions;
+		old_mem = dev->mem;
+		dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode);
+		if (!dev->mem) {
+			dev->mem = old_mem;
+			goto out;
+		}
+
+		memcpy(dev->mem, old_mem, mem_size);
+		rte_free(old_mem);
 	}
 
 out:
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v5 2/4] vhost: fix missing guest pages table NUMA realloc
  2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc Maxime Coquelin
@ 2021-06-18 13:57 ` Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 3/4] vhost: fix missing cache logging " Maxime Coquelin
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 13:57 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia; +Cc: Maxime Coquelin, stable

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the guest pages table was missing, which
likely causes at least one cross-NUMA accesses for every burst
of packets.

This patch reallocates this table on the same NUMA node as the
other metadata.

Fixes: e246896178e6 ("vhost: get guest/host physical address mappings")
Cc: stable@dpdk.org

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 lib/vhost/vhost_user.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
index b5a84f3dcd..5fb055ea2e 100644
--- a/lib/vhost/vhost_user.c
+++ b/lib/vhost/vhost_user.c
@@ -558,7 +558,8 @@ numa_realloc(struct virtio_net *dev, int index)
 	}
 	if (oldnode != newnode) {
 		struct rte_vhost_memory *old_mem;
-		ssize_t mem_size;
+		struct guest_page *old_gp;
+		ssize_t mem_size, gp_size;
 
 		VHOST_LOG_CONFIG(INFO,
 			"reallocate dev from %d to %d node\n",
@@ -583,6 +584,17 @@ numa_realloc(struct virtio_net *dev, int index)
 
 		memcpy(dev->mem, old_mem, mem_size);
 		rte_free(old_mem);
+
+		gp_size = dev->max_guest_pages * sizeof(*dev->guest_pages);
+		old_gp = dev->guest_pages;
+		dev->guest_pages = rte_malloc_socket(NULL, gp_size, RTE_CACHE_LINE_SIZE, newnode);
+		if (!dev->guest_pages) {
+			dev->guest_pages = old_gp;
+			goto out;
+		}
+
+		memcpy(dev->guest_pages, old_gp, gp_size);
+		rte_free(old_gp);
 	}
 
 out:
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v5 3/4] vhost: fix missing cache logging NUMA realloc
  2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 2/4] vhost: fix missing guest pages " Maxime Coquelin
@ 2021-06-18 13:57 ` Maxime Coquelin
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 4/4] vhost: fix NUMA reallocation with multiqueue Maxime Coquelin
  2021-06-18 14:04 ` [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
  4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 13:57 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia; +Cc: Maxime Coquelin

When the guest allocates virtqueues on a different NUMA node
than the one the Vhost metadata are allocated, both the Vhost
device struct and the virtqueues struct are reallocated.

However, reallocating the log cache on the new NUMA node was
not done. This patch fixes this by reallocating it if it has
been allocated already, which means a live-migration is
on-going.

Fixes: 1818a63147fb ("vhost: move dirty logging cache out of virtqueue")

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 lib/vhost/vhost_user.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
index 5fb055ea2e..82adf80fe5 100644
--- a/lib/vhost/vhost_user.c
+++ b/lib/vhost/vhost_user.c
@@ -545,6 +545,16 @@ numa_realloc(struct virtio_net *dev, int index)
 			vq->batch_copy_elems = new_batch_copy_elems;
 		}
 
+		if (vq->log_cache) {
+			struct log_cache_entry *log_cache;
+
+			log_cache = rte_realloc_socket(vq->log_cache,
+					sizeof(struct log_cache_entry) * VHOST_LOG_CACHE_NR,
+					0, newnode);
+			if (log_cache)
+				vq->log_cache = log_cache;
+		}
+
 		rte_free(old_vq);
 	}
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-dev] [PATCH v5 4/4] vhost: fix NUMA reallocation with multiqueue
  2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
                   ` (2 preceding siblings ...)
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 3/4] vhost: fix missing cache logging " Maxime Coquelin
@ 2021-06-18 13:57 ` Maxime Coquelin
  2021-06-18 14:04 ` [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
  4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 13:57 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia; +Cc: Maxime Coquelin, stable

Since the Vhost-user device initialization has been reworked,
enabling the application to start using the device as soon as
the first queue pair is ready, NUMA reallocation no more
happened on queue pairs other than the first one since
numa_realloc() was returning early if the device was running.

This patch fixes this issue by only preventing the device
metadata to be allocated if the device is running. For the
virtqueues, a vring state change notification is sent to
notify the application of its disablement. Since the callback
is supposed to be blocking, it is safe to reallocate it
afterwards.

Fixes: d0fcc38f5fa4 ("vhost: improve device readiness notifications")
Cc: stable@dpdk.org

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 lib/vhost/vhost_user.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c
index 82adf80fe5..51b96a0716 100644
--- a/lib/vhost/vhost_user.c
+++ b/lib/vhost/vhost_user.c
@@ -488,12 +488,16 @@ numa_realloc(struct virtio_net *dev, int index)
 	struct batch_copy_elem *new_batch_copy_elems;
 	int ret;
 
-	if (dev->flags & VIRTIO_DEV_RUNNING)
-		return dev;
-
 	old_dev = dev;
 	vq = old_vq = dev->virtqueue[index];
 
+	/*
+	 * If VQ is ready, it is too late to reallocate, it certainly already
+	 * happened anyway on VHOST_USER_SET_VRING_ADRR.
+	 */
+	if (vq->ready)
+		return dev;
+
 	ret = get_mempolicy(&newnode, NULL, 0, old_vq->desc,
 			    MPOL_F_NODE | MPOL_F_ADDR);
 
@@ -558,6 +562,9 @@ numa_realloc(struct virtio_net *dev, int index)
 		rte_free(old_vq);
 	}
 
+	if (dev->flags & VIRTIO_DEV_RUNNING)
+		goto out;
+
 	/* check if we need to reallocate dev */
 	ret = get_mempolicy(&oldnode, NULL, 0, old_dev,
 			    MPOL_F_NODE | MPOL_F_ADDR);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation
  2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
                   ` (3 preceding siblings ...)
  2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 4/4] vhost: fix NUMA reallocation with multiqueue Maxime Coquelin
@ 2021-06-18 14:04 ` Maxime Coquelin
  4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2021-06-18 14:04 UTC (permalink / raw)
  To: dev, david.marchand, chenbo.xia



On 6/18/21 3:57 PM, Maxime Coquelin wrote:
> This patch series first fixes missing reallocations of some
> Virtqueue and device metadata.
> 
> Then, it improves the numa_realloc function by using
> rte_realloc_socket API that takes care of the memcpy &
> freeing. The VQs NUMA IDs are also saved in the VQ metadata
> and used for every allocations so that all allocations
> before NUMA realloc are on the same VQ, later ones are
> allocated on the proper one.
> 
> Finally inflight feature metada are converted from calloc()
> to rte_zmalloc_socket() and their reallocation is handled
> in numa_realloc().
> 
> Changes in v5:
> ==============
> - Do not reallocate if VS is ready (Chenbo)
> - Fix typos & cosmetics (Chenbo)
> - Improve numa_realloc() comment (Chenbo)
> 
> Changes in v4:
> ==============
> - Check Vhose device numa node to avoid rte_realloc_socket
>   to realloc even if already right node/size/align.
> 
> Changes in v3:
> ==============
> - Fix copy/paste issues (David)
> - Ad new patch to fix multiqueue reallocation
> 
> Changes in v2:
> ==============
> - Add missing NUMA realloc in patch 6
> 
> Maxime Coquelin (4):
>   vhost: fix missing memory table NUMA realloc
>   vhost: fix missing guest pages table NUMA realloc
>   vhost: fix missing cache logging NUMA realloc
>   vhost: fix NUMA reallocation with multiqueue
> 
>  lib/vhost/vhost_user.c | 54 ++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 49 insertions(+), 5 deletions(-)
> 

Sorry, please discard this revision, 3 patches are missing.
V6 sent.

Maxime


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-06-18 14:05 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-18 13:57 [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin
2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 1/4] vhost: fix missing memory table NUMA realloc Maxime Coquelin
2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 2/4] vhost: fix missing guest pages " Maxime Coquelin
2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 3/4] vhost: fix missing cache logging " Maxime Coquelin
2021-06-18 13:57 ` [dpdk-dev] [PATCH v5 4/4] vhost: fix NUMA reallocation with multiqueue Maxime Coquelin
2021-06-18 14:04 ` [dpdk-dev] [PATCH v5 0/4] vhost: Fix and improve NUMA reallocation Maxime Coquelin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).