* [PATCH 1/4] net/virtio: rename Virtio-user queue iterator
2024-03-27 9:40 [PATCH 0/4] Virtio-user queues setup fixes Maxime Coquelin
@ 2024-03-27 9:40 ` Maxime Coquelin
2024-03-27 9:40 ` [PATCH 2/4] net/virtio: use iterator to destroy Virtio-user queues Maxime Coquelin
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2024-03-27 9:40 UTC (permalink / raw)
To: dev, david.marchand, chenbox; +Cc: Maxime Coquelin
This is a preliminary rework to prepare for iterating
over queues for non-setup operations.
Also, remove the error log that does not provide much
information given the callbacks already provide one.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 4fdfe70e7c..c3d44880f5 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -129,7 +129,7 @@ virtio_user_kick_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
}
static int
-virtio_user_queue_setup(struct virtio_user_dev *dev,
+virtio_user_foreach_queue(struct virtio_user_dev *dev,
int (*fn)(struct virtio_user_dev *, uint32_t))
{
uint32_t i, nr_vq;
@@ -138,12 +138,9 @@ virtio_user_queue_setup(struct virtio_user_dev *dev,
if (dev->hw_cvq)
nr_vq++;
- for (i = 0; i < nr_vq; i++) {
- if (fn(dev, i) < 0) {
- PMD_DRV_LOG(ERR, "(%s) setup VQ %u failed", dev->path, i);
+ for (i = 0; i < nr_vq; i++)
+ if (fn(dev, i) < 0)
return -1;
- }
- }
return 0;
}
@@ -157,7 +154,7 @@ virtio_user_dev_set_features(struct virtio_user_dev *dev)
pthread_mutex_lock(&dev->mutex);
/* Step 0: tell vhost to create queues */
- if (virtio_user_queue_setup(dev, virtio_user_create_queue) < 0)
+ if (virtio_user_foreach_queue(dev, virtio_user_create_queue) < 0)
goto error;
features = dev->features;
@@ -205,7 +202,7 @@ virtio_user_start_device(struct virtio_user_dev *dev)
goto error;
/* Step 3: kick queues */
- ret = virtio_user_queue_setup(dev, virtio_user_kick_queue);
+ ret = virtio_user_foreach_queue(dev, virtio_user_kick_queue);
if (ret < 0)
goto error;
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 2/4] net/virtio: use iterator to destroy Virtio-user queues
2024-03-27 9:40 [PATCH 0/4] Virtio-user queues setup fixes Maxime Coquelin
2024-03-27 9:40 ` [PATCH 1/4] net/virtio: rename Virtio-user queue iterator Maxime Coquelin
@ 2024-03-27 9:40 ` Maxime Coquelin
2024-03-27 9:40 ` [PATCH 3/4] net/virtio: fix shadow control queue notification init Maxime Coquelin
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2024-03-27 9:40 UTC (permalink / raw)
To: dev, david.marchand, chenbox; +Cc: Maxime Coquelin, stable
This patch uses the freshly renamed iterator to destroy
queues at stop time. Doing this, we fix the missing
control queue destroy.
Fixes: 90966e8e5b67 ("net/virtio-user: send shadow virtqueue info to the backend")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
.../net/virtio/virtio_user/virtio_user_dev.c | 27 ++++++++++++-------
1 file changed, 18 insertions(+), 9 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index c3d44880f5..0776c54deb 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -33,6 +33,22 @@ const char * const virtio_user_backend_strings[] = {
[VIRTIO_USER_BACKEND_VHOST_VDPA] = "VHOST_VDPA",
};
+static int
+virtio_user_destroy_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
+{
+ struct vhost_vring_state state;
+ int ret;
+
+ state.index = queue_sel;
+ ret = dev->ops->get_vring_base(dev, &state);
+ if (ret < 0) {
+ PMD_DRV_LOG(ERR, "(%s) Failed to destroy queue %u", dev->path, queue_sel);
+ return -1;
+ }
+
+ return 0;
+}
+
static int
virtio_user_create_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
{
@@ -237,7 +253,6 @@ virtio_user_start_device(struct virtio_user_dev *dev)
int virtio_user_stop_device(struct virtio_user_dev *dev)
{
- struct vhost_vring_state state;
uint32_t i;
int ret;
@@ -258,14 +273,8 @@ int virtio_user_stop_device(struct virtio_user_dev *dev)
}
/* Stop the backend. */
- for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
- state.index = i;
- ret = dev->ops->get_vring_base(dev, &state);
- if (ret < 0) {
- PMD_DRV_LOG(ERR, "(%s) get_vring_base failed, index=%u", dev->path, i);
- goto err;
- }
- }
+ if (virtio_user_foreach_queue(dev, virtio_user_destroy_queue) < 0)
+ goto err;
dev->started = false;
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 3/4] net/virtio: fix shadow control queue notification init
2024-03-27 9:40 [PATCH 0/4] Virtio-user queues setup fixes Maxime Coquelin
2024-03-27 9:40 ` [PATCH 1/4] net/virtio: rename Virtio-user queue iterator Maxime Coquelin
2024-03-27 9:40 ` [PATCH 2/4] net/virtio: use iterator to destroy Virtio-user queues Maxime Coquelin
@ 2024-03-27 9:40 ` Maxime Coquelin
2024-03-27 9:40 ` [PATCH 4/4] net/virtio: fix shadow control queue allocation Maxime Coquelin
2024-03-27 10:04 ` [PATCH 0/4] Virtio-user queues setup fixes David Marchand
4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2024-03-27 9:40 UTC (permalink / raw)
To: dev, david.marchand, chenbox; +Cc: Maxime Coquelin, stable
The Virtio-user control queue kick and call FDs were not
uninitialized at device stop time.
This patch fixes this using the queues iterator helper for
both initialization and uninitialization.
Fixes: 90966e8e5b67 ("net/virtio-user: send shadow virtqueue info to the backend")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
.../net/virtio/virtio_user/virtio_user_dev.c | 90 +++++++++----------
1 file changed, 43 insertions(+), 47 deletions(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 0776c54deb..912e87fecf 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -33,6 +33,45 @@ const char * const virtio_user_backend_strings[] = {
[VIRTIO_USER_BACKEND_VHOST_VDPA] = "VHOST_VDPA",
};
+static int
+virtio_user_uninit_notify_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
+{
+ if (dev->kickfds[queue_sel] >= 0) {
+ close(dev->kickfds[queue_sel]);
+ dev->kickfds[queue_sel] = -1;
+ }
+
+ if (dev->callfds[queue_sel] >= 0) {
+ close(dev->callfds[queue_sel]);
+ dev->callfds[queue_sel] = -1;
+ }
+
+ return 0;
+}
+
+static int
+virtio_user_init_notify_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
+{
+ /* May use invalid flag, but some backend uses kickfd and
+ * callfd as criteria to judge if dev is alive. so finally we
+ * use real event_fd.
+ */
+ dev->callfds[queue_sel] = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
+ if (dev->callfds[queue_sel] < 0) {
+ PMD_DRV_LOG(ERR, "(%s) Failed to setup callfd for queue %u: %s",
+ dev->path, queue_sel, strerror(errno));
+ return -1;
+ }
+ dev->kickfds[queue_sel] = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
+ if (dev->kickfds[queue_sel] < 0) {
+ PMD_DRV_LOG(ERR, "(%s) Failed to setup kickfd for queue %u: %s",
+ dev->path, queue_sel, strerror(errno));
+ return -1;
+ }
+
+ return 0;
+}
+
static int
virtio_user_destroy_queue(struct virtio_user_dev *dev, uint32_t queue_sel)
{
@@ -423,33 +462,9 @@ virtio_user_dev_init_mac(struct virtio_user_dev *dev, const char *mac)
static int
virtio_user_dev_init_notify(struct virtio_user_dev *dev)
{
- uint32_t i, j, nr_vq;
- int callfd;
- int kickfd;
-
- nr_vq = dev->max_queue_pairs * 2;
- if (dev->hw_cvq)
- nr_vq++;
- for (i = 0; i < nr_vq; i++) {
- /* May use invalid flag, but some backend uses kickfd and
- * callfd as criteria to judge if dev is alive. so finally we
- * use real event_fd.
- */
- callfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
- if (callfd < 0) {
- PMD_DRV_LOG(ERR, "(%s) callfd error, %s", dev->path, strerror(errno));
- goto err;
- }
- kickfd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK);
- if (kickfd < 0) {
- close(callfd);
- PMD_DRV_LOG(ERR, "(%s) kickfd error, %s", dev->path, strerror(errno));
- goto err;
- }
- dev->callfds[i] = callfd;
- dev->kickfds[i] = kickfd;
- }
+ if (virtio_user_foreach_queue(dev, virtio_user_init_notify_queue) < 0)
+ goto err;
if (dev->device_features & (1ULL << VIRTIO_F_NOTIFICATION_DATA))
if (dev->ops->map_notification_area &&
@@ -458,16 +473,7 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
return 0;
err:
- for (j = 0; j < i; j++) {
- if (dev->kickfds[j] >= 0) {
- close(dev->kickfds[j]);
- dev->kickfds[j] = -1;
- }
- if (dev->callfds[j] >= 0) {
- close(dev->callfds[j]);
- dev->callfds[j] = -1;
- }
- }
+ virtio_user_foreach_queue(dev, virtio_user_uninit_notify_queue);
return -1;
}
@@ -475,18 +481,8 @@ virtio_user_dev_init_notify(struct virtio_user_dev *dev)
static void
virtio_user_dev_uninit_notify(struct virtio_user_dev *dev)
{
- uint32_t i;
+ virtio_user_foreach_queue(dev, virtio_user_uninit_notify_queue);
- for (i = 0; i < dev->max_queue_pairs * 2; ++i) {
- if (dev->kickfds[i] >= 0) {
- close(dev->kickfds[i]);
- dev->kickfds[i] = -1;
- }
- if (dev->callfds[i] >= 0) {
- close(dev->callfds[i]);
- dev->callfds[i] = -1;
- }
- }
if (dev->ops->unmap_notification_area && dev->notify_area)
dev->ops->unmap_notification_area(dev);
}
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 4/4] net/virtio: fix shadow control queue allocation
2024-03-27 9:40 [PATCH 0/4] Virtio-user queues setup fixes Maxime Coquelin
` (2 preceding siblings ...)
2024-03-27 9:40 ` [PATCH 3/4] net/virtio: fix shadow control queue notification init Maxime Coquelin
@ 2024-03-27 9:40 ` Maxime Coquelin
2024-03-27 10:04 ` [PATCH 0/4] Virtio-user queues setup fixes David Marchand
4 siblings, 0 replies; 6+ messages in thread
From: Maxime Coquelin @ 2024-03-27 9:40 UTC (permalink / raw)
To: dev, david.marchand, chenbox; +Cc: Maxime Coquelin, stable
It is possible to have the control queue without the vDPA
device advertising VIRTIO_NET_F_MQ.
Let's just rely on hw_cvq flag as in other places.
Fixes: 6fdf32d1e318 ("net/virtio-user: remove max queues limitation")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
drivers/net/virtio/virtio_user/virtio_user_dev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 912e87fecf..5240b44481 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -615,7 +615,7 @@ virtio_user_alloc_vrings(struct virtio_user_dev *dev)
bool packed_ring = !!(dev->device_features & (1ull << VIRTIO_F_RING_PACKED));
nr_vrings = dev->max_queue_pairs * 2;
- if (dev->device_features & (1ull << VIRTIO_NET_F_MQ))
+ if (dev->hw_cvq)
nr_vrings++;
dev->callfds = rte_zmalloc("virtio_user_dev", nr_vrings * sizeof(*dev->callfds), 0);
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0/4] Virtio-user queues setup fixes
2024-03-27 9:40 [PATCH 0/4] Virtio-user queues setup fixes Maxime Coquelin
` (3 preceding siblings ...)
2024-03-27 9:40 ` [PATCH 4/4] net/virtio: fix shadow control queue allocation Maxime Coquelin
@ 2024-03-27 10:04 ` David Marchand
4 siblings, 0 replies; 6+ messages in thread
From: David Marchand @ 2024-03-27 10:04 UTC (permalink / raw)
To: Maxime Coquelin; +Cc: dev, chenbox
Hello Maxime,
On Wed, Mar 27, 2024 at 10:40 AM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> This series aims at fixing several issues found in
> Virtio-user PMD related to queues setup and cleanup.
>
> It has been tested with Vhost-vDPA backend using Nvidia
> Cx6-Dx vDPA VF.
>
> First patch in the series renames the queues iterator
> helper, so it is not a fix. But I would suggest to have it
> backported to ease backporting of the fixes.
>
> Maxime Coquelin (4):
> net/virtio: rename Virtio-user queue iterator
> net/virtio: use iterator to destroy Virtio-user queues
> net/virtio: fix shadow control queue notification init
> net/virtio: fix shadow control queue allocation
>
> .../net/virtio/virtio_user/virtio_user_dev.c | 132 +++++++++---------
> 1 file changed, 67 insertions(+), 65 deletions(-)
I would prefix those titles with net/virtio-user: as was done in the past.
This will show the changes only affect virtio-user.
For patch 2, it is better to highlight the functional change/fix in
the title, so something like:
net/virtio-user: fix control queue destruction
With this fixed, you can add on the series:
Acked-by: David Marchand <david.marchand@redhat.com>
--
David Marchand
^ permalink raw reply [flat|nested] 6+ messages in thread