[dpdk-dev] [PATCH] vhost-user: fix enabling of queue pair

[dpdk-dev] [PATCH] vhost-user: fix enabling of queue pair

[Victor Kaplansky]

The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in
queue-pair separately. So, a queue-pair should be considered
enabled only when both RX and TX queues are enabled.

The old code caused segfault when last TX queue was enabled.

Signed-off-by: Victor Kaplansky <victork@redhat.com>
---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c
index d07452a..c06e53e 100644
--- a/lib/librte_vhost/vhost_user/virtio-net-user.c
+++ b/lib/librte_vhost/vhost_user/virtio-net-user.c
@@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
+	uint16_t idx = state->index;
+	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
 	int enable = (int)state->num;
+	int qpair_enabled;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
 		"set queue enable: %d to qp idx: %d\n",
 		enable, state->index);
 
+	dev->virtqueue[idx]->enabled = enable;
+	qpair_enabled =
+		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
+		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
+
 	if (notify_ops->vring_state_changed) {
 		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+						qpair_enabled);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
-
 	return 0;
 }
 
-- 
--Victor

Re: [dpdk-dev] [PATCH] vhost-user: fix enabling of queue pair

[Victor Kaplansky]

On Fri, Nov 20, 2015 at 03:45:00PM +0200, Victor Kaplansky wrote:
> The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in
> queue-pair separately. So, a queue-pair should be considered
> enabled only when both RX and TX queues are enabled.
> 
> The old code caused segfault when last TX queue was enabled.

Since the problem causes segfault, it's important to review this
soon before QEMU 2.5 is out.

Thanks,
-- Victor

> 
> Signed-off-by: Victor Kaplansky <victork@redhat.com>
> ---
>  lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c
> index d07452a..c06e53e 100644
> --- a/lib/librte_vhost/vhost_user/virtio-net-user.c
> +++ b/lib/librte_vhost/vhost_user/virtio-net-user.c
> @@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
>  		      struct vhost_vring_state *state)
>  {
>  	struct virtio_net *dev = get_device(ctx);
> -	uint16_t base_idx = state->index;
> +	uint16_t idx = state->index;
> +	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
>  	int enable = (int)state->num;
> +	int qpair_enabled;
>  
>  	RTE_LOG(INFO, VHOST_CONFIG,
>  		"set queue enable: %d to qp idx: %d\n",
>  		enable, state->index);
>  
> +	dev->virtqueue[idx]->enabled = enable;
> +	qpair_enabled =
> +		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
> +		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
> +
>  	if (notify_ops->vring_state_changed) {
>  		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
> -						enable);
> +						qpair_enabled);
>  	}
>  
> -	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
> -	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
> -
>  	return 0;
>  }
>  
> -- 
> --Victor

Re: [dpdk-dev] [PATCH] vhost-user: fix enabling of queue pair

[Yuanhan Liu]

On Fri, Nov 20, 2015 at 03:45:00PM +0200, Victor Kaplansky wrote:
> The VHOST_USER_SET_VRING_ENABLE request is sent for each queue in
> queue-pair separately. So, a queue-pair should be considered
> enabled only when both RX and TX queues are enabled.

Hi Victor,

It was sent per queue-pair, and I found that Michael changed it to
per vring. The reason I made it per queue-pair is that the backend
can handle both queues in one queue-pair. However, I agree that
making it per queue (vring) is better here, to keep the consistency
of sending all requests (say vring_call) per vring.

> 
> The old code caused segfault when last TX queue was enabled.
> 
> Signed-off-by: Victor Kaplansky <victork@redhat.com>
> ---
>  lib/librte_vhost/vhost_user/virtio-net-user.c | 14 +++++++++-----
>  1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c
> index d07452a..c06e53e 100644
> --- a/lib/librte_vhost/vhost_user/virtio-net-user.c
> +++ b/lib/librte_vhost/vhost_user/virtio-net-user.c
> @@ -317,21 +317,25 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
>  		      struct vhost_vring_state *state)
>  {
>  	struct virtio_net *dev = get_device(ctx);
> -	uint16_t base_idx = state->index;
> +	uint16_t idx = state->index;
> +	uint16_t base_idx = (idx / VIRTIO_QNUM) * VIRTIO_QNUM;
>  	int enable = (int)state->num;
> +	int qpair_enabled;
>  
>  	RTE_LOG(INFO, VHOST_CONFIG,
>  		"set queue enable: %d to qp idx: %d\n",
>  		enable, state->index);
>  
> +	dev->virtqueue[idx]->enabled = enable;
> +	qpair_enabled =
> +		dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled &&
> +		dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled;
> +
>  	if (notify_ops->vring_state_changed) {
>  		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
> -						enable);
> +						qpair_enabled);

Here we could make the vring_state_changed callback be per vring then,
instead of per queue-pair. That make the code simpler, as well as more
consistent.

Thanks.

	--yliu
>  	}
>  
> -	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
> -	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
> -
>  	return 0;
>  }
>  
> -- 
> --Victor

[dpdk-dev] [PATCH v2] vhost-user: fix enabling of queue pair

[Yuanhan Liu]

From: Victor Kaplansky <victork@redhat.com>

The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair.
However, it's changed to be sent per queue in the queue-pair by QEMU
commit dc3db6ad ("vhost-user: start/stop all rings"). The change
is reasonable, as we send all other request per queue, instead of
queue-pair.

Hence we should do proper changes to adapt to the QEMU change here.
Otherwise, a segfault will be triggered when last TX queue was enabled.

[ commit log reword --  Yuanhan Liu ]

Signed-off-by: Victor Kaplansky <victork@redhat.com>
Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
---

v2: invoke vring_state_changed() callback per-vring as well

This fixes a fatal issue, without that MQ will not work properly.
Seems that Victor is busy at something else (or out of office so
far), hence I sent v2 on behalf of him.

---
 lib/librte_vhost/vhost_user/virtio-net-user.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c
index d07452a..b2ada9f 100644
--- a/lib/librte_vhost/vhost_user/virtio-net-user.c
+++ b/lib/librte_vhost/vhost_user/virtio-net-user.c
@@ -317,7 +317,6 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		      struct vhost_vring_state *state)
 {
 	struct virtio_net *dev = get_device(ctx);
-	uint16_t base_idx = state->index;
 	int enable = (int)state->num;
 
 	RTE_LOG(INFO, VHOST_CONFIG,
@@ -325,12 +324,10 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
 		enable, state->index);
 
 	if (notify_ops->vring_state_changed) {
-		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
-						enable);
+		notify_ops->vring_state_changed(dev, state->index, enable);
 	}
 
-	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
-	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
+	dev->virtqueue[state->index]->enabled = enable;
 
 	return 0;
 }
-- 
1.9.3

Re: [dpdk-dev] [PATCH v2] vhost-user: fix enabling of queue pair

[Victor Kaplansky]

Yuanhan, looks fine. Thanks for handling this.
-- Victor

On Tue, Nov 24, 2015 at 03:25:35PM +0800, Yuanhan Liu wrote:
> From: Victor Kaplansky <victork@redhat.com>
> 
> The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair.
> However, it's changed to be sent per queue in the queue-pair by QEMU
> commit dc3db6ad ("vhost-user: start/stop all rings"). The change
> is reasonable, as we send all other request per queue, instead of
> queue-pair.
> 
> Hence we should do proper changes to adapt to the QEMU change here.
> Otherwise, a segfault will be triggered when last TX queue was enabled.
> 
> [ commit log reword --  Yuanhan Liu ]
> 
> Signed-off-by: Victor Kaplansky <victork@redhat.com>
> Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> ---
> 
> v2: invoke vring_state_changed() callback per-vring as well
> 
> This fixes a fatal issue, without that MQ will not work properly.
> Seems that Victor is busy at something else (or out of office so
> far), hence I sent v2 on behalf of him.
> 
> ---
>  lib/librte_vhost/vhost_user/virtio-net-user.c | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user/virtio-net-user.c b/lib/librte_vhost/vhost_user/virtio-net-user.c
> index d07452a..b2ada9f 100644
> --- a/lib/librte_vhost/vhost_user/virtio-net-user.c
> +++ b/lib/librte_vhost/vhost_user/virtio-net-user.c
> @@ -317,7 +317,6 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
>  		      struct vhost_vring_state *state)
>  {
>  	struct virtio_net *dev = get_device(ctx);
> -	uint16_t base_idx = state->index;
>  	int enable = (int)state->num;
>  
>  	RTE_LOG(INFO, VHOST_CONFIG,
> @@ -325,12 +324,10 @@ user_set_vring_enable(struct vhost_device_ctx ctx,
>  		enable, state->index);
>  
>  	if (notify_ops->vring_state_changed) {
> -		notify_ops->vring_state_changed(dev, base_idx / VIRTIO_QNUM,
> -						enable);
> +		notify_ops->vring_state_changed(dev, state->index, enable);
>  	}
>  
> -	dev->virtqueue[base_idx + VIRTIO_RXQ]->enabled = enable;
> -	dev->virtqueue[base_idx + VIRTIO_TXQ]->enabled = enable;
> +	dev->virtqueue[state->index]->enabled = enable;
>  
>  	return 0;
>  }
> -- 
> 1.9.3

Re: [dpdk-dev] [PATCH v2] vhost-user: fix enabling of queue pair

[Thomas Monjalon]

2015-11-24 09:43, Victor Kaplansky:
> Yuanhan, looks fine. Thanks for handling this.
> -- Victor
> 
> On Tue, Nov 24, 2015 at 03:25:35PM +0800, Yuanhan Liu wrote:
> > From: Victor Kaplansky <victork@redhat.com>
> > 
> > The VHOST_USER_SET_VRING_ENABLE request was sent for each queue-pair.
> > However, it's changed to be sent per queue in the queue-pair by QEMU
> > commit dc3db6ad ("vhost-user: start/stop all rings"). The change
> > is reasonable, as we send all other request per queue, instead of
> > queue-pair.
> > 
> > Hence we should do proper changes to adapt to the QEMU change here.
> > Otherwise, a segfault will be triggered when last TX queue was enabled.
> > 
> > [ commit log reword --  Yuanhan Liu ]
> > 
> > Signed-off-by: Victor Kaplansky <victork@redhat.com>
> > Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> > ---
> > 
> > v2: invoke vring_state_changed() callback per-vring as well
> > 
> > This fixes a fatal issue, without that MQ will not work properly.
> > Seems that Victor is busy at something else (or out of office so
> > far), hence I sent v2 on behalf of him.

Applied, thanks