[PATCH 1/1] vhost: support VIRTIO_F_RING_RESET for vhost-user

[PATCH 1/1] vhost: support VIRTIO_F_RING_RESET for vhost-user

[Kangjie Xu]

Add VIRTIO_F_RING_RESET, which indicates that the driver can reset a
queue individually.

VIRTIO_F_RING_RESET feature is added to virtio-spec 1.2. The relevant
information is in
    https://github.com/oasis-tcs/virtio-spec/issues/124
    https://github.com/oasis-tcs/virtio-spec/issues/139

The implementation only adds the feature bit in supported features. It
does not require any other changes because we reuse the existing vhost
protocol.

The virtqueue reset process can be concluded as two parts:
1. The driver can reset a virtqueue. When it is triggered,
VHOST_USER_SET_VRING_ENABLE message with "payload.state.num" set to 0
is sent to DPDK. Then the virtqueue will be disabled in DPDK.
2. After the virtqueue is disabled, the driver may optionally re-enable
it. To avoid confusion with VHOST_USER_SET_VRING_ENABLE, we call this
part as "restart". The virtqueue's information may be changed when
restarting it. Thus, the information of the reset virtqueue should be
updated. This part is basically similar to when the virtqueue is started
for the first time, except that the restart process does not need to set
features and set mem table since they does not change. QEMU will send
messages containing size, base, addr, kickfd and callfd of the virtqueue
in order. Specifically, the DPDK will receive these messages in order:
    a. VHOST_USER_SET_VRING_NUM
    b. VHOST_USER_SET_VRING_BASE
    c. VHOST_USER_SET_VRING_ADDR
    d. VHOST_USER_SET_VRING_KICK
    e. VHOST_USER_SET_VRING_CALL
    f. VHOST_USER_SET_VRING_ENABLE
The last VHOST_USER_SET_VRING_ENABLE message with "payload.state.num" set
to 1, will be sent to enable the virtqueue and the restart process is
finished.

Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
---
Test environment:
    Host: 5.4.189
    Qemu: QEMU emulator version 7.0.50 (With vq reset support)
    Guest: 5.19.0-rc3 (With vq reset support)
    DPDK: 22.07-rc1
    Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2;
            ethtool -g eth1;

    The driver can resize the virtio queue, then virtio queue reset
    function should be triggered.

Guest Kernel Patch:
    https://lore.kernel.org/bpf/20220629065656.54420-1-xuanzhuo@linux.alibaba.com

QEMU Patch:
    https://lore.kernel.org/qemu-devel/cover.1658141552.git.kangjie.xu@linux.alibaba.com

 lib/vhost/vhost.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
index 14235aaf81..9711ded7dd 100644
--- a/lib/vhost/vhost.h
+++ b/lib/vhost/vhost.h
@@ -378,6 +378,11 @@ struct vhost_msg {
  #define VIRTIO_F_VERSION_1 32
 #endif
 
+/* This feature indicates that the driver can reset a queue individually. */
+#ifndef VIRTIO_F_RING_RESET
+#define VIRTIO_F_RING_RESET 40
+#endif
+
 /* Declare packed ring related bits for older kernels */
 #ifndef VIRTIO_F_RING_PACKED
 
@@ -440,7 +445,8 @@ struct vring_packed_desc_event {
 				(1ULL << VIRTIO_NET_F_MTU)  | \
 				(1ULL << VIRTIO_F_IN_ORDER) | \
 				(1ULL << VIRTIO_F_IOMMU_PLATFORM) | \
-				(1ULL << VIRTIO_F_RING_PACKED))
+				(1ULL << VIRTIO_F_RING_PACKED)	| \
+				(1ULL << VIRTIO_F_RING_RESET))
 
 
 struct guest_page {
-- 
2.32.0

Re: [PATCH 1/1] vhost: support VIRTIO_F_RING_RESET for vhost-user

[Kangjie Xu]

This patch will be updated in the future. We'll fix some issues based on 
some review comments from QEMU community.

We intend to add a new vhost protocol message for virtqueue reset.

Please let me know any questions or suggestions you might have.

Thanks

在 2022/7/19 15:26, Kangjie Xu 写道:
> Add VIRTIO_F_RING_RESET, which indicates that the driver can reset a
> queue individually.
>
> VIRTIO_F_RING_RESET feature is added to virtio-spec 1.2. The relevant
> information is in
>      https://github.com/oasis-tcs/virtio-spec/issues/124
>      https://github.com/oasis-tcs/virtio-spec/issues/139
>
> The implementation only adds the feature bit in supported features. It
> does not require any other changes because we reuse the existing vhost
> protocol.
>
> The virtqueue reset process can be concluded as two parts:
> 1. The driver can reset a virtqueue. When it is triggered,
> VHOST_USER_SET_VRING_ENABLE message with "payload.state.num" set to 0
> is sent to DPDK. Then the virtqueue will be disabled in DPDK.
> 2. After the virtqueue is disabled, the driver may optionally re-enable
> it. To avoid confusion with VHOST_USER_SET_VRING_ENABLE, we call this
> part as "restart". The virtqueue's information may be changed when
> restarting it. Thus, the information of the reset virtqueue should be
> updated. This part is basically similar to when the virtqueue is started
> for the first time, except that the restart process does not need to set
> features and set mem table since they does not change. QEMU will send
> messages containing size, base, addr, kickfd and callfd of the virtqueue
> in order. Specifically, the DPDK will receive these messages in order:
>      a. VHOST_USER_SET_VRING_NUM
>      b. VHOST_USER_SET_VRING_BASE
>      c. VHOST_USER_SET_VRING_ADDR
>      d. VHOST_USER_SET_VRING_KICK
>      e. VHOST_USER_SET_VRING_CALL
>      f. VHOST_USER_SET_VRING_ENABLE
> The last VHOST_USER_SET_VRING_ENABLE message with "payload.state.num" set
> to 1, will be sent to enable the virtqueue and the restart process is
> finished.
>
> Signed-off-by: Kangjie Xu <kangjie.xu@linux.alibaba.com>
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
> ---
> Test environment:
>      Host: 5.4.189
>      Qemu: QEMU emulator version 7.0.50 (With vq reset support)
>      Guest: 5.19.0-rc3 (With vq reset support)
>      DPDK: 22.07-rc1
>      Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2;
>              ethtool -g eth1;
>
>      The driver can resize the virtio queue, then virtio queue reset
>      function should be triggered.
>
> Guest Kernel Patch:
>      https://lore.kernel.org/bpf/20220629065656.54420-1-xuanzhuo@linux.alibaba.com
>
> QEMU Patch:
>      https://lore.kernel.org/qemu-devel/cover.1658141552.git.kangjie.xu@linux.alibaba.com
>
>   lib/vhost/vhost.h | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 14235aaf81..9711ded7dd 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -378,6 +378,11 @@ struct vhost_msg {
>    #define VIRTIO_F_VERSION_1 32
>   #endif
>   
> +/* This feature indicates that the driver can reset a queue individually. */
> +#ifndef VIRTIO_F_RING_RESET
> +#define VIRTIO_F_RING_RESET 40
> +#endif
> +
>   /* Declare packed ring related bits for older kernels */
>   #ifndef VIRTIO_F_RING_PACKED
>   
> @@ -440,7 +445,8 @@ struct vring_packed_desc_event {
>   				(1ULL << VIRTIO_NET_F_MTU)  | \
>   				(1ULL << VIRTIO_F_IN_ORDER) | \
>   				(1ULL << VIRTIO_F_IOMMU_PLATFORM) | \
> -				(1ULL << VIRTIO_F_RING_PACKED))
> +				(1ULL << VIRTIO_F_RING_PACKED)	| \
> +				(1ULL << VIRTIO_F_RING_RESET))
>   
>   
>   struct guest_page {