Re: [dpdk-dev] [PATCH v4] net/virtio-user: fix packed ring server mode

[Maxime Coquelin <maxime.coquelin@redhat.com>]

On 12/23/19 8:25 AM, Xuan Ding wrote:
> This patch fixes the situation where datapath does not work properly when
> vhost reconnects to virtio in server mode with packed ring.
> 
> Currently, virtio and vhost share memory of vring. For split ring, vhost
> can read the status of discriptors directly from the available ring and

s/discriptors/descriptors/

> the used ring during reconnection. Therefore, the datapath can continue.
> 
> But for packed ring, when reconnecting to virtio, vhost cannot get the
> status of discriptors only through the descriptor ring. By resetting Tx

s/discriptor only through/descriptor via/

> and Rx queues, the datapath can restart from the beginning.
> 
> Fixes: 4c3f5822eb214 ("net/virtio: add packed virtqueue defines")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Xuan Ding <xuan.ding@intel.com>
> ---
> 
> v4:
> * Moved change log below '---' marker.
> 
> v3:
> * Removed an extra asterisk from a comment.
> * Renamed device reset function and moved it to virtio_user_ethdev.c.
> 
> v2:
> * Renamed queue reset functions and moved them to virtqueue.c.
> 
>  drivers/net/virtio/virtio_ethdev.c      |  4 +-
>  drivers/net/virtio/virtio_user_ethdev.c | 40 ++++++++++++++
>  drivers/net/virtio/virtqueue.c          | 71 +++++++++++++++++++++++++
>  drivers/net/virtio/virtqueue.h          |  4 ++
>  4 files changed, 117 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
> index 044eb10a7..f9d0ea70d 100644
> --- a/drivers/net/virtio/virtio_ethdev.c
> +++ b/drivers/net/virtio/virtio_ethdev.c
> @@ -1913,6 +1913,8 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev)
>  			goto err_vtpci_init;
>  	}
>  
> +	rte_spinlock_init(&hw->state_lock);
> +
>  	/* reset device and negotiate default features */
>  	ret = virtio_init_device(eth_dev, VIRTIO_PMD_DEFAULT_GUEST_FEATURES);
>  	if (ret < 0)
> @@ -2155,8 +2157,6 @@ virtio_dev_configure(struct rte_eth_dev *dev)
>  			return -EBUSY;
>  		}
>  
> -	rte_spinlock_init(&hw->state_lock);
> -
>  	hw->use_simple_rx = 1;
>  
>  	if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) {
> diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
> index 3fc172573..425f48230 100644
> --- a/drivers/net/virtio/virtio_user_ethdev.c
> +++ b/drivers/net/virtio/virtio_user_ethdev.c
> @@ -25,12 +25,48 @@
>  #define virtio_user_get_dev(hw) \
>  	((struct virtio_user_dev *)(hw)->virtio_user_dev)
>  
> +static void
> +virtio_user_reset_queues_packed(struct rte_eth_dev *dev)
> +{
> +	struct virtio_hw *hw = dev->data->dev_private;
> +	struct virtnet_rx *rxvq;
> +	struct virtnet_tx *txvq;
> +	uint16_t i;
> +
> +	/* Add lock to avoid queue contention. */
> +	rte_spinlock_lock(&hw->state_lock);
> +	hw->started = 0;
> +
> +	/*
> +	 * Waitting for datapath to complete before resetting queues.
> +	 * 1 ms should be enough for the ongoing Tx/Rx function to finish.
> +	 */
> +	rte_delay_ms(1);
> +
> +	/* Vring reset for each Tx queue and Rx queue. */
> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> +		rxvq = dev->data->rx_queues[i];
> +		virtqueue_rxvq_reset_packed(rxvq->vq);
> +		virtio_dev_rx_queue_setup_finish(dev, i);
> +	}
> +
> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
> +		txvq = dev->data->tx_queues[i];
> +		virtqueue_txvq_reset_packed(txvq->vq);
> +	}
> +
> +	hw->started = 1;
> +	rte_spinlock_unlock(&hw->state_lock);
> +}
> +
> +
>  static int
>  virtio_user_server_reconnect(struct virtio_user_dev *dev)
>  {
>  	int ret;
>  	int connectfd;
>  	struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id];
> +	struct virtio_hw *hw = eth_dev->data->dev_private;
>  
>  	connectfd = accept(dev->listenfd, NULL, NULL);
>  	if (connectfd < 0)
> @@ -51,6 +87,10 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev)
>  
>  	dev->features &= dev->device_features;
>  
> +	/* For packed ring, resetting queues is required in reconnection. */
> +	if (vtpci_packed_queue(hw))
> +		virtio_user_reset_queues_packed(eth_dev);

I think we should log an error here to state that some packets are
likely to be dropped.

Other than that, I don't think we have other choices than doing that
right now, so I'll ack your patch once commit message is fixed and
error message is printed on vrings reset.

Thanks,
Maxime