Re: [dpdk-dev] [PATCH] net/virtio: fix virtio-user init when using existing tap

[Olivier Matz <olivier.matz@6wind.com>]

Hi David,

On Fri, Sep 17, 2021 at 11:33:44AM +0200, David Marchand wrote:
> When attaching to an existing mono queue tap, the virtio-user was not
> reporting that the virtio device was not properly initialised which
> prevented from starting the port later.
> 
> $ ip tuntap add test mode tap
> $ dpdk-testpmd --vdev \
>   net_virtio_user0,iface=test,path=/dev/vhost-net,queues=2 -- -i
> 
> ...
> virtio_user_dev_init_mac(): (/dev/vhost-net) No valid MAC in devargs or
> device, use random
> vhost_kernel_open_tap(): TUNSETIFF failed: Invalid argument
> vhost_kernel_enable_queue_pair(): fail to open tap for vhost kernel
> virtio_user_start_device(): (/dev/vhost-net) Failed to start device
> ...
> Configuring Port 0 (socket 0)
> vhost_kernel_open_tap(): TUNSETIFF failed: Invalid argument
> vhost_kernel_enable_queue_pair(): fail to open tap for vhost kernel
> virtio_set_multiple_queues(): Multiqueue configured but send command
> failed, this is too late now...
> Fail to start port 0: Invalid argument
> Please stop the ports first
> Done
> 
> The virtio-user with vhost-kernel backend was going through a lot
> of complications to initialise tap fds only when using them.
> 
> For each qp enabled for the first time, a tapfd was created via
> TUNSETIFF with unneeded additional steps (see below) and then mapped to
> the right qp in the vhost-net backend.
> Unneeded steps (as long as it has been done once for the port):
> - tap features were queried while this is a constant on a running
>   system,
> - the device name in DPDK was updated,
> - the mac address of the tap was set,
> 
> On subsequent qps state change, the vhost-net backend fd mapping was
> updated and the associated queue/tapfd were disabled/enabled via
> TUNSETQUEUE.
> 
> Now, this patch simplifies the whole logic by keeping all tapfds opened
> and in enabled state (from the tap point of view) at all time.
> 
> Unused ioctl defines are removed.
> 
> Tap features are validated earlier to fail initialisation asap.
> Tap name discovery and mac address configuration are moved when
> configuring qp 0.
> 
> To support attaching to mono queue tap, the virtio-user driver now tries
> to attach in multi queue first, then fallbacks to mono queue.
> 
> Finally (but this is more for consistency), VIRTIO_NET_F_MQ feature is
> exposed only if the underlying tap supports multi queue.
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

It also solves our use-case, which is a bit different:

- our application creates a tap with the IFF_MULTI_QUEUE
- we bind this tap to a virtio-user pmd

Previously, it was not working when passing queues=1 to the pmd, the
TUNSETIFF ioctl() in the pmd was failing because the IFF_MULTI_QUEUE
flag was not passed.

With your patch, it works fine, thanks.

(...)

> @@ -414,18 +425,40 @@ vhost_kernel_setup(struct virtio_user_dev *dev)
>  			PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path, strerror(errno));
>  			goto err_tapfds;
>  		}
> -
>  		data->vhostfds[i] = vhostfd;
>  	}
>  
> +	ifname = dev->ifname != NULL ? dev->ifname : "tap%d";
> +	if (tap_features & IFF_MULTI_QUEUE)
> +		data->tapfds[0] = tap_open(ifname, true);
> +	if (data->tapfds[0] < 0)
> +		data->tapfds[0] = tap_open(ifname, false);

As discussed off-list, since tap_open() is called twice, it can in some
conditions log failures twice. And since tun/tap multi queue feature is
available from 3.8 kernel, we may not want to bother with that.


Olivier