[dpdk-dev] cannot use virtio-user pmd in IOVA=PA mode anymore

[Olivier Matz <olivier.matz@6wind.com>]

Hello,

I recently tested a use-case with our application using the main
branch of dpdk.org.

- the application runs inside a standard x86 VM (no IOMMU)
- there are emulated physical NICs inside the VM
- we use virtio-user pmds connected to tap interfaces through the
  vhost-net backend for exception path

This use-case works with the the stable 20.11 branch, but not with the
current main dpdk.org branch. The virtio-user driver refuses to start in
IOVA=PA mode:

  vdev_probe_all_drivers(): virtio_user0 requires VA IOVA mode but current mode is PA, not initializing
  EAL: Driver cannot attach the device (virtio_user0)

This is likely due to these commits:

  8d935fff5546 ("bus/vdev: add driver IOVA VA mode requirement")
  17043a2909bb ("net/virtio: force IOVA as VA mode for virtio-user")

We didn't see this problem before because we are only testing dpdk.org
main branch on physical machines that have an IOMMU.

I'm not sure to understand the reasons for which the ability to run a
virtio-user pmd in IOVA=PA mode was removed.

The commitlog of 17043a2909bb says:

  At least Vhost-user backend of Virtio-user PMD requires
  IOVA as VA mode. Until now, it was implemented as a hack
  by forcing to use mbuf's buf_addr field instead of buf_iova.

I don't get why vhost-user backend would require IOVA=VA. Note that we
also have a use-case where a virtio-user pmd is connected to our
pmd-vhost in another application, which was working with 20.11.

If there is a constraint with vhost-user backend, what about vhost-net
backend?

Would it make sense to re-allow this feature by somehow reverting
17043a2909bb with some additional cleanup?

I am aware that a solution can be to configure qemu to enable a vIOMMU,
but it is not my preferred solution yet, as it would impact our users
that do not do this currently.

Thanks!
Olivier