Re: [dpdk-dev] [RFC v3 0/6] Add mdev (Mediated device) support in DPDK

[Thomas Monjalon <thomas@monjalon.net>]

01/06/2021 05:06, Chenbo Xia:
> Hi everyone,
> 
> This is a draft implementation of the mdev (Mediated device [1])
> support in DPDK PCI bus driver. Mdev is a way to virtualize devices
> in Linux kernel. Based on the device-api (mdev_type/device_api),
> there could be different types of mdev devices (e.g. vfio-pci).

Please could you illustrate with an usage of mdev in DPDK?
What does it enable which is not possible today?

> In this patchset, the PCI bus driver is extended to support scanning
> and probing the mdev devices whose device-api is "vfio-pci".
> 
>                      +---------+
>                      | PCI bus |
>                      +----+----+
>                           |
>          +--------+-------+-------+--------+
>          |        |               |        |
>   Physical PCI devices ...   Mediated PCI devices ...
> 
> The first four patches in this patchset are mainly preparation of mdev
> bus support. The left two patches are the key implementation of mdev bus.
> 
> The implementation of mdev bus in DPDK has several options:
> 
> 1: Embed mdev bus in current pci bus
> 
>    This patchset takes this option for an example. Mdev has several
>    device types: pci/platform/amba/ccw/ap. DPDK currently only cares
>    pci devices in all mdev device types so we could embed the mdev bus
>    into current pci bus. Then pci bus with mdev support will scan/plug/
>    unplug/.. not only normal pci devices but also mediated pci devices.

I think it is a different bus.
It would be cleaner to not touch the PCI bus.
Having a separate bus will allow an easy way to identify a device
with the new generic devargs syntax, example:
	bus=mdev,uuid=XXX
or more complex:
	bus=mdev,uuid=XXX/class=crypto/driver=qat,foo=bar

> 2: A new mdev bus that scans mediated pci devices and probes mdev driver to
>    plug-in pci devices to pci bus
> 
>    If we took this option, a new mdev bus will be implemented to scan
>    mediated pci devices and a new mdev driver for pci devices will be
>    implemented in pci bus to plug-in mediated pci devices to pci bus.
> 
>    Our RFC v1 takes this option:
>    http://patchwork.dpdk.org/project/dpdk/cover/20190403071844.21126-1-tiwei.bie@intel.com/
> 
>    Note that: for either option 1 or 2, device drivers do not know the
>    implementation difference but only use structs/functions exposed by
>    pci bus. Mediated pci devices are different from normal pci devices
>    on: 1. Mediated pci devices use UUID as address but normal ones use BDF.
>    2. Mediated pci devices may have some capabilities that normal pci
>    devices do not have. For example, mediated pci devices could have
>    regions that have sparse mmap capability, which allows a region to have
>    multiple mmap areas. Another example is mediated pci devices may have
>    regions/part of regions not mmaped but need to access them. Above
>    difference will change the current ABI (i.e., struct rte_pci_device).
>    Please check 5th and 6th patch for details.
> 
> 3. A brand new mdev bus that does everything
> 
>    This option will implement a new and standalone mdev bus. This option
>    does not need any changes in current pci bus but only needs some shared
>    code (linux vfio part) in pci bus. Drivers of devices that support mdev
>    will register itself as a mdev driver and do not rely on pci bus anymore.
>    This option, IMHO, will make the code clean. The only potential problem
>    may be code duplication, which could be solved by making code of linux
>    vfio part of pci bus common and shared.

Yes I prefer this third option.
We can find an elegant way of sharing some VFIO code between buses.