Re: [dpdk-dev] [RFC 0/3] Add mdev (Mediated device) support in DPDK

[Alejandro Lucero <alejandro.lucero@netronome.com>]

On Wed, Apr 3, 2019 at 8:19 AM Tiwei Bie <tiwei.bie@intel.com> wrote:

> Hi everyone,
>
> This is a draft implementation of the mdev (Mediated device [1])
> bus support in DPDK. Mdev is a way to virtualize devices in Linux
> kernel. Based on the device-api (mdev_type/device_api), there could
> be different types of mdev devices (e.g. vfio-pci). In this RFC,
> one mdev bus is introduced to scan the mdev devices in the system
> and do the probe based on the device-api.
>
> Take the mdev devices whose device-api is "vfio-pci" as an example,
> in this RFC, these devices will be probed by a mdev driver provided
> by PCI bus, which will plug them to the PCI bus. And they will be
> probed with the drivers registered on the PCI bus based on VendorID/
> DeviceID/... then.
>
>                      +----------+
>                      | mdev bus |
>                      +----+-----+
>                           |
>          +----------------+----+------+------+
>          |                     |      |      |
>    mdev_vfio_pci               ......
> (device-api: vfio-pci)
>
> There are also other ways to add mdev device support in DPDK (e.g.
> let PCI bus scan /sys/bus/mdev/devices directly). Comments would be
> appreciated!
>
>
Hi Tiwei,

Thanks for the patchset. I was close to send a patchset with the same mdev
support, but I'm glad to see your patchset first because I think it is
interesting to see another view of how to implemented this.

After going through your patch I was a bit confused about how the mdev
device to mdev driver match was done. But then I realized the approach you
are following is different to my implementation, likely due to having
different purposes. If I understand the idea behind, you want to have same
PCI PMD drivers working with devices, PCI devices, created from mediated
devices. That is the reason there is just one mdev driver, the one for
vfio-pci mediated devices type.

My approach was different and I though having specific PMD mdev support was
necessary, with the PMD requiring to register a mdev driver. I can see,
after reading your patch, it can be perfectly possible to have the same
PMDs for "pure" PCI devices and PCI devices made from mediated devices, and
if the PMD requires to do something different due to the mediated devices
intrinsics, then explicitly supporting that per PMD. I got specific ioctl
calls between the PMD and the mediating driver but this can also be done
with your approach.

I'm working on having a mediated PF, what is a different purpose than the
Intel scalable I/O idea, so I will merge this patchset with my code and see
if it works.

Thanks!


> [1]
> https://github.com/torvalds/linux/blob/master/Documentation/vfio-mediated-device.txt
>
> Thanks,
> Tiwei
>
> Tiwei Bie (3):
>   eal: add a helper for reading string from sysfs
>   bus/mdev: add mdev bus support
>   bus/pci: add mdev support
>
>  config/common_base                        |   5 +
>  config/common_linux                       |   1 +
>  drivers/bus/Makefile                      |   1 +
>  drivers/bus/mdev/Makefile                 |  41 +++
>  drivers/bus/mdev/linux/Makefile           |   6 +
>  drivers/bus/mdev/linux/mdev.c             | 117 ++++++++
>  drivers/bus/mdev/mdev.c                   | 310 ++++++++++++++++++++++
>  drivers/bus/mdev/meson.build              |  15 ++
>  drivers/bus/mdev/private.h                |  90 +++++++
>  drivers/bus/mdev/rte_bus_mdev.h           | 141 ++++++++++
>  drivers/bus/mdev/rte_bus_mdev_version.map |  12 +
>  drivers/bus/meson.build                   |   2 +-
>  drivers/bus/pci/Makefile                  |   3 +
>  drivers/bus/pci/linux/Makefile            |   4 +
>  drivers/bus/pci/linux/pci_vfio.c          |  35 ++-
>  drivers/bus/pci/linux/pci_vfio_mdev.c     | 305 +++++++++++++++++++++
>  drivers/bus/pci/meson.build               |   4 +-
>  drivers/bus/pci/pci_common.c              |  17 +-
>  drivers/bus/pci/private.h                 |   9 +
>  drivers/bus/pci/rte_bus_pci.h             |  11 +-
>  lib/librte_eal/common/eal_filesystem.h    |   7 +
>  lib/librte_eal/freebsd/eal/eal.c          |  22 ++
>  lib/librte_eal/linux/eal/eal.c            |  22 ++
>  lib/librte_eal/rte_eal_version.map        |   1 +
>  mk/rte.app.mk                             |   1 +
>  25 files changed, 1163 insertions(+), 19 deletions(-)
>  create mode 100644 drivers/bus/mdev/Makefile
>  create mode 100644 drivers/bus/mdev/linux/Makefile
>  create mode 100644 drivers/bus/mdev/linux/mdev.c
>  create mode 100644 drivers/bus/mdev/mdev.c
>  create mode 100644 drivers/bus/mdev/meson.build
>  create mode 100644 drivers/bus/mdev/private.h
>  create mode 100644 drivers/bus/mdev/rte_bus_mdev.h
>  create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map
>  create mode 100644 drivers/bus/pci/linux/pci_vfio_mdev.c
>
> --
> 2.17.1
>
>