Re: [dpdk-dev] [RFC 0/3] Add mdev (Mediated device) support in DPDK

[Tiwei Bie <tiwei.bie@intel.com>]

On Mon, Apr 08, 2019 at 09:44:07AM +0100, Alejandro Lucero wrote:
> On Wed, Apr 3, 2019 at 8:19 AM Tiwei Bie <tiwei.bie@intel.com> wrote:
> > Hi everyone,
> >
> > This is a draft implementation of the mdev (Mediated device [1])
> > bus support in DPDK. Mdev is a way to virtualize devices in Linux
> > kernel. Based on the device-api (mdev_type/device_api), there could
> > be different types of mdev devices (e.g. vfio-pci). In this RFC,
> > one mdev bus is introduced to scan the mdev devices in the system
> > and do the probe based on the device-api.
> >
> > Take the mdev devices whose device-api is "vfio-pci" as an example,
> > in this RFC, these devices will be probed by a mdev driver provided
> > by PCI bus, which will plug them to the PCI bus. And they will be
> > probed with the drivers registered on the PCI bus based on VendorID/
> > DeviceID/... then.
> >
> >                      +----------+
> >                      | mdev bus |
> >                      +----+-----+
> >                           |
> >          +----------------+----+------+------+
> >          |                     |      |      |
> >    mdev_vfio_pci               ......
> > (device-api: vfio-pci)
> >
> > There are also other ways to add mdev device support in DPDK (e.g.
> > let PCI bus scan /sys/bus/mdev/devices directly). Comments would be
> > appreciated!
> 
> Hi Tiwei,
> 
> Thanks for the patchset. I was close to send a patchset with the same mdev
> support, but I'm glad to see your patchset first because I think it is
> interesting to see another view of how to implemented this.
> 
> After going through your patch I was a bit confused about how the mdev device
> to mdev driver match was done. But then I realized the approach you are
> following is different to my implementation, likely due to having different
> purposes. If I understand the idea behind, you want to have same PCI PMD
> drivers working with devices, PCI devices, created from mediated devices.

Exactly!

> That
> is the reason there is just one mdev driver, the one for vfio-pci mediated
> devices type.
> 
> My approach was different and I though having specific PMD mdev support was
> necessary, with the PMD requiring to register a mdev driver. I can see, after
> reading your patch, it can be perfectly possible to have the same PMDs for
> "pure" PCI devices and PCI devices made from mediated devices, and if the PMD
> requires to do something different due to the mediated devices intrinsics, then
> explicitly supporting that per PMD. I got specific ioctl calls between the PMD
> and the mediating driver but this can also be done with your approach.
> 
> I'm working on having a mediated PF, what is a different purpose than the Intel
> scalable I/O idea, so I will merge this patchset with my code and see if it
> works. 

Cool! Thanks!

> 
> Thanks!
>  
> 
> > [1] https://github.com/torvalds/linux/blob/master/Documentation/vfio-mediated-device.txt
> > 
> > Thanks,
> > Tiwei
> > 
> > Tiwei Bie (3):
> >   eal: add a helper for reading string from sysfs
> >   bus/mdev: add mdev bus support
> >   bus/pci: add mdev support
> > 
> >  config/common_base                        |   5 +
> >  config/common_linux                       |   1 +
> >  drivers/bus/Makefile                      |   1 +
> >  drivers/bus/mdev/Makefile                 |  41 +++
> >  drivers/bus/mdev/linux/Makefile           |   6 +
> >  drivers/bus/mdev/linux/mdev.c             | 117 ++++++++
> >  drivers/bus/mdev/mdev.c                   | 310 ++++++++++++++++++++++
> >  drivers/bus/mdev/meson.build              |  15 ++
> >  drivers/bus/mdev/private.h                |  90 +++++++
> >  drivers/bus/mdev/rte_bus_mdev.h           | 141 ++++++++++
> >  drivers/bus/mdev/rte_bus_mdev_version.map |  12 +
> >  drivers/bus/meson.build                   |   2 +-
> >  drivers/bus/pci/Makefile                  |   3 +
> >  drivers/bus/pci/linux/Makefile            |   4 +
> >  drivers/bus/pci/linux/pci_vfio.c          |  35 ++-
> >  drivers/bus/pci/linux/pci_vfio_mdev.c     | 305 +++++++++++++++++++++
> >  drivers/bus/pci/meson.build               |   4 +-
> >  drivers/bus/pci/pci_common.c              |  17 +-
> >  drivers/bus/pci/private.h                 |   9 +
> >  drivers/bus/pci/rte_bus_pci.h             |  11 +-
> >  lib/librte_eal/common/eal_filesystem.h    |   7 +
> >  lib/librte_eal/freebsd/eal/eal.c          |  22 ++
> >  lib/librte_eal/linux/eal/eal.c            |  22 ++
> >  lib/librte_eal/rte_eal_version.map        |   1 +
> >  mk/rte.app.mk                             |   1 +
> >  25 files changed, 1163 insertions(+), 19 deletions(-)
> >  create mode 100644 drivers/bus/mdev/Makefile
> >  create mode 100644 drivers/bus/mdev/linux/Makefile
> >  create mode 100644 drivers/bus/mdev/linux/mdev.c
> >  create mode 100644 drivers/bus/mdev/mdev.c
> >  create mode 100644 drivers/bus/mdev/meson.build
> >  create mode 100644 drivers/bus/mdev/private.h
> >  create mode 100644 drivers/bus/mdev/rte_bus_mdev.h
> >  create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map
> >  create mode 100644 drivers/bus/pci/linux/pci_vfio_mdev.c
> > 
> > --
> > 2.17.1
> 
>