From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id C584E2BD8 for ; Mon, 8 Apr 2019 11:36:30 +0200 (CEST) X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Apr 2019 02:36:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,324,1549958400"; d="scan'208";a="140882559" Received: from dpdk-tbie.sh.intel.com ([10.67.104.173]) by orsmga003.jf.intel.com with ESMTP; 08 Apr 2019 02:36:28 -0700 Date: Mon, 8 Apr 2019 17:36:01 +0800 From: Tiwei Bie To: Alejandro Lucero Cc: dev , "Liang, Cunming" , Bruce Richardson Message-ID: <20190408093601.GA12313@dpdk-tbie.sh.intel.com> References: <20190403071844.21126-1-tiwei.bie@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Subject: Re: [dpdk-dev] [RFC 0/3] Add mdev (Mediated device) support in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Apr 2019 09:36:31 -0000 On Mon, Apr 08, 2019 at 09:44:07AM +0100, Alejandro Lucero wrote: > On Wed, Apr 3, 2019 at 8:19 AM Tiwei Bie wrote: > > Hi everyone, > > > > This is a draft implementation of the mdev (Mediated device [1]) > > bus support in DPDK. Mdev is a way to virtualize devices in Linux > > kernel. Based on the device-api (mdev_type/device_api), there could > > be different types of mdev devices (e.g. vfio-pci). In this RFC, > > one mdev bus is introduced to scan the mdev devices in the system > > and do the probe based on the device-api. > > > > Take the mdev devices whose device-api is "vfio-pci" as an example, > > in this RFC, these devices will be probed by a mdev driver provided > > by PCI bus, which will plug them to the PCI bus. And they will be > > probed with the drivers registered on the PCI bus based on VendorID/ > > DeviceID/... then. > > > > +----------+ > > | mdev bus | > > +----+-----+ > > | > > +----------------+----+------+------+ > > | | | | > > mdev_vfio_pci ...... > > (device-api: vfio-pci) > > > > There are also other ways to add mdev device support in DPDK (e.g. > > let PCI bus scan /sys/bus/mdev/devices directly). Comments would be > > appreciated! > > Hi Tiwei, > > Thanks for the patchset. I was close to send a patchset with the same mdev > support, but I'm glad to see your patchset first because I think it is > interesting to see another view of how to implemented this. > > After going through your patch I was a bit confused about how the mdev device > to mdev driver match was done. But then I realized the approach you are > following is different to my implementation, likely due to having different > purposes. If I understand the idea behind, you want to have same PCI PMD > drivers working with devices, PCI devices, created from mediated devices. Exactly! > That > is the reason there is just one mdev driver, the one for vfio-pci mediated > devices type. > > My approach was different and I though having specific PMD mdev support was > necessary, with the PMD requiring to register a mdev driver. I can see, after > reading your patch, it can be perfectly possible to have the same PMDs for > "pure" PCI devices and PCI devices made from mediated devices, and if the PMD > requires to do something different due to the mediated devices intrinsics, then > explicitly supporting that per PMD. I got specific ioctl calls between the PMD > and the mediating driver but this can also be done with your approach. > > I'm working on having a mediated PF, what is a different purpose than the Intel > scalable I/O idea, so I will merge this patchset with my code and see if it > works. Cool! Thanks! > > Thanks! > > > > [1] https://github.com/torvalds/linux/blob/master/Documentation/vfio-mediated-device.txt > > > > Thanks, > > Tiwei > > > > Tiwei Bie (3): > > eal: add a helper for reading string from sysfs > > bus/mdev: add mdev bus support > > bus/pci: add mdev support > > > > config/common_base | 5 + > > config/common_linux | 1 + > > drivers/bus/Makefile | 1 + > > drivers/bus/mdev/Makefile | 41 +++ > > drivers/bus/mdev/linux/Makefile | 6 + > > drivers/bus/mdev/linux/mdev.c | 117 ++++++++ > > drivers/bus/mdev/mdev.c | 310 ++++++++++++++++++++++ > > drivers/bus/mdev/meson.build | 15 ++ > > drivers/bus/mdev/private.h | 90 +++++++ > > drivers/bus/mdev/rte_bus_mdev.h | 141 ++++++++++ > > drivers/bus/mdev/rte_bus_mdev_version.map | 12 + > > drivers/bus/meson.build | 2 +- > > drivers/bus/pci/Makefile | 3 + > > drivers/bus/pci/linux/Makefile | 4 + > > drivers/bus/pci/linux/pci_vfio.c | 35 ++- > > drivers/bus/pci/linux/pci_vfio_mdev.c | 305 +++++++++++++++++++++ > > drivers/bus/pci/meson.build | 4 +- > > drivers/bus/pci/pci_common.c | 17 +- > > drivers/bus/pci/private.h | 9 + > > drivers/bus/pci/rte_bus_pci.h | 11 +- > > lib/librte_eal/common/eal_filesystem.h | 7 + > > lib/librte_eal/freebsd/eal/eal.c | 22 ++ > > lib/librte_eal/linux/eal/eal.c | 22 ++ > > lib/librte_eal/rte_eal_version.map | 1 + > > mk/rte.app.mk | 1 + > > 25 files changed, 1163 insertions(+), 19 deletions(-) > > create mode 100644 drivers/bus/mdev/Makefile > > create mode 100644 drivers/bus/mdev/linux/Makefile > > create mode 100644 drivers/bus/mdev/linux/mdev.c > > create mode 100644 drivers/bus/mdev/mdev.c > > create mode 100644 drivers/bus/mdev/meson.build > > create mode 100644 drivers/bus/mdev/private.h > > create mode 100644 drivers/bus/mdev/rte_bus_mdev.h > > create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map > > create mode 100644 drivers/bus/pci/linux/pci_vfio_mdev.c > > > > -- > > 2.17.1 > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id A0A5EA0096 for ; Mon, 8 Apr 2019 11:36:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 215BD2C24; Mon, 8 Apr 2019 11:36:32 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id C584E2BD8 for ; Mon, 8 Apr 2019 11:36:30 +0200 (CEST) X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Apr 2019 02:36:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,324,1549958400"; d="scan'208";a="140882559" Received: from dpdk-tbie.sh.intel.com ([10.67.104.173]) by orsmga003.jf.intel.com with ESMTP; 08 Apr 2019 02:36:28 -0700 Date: Mon, 8 Apr 2019 17:36:01 +0800 From: Tiwei Bie To: Alejandro Lucero Cc: dev , "Liang, Cunming" , Bruce Richardson Message-ID: <20190408093601.GA12313@dpdk-tbie.sh.intel.com> References: <20190403071844.21126-1-tiwei.bie@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Subject: Re: [dpdk-dev] [RFC 0/3] Add mdev (Mediated device) support in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190408093601.so3Ep21gob7FN9XViR4o873Ml2aeueuyrm79Tal2yio@z> On Mon, Apr 08, 2019 at 09:44:07AM +0100, Alejandro Lucero wrote: > On Wed, Apr 3, 2019 at 8:19 AM Tiwei Bie wrote: > > Hi everyone, > > > > This is a draft implementation of the mdev (Mediated device [1]) > > bus support in DPDK. Mdev is a way to virtualize devices in Linux > > kernel. Based on the device-api (mdev_type/device_api), there could > > be different types of mdev devices (e.g. vfio-pci). In this RFC, > > one mdev bus is introduced to scan the mdev devices in the system > > and do the probe based on the device-api. > > > > Take the mdev devices whose device-api is "vfio-pci" as an example, > > in this RFC, these devices will be probed by a mdev driver provided > > by PCI bus, which will plug them to the PCI bus. And they will be > > probed with the drivers registered on the PCI bus based on VendorID/ > > DeviceID/... then. > > > > +----------+ > > | mdev bus | > > +----+-----+ > > | > > +----------------+----+------+------+ > > | | | | > > mdev_vfio_pci ...... > > (device-api: vfio-pci) > > > > There are also other ways to add mdev device support in DPDK (e.g. > > let PCI bus scan /sys/bus/mdev/devices directly). Comments would be > > appreciated! > > Hi Tiwei, > > Thanks for the patchset. I was close to send a patchset with the same mdev > support, but I'm glad to see your patchset first because I think it is > interesting to see another view of how to implemented this. > > After going through your patch I was a bit confused about how the mdev device > to mdev driver match was done. But then I realized the approach you are > following is different to my implementation, likely due to having different > purposes. If I understand the idea behind, you want to have same PCI PMD > drivers working with devices, PCI devices, created from mediated devices. Exactly! > That > is the reason there is just one mdev driver, the one for vfio-pci mediated > devices type. > > My approach was different and I though having specific PMD mdev support was > necessary, with the PMD requiring to register a mdev driver. I can see, after > reading your patch, it can be perfectly possible to have the same PMDs for > "pure" PCI devices and PCI devices made from mediated devices, and if the PMD > requires to do something different due to the mediated devices intrinsics, then > explicitly supporting that per PMD. I got specific ioctl calls between the PMD > and the mediating driver but this can also be done with your approach. > > I'm working on having a mediated PF, what is a different purpose than the Intel > scalable I/O idea, so I will merge this patchset with my code and see if it > works. Cool! Thanks! > > Thanks! > > > > [1] https://github.com/torvalds/linux/blob/master/Documentation/vfio-mediated-device.txt > > > > Thanks, > > Tiwei > > > > Tiwei Bie (3): > > eal: add a helper for reading string from sysfs > > bus/mdev: add mdev bus support > > bus/pci: add mdev support > > > > config/common_base | 5 + > > config/common_linux | 1 + > > drivers/bus/Makefile | 1 + > > drivers/bus/mdev/Makefile | 41 +++ > > drivers/bus/mdev/linux/Makefile | 6 + > > drivers/bus/mdev/linux/mdev.c | 117 ++++++++ > > drivers/bus/mdev/mdev.c | 310 ++++++++++++++++++++++ > > drivers/bus/mdev/meson.build | 15 ++ > > drivers/bus/mdev/private.h | 90 +++++++ > > drivers/bus/mdev/rte_bus_mdev.h | 141 ++++++++++ > > drivers/bus/mdev/rte_bus_mdev_version.map | 12 + > > drivers/bus/meson.build | 2 +- > > drivers/bus/pci/Makefile | 3 + > > drivers/bus/pci/linux/Makefile | 4 + > > drivers/bus/pci/linux/pci_vfio.c | 35 ++- > > drivers/bus/pci/linux/pci_vfio_mdev.c | 305 +++++++++++++++++++++ > > drivers/bus/pci/meson.build | 4 +- > > drivers/bus/pci/pci_common.c | 17 +- > > drivers/bus/pci/private.h | 9 + > > drivers/bus/pci/rte_bus_pci.h | 11 +- > > lib/librte_eal/common/eal_filesystem.h | 7 + > > lib/librte_eal/freebsd/eal/eal.c | 22 ++ > > lib/librte_eal/linux/eal/eal.c | 22 ++ > > lib/librte_eal/rte_eal_version.map | 1 + > > mk/rte.app.mk | 1 + > > 25 files changed, 1163 insertions(+), 19 deletions(-) > > create mode 100644 drivers/bus/mdev/Makefile > > create mode 100644 drivers/bus/mdev/linux/Makefile > > create mode 100644 drivers/bus/mdev/linux/mdev.c > > create mode 100644 drivers/bus/mdev/mdev.c > > create mode 100644 drivers/bus/mdev/meson.build > > create mode 100644 drivers/bus/mdev/private.h > > create mode 100644 drivers/bus/mdev/rte_bus_mdev.h > > create mode 100644 drivers/bus/mdev/rte_bus_mdev_version.map > > create mode 100644 drivers/bus/pci/linux/pci_vfio_mdev.c > > > > -- > > 2.17.1 > >