From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] VFIO no-iommu
Date: Wed, 16 Dec 2015 17:22:11 +0000 [thread overview]
Message-ID: <C6ECDF3AB251BE4894318F4E45123697820B2B8B@IRSMSX109.ger.corp.intel.com> (raw)
In-Reply-To: <1450285912.2674.22.camel@redhat.com>
Hi Alex,
> On Wed, 2015-12-16 at 08:35 +0000, Burakov, Anatoly wrote:
> > Hi Alex,
> >
> > > On Wed, 2015-12-16 at 04:04 +0000, Ferruh Yigit wrote:
> > > > On Tue, Dec 15, 2015 at 09:53:18AM -0700, Alex Williamson wrote:
> > > > I tested the DPDK (HEAD of master) with the patch, with help of
> > > > Anatoly, and DPDK works in no-iommu environment with a little
> > > > modification.
> > > >
> > > > Basically the only modification is adapt new group naming
> > > > (noiommu-$)
> > > > and
> > >
> > > Sorry, forgot to mention that one. The intention with the modified
> > > group name is that I want to be very certain that a user intending
> > > to only support properly iommu isolated devices doesn't accidentally
> > > need to deal with these no-iommu mode devices.
> > >
> > > > disable dma mapping (VFIO_IOMMU_MAP_DMA)
> > > >
> > > > Also I need to disable VFIO_CHECK_EXTENSION ioctl, because in vfio
> > > > module,
> > > > container->noiommu is not set before doing a
> > > > vfio_group_set_container()
> > > > and vfio_for_each_iommu_driver selects wrong driver.
> > >
> > > Running CHECK_EXTENSION on a container without the group attached is
> > > only going to tell you what extensions vfio is capable of, not
> > > necessarily what extensions are available to you with that group.
> > > Is this just a general dpdk- vfio ordering bug?
> >
> > Yes, that is how VFIO was implemented in DPDK. I was under the
> > impression that checking extension before assigning devices was the
> > correct way to do things, so as to not to try anything we know would
> > fail anyway. Does this imply that CHECK_EXTENSION needs to be called
> > on both container and groups (or just on groups)?
>
> Hmm, in Documentation/vfio.txt we do give the following algorithm:
>
> if (ioctl(container, VFIO_GET_API_VERSION) != VFIO_API_VERSION)
> /* Unknown API version */
>
> if (!ioctl(container, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU))
> /* Doesn't support the IOMMU driver we want. */
> ...
>
> That's just going to query each iommu driver and we can't yet say whether
> the group the user attaches to the container later will actually support that
> extension until we try to do it, that would come at VFIO_SET_IOMMU. So is
> it perhaps a vfio bug that we're not advertising no-iommu until the group is
> attached? After all, we are capable of it with just an empty container, just
> like we are with type1, but we're going to fail SET_IOMMU for the wrong
> combination.
> This is exactly the sort of thing that makes me glad we reverted it without
> feedback from a working user driver. Thanks,
Whether it should be considered a "bug" in VFIO or "by design" is up to you, of course, but at least according to the VFIO documentation, we are meant to check for type 1 extension and then attach devices, so it would be expected to get VFIO_NOIOMMU_IOMMU marked as supported even without any devices attached to the container (just like we get type 1 as supported without any devices attached). Having said that, if it was meant to attach devices first and then check the extensions, then perhaps the documentation should also point out that fact (or perhaps I missed that detail in my readings of the docs, in which case my apologies).
Thanks,
Anatoly
next prev parent reply other threads:[~2015-12-16 17:22 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-12-11 16:28 Thomas Monjalon
2015-12-11 22:12 ` Vincent JARDIN
2015-12-11 23:02 ` Alex Williamson
2015-12-15 13:43 ` O'Driscoll, Tim
2015-12-15 16:53 ` Alex Williamson
2015-12-16 4:04 ` Ferruh Yigit
2015-12-16 4:38 ` Alex Williamson
2015-12-16 8:35 ` Burakov, Anatoly
2015-12-16 16:23 ` Burakov, Anatoly
2015-12-16 23:17 ` Thomas Monjalon
2015-12-17 9:52 ` Burakov, Anatoly
2015-12-17 10:09 ` Thomas Monjalon
2015-12-17 19:38 ` Jan Viktorin
2015-12-17 21:16 ` Vincent JARDIN
2015-12-17 23:29 ` Stephen Hemminger
2015-12-16 17:11 ` Alex Williamson
2015-12-16 17:22 ` Burakov, Anatoly [this message]
2015-12-17 16:43 ` Alex Williamson
2015-12-18 10:43 ` Yigit, Ferruh
2015-12-18 14:38 ` Alex Williamson
2015-12-18 21:50 ` Alex Williamson
2015-12-21 11:46 ` Yigit, Ferruh
2015-12-21 12:18 ` [dpdk-dev] [PATCH] vfio: add no-iommu support Ferruh Yigit
2015-12-21 15:15 ` Burakov, Anatoly
2015-12-21 15:26 ` Yigit, Ferruh
2015-12-21 15:28 ` Burakov, Anatoly
2015-12-21 19:22 ` [dpdk-dev] VFIO no-iommu Alex Williamson
2015-12-22 20:20 ` Alex Williamson
2015-12-23 11:19 ` Burakov, Anatoly
2015-12-31 14:30 ` Santosh Shukla
2016-01-14 6:03 ` Jike Song
2016-01-14 6:52 ` Alex Williamson
2016-01-14 8:12 ` Jike Song
2015-12-11 23:20 ` Jan Viktorin
2015-12-15 11:20 ` Alejandro Lucero
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=C6ECDF3AB251BE4894318F4E45123697820B2B8B@IRSMSX109.ger.corp.intel.com \
--to=anatoly.burakov@intel.com \
--cc=alex.williamson@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).