Re: [dpdk-dev] Hotplug support for VFIO

[Alejandro Lucero <alejandro.lucero@netronome.com>]

On Thu, Feb 9, 2017 at 3:16 AM, Tetsuya Mukawa <mtetsuyah@gmail.com> wrote:

> 2017-02-09 0:48 GMT+09:00 Alejandro Lucero <alejandro.lucero@netronome.com
> >:
> > I just wanted to clarify the hotplug VFIO is not the problem, as I can
> see
> > it, but the unplug. When attaching a device the current VFIO code will be
> > used, but there is no code for doing the IOMMU unmapping when unplugging.
> >
> > On Wed, Feb 8, 2017 at 3:43 PM, Alejandro Lucero
> > <alejandro.lucero@netronome.com> wrote:
> >>
> >> Hi Eelco,
> >>
> >> On Wed, Feb 8, 2017 at 9:41 AM, Eelco Chaudron <echaudron@redhat.com>
> >> wrote:
> >>>
> >>> Hi Anatoly,
> >>>
> >>> This will be great... If you want I can do some testing on (early)
> >>> patches if required.
> >>>
> >>> Also do you know why it is currently not supported, i.e. what are the
> >>> limitations?
> >>>
> >>> Thanks,
> >>
> >>
> >> I assume your reply was to my previous email and not to Anatoly's one.
> My
> >> apologies if this assumption is wrong.
> >> By the way, you did not reply to the list, but I'm doing to do so
> because
> >> this could be interesting to other people. Also, I'm CCing hotplug UIO
> >> developer, Tetsuya Mukawa, since there is no a specific maintainer for
> this
> >> functionality.
>
> Hi,
>
> Sorry for late reply.
> Honestly, I am not so familiar with VFIO, so I may be wrong.
> But please check my below comments.
>
> >>
> >> I think the main problem for supporting VFIO hotplug is to deal with
> what
> >> VFIO does, this is IOMMU mappings.
> >> There is no code for doing so, and although the implementation would use
> >> most of the code already there for UIO hotplug, which is basically
> related
> >> to unmapping PCI resources, and the interrupt part will be likely quite
> >> similar, the IOMMU unmapping requires more work.
> >>
> >> I dare to say this is the main reason for not having VFIO hotplug
> support
> >> right now. Maybe Tetsuya can confirm this or give us other reasons.
>
> Yes, this is the main reason.
> When I implement hot plugging, I also needed to implement UIO detaching
> code.
> And at the time, I also tried to implement VFIO code as well, but I
> gave up with below reasons.
>  - Patch will be bigger and more difficult to be merged in mainline code.
>  - I don't have enough time to check actually what kind of code is
> needed to detach it.
>
>
I'm quite familiar with the VFIO code so I know what it requires. At least
the main part. You know there are details not so trivial to see initially
until you start doing the work.


> So above were not technical reasons.
>
> One thing I am not clear is whether we may need to do something
> special for multi process case.
> For example, if primary process is died suddenly, what kind of error
> will be happen to slave process, and what is good way to handle it.
>
>
I do no think this is a problem specific to the hotplug mechanism, but
thanks for the heads up.

I will start working on this. Hopefully it could be part of the 17.05
release.

Thank you


> Thanks,
> Tetsuya
>
> >>
> >>>
> >>>
> >>> Eelco
> >>>
> >>>
> >>>
> >>> On 07/02/17 10:29, Alejandro Lucero wrote:
> >>>
> >>> It seems none is working on this VFIO support.
> >>>
> >>> I will work on this if there is no reply to this thread saying the
> >>> opposite the next days.
> >>>
> >>> On Thu, Feb 2, 2017 at 12:58 PM, Eelco Chaudron <echaudron@redhat.com>
> >>> wrote:
> >>>>
> >>>> On 02/02/17 13:05, Burakov, Anatoly wrote:
> >>>>>
> >>>>> Hi Eelco,
> >>>>>
> >>>>> Please forgive me my ignorance on this matter, but doesn't it work at
> >>>>> the moment? I would assume that if regular PCI hotplug works (with
> igb_uio),
> >>>>> then so would hotplug with VFIO, as it basically utilizes the same
> PCI
> >>>>> infrastructure igb_uio does. That said, I'm not aware of any patches
> >>>>> submitted that had to do with VFIO and hotplug, so I guess the
> answer is,
> >>>>> not at the moment.
> >>>>>
> >>>>> Thanks,
> >>>>> Anatoly
> >>>>
> >>>> I was asking as the documentation explicitly mentions its not
> supported.
> >>>>
> >>>> http://dpdk.org/doc/guides/prog_guide/port_hotplug_
> framework.html#hotplug
> >>>>
> >>>> - "To detach a port, the port should be backed by a device that
> igb_uio
> >>>> manages. VFIO is not supported."
> >>>>
> >>>> I could not find any specific reason why it's not supported, so if
> some
> >>>> one can explain this it would help also...
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Eelco
> >>>>
> >>>>
> >>>
> >>>
> >>
> >
>