Re: [dpdk-dev] kernel binding of devices + hotplug

[Matan Azrad <matan@mellanox.com>]

Hi Konstantin

From: Ananyev, Konstantin, Tuesday, April 17, 2018 2:00 PM
> Hi Matan,
> > Hi Konstantin
> > From: Ananyev, Konstantin, Tuesday, April 17, 2018 12:23 PM

> > > > Actually you say that the whitelist\blacklist mechanism is not
> > > > good enough
> > > and the binding workarounds it.
> > > > The user need to specify somehow the devices it want to run, I
> > > > think that specifying the device you want by -w option (no need to
> > > > specify what you don't want in -w case) is really simpler  and
> > > > more descriptive than
> > > binding each device you want by prior process to its correct driver.
> > >
> > > But what if user changes his mind and decides to give particular
> > > device back to the kernel?
> > > Should he restart the dpdk application? In some cases it might be
> > > not desirable.
> >
> > And what is the behavior now in this case?
> 
> Now we don't have hotplug support at all, so nothing to worry about, I guess
> :)

Looks like we are going to support it soon (Guo series have started it) :)
We have already fail-safe driver to support hot-plug, and a real use case to manage Azure dpdk system.
I think we must take it into account when we discuss on binding. 

> > Looks like if we want to solve it we need to add mechanism to stop these
> particular devices DPDK management in any case.
> 
> I am not talking about managing a device itself (start/stop/attach/detach).
> Let say you start dpdk with '-w dev -w dev2'.
> dev1 is active, traffic is going through it, etc., while dev2 is unplugged.
> Now user is about to plug dev2, though just before that he changed his mind
> and decided to give that device to kernel (or different dpdk app), ideally
> without re-starting the app.
> With just command-line support there is no option to do that.
> Same for opposite case - you started dpdk app with '-w dev1' and then
> decided that you also want that app to manage hot-pluggable dev2 too.

And what if the user wants to switch the management from a dpdk application to other user-space\dpdk application?
Here the binding method doesn't work.
Looks like you are suggesting a new feature regardless binding. 

> > > > > It also allows better usability across systems, since the same
> > > > > commandline can be used on multiple systems with different
> > > > > hardware, with the actual device management rules having been
> > > > > already configured at system install/setup time in udev.
> > > >
> > > > But the user still needs to configure the udev per device for each
> > > > system, I
> > > think that command line is better.
> > > >
> > > > > > Regarding the conflict of system rules for a device, it is
> > > > > > again the user
> > > > > responsibility, whatever we will decide for the binding
> > > > > procedure of DPDK application the user needs to take it into
> > > > > account and to solve such like conflicts.
> > > > > > One option is to remove any binding rules of a DPDK device in
> > > > > > the DPDK
> > > > > application initialization and adjust the new rules by the PMDs,
> > > > > then any conflict should not disturb the user.
> > > > >
> > > > > If the device management is only managed in one place, i.e. not
> > > > > in DPDK, then there is no conflict to manage.
> > > >
> > > > I can't agree with this statement, The essence of DPDK is to give
> > > > a good alternative to managing network devices, DPDK actually
> > > > takes a lot of management area to manage by itself to do the user
> > > > life better :)
> > >
> > > From my point - it is not about managing particular device.
> > > It is about making decision who (kenel/dpdk) will manage that device.
> >
> > Doesn't the w\b mechanism comes to solve it?
> 
> It does till some extent, but I don't think it is the best possible way.

Why? pros and cons against alternative ways...

> >
> > > From usability perspective it seems to me that better to keep it in
> > > one place for all devices.
> > > So udev (or any other sysadmin tool) seems like a right choice here.
> >
> > Please explain why?
> > And if you are talking about 1 place:
> > Why not to do the binding in the same place where we define which device
> to run?
> 
> Not all devices are always managed by dpdk apps.
> Some of them will still be managed by kernel.
> Again there could be several independent dpdk apps running on the same
> system.
> Obviously dpdk app itself can't be a centric place to handle all that varieties.

You can use whitelist way to specify the devices for each app and that's it.

The binding to UIO driver cannot choose the specific user-space\dpdk application for the device management.

> Again - if there exists a system tool that provides that functionality - why not
> to use it?

Use it by dpdk or any other user-space application needs to manage devices.