Re: [dpdk-dev] [dpdk-techboard] RFC: DPDK drivers for DPDK bus types

[Thomas Monjalon <thomas@monjalon.net>]

11/03/2021 17:45, Bruce Richardson:
> On Thu, Mar 11, 2021 at 04:40:14PM +0000, Bruce Richardson wrote:
> > On Thu, Mar 11, 2021 at 04:44:49PM +0100, Thomas Monjalon wrote:
> > > 11/03/2021 16:19, Bruce Richardson:
> > > > Hi all,
> > > > 
> > > > looking for input here into the area of bus-type drivers and interaction
> > > > with other drivers in DPDK.
> > > > 
> > > > By way of context, I'm looking at extending the vdev support in the
> > > > "raw/ioat" driver (file raw/ioat/idxd_vdev.c) to make it more user
> > > > friendly.  These devices are accessed by DPDK via nodes in /dev and
> > > > paths in /sys, with the vdev parameters being passed identifying the
> > > > particular devices to use. However, the presence of these devices can be
> > > > detected at runtime by a scan of /dev and /sys, and so it's easy enough
> > > > to implement a custom bus-type driver in DPDK to detect these, rather
> > > > than having the user pass in vdev parameters (which can get awkward to
> > > > use as the number of devices increases).
> > > 
> > > I agree. vdev bus should be used for creating device from the void.
> > > If the device has its roots in the system (HW or SW), there should be
> > > a bus for that.
> > > 
> > > 
> > > > However, looking through a few other drivers in the "bus" directory, it
> > > > appears that scanning system paths, i.e. /sys, is fairly common, so I'm
> > > > wondering if it's possible to have some sharing of functionality here.
> > > > Unfortunately, the use of /sys in each of these drivers I've looked at
> > > > seem sufficiently different to me that I've not immediately seen a
> > > > common level of abstraction we can use. Therefore I'm looking for
> > > > suggestions here that those in the community might have.
> > > 
> > > Not sure it's worth looking for such sharing between bus.
> > > 
> > > 
> > > > On a related note, I'm also concerned about the need for a single device
> > > > type, e.g. one used by DPDK and shared with the kernel, to require two
> > > > separate drivers to work together to support it - a bus driver for
> > > > scanning and a type-specific driver for the actual functional
> > > > implementation. Can we not find a way to reduce the number of drivers
> > > > needing to be supported?
> > > 
> > > The bus driver is managing the device life.
> > > The device driver implements a functional class.
> > > I don't see what to save.
> > > Maybe you are biased because the rawdev class is fake.
> > > 
> > 
> > The fact of the rawdev class being fake is irrelevant to this, the same
> > would apply to any device class which as you say - "has it's roots in the
> > system". However, for some of these devices, the roots in the system may
> > well be unique, and therefore - irrespective of device class - one needs
> > two separate drivers to work with such a device, a bus driver to do system
> > scanning, and a device driver to manage the instances.
> > 
> > > 
> > > > Following on from this, and if we can't find a good way of doing a
> > > > generic driver for scanning /sys nodes, I wonder if there is value in
> > > > providing a "generic" bus implementation in DPDK, as a catch-all for
> > > > device drivers which need their own custom probing, but that do not
> > > > neatly fall into the other types. The way this might work is to have the
> > > > scanning and probing of devices left entirely to the device driver
> > > > implementation itself. For example, rather than creating an idxd
> > > > bus driver, it would be easier and more self-contained to have the
> > > > rawdev driver itself able to perform scanning and probing - keeping the
> > > > code all in one place. All the bus driver would have to do is maintain a
> > > > list of drivers and found devices reported by the individual driver
> > > > after they have done their own probing.
> > > 
> > > So you mean there is a single user of the bus,
> > > so the implementation could be moved into the device driver,
> > > relying on a fake bus?
> > > 
> > 
> > Exactly. If the bus exists for only one device instance - and will only
> > ever exist for one device - why not allow the driver itself to have an
> > extra function called "scan", rather than having to create a custom bus
> > driver just to do that scanning.
> > 
> > > 
> > > > Other possible candidates to think about that might be able to use
> > > > their own probing from a generic bus might be, e.g. af_xdp driver, or a
> > > > TAP or memif driver.
> > > 
> > > These devices don't exist naturally in the system, so I think they should
> > > be vdev.
> > > 
> > AF_XDP devices certainly exist in the system, and serves at least as a good
> > theoretical model of exactly what I am describing. Drivers for TAP or memif
> > could be written to scan for and attach to already-existing instances on
> > the system too. 
> > 
> > For the AF_XDP case, suppose that the kernel was extended to allow for
> > certain queues to be marked or named, one could then create an AF_XDP bus
> > driver to look for named queues on each NIC interface. Similarly a TAP bus
> > device could scan for TAP instances previously created and configured and
> > attach to them. The key thing is that in each case, the scanning done is
> > unique and applies to only one net driver - nothing rawdev related about
> > this! Therefore, while I don't think we would ever really implement bus
> > drivers for these specific cases, they are examples which I think shows the
> > value in having a bus type to allow generic scanning, rather than forcing a
> > new bus driver to be created for each individual type of scanning
> > necessary.
> > 
> 
> Just to finally add that I'm happy enough to do a bus driver for the
> specific case I'm looking at - it's actually far less work, I believe.
> However, I'm just trying to avoid us having more components to maintain in
> the future.

I understand and I agree that creating a bus driver for scanning
an item which will be ever managed by a single driver is meaningless.

What you want is calling RTE_REGISTER_BUS() from the device driver.
What else is needed?