From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BAB7AA056A; Thu, 11 Mar 2021 18:28:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DD50406A3; Thu, 11 Mar 2021 18:28:18 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 059994014D; Thu, 11 Mar 2021 18:28:16 +0100 (CET) IronPort-SDR: yICVv1amUpO61eOX/iU1us5XK2Z3JvKuiY+lfOeX6S/3xIOW/iYGHHLMVbde73Ez29Tia6nH9r imHeu9oab/xw== X-IronPort-AV: E=McAfee;i="6000,8403,9920"; a="252721774" X-IronPort-AV: E=Sophos;i="5.81,241,1610438400"; d="scan'208";a="252721774" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Mar 2021 09:28:15 -0800 IronPort-SDR: 3HWzCTa7PCc6lsDN1/tdNI4pBPsy3wsrAWhMKH+irKgcWUjzFHk8qUQhzkIE6rMb0oCkdru+Up jDYszJCQiABA== X-IronPort-AV: E=Sophos;i="5.81,241,1610438400"; d="scan'208";a="510079851" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.23.238]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 11 Mar 2021 09:28:14 -0800 Date: Thu, 11 Mar 2021 17:28:11 +0000 From: Bruce Richardson To: Thomas Monjalon Cc: dev@dpdk.org, techboard@dpdk.org Message-ID: <20210311171913.GD1534@bricha3-MOBL.ger.corp.intel.com> References: <20210311151915.GA1534@bricha3-MOBL.ger.corp.intel.com> <20210311164014.GB1534@bricha3-MOBL.ger.corp.intel.com> <20210311164530.GC1534@bricha3-MOBL.ger.corp.intel.com> <2040842.lu7LhsAh0k@thomas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2040842.lu7LhsAh0k@thomas> Subject: Re: [dpdk-dev] [dpdk-techboard] RFC: DPDK drivers for DPDK bus types X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Mar 11, 2021 at 06:06:48PM +0100, Thomas Monjalon wrote: > 11/03/2021 17:45, Bruce Richardson: > > On Thu, Mar 11, 2021 at 04:40:14PM +0000, Bruce Richardson wrote: > > > On Thu, Mar 11, 2021 at 04:44:49PM +0100, Thomas Monjalon wrote: > > > > 11/03/2021 16:19, Bruce Richardson: > > > > > Hi all, > > > > > > > > > > looking for input here into the area of bus-type drivers and interaction > > > > > with other drivers in DPDK. > > > > > > > > > > By way of context, I'm looking at extending the vdev support in the > > > > > "raw/ioat" driver (file raw/ioat/idxd_vdev.c) to make it more user > > > > > friendly. These devices are accessed by DPDK via nodes in /dev and > > > > > paths in /sys, with the vdev parameters being passed identifying the > > > > > particular devices to use. However, the presence of these devices can be > > > > > detected at runtime by a scan of /dev and /sys, and so it's easy enough > > > > > to implement a custom bus-type driver in DPDK to detect these, rather > > > > > than having the user pass in vdev parameters (which can get awkward to > > > > > use as the number of devices increases). > > > > > > > > I agree. vdev bus should be used for creating device from the void. > > > > If the device has its roots in the system (HW or SW), there should be > > > > a bus for that. > > > > > > > > > > > > > However, looking through a few other drivers in the "bus" directory, it > > > > > appears that scanning system paths, i.e. /sys, is fairly common, so I'm > > > > > wondering if it's possible to have some sharing of functionality here. > > > > > Unfortunately, the use of /sys in each of these drivers I've looked at > > > > > seem sufficiently different to me that I've not immediately seen a > > > > > common level of abstraction we can use. Therefore I'm looking for > > > > > suggestions here that those in the community might have. > > > > > > > > Not sure it's worth looking for such sharing between bus. > > > > > > > > > > > > > On a related note, I'm also concerned about the need for a single device > > > > > type, e.g. one used by DPDK and shared with the kernel, to require two > > > > > separate drivers to work together to support it - a bus driver for > > > > > scanning and a type-specific driver for the actual functional > > > > > implementation. Can we not find a way to reduce the number of drivers > > > > > needing to be supported? > > > > > > > > The bus driver is managing the device life. > > > > The device driver implements a functional class. > > > > I don't see what to save. > > > > Maybe you are biased because the rawdev class is fake. > > > > > > > > > > The fact of the rawdev class being fake is irrelevant to this, the same > > > would apply to any device class which as you say - "has it's roots in the > > > system". However, for some of these devices, the roots in the system may > > > well be unique, and therefore - irrespective of device class - one needs > > > two separate drivers to work with such a device, a bus driver to do system > > > scanning, and a device driver to manage the instances. > > > > > > > > > > > > Following on from this, and if we can't find a good way of doing a > > > > > generic driver for scanning /sys nodes, I wonder if there is value in > > > > > providing a "generic" bus implementation in DPDK, as a catch-all for > > > > > device drivers which need their own custom probing, but that do not > > > > > neatly fall into the other types. The way this might work is to have the > > > > > scanning and probing of devices left entirely to the device driver > > > > > implementation itself. For example, rather than creating an idxd > > > > > bus driver, it would be easier and more self-contained to have the > > > > > rawdev driver itself able to perform scanning and probing - keeping the > > > > > code all in one place. All the bus driver would have to do is maintain a > > > > > list of drivers and found devices reported by the individual driver > > > > > after they have done their own probing. > > > > > > > > So you mean there is a single user of the bus, > > > > so the implementation could be moved into the device driver, > > > > relying on a fake bus? > > > > > > > > > > Exactly. If the bus exists for only one device instance - and will only > > > ever exist for one device - why not allow the driver itself to have an > > > extra function called "scan", rather than having to create a custom bus > > > driver just to do that scanning. > > > > > > > > > > > > Other possible candidates to think about that might be able to use > > > > > their own probing from a generic bus might be, e.g. af_xdp driver, or a > > > > > TAP or memif driver. > > > > > > > > These devices don't exist naturally in the system, so I think they should > > > > be vdev. > > > > > > > AF_XDP devices certainly exist in the system, and serves at least as a good > > > theoretical model of exactly what I am describing. Drivers for TAP or memif > > > could be written to scan for and attach to already-existing instances on > > > the system too. > > > > > > For the AF_XDP case, suppose that the kernel was extended to allow for > > > certain queues to be marked or named, one could then create an AF_XDP bus > > > driver to look for named queues on each NIC interface. Similarly a TAP bus > > > device could scan for TAP instances previously created and configured and > > > attach to them. The key thing is that in each case, the scanning done is > > > unique and applies to only one net driver - nothing rawdev related about > > > this! Therefore, while I don't think we would ever really implement bus > > > drivers for these specific cases, they are examples which I think shows the > > > value in having a bus type to allow generic scanning, rather than forcing a > > > new bus driver to be created for each individual type of scanning > > > necessary. > > > > > > > Just to finally add that I'm happy enough to do a bus driver for the > > specific case I'm looking at - it's actually far less work, I believe. > > However, I'm just trying to avoid us having more components to maintain in > > the future. > > I understand and I agree that creating a bus driver for scanning > an item which will be ever managed by a single driver is meaningless. > > What you want is calling RTE_REGISTER_BUS() from the device driver. > What else is needed? > That could work indeed, only issue I see is that each driver that does this will need to implement more of the bus APIs than just a scan method. However, it's probably a good enough compromise for now, so I'll work on implementing it that way, as it allows keeping things pretty self contained and more maintainable. Thanks, /Bruce