From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id B824758CF for ; Tue, 7 Mar 2017 12:12:36 +0100 (CET) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Mar 2017 03:12:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,258,1484035200"; d="scan'208";a="57045146" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61]) by orsmga002.jf.intel.com with SMTP; 07 Mar 2017 03:12:32 -0800 Received: by (sSMTP sendmail emulation); Tue, 07 Mar 2017 11:12:32 +0000 Date: Tue, 7 Mar 2017 11:12:31 +0000 From: Bruce Richardson To: "Chen, Jing D" Cc: Thomas Monjalon , "dev@dpdk.org" , "Liang, Cunming" , "Rogers, Gerald" , "Wiles, Keith" , "Mcnamara, John" Message-ID: <20170307111231.GA254436@bricha3-MOBL3.ger.corp.intel.com> References: <1488427438-13646-1-git-send-email-jing.d.chen@intel.com> <1488427438-13646-7-git-send-email-jing.d.chen@intel.com> <1488825967.6DAzsKhpGM@xps13> <4341B239C0EFF9468EE453F9E9F4604D3C5FEB70@shsmsx102.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4341B239C0EFF9468EE453F9E9F4604D3C5FEB70@shsmsx102.ccr.corp.intel.com> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.8.0 (2017-02-23) Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Mar 2017 11:12:37 -0000 On Tue, Mar 07, 2017 at 10:34:06AM +0000, Chen, Jing D wrote: > Hi, Thomas, > > > -----Original Message----- > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > Sent: Monday, March 6, 2017 11:27 PM > > To: Chen, Jing D > > Cc: dev@dpdk.org; Liang, Cunming ; Rogers, > > Gerald ; Wiles, Keith ; > > Richardson, Bruce ; Mcnamara, John > > > > Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev > > > > 2017-03-02 12:03, Chen Jing D: > > > +Overview > > > +======== > > > > I think the first review pass of this series must be focused on the overview > > you give here (thanks for the detailed explanations). > > > > I'll try to sum up while commenting. > > > > [...] > > The target is programmable devices. > > > > > +The major purpose of prgdev is to help DPDK users to load/upgrade RTL > > > +images for FPGA devices, or upgrade firmware for programmble NICs. > > > > This is a control plane feature. > > You want to have it in DPDK to allow dynamic programmation while running > > the device, right? > > Exactly right. > > > > > [...] > > > +When the set of APIs is introduced, a general question is why we need > > > +it in DPDK community? > > > > Good question :) > > > > [...] > > > +Any devices, having the capability to store/load a piece of info > > > +to/from the deivce then changed hardware behavior, and applicable to > > > +prgdev programming model, could be registered as a prgdev device. > > > + > > > +The device can be programmed (dynamic) within DPDK or have been prior > > > +programmed > > > +(static) prior to a DPDK application being launched. > > [...] > > > +Besides the simple API to upload/download image, the prgdev also > > > +introduces a mechanism internally to switch drivers and > > > +register/unregister device on the fly. With this mechanism, apps can > > > +use the programmable device, unbind other personalities on demand, > > then program it and bind it back with new personalities. > > > +Apps can follow below examples to simply complete the whole process. > > > > I disagree about the specific bind/unbind for prgdev. > > We must improve binding inside DPDK for every devices. > > First of all, let us try to imagine what is the safe status for device before apps > intending to download some image to it - apps wouldn't operate on the device, > any behaviors like configuring registers, receive/transmit data may impair the > device or make the download failed. > Following first answer to prevent app accessing device during image > downloading, how can we achieve that? Detach drivers with device is a smart > idea, right? But the problem is how can apps use prgdev API to download image > after all drivers detached with the device? > So, the final question is how can we just detached others driver except prgdev > one? I can't find answer in current DPDK framework, that's why I'd like to introduce > bind/unbind functions to detach other drivers from the device. > > I'm open to this problem. If any suggestion or mechanism can help on it, I can > remove these 2. > > BTW, not all devices or image download actions need the device to detach the > device, depending on hardware implementation. > > > > > > +Note that bind/unbind actions are different concept from that a whole > > > +device attach/detach. For example, ``rte_eal_dev_detach()``, which > > > +will try to detach the drivers with device and remove the whole > > > +device from specific class of devices (ethdev, for example). Then, the > > whole device is invisible until a new 'probe' > > > +is activated. > > > > I do not understand. > > See above explanations. > > > > > > +During the whole procedure of image upload/download, prgdev handler > > > +is always valid and apps can use it operate on programmable device. > > > +The reason why unbind is necessary is it may break the hardware when > > > +programming is in progress while other functions are active. Using > > > +programmble NIC as an example, it's a little impossible to > > > +receive/forward packets for hardware while updating firmware. In this > > > +case, apps need to detach ethdev function before firmware upgrading. > > > +Again, prgdev handler is still valid, it's function-level detach, > > > +different from device-level detach. After finishing image download, > > > +original function needs to attach back, either use same or different > > > +drivers, depends on personalities changed or not. For a programmble NIC, > > the driver may be the same. For FPGA, it may not, see below examples to get > > more details. > > > > If the personality of the device is changed, it must be seen as a new device > > from e.g. the ethdev point of view, while keeping the same prgdev device. > > In other words, a device can have several interfaces at the same time (ethdev, > > cryptodev, eventdev, prgdev, whatever). > > I think we can dynamically create/destroy some interfaces while keeping > > track of the underlying device. > > Fully agree. But current PCI device with 'vendor_id' and 'device_id' ony can > be bind to single driver at a time. So I add 'rte_prgdev_allocate/release' to > support register/unregister prgdev dynamically without BDF. > > > > > > +Another reason to provide bind/unbind action is programmble devices, > > > +like FPGA, are not identified driver by 'vendor ID' and 'device ID', > > > +they might not be changed in all the ways, even FPGA is fully > > > +programmed. So, it depends on internal mechanism of FPGA, not 'vendor > > > +ID' and 'device ID' to identify proper drivers, either a register > > > +value or signature, depending on FPGA hardware design. In this case, > > > +EAL or other bus driver doesn't have the capability to identify > > > +proper driver for FPGA device. With prgdev introduced, while FPGA is > > > +always a prgdev, FPGA can use prgdev as primary driver, to find proper > > function driver. > > > > You mean prgdev should help the bus layer to map the right driver interface? > > It looks weird and dangerous. The standard way to identify a PCI device is to > > look at its IDs. Other unknown methods must be strongly discussed. > > For programmable Ethernet device, it's not truce. But for FPGA, it's. When FPGA > is produced, the device ID indicate what model it is and won't be changed > anyway, even being reprogrammed. It used some not-generic mechanism, like > AFU id to distinguish the personalities. So, for FPGA, a prgdev driver can be used > as primary driver to identify personalities and then register to specific devices. Sounds like we would need an FPGA bus driver in that case. I think that would be a better solution than having a specific device driver loading other drivers. /Bruce