From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id CD8CF952 for ; Tue, 7 Mar 2017 11:34:12 +0100 (CET) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Mar 2017 02:34:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,258,1484035200"; d="scan'208";a="1138954960" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga002.fm.intel.com with ESMTP; 07 Mar 2017 02:34:11 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.248.2; Tue, 7 Mar 2017 02:34:10 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.88]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.204]) with mapi id 14.03.0248.002; Tue, 7 Mar 2017 18:34:07 +0800 From: "Chen, Jing D" To: Thomas Monjalon CC: "dev@dpdk.org" , "Liang, Cunming" , "Rogers, Gerald" , "Wiles, Keith" , "Richardson, Bruce" , "Mcnamara, John" Thread-Topic: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev Thread-Index: AQHSk0U8EZiYFBGnPUiwkBf6WHmbxaGHb3CAgAGwBCA= Date: Tue, 7 Mar 2017 10:34:06 +0000 Message-ID: <4341B239C0EFF9468EE453F9E9F4604D3C5FEB70@shsmsx102.ccr.corp.intel.com> References: <1488427438-13646-1-git-send-email-jing.d.chen@intel.com> <1488427438-13646-7-git-send-email-jing.d.chen@intel.com> <1488825967.6DAzsKhpGM@xps13> In-Reply-To: <1488825967.6DAzsKhpGM@xps13> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Mar 2017 10:34:13 -0000 Hi, Thomas, > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, March 6, 2017 11:27 PM > To: Chen, Jing D > Cc: dev@dpdk.org; Liang, Cunming ; Rogers, > Gerald ; Wiles, Keith ; > Richardson, Bruce ; Mcnamara, John > > Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev >=20 > 2017-03-02 12:03, Chen Jing D: > > +Overview > > +=3D=3D=3D=3D=3D=3D=3D=3D >=20 > I think the first review pass of this series must be focused on the overv= iew > you give here (thanks for the detailed explanations). >=20 > I'll try to sum up while commenting. >=20 > [...] > The target is programmable devices. >=20 > > +The major purpose of prgdev is to help DPDK users to load/upgrade RTL > > +images for FPGA devices, or upgrade firmware for programmble NICs. >=20 > This is a control plane feature. > You want to have it in DPDK to allow dynamic programmation while running > the device, right? Exactly right. >=20 > [...] > > +When the set of APIs is introduced, a general question is why we need > > +it in DPDK community? >=20 > Good question :) >=20 > [...] > > +Any devices, having the capability to store/load a piece of info > > +to/from the deivce then changed hardware behavior, and applicable to > > +prgdev programming model, could be registered as a prgdev device. > > + > > +The device can be programmed (dynamic) within DPDK or have been prior > > +programmed > > +(static) prior to a DPDK application being launched. > [...] > > +Besides the simple API to upload/download image, the prgdev also > > +introduces a mechanism internally to switch drivers and > > +register/unregister device on the fly. With this mechanism, apps can > > +use the programmable device, unbind other personalities on demand, > then program it and bind it back with new personalities. > > +Apps can follow below examples to simply complete the whole process. >=20 > I disagree about the specific bind/unbind for prgdev. > We must improve binding inside DPDK for every devices. First of all, let us try to imagine what is the safe status for device befo= re apps intending to download some image to it - apps wouldn't operate on the devic= e,=20 any behaviors like configuring registers, receive/transmit data may impair = the device or make the download failed.=20 Following first answer to prevent app accessing device during image downloading, how can we achieve that? Detach drivers with device is a smart idea, right? But the problem is how can apps use prgdev API to download ima= ge after all drivers detached with the device? So, the final question is how can we just detached others driver except prg= dev one? I can't find answer in current DPDK framework, that's why I'd like to = introduce bind/unbind functions to detach other drivers from the device. I'm open to this problem. If any suggestion or mechanism can help on it, I = can remove these 2. BTW, not all devices or image download actions need the device to detach th= e=20 device, depending on hardware implementation. >=20 > > +Note that bind/unbind actions are different concept from that a whole > > +device attach/detach. For example, ``rte_eal_dev_detach()``, which > > +will try to detach the drivers with device and remove the whole > > +device from specific class of devices (ethdev, for example). Then, the > whole device is invisible until a new 'probe' > > +is activated. >=20 > I do not understand. See above explanations. >=20 > > +During the whole procedure of image upload/download, prgdev handler > > +is always valid and apps can use it operate on programmable device. > > +The reason why unbind is necessary is it may break the hardware when > > +programming is in progress while other functions are active. Using > > +programmble NIC as an example, it's a little impossible to > > +receive/forward packets for hardware while updating firmware. In this > > +case, apps need to detach ethdev function before firmware upgrading. > > +Again, prgdev handler is still valid, it's function-level detach, > > +different from device-level detach. After finishing image download, > > +original function needs to attach back, either use same or different > > +drivers, depends on personalities changed or not. For a programmble NI= C, > the driver may be the same. For FPGA, it may not, see below examples to g= et > more details. >=20 > If the personality of the device is changed, it must be seen as a new dev= ice > from e.g. the ethdev point of view, while keeping the same prgdev device. > In other words, a device can have several interfaces at the same time (et= hdev, > cryptodev, eventdev, prgdev, whatever). > I think we can dynamically create/destroy some interfaces while keeping > track of the underlying device. Fully agree. But current PCI device with 'vendor_id' and 'device_id' ony ca= n be bind to single driver at a time. So I add 'rte_prgdev_allocate/release' = to support register/unregister prgdev dynamically without BDF. >=20 > > +Another reason to provide bind/unbind action is programmble devices, > > +like FPGA, are not identified driver by 'vendor ID' and 'device ID', > > +they might not be changed in all the ways, even FPGA is fully > > +programmed. So, it depends on internal mechanism of FPGA, not 'vendor > > +ID' and 'device ID' to identify proper drivers, either a register > > +value or signature, depending on FPGA hardware design. In this case, > > +EAL or other bus driver doesn't have the capability to identify > > +proper driver for FPGA device. With prgdev introduced, while FPGA is > > +always a prgdev, FPGA can use prgdev as primary driver, to find proper > function driver. >=20 > You mean prgdev should help the bus layer to map the right driver interfa= ce? > It looks weird and dangerous. The standard way to identify a PCI device i= s to > look at its IDs. Other unknown methods must be strongly discussed. For programmable Ethernet device, it's not truce. But for FPGA, it's. When = FPGA is produced, the device ID indicate what model it is and won't be changed anyway, even being reprogrammed. It used some not-generic mechanism, like AFU id to distinguish the personalities. So, for FPGA, a prgdev driver can = be used as primary driver to identify personalities and then register to specific d= evices.