From: Bruce Richardson <bruce.richardson@intel.com>
To: "Chen, Jing D" <jing.d.chen@intel.com>
Cc: Thomas Monjalon <thomas.monjalon@6wind.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Liang, Cunming" <cunming.liang@intel.com>,
"Rogers, Gerald" <gerald.rogers@intel.com>,
"Wiles, Keith" <keith.wiles@intel.com>,
"Mcnamara, John" <john.mcnamara@intel.com>
Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev
Date: Tue, 7 Mar 2017 11:12:31 +0000 [thread overview]
Message-ID: <20170307111231.GA254436@bricha3-MOBL3.ger.corp.intel.com> (raw)
In-Reply-To: <4341B239C0EFF9468EE453F9E9F4604D3C5FEB70@shsmsx102.ccr.corp.intel.com>
On Tue, Mar 07, 2017 at 10:34:06AM +0000, Chen, Jing D wrote:
> Hi, Thomas,
>
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > Sent: Monday, March 6, 2017 11:27 PM
> > To: Chen, Jing D <jing.d.chen@intel.com>
> > Cc: dev@dpdk.org; Liang, Cunming <cunming.liang@intel.com>; Rogers,
> > Gerald <gerald.rogers@intel.com>; Wiles, Keith <keith.wiles@intel.com>;
> > Richardson, Bruce <bruce.richardson@intel.com>; Mcnamara, John
> > <john.mcnamara@intel.com>
> > Subject: Re: [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev
> >
> > 2017-03-02 12:03, Chen Jing D:
> > > +Overview
> > > +========
> >
> > I think the first review pass of this series must be focused on the overview
> > you give here (thanks for the detailed explanations).
> >
> > I'll try to sum up while commenting.
> >
> > [...]
> > The target is programmable devices.
> >
> > > +The major purpose of prgdev is to help DPDK users to load/upgrade RTL
> > > +images for FPGA devices, or upgrade firmware for programmble NICs.
> >
> > This is a control plane feature.
> > You want to have it in DPDK to allow dynamic programmation while running
> > the device, right?
>
> Exactly right.
>
> >
> > [...]
> > > +When the set of APIs is introduced, a general question is why we need
> > > +it in DPDK community?
> >
> > Good question :)
> >
> > [...]
> > > +Any devices, having the capability to store/load a piece of info
> > > +to/from the deivce then changed hardware behavior, and applicable to
> > > +prgdev programming model, could be registered as a prgdev device.
> > > +
> > > +The device can be programmed (dynamic) within DPDK or have been prior
> > > +programmed
> > > +(static) prior to a DPDK application being launched.
> > [...]
> > > +Besides the simple API to upload/download image, the prgdev also
> > > +introduces a mechanism internally to switch drivers and
> > > +register/unregister device on the fly. With this mechanism, apps can
> > > +use the programmable device, unbind other personalities on demand,
> > then program it and bind it back with new personalities.
> > > +Apps can follow below examples to simply complete the whole process.
> >
> > I disagree about the specific bind/unbind for prgdev.
> > We must improve binding inside DPDK for every devices.
>
> First of all, let us try to imagine what is the safe status for device before apps
> intending to download some image to it - apps wouldn't operate on the device,
> any behaviors like configuring registers, receive/transmit data may impair the
> device or make the download failed.
> Following first answer to prevent app accessing device during image
> downloading, how can we achieve that? Detach drivers with device is a smart
> idea, right? But the problem is how can apps use prgdev API to download image
> after all drivers detached with the device?
> So, the final question is how can we just detached others driver except prgdev
> one? I can't find answer in current DPDK framework, that's why I'd like to introduce
> bind/unbind functions to detach other drivers from the device.
>
> I'm open to this problem. If any suggestion or mechanism can help on it, I can
> remove these 2.
>
> BTW, not all devices or image download actions need the device to detach the
> device, depending on hardware implementation.
>
> >
> > > +Note that bind/unbind actions are different concept from that a whole
> > > +device attach/detach. For example, ``rte_eal_dev_detach()``, which
> > > +will try to detach the drivers with device and remove the whole
> > > +device from specific class of devices (ethdev, for example). Then, the
> > whole device is invisible until a new 'probe'
> > > +is activated.
> >
> > I do not understand.
>
> See above explanations.
>
> >
> > > +During the whole procedure of image upload/download, prgdev handler
> > > +is always valid and apps can use it operate on programmable device.
> > > +The reason why unbind is necessary is it may break the hardware when
> > > +programming is in progress while other functions are active. Using
> > > +programmble NIC as an example, it's a little impossible to
> > > +receive/forward packets for hardware while updating firmware. In this
> > > +case, apps need to detach ethdev function before firmware upgrading.
> > > +Again, prgdev handler is still valid, it's function-level detach,
> > > +different from device-level detach. After finishing image download,
> > > +original function needs to attach back, either use same or different
> > > +drivers, depends on personalities changed or not. For a programmble NIC,
> > the driver may be the same. For FPGA, it may not, see below examples to get
> > more details.
> >
> > If the personality of the device is changed, it must be seen as a new device
> > from e.g. the ethdev point of view, while keeping the same prgdev device.
> > In other words, a device can have several interfaces at the same time (ethdev,
> > cryptodev, eventdev, prgdev, whatever).
> > I think we can dynamically create/destroy some interfaces while keeping
> > track of the underlying device.
>
> Fully agree. But current PCI device with 'vendor_id' and 'device_id' ony can
> be bind to single driver at a time. So I add 'rte_prgdev_allocate/release' to
> support register/unregister prgdev dynamically without BDF.
>
> >
> > > +Another reason to provide bind/unbind action is programmble devices,
> > > +like FPGA, are not identified driver by 'vendor ID' and 'device ID',
> > > +they might not be changed in all the ways, even FPGA is fully
> > > +programmed. So, it depends on internal mechanism of FPGA, not 'vendor
> > > +ID' and 'device ID' to identify proper drivers, either a register
> > > +value or signature, depending on FPGA hardware design. In this case,
> > > +EAL or other bus driver doesn't have the capability to identify
> > > +proper driver for FPGA device. With prgdev introduced, while FPGA is
> > > +always a prgdev, FPGA can use prgdev as primary driver, to find proper
> > function driver.
> >
> > You mean prgdev should help the bus layer to map the right driver interface?
> > It looks weird and dangerous. The standard way to identify a PCI device is to
> > look at its IDs. Other unknown methods must be strongly discussed.
>
> For programmable Ethernet device, it's not truce. But for FPGA, it's. When FPGA
> is produced, the device ID indicate what model it is and won't be changed
> anyway, even being reprogrammed. It used some not-generic mechanism, like
> AFU id to distinguish the personalities. So, for FPGA, a prgdev driver can be used
> as primary driver to identify personalities and then register to specific devices.
Sounds like we would need an FPGA bus driver in that case. I think that
would be a better solution than having a specific device driver loading
other drivers.
/Bruce
next prev parent reply other threads:[~2017-03-07 11:12 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-02 4:03 [dpdk-dev] [PATCH 0/6] introduce prgdev abstraction library Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 1/6] prgdev: introduce new library Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 2/6] prgdev: add debug macro for prgdev Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 3/6] prgdev: add bus probe and remove functions Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 4/6] prgdev: add prgdev API exposed to application Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 5/6] prgdev: add ABI control info Chen Jing D(Mark)
2017-03-02 4:03 ` [dpdk-dev] [PATCH 6/6] doc: introduction to prgdev Chen Jing D(Mark)
2017-03-06 15:26 ` Thomas Monjalon
2017-03-07 10:34 ` Chen, Jing D
2017-03-07 11:12 ` Bruce Richardson [this message]
2017-03-07 13:05 ` Thomas Monjalon
2017-03-07 13:45 ` Chen, Jing D
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170307111231.GA254436@bricha3-MOBL3.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=cunming.liang@intel.com \
--cc=dev@dpdk.org \
--cc=gerald.rogers@intel.com \
--cc=jing.d.chen@intel.com \
--cc=john.mcnamara@intel.com \
--cc=keith.wiles@intel.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).