Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management

[Alejandro Lucero <alejandro.lucero@netronome.com>]

On Thu, Jan 19, 2017 at 4:36 PM, Ferruh Yigit <ferruh.yigit@intel.com>
wrote:

> On 1/13/2017 11:10 AM, Alejandro Lucero wrote:
> > I completely misread the patch, and I wrongly thought that code was
> > linked to module removal, but I see this is not about that, but about
> > releasing the /dev/uio file calling release function, what is done by
> > the kernel when the process exits.
> >
> > So yes, the patch avoids the problem I talked about.
> >
> > However, calling that specific ixgbe driver function will break other
> > devices relying on igb_uio. What about implementing a notifier chain for
> > this? The igb_uio code would be agnostic and each interested driver,
> > like ixgbe or nfp_net, could execute the specific port close code when
> > the notifier chain triggers.
>
> Can you please elaborate, how can this be done?
> This is kernel code, and interested drivers are in userspace. This patch
> benefit from Linux kernel driver of same device.
>
>
The patch seems to add  a function to the ixgve_vf kernel driver which will
be called from igb_uio. The patch does not show the usual path to the
kernel ethernet drivers because, I guess, it is a patch for a specific
software build. So the interested drivers are not in user space, and a
driver, igb_uio, calling a userspace function? No way.

Any netdev driver interested to close ports, those left working by a
crashed DPDK app, could register a notifier like this:


int device_notifier(struct notifier_block *nb, unsigned long action, void
*data) {

...
    if (action == NETDEV_GOING_DOWN) {

...
}


struct notifier_block device_nb = {
   .notifier_call =device_notifier,
};

...

register_netdev_notifer(&device_nb);

...


Inside igb_uio, something like this:

raw_notifier_call_chain(&netdev_chain, val, dev);

which would trigger the invocation of all registered notifier functions.

Please, note that this is just a possibility, and the notifier chain to use
could not be the best one, same for the event NETDEV_GOING_DOWN.

Also, how can the drivers know if that event is for them?, This is
something to work on.

Because UIO is part of the kernel, maybe it could be pushed a specific
chain for this purpose if we can not use any currently available.


> >
> >
> > On Fri, Jan 13, 2017 at 5:33 AM, Tan, Jianfeng <jianfeng.tan@intel.com
> > <mailto:jianfeng.tan@intel.com>> wrote:
> >
> >
> >
> >     > -----Original Message-----
> >     > From: Yigit, Ferruh
> >     > Sent: Friday, January 13, 2017 10:05 AM
> >     > To: Tan, Jianfeng; Alejandro Lucero
> >     > Cc: Gregory Etelson; dev; users@dpdk.org <mailto:users@dpdk.org>
> >     > Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources
> Management
> >     >
> >     > On 1/13/2017 1:51 AM, Tan, Jianfeng wrote:
> >     > >
> >     > >
> >     > >> -----Original Message-----
> >     > >> From: users [mailto:users-bounces@dpdk.org
> >     <mailto:users-bounces@dpdk.org>] On Behalf Of Ferruh Yigit
> >     > >> Sent: Thursday, January 12, 2017 8:22 PM
> >     > >> To: Alejandro Lucero
> >     > >> Cc: Gregory Etelson; dev; users@dpdk.org <mailto:users@dpdk.org
> >
> >     > >> Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources
> >     > Management
> >     > >>
> >     > >> On 1/12/2017 12:12 PM, Alejandro Lucero wrote:
> >     > >>>
> >     > >>>
> >     > >>> On Thu, Jan 12, 2017 at 11:55 AM, Ferruh Yigit
> >     <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>
> >     > >>> <mailto:ferruh.yigit@intel.com
> >     <mailto:ferruh.yigit@intel.com>>> wrote:
> >     > >>>
> >     > >>>     On 12/9/2016 8:54 AM, Gregory Etelson wrote:
> >     > >>>     > Hello,
> >     > >>>     >
> >     > >>>     > IGB_UIO driver does not close port PCI activities after
> >     DPDK process
> >     > >> exits.
> >     > >>>     > DPDK API provides rte_eth_dev_close() to manage port PCI,
> >     > >>>     > but it can be skipped if process receives SIGKILL signal
> >     > >>>
> >     > >>>     I guess I understand the problem.
> >     > >>>
> >     > >>>
> >     > >>> This is a known problem, but it is not just a UIO problem, and
> >     this
> >     > >>> patch does not solve it, maybe it just solves part of it.
> >     > >>>
> >     > >>> In fact, a DPDK program crashing could imply the NIC DMAing
> >     after that
> >     > >>> and after that memory was assigned to another program.
> >     > >>
> >     > >> Yes.
> >     > >> Can there be a way to stop NIC DMA, (or prevent it access to mem
> >     > >> anymore) when app crashes?
> >     > >> I think that is what this patch is looking for.
> >     > >
> >     > > If I understand it correctly, you are looking for this patch?
> >     > > http://dpdk.org/dev/patchwork/patch/17495/
> >     <http://dpdk.org/dev/patchwork/patch/17495/>
> >     > >
> >     >
> >     > That is good, thanks Jianfeng, I will check it.
> >     >
> >     > btw, patch's current state is rejected, which is by mistake, it
> >     seems I
> >     > confused it with "iomem and ioport mapping" patch, sorry about it,
> I
> >     > will update its status immediately.
> >
> >     No problem at all. This patch is rejected as it's based on "iomem
> >     and ioport mapping" patch. As "iomem and ioport mapping" patch has
> >     backward compatibility issue, we need to figure out a way to
> >     resubmit this patch without changing the original "iomem and ioport
> >     mapping" in igb_uio.
> >
> >     Thanks,
> >     Jianfeng
> >
> >
>
>