Re: [dpdk-dev] [PATCH v3 2/4] nfp-uio: new uio driver for netronome nfp6000 card

[Alejandro Lucero <alejandro.lucero@netronome.com>]

Hi David,

On Wed, Oct 21, 2015 at 6:24 AM, David Marchand <david.marchand@6wind.com>
wrote:

> Hello Alejandro,
>
> On Fri, Oct 16, 2015 at 12:45 PM, Alejandro.Lucero <
> alejandro.lucero@netronome.com> wrote:
>
>> From: "Alejandro.Lucero" <alejandro.lucero@netronome.com>
>>
>> This patch adds a new UIO kernel driver for supporting PCI VFs with
>> Netronome
>> nfp6000 card. Future PCI PF support will be based on changes to this
>> module.
>>
>> Signed-off-by: Alejandro.Lucero <alejandro.lucero@netronome.com>
>> Signed-off-by: Rolf.Neugebauer <rolf.neugebauer@netronome.com>
>>
>
> Please, can you elaborate on the need for (yet another) uio driver, rather
> than make igb_uio work with your hardware ?
>
>
We have two different needs not covered by igb_uio:

  - pci mask needs to be 40 bits instead of 64
  - some old kernel have buggy IOMMU with devices with pci masks as those
40 bits. This can be fixed just using kernel DMA API for a harmless dma
memory allocation. We have such a allocation along with the release in our
nfp_uio driver.

I thought to modify igb_uio adding a kernel param for setting another pci
mask than igb default one, but this is to leave to the user the right
initialisation.

But there is another more important reason for having a different driver:
future PF support. Our device is programmable with processing units and
functional units being configured by firmware code. Every part of the chip
is addressable from the host which is possible through a BSP kernel driver.
Device PCI BARs are not the same than network device (PF or VF) PCI BARs:
the latter is a subregion of the former. VFs subregions are easy to get
thanks to SRIOV. But in order to get the right subregions sizes and
addresses for the PF, special firmware symbols are needed. User space
programs can use the BSP through specific libraries but that requires
support from the BSP, or from another driver doing the same. So nfp_uio
will likely add that support.

I have been looking at the possibility of getting rid of nfp_uio. The fact
is our PMD can work without it, both for the PF and VF (not the PMD version
already submitted but one under development).  The PF support requires not
using UIO at all, because the device is attached to the BSP driver. The
only problem with this approach is we do not have support for interrupts,
what is not critical (I can see other PMDs not having support for Link
Status Changes) but we do not like it as programs can register callbacks
for these interrupts which would not work at all.

Interrupt support could be implemented in the BSP, doing the same UIO or
VFIO do, but this will require (minor) changes to DPDK for having another
intr_handle (not UIO, not VFIO). I do not know if other PMDs could also
make use of such a change but I guess that would help to accept those
changes.

Therefore, the easiest thing to do is to implement the PF support in the
nfp_uio by now.

I hope this explanation can help to justify nfp_uio.

Thanks


> Thanks.
>
> --
> David Marchand
>