Re: [dpdk-dev] bifurcated driver

[Alex Markuze <alex@weka.io>]

On Wed, Nov 5, 2014 at 3:00 PM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:

> Hi Danny,
>
> 2014-10-31 17:36, O'driscoll, Tim:
> > Bifurcated Driver (Danny.Zhou@intel.com)
>
> Thanks for the presentation of bifurcated driver during the community call.
> I asked if you looked at ibverbs and you wanted a link to check.
> The kernel module is here:
>
> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core
> The userspace library:
>         http://git.kernel.org/cgit/libs/infiniband/libibverbs.git
>
> Extract from Kconfig:
> "
> config INFINIBAND_USER_ACCESS
>         tristate "InfiniBand userspace access (verbs and CM)"
>         select ANON_INODES
>         ---help---
>           Userspace InfiniBand access support.  This enables the
>           kernel side of userspace verbs and the userspace
>           communication manager (CM).  This allows userspace processes
>           to set up connections and directly access InfiniBand
>           hardware for fast-path operations.  You will also need
>           libibverbs, libibcm and a hardware driver library from
>           <http://www.openfabrics.org/git/>.
> "
>
> It seems to be close to the bifurcated driver needs.
> Not sure if it can solve the security issues if there is no dedicated MMU
> in the NIC.
>

Mellanox NIC's and other  RDMA HW (Infiniband/RoCE/iWARP) have MTT units -
memory translation units - a dedicated MMU. These are filled via an
ibv_reg_mr sys calls - this creates a Process VM to physical/iova memory
mapping in the NIC. Thus each process can access only its own memory via
the NIC. This is the way RNICs resolve the security issue I'm not sure how
standard intel nics could support this scheme.

There is already a 6wind PMD for mellanox Nics. I'm assuming this PMD is
verbs based and behaves similar to the bifurcated driver proposed.
http://www.mellanox.com/page/press_release_item?id=979

One, thing that I don't understand (And will be happy if some one could
shed some light on), is how does the NIC supposed do distinguish between
packets that need to go to the kernel driver rings and packets going to
user space rings.

I feel we should sum up pros and cons of
>         - igb_uio
>         - uio_pci_generic
>         - VFIO
>         - ibverbs
>         - bifurcated driver
> I suggest to consider these criterias:
>         - upstream status
>         - usable with kernel netdev
>         - usable in a vm
>         - usable for ethernet
>         - hardware requirements
>         - security protection
>         - performance
>
> Regarding obverts - I'm not sure how its relevant to future DPDK
development , but this is the run down as I know It.
 This is a veteran package called OFED , or its counterpart Mellanox OFED.
   ---- The kernel drivers are upstream
   ---- The PCI dev stays in the kernels care trough out its life span
   ---- SRIOV support exists, paravirt support exists only(AFAIK) as an
Office of the CTO(VMware) project called vRDMA
   ---- Eth/RoCE (RDMA over Converged Ethernet)/IB
   === HW === RDMA capable HW ONLY.
   ---- Security is designed into RDMA HW
   ---- Stellar performance - Favored by HPC.



> --
> Thomas
>