From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f169.google.com (mail-lb0-f169.google.com [209.85.217.169]) by dpdk.org (Postfix) with ESMTP id 56B6C7EC4 for ; Wed, 5 Nov 2014 16:05:34 +0100 (CET) Received: by mail-lb0-f169.google.com with SMTP id 10so835177lbg.0 for ; Wed, 05 Nov 2014 07:14:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=hd/YAYqyBZ3Z/jCbXnTl6e2mrrk+Vaxqp8c2c5Tvs2c=; b=bVoE6NBwlDRQ9ulGYSoDWsovaQLZBuKkFg+3pqM9cmQwi2vbOWIrXcOBCNB+evJDt3 rls/NjjO/5edH64dBl6jjuX+SG6RMGhj4QKpw/u8jXmUyMHmtaXgJP+tbghW5+pp3iw2 dU0i/Z27LH8CU7qF2WO1hoAvEbWN3zBx6iJALke/mowqlru+xD80eAVOhZ2VSrTj9sIr Pq2dEo2HJG0uD7R5ly5Ug3Qx9DSfOhBtytForEb6MoHYIW9k8IKFJgM6e3o4wiqQ+bWX cZ8QZ6FddzX9iqjeCqVjM+weg0aruKare0xE3Hj7MaJ9QES7Dk+AgO7tPUEEIh7iGyvg Nm4g== X-Gm-Message-State: ALoCoQk+ZU+dLUCgpEUyrt2rIbArBw4eOBYQtaA5MaULY/z4Xa/X7qqzgUz5cKG1XkhYLv+HIsEj MIME-Version: 1.0 X-Received: by 10.112.199.40 with SMTP id jh8mr62823667lbc.5.1415200497403; Wed, 05 Nov 2014 07:14:57 -0800 (PST) Received: by 10.25.215.157 with HTTP; Wed, 5 Nov 2014 07:14:57 -0800 (PST) In-Reply-To: <4163036.7ZLDrAu3iM@xps13> References: <26FA93C7ED1EAA44AB77D62FBE1D27BA54C42C29@IRSMSX102.ger.corp.intel.com> <26FA93C7ED1EAA44AB77D62FBE1D27BA54C4620C@IRSMSX102.ger.corp.intel.com> <26FA93C7ED1EAA44AB77D62FBE1D27BA54C46444@IRSMSX102.ger.corp.intel.com> <4163036.7ZLDrAu3iM@xps13> Date: Wed, 5 Nov 2014 17:14:57 +0200 Message-ID: From: Alex Markuze To: Thomas Monjalon Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" , john.r.fastabend@intel.com Subject: Re: [dpdk-dev] bifurcated driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2014 15:05:34 -0000 On Wed, Nov 5, 2014 at 3:00 PM, Thomas Monjalon wrote: > Hi Danny, > > 2014-10-31 17:36, O'driscoll, Tim: > > Bifurcated Driver (Danny.Zhou@intel.com) > > Thanks for the presentation of bifurcated driver during the community call. > I asked if you looked at ibverbs and you wanted a link to check. > The kernel module is here: > > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core > The userspace library: > http://git.kernel.org/cgit/libs/infiniband/libibverbs.git > > Extract from Kconfig: > " > config INFINIBAND_USER_ACCESS > tristate "InfiniBand userspace access (verbs and CM)" > select ANON_INODES > ---help--- > Userspace InfiniBand access support. This enables the > kernel side of userspace verbs and the userspace > communication manager (CM). This allows userspace processes > to set up connections and directly access InfiniBand > hardware for fast-path operations. You will also need > libibverbs, libibcm and a hardware driver library from > . > " > > It seems to be close to the bifurcated driver needs. > Not sure if it can solve the security issues if there is no dedicated MMU > in the NIC. > Mellanox NIC's and other RDMA HW (Infiniband/RoCE/iWARP) have MTT units - memory translation units - a dedicated MMU. These are filled via an ibv_reg_mr sys calls - this creates a Process VM to physical/iova memory mapping in the NIC. Thus each process can access only its own memory via the NIC. This is the way RNICs resolve the security issue I'm not sure how standard intel nics could support this scheme. There is already a 6wind PMD for mellanox Nics. I'm assuming this PMD is verbs based and behaves similar to the bifurcated driver proposed. http://www.mellanox.com/page/press_release_item?id=979 One, thing that I don't understand (And will be happy if some one could shed some light on), is how does the NIC supposed do distinguish between packets that need to go to the kernel driver rings and packets going to user space rings. I feel we should sum up pros and cons of > - igb_uio > - uio_pci_generic > - VFIO > - ibverbs > - bifurcated driver > I suggest to consider these criterias: > - upstream status > - usable with kernel netdev > - usable in a vm > - usable for ethernet > - hardware requirements > - security protection > - performance > > Regarding obverts - I'm not sure how its relevant to future DPDK development , but this is the run down as I know It. This is a veteran package called OFED , or its counterpart Mellanox OFED. ---- The kernel drivers are upstream ---- The PCI dev stays in the kernels care trough out its life span ---- SRIOV support exists, paravirt support exists only(AFAIK) as an Office of the CTO(VMware) project called vRDMA ---- Eth/RoCE (RDMA over Converged Ethernet)/IB === HW === RDMA capable HW ONLY. ---- Security is designed into RDMA HW ---- Stellar performance - Favored by HPC. > -- > Thomas >