From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by dpdk.org (Postfix) with ESMTP id 853C37F85 for ; Thu, 6 Nov 2014 02:21:11 +0100 (CET) Received: by mail-wi0-f182.google.com with SMTP id d1so110056wiv.15 for ; Wed, 05 Nov 2014 17:30:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:organization:user-agent :mime-version:to:cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=A0LG/uegkLvTSzhz5HWUE3+EdSO5mqiBvGhJ5Cq4G68=; b=FP6d2m+bWyGyftNfdv4Sp+g76FyjnZuXRlPP+QbZEIDStJs7TTNetZiGws2rkfnoaO CzD5JyLwICnm7i+9KvMA74XAKw90pZZ8dHNNMa6QDp7iGoZr5xzbtBFFEnND4MI+VbOj pEMIYIhgQzXZfgT5u7GgDB6UUXDFo6NlxyiacWU/lMz14vUabvboXSceQpgeyXuKYxrs 1eWae8GmuciCPMgylMTjrlOlCkSU3CxfM7Ud3MPW4jwp9CCZwKpkEDRvoQTZC0PMqz5O tx7wrFaHISfzm2xj8fWm2MKHDLxUxQy8aWitZmWDMmr5Ktsg2YANcpUtiAFn/jhOTkhJ 2ohg== X-Gm-Message-State: ALoCoQkUGAyCSE1DAVN6XxrXBNsRgkIfalK6H14Ebg3a86Nz58iczcOFrNJ7WJfVP62A4EYwtkXH X-Received: by 10.180.107.1 with SMTP id gy1mr10166877wib.8.1415237436806; Wed, 05 Nov 2014 17:30:36 -0800 (PST) Received: from saturne.dev.6wind.com ([2a01:e35:8a8e:1b70:226:b9ff:fec7:e133]) by mx.google.com with ESMTPSA id jw2sm9441954wid.3.2014.11.05.17.30.35 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 05 Nov 2014 17:30:36 -0800 (PST) Message-ID: <545ACF39.5000507@6wind.com> Date: Thu, 06 Nov 2014 02:30:33 +0100 From: Vincent JARDIN Organization: www.6wind.com User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7 MIME-Version: 1.0 To: "Zhou, Danny" References: <26FA93C7ED1EAA44AB77D62FBE1D27BA54C42C29@IRSMSX102.ger.corp.intel.com> <26FA93C7ED1EAA44AB77D62FBE1D27BA54C4620C@IRSMSX102.ger.corp.intel.com> <26FA93C7ED1EAA44AB77D62FBE1D27BA54C46444@IRSMSX102.ger.corp.intel.com> <4163036.7ZLDrAu3iM@xps13> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" , "Fastabend, John R" , Or Gerlitz Subject: Re: [dpdk-dev] bifurcated driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 01:21:11 -0000 +Or On 05/11/2014 23:48, Zhou, Danny wrote: > Hi Thomas, > > Thanks for sharing the links to ibverbs, I will take a close look at it and compare it to bifurcated driver. My take > after a rough review is that idea is very much similar, but bifurcated driver implementation is generic for any > Ethernet device based on existing af_packet mechanism, with extension of exchanging the messages between > user space and kernel space driver. > > I have an internal document to summary the pros and cons of below solutions, except for ibvers, but > will be adding it shortly. > > - igb_uio > - uio_pci_generic > - VFIO > - bifurcated driver > > Short answers to your questions: >> - upstream status > Adding IOMMU based memory protection and generic descriptor description support now, into version 2 > kernel patches. > >> - usable with kernel netdev > af_packet based, and relevant patchset will be submitted to netdev for sure. > >> - usable in a vm > No, it does no coexist with SRIOV for number of reasons. but if you pass-through a PF to a VM, it works perfect. > >> - usable for Ethernet > It could work with all Ethernet NICs, as flow director is available and NIC driver support new net_ops to split off > queue pairs for user space. > >> - hardware requirements > No specific hardware requirements. All mainstream NICs have multiple qpairs and flow director support. > >> - security protection > Leverage IOMMU to provide memory protection on Intel platform. Other archs provide similar memory protection > mechanism, so we only use arch-agnostic DMA memory allocation APIs in kernel to support memory protection. > >> - performance > DPDK native performance on user space queues, as long as drop_en is enabled to avoid head-of-line blocking. > > -Danny > >> -----Original Message----- >> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] >> Sent: Wednesday, November 05, 2014 9:01 PM >> To: Zhou, Danny >> Cc: dev@dpdk.org; Fastabend, John R >> Subject: Re: [dpdk-dev] bifurcated driver >> >> Hi Danny, >> >> 2014-10-31 17:36, O'driscoll, Tim: >>> Bifurcated Driver (Danny.Zhou@intel.com) >> >> Thanks for the presentation of bifurcated driver during the community call. >> I asked if you looked at ibverbs and you wanted a link to check. >> The kernel module is here: >> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core >> The userspace library: >> http://git.kernel.org/cgit/libs/infiniband/libibverbs.git >> >> Extract from Kconfig: >> " >> config INFINIBAND_USER_ACCESS >> tristate "InfiniBand userspace access (verbs and CM)" >> select ANON_INODES >> ---help--- >> Userspace InfiniBand access support. This enables the >> kernel side of userspace verbs and the userspace >> communication manager (CM). This allows userspace processes >> to set up connections and directly access InfiniBand >> hardware for fast-path operations. You will also need >> libibverbs, libibcm and a hardware driver library from >> . >> " >> >> It seems to be close to the bifurcated driver needs. >> Not sure if it can solve the security issues if there is no dedicated MMU >> in the NIC. >> >> I feel we should sum up pros and cons of >> - igb_uio >> - uio_pci_generic >> - VFIO >> - ibverbs >> - bifurcated driver >> I suggest to consider these criterias: >> - upstream status >> - usable with kernel netdev >> - usable in a vm >> - usable for ethernet >> - hardware requirements >> - security protection >> - performance >> >> -- >> Thomas