From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f169.google.com (mail-wi0-f169.google.com [209.85.212.169]) by dpdk.org (Postfix) with ESMTP id 5A0B18E69 for ; Wed, 30 Sep 2015 22:00:54 +0200 (CEST) Received: by wicfx3 with SMTP id fx3so2994302wic.1 for ; Wed, 30 Sep 2015 13:00:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to; bh=gi00BMvWs09aTdiscvfTfG2eXw1yl8w72/ltxsPqcP0=; b=JdHY8vJQG9HrCIpiG4GU3w6iudwQqmMj/XQQDhhT8BRBqVGoy8BTneC2iWGARS10Rz uGGg44fb7rbryGlDKuJtb+7K7+r9tT+l7tWGek/vDCfkMseAZdXUFwY2XCYJSGbeOXwQ h1cjQhOtcCOrZMw9zLqZ9+O+GV1I9UnLZyUNUuJSB3UEDmpR3UJlytGUa8r1tRPGk6vU P4+DM+bpLlb+pa4alhsm6kUgeIXwx+Vxw8RMUf1BN6TU0LDZGSbWibf6M98VflwOmLkK adAY+cALqTJ0a1iED9LfjKHEtDpzBPBIJIrgsI39EVIILxVbME6DVxkjyNY3JOlmqa98 K7Qw== X-Gm-Message-State: ALoCoQlO2dGE1R0NVlM6KmKUwYbyQtMdpAgw4NieQAbANeIuRRP7BQxJslzj8jkvAmCjItHwcyEf X-Received: by 10.194.76.7 with SMTP id g7mr6359326wjw.44.1443643254008; Wed, 30 Sep 2015 13:00:54 -0700 (PDT) Received: from trex.cloudius-systems.com (bzq-84-111-155-225.cablep.bezeqint.net. [84.111.155.225]) by smtp.gmail.com with ESMTPSA id fx2sm31256783wib.24.2015.09.30.13.00.51 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Sep 2015 13:00:52 -0700 (PDT) Received: by trex.cloudius-systems.com (Postfix, from userid 1042) id 69F3D83F52; Wed, 30 Sep 2015 23:00:49 +0300 (IDT) Date: Wed, 30 Sep 2015 23:00:49 +0300 From: Gleb Natapov To: "Michael S. Tsirkin" Message-ID: <20150930200049.GC27881@scylladb.com> References: <20150927123914-mutt-send-email-mst@redhat.com> <560ABF25.9030300@cloudius-systems.com> <20150929235122-mutt-send-email-mst@redhat.com> <20150929144616.4e70b44c@urahara> <20150930004714-mutt-send-email-mst@redhat.com> <560BBB62.3050502@cloudius-systems.com> <20150930102807.6e681bca@urahara> <20150930203712-mutt-send-email-mst@redhat.com> <20150930104304.7a8c8e56@urahara> <20150930212553-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150930212553-mutt-send-email-mst@redhat.com> Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2015 20:00:54 -0000 On Wed, Sep 30, 2015 at 09:50:08PM +0300, Michael S. Tsirkin wrote: > On Wed, Sep 30, 2015 at 10:43:04AM -0700, Stephen Hemminger wrote: > > On Wed, 30 Sep 2015 20:39:43 +0300 > > "Michael S. Tsirkin" wrote: > > > > > On Wed, Sep 30, 2015 at 10:28:07AM -0700, Stephen Hemminger wrote: > > > > On Wed, 30 Sep 2015 13:37:22 +0300 > > > > Vlad Zolotarov wrote: > > > > > > > > > > > > > > > > > > > On 09/30/15 00:49, Michael S. Tsirkin wrote: > > > > > > On Tue, Sep 29, 2015 at 02:46:16PM -0700, Stephen Hemminger wrote: > > > > > >> On Tue, 29 Sep 2015 23:54:54 +0300 > > > > > >> "Michael S. Tsirkin" wrote: > > > > > >> > > > > > >>> On Tue, Sep 29, 2015 at 07:41:09PM +0300, Vlad Zolotarov wrote: > > > > > >>>> The security breach motivation u brought in "[RFC PATCH] uio: > > > > > >>>> uio_pci_generic: Add support for MSI interrupts" thread seems a bit weak > > > > > >>>> since one u let the userland access to the bar it may do any funny thing > > > > > >>>> using the DMA engine of the device. This kind of stuff should be prevented > > > > > >>>> using the iommu and if it's enabled then any funny tricks using MSI/MSI-X > > > > > >>>> configuration will be prevented too. > > > > > >>>> > > > > > >>>> I'm about to send the patch to main Linux mailing list. Let's continue this > > > > > >>>> discussion there. > > > > > >>>> > > > > > >>> Basically UIO shouldn't be used with devices capable of DMA. > > > > > >>> Use VFIO for that (yes, this implies an emulated or PV IOMMU). > > > > > > > > > > If there is an IOMMU in the picture there shouldn't be any problem to > > > > > use UIO with DMA capable devices. > > > > > > > > > > >>> I don't think this can change. > > > > > >> Given there is no PV IOMMU and even if there was it would be too slow for DPDK > > > > > >> use, I can't accept that. > > > > > > QEMU does allow emulating an iommu. > > > > > > > > > > Amazon's EC2 xen HV doesn't. At least today. Therefore VFIO is not an > > > > > option there. And again, it's a general issue not DPDK specific. > > > > > Today one has to develop some proprietary modules (like igb_uio) to > > > > > workaround the issue and this is lame. IMHO uio_pci_generic should > > > > > be fixed to be able to properly work within any virtualized environment > > > > > and not only with KVM. > > > > > > > > > > > > > Also VMware (bigger problem) has no IOMMU emulation. > > > > Other environments as well (Windriver, GCE) have noe IOMMU. > > > > > > Because the use-case of userspace drivers is not important enough? > > > Without an IOMMU, there's no way to have secure userspace drivers. > > > > Look at Cloudius, there is no necessity of security in guest. > > It's an interesting concept, isn't it? > It is. > So why not do what Cloudius does, and run this task code in ring 0 then, > allocating all memory in the kernel range? > Except this is not what Cloudius does. The idea of OSv is that it can run your regular userspace application, but remove unneeded level of indirection by bypassing userspace/kernelspace communication (among other things). Application still uses virtual, not directly mapped physical memory like Linux ring 0 has. You can achieve most of the benefits of kernel bypass on Linux too, but unlike OSv you need to code for it. UIO is one of those things that allows that. > You are increasing interrupt latency by a huge factor by channeling > interrupts through a scheduler. Let user install an > interrupt handler function, and be done with it. > Interrupt latency is not always hugely important. If you enter interrupt mode only when idle hundred more us on a first packet will not kill you. If interrupt latency is important then uio may be not the right solution, but then neither is vfio. -- Gleb.