From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ig0-f170.google.com (mail-ig0-f170.google.com [209.85.213.170]) by dpdk.org (Postfix) with ESMTP id 3A57B7E7B for ; Wed, 22 Oct 2014 09:27:19 +0200 (CEST) Received: by mail-ig0-f170.google.com with SMTP id hn18so424025igb.1 for ; Wed, 22 Oct 2014 00:35:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=wMG4t4RMOBsAVW0Pux+tIP90oamEQs+kihTGvMKACUo=; b=f+e/Z8n9lRBiJ3oYxEODmz56LfNRqxCPIWSsouaxlBg5K1S9K1HHoDJtTlqbuqG4e/ 7zAsBO3H6frPNiqtTbIYbZ5Afz7CTRR5RE2+Q92rqpkjwhnqUL/vrX0vc41z3G9nFdel FLBGlyPKGQbuPmSqAttbcS/MM/3f/q7kTV42UPZ8jUnhuXEURc+l8b0YZGK5s5FmDMxK LQepXQLXrDJG+LfTrS8SGaMZaaV+e14pJr97Kvq8YRjPFycOyRKpPnpKG1CkrpiOVwqz Gwz0zobXVKhE0KtMVArlQhAsKO9xnOsKaaI5ZAQ2QH7rq+bFYIaKHS0L1baisNlzg1pc CPfg== X-Gm-Message-State: ALoCoQnML+fBAdIaG0BDzc6gJtAowHQzoMbn35+nXyov/Mv4vPdyM544C+c5rCp/rhG29wi42nFL MIME-Version: 1.0 X-Received: by 10.50.143.65 with SMTP id sc1mr32933285igb.27.1413963337780; Wed, 22 Oct 2014 00:35:37 -0700 (PDT) Received: by 10.107.132.69 with HTTP; Wed, 22 Oct 2014 00:35:37 -0700 (PDT) In-Reply-To: References: Date: Wed, 22 Oct 2014 10:35:37 +0300 Message-ID: From: alex To: "Zhou, Danny" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Why do we need iommu=pt? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Oct 2014 07:27:19 -0000 Shiva. The cost of disabling iommu=pt when intel_iommu=on is dire. DPDK won't work as the RX/TX descriptors will be useless. Any dam access by the device will be dropped as no dam-mapping will exists. Danny. The IOMMU hurts performance in kernel drivers which perform a map and umap operation for each e/ingress packet. The costs of unmapping when under strict protection limit a +10Gb to 3Gb with cpu maxed out at 100%. DPDK apps shouldn't feel any difference IFF the rx descriptors contain iova and not real physical addresses which are used currently. On Tue, Oct 21, 2014 at 10:10 PM, Zhou, Danny wrote: > IMHO, if memory protection with IOMMU is needed or not really depends on > how you use > and deploy your DPDK based applications. For Telco network middle boxes, > which adopts > a "close model" solution to achieve extremely high performance, the entire > system including > HW, software in kernel and userspace are controlled by Telco vendors and > assumed trustable, so > memory protection is not so important. While for Datacenters, which > generally adopts a "open model" > solution allows running user space applications(e.g. tenant applications > and VMs) which could > direct access NIC and DMA engine inside the NIC using modified DPDK PMD > are not trustable > as they can potentially DAM to/from arbitrary memory regions using > physical addresses, so IOMMU > is needed to provide strict memory protection, at the cost of negative > performance impact. > > So if you want to seek high performance, disable IOMMU in BIOS or OS. And > if security is a major > concern, tune it on and tradeoff between performance and security. But I > do NOT think is comes with > an extremely high performance costs according to our performance > measurement, but it probably true > for 100G NIC. > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shivapriya Hiremath > > Sent: Wednesday, October 22, 2014 12:54 AM > > To: Alex Markuze > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] Why do we need iommu=pt? > > > > Hi, > > > > Thank you for all the replies. > > I am trying to understand the impact of this on DPDK. What will be the > > repercussions of disabling "iommu=pt" on the DPDK performance? > > > > > > On Tue, Oct 21, 2014 at 12:32 AM, Alex Markuze wrote: > > > > > DPDK uses a 1:1 mapping and doesn't support IOMMU. IOMMU allows for > > > simpler VM physical address translation. > > > The second role of IOMMU is to allow protection from unwanted memory > > > access by an unsafe devise that has DMA privileges. Unfortunately this > > > protection comes with an extremely high performance costs for high > speed > > > nics. > > > > > > To your question iommu=pt disables IOMMU support for the hypervisor. > > > > > > On Tue, Oct 21, 2014 at 1:39 AM, Xie, Huawei > wrote: > > > > > >> > > >> > > >> > -----Original Message----- > > >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shivapriya > > >> Hiremath > > >> > Sent: Monday, October 20, 2014 2:59 PM > > >> > To: dev@dpdk.org > > >> > Subject: [dpdk-dev] Why do we need iommu=pt? > > >> > > > >> > Hi, > > >> > > > >> > My question is that if the Poll mode driver used the DMA kernel > > >> interface > > >> > to set up its mappings appropriately, would it still require that > > >> iommu=pt > > >> > be set? > > >> > What is the purpose of setting iommu=pt ? > > >> PMD allocates memory though hugetlb file system, and fills the > physical > > >> address > > >> into the descriptor. > > >> pt is used to pass through iotlb translation. Refer to the below link. > > >> http://lkml.iu.edu/hypermail/linux/kernel/0906.2/02129.html > > >> > > > >> > Thank you. > > >> > > > > > > >