Re: [dpdk-dev] Why do we need iommu=pt?

[alex <alex@weka.io>]

Shiva.
The cost of disabling iommu=pt when intel_iommu=on is dire. DPDK won't work
as the RX/TX descriptors will be useless.
Any dam access by the device will be dropped as no dam-mapping will exists.

Danny.
The IOMMU hurts performance in kernel drivers which perform a map and umap
operation for each e/ingress packet.
The costs of unmapping when under strict protection limit a +10Gb to 3Gb
with cpu maxed out at 100%. DPDK apps shouldn't feel any difference IFF the
rx descriptors contain iova and not real physical addresses which are used
currently.


On Tue, Oct 21, 2014 at 10:10 PM, Zhou, Danny <danny.zhou@intel.com> wrote:

> IMHO, if memory protection with IOMMU is needed or not really depends on
> how you use
> and deploy your DPDK based applications. For Telco network middle boxes,
> which adopts
> a "close model" solution to achieve extremely high performance, the entire
> system including
> HW, software in kernel and userspace are controlled by Telco vendors and
> assumed trustable, so
> memory protection is not so important. While for Datacenters, which
> generally adopts a "open model"
> solution allows running user space applications(e.g. tenant applications
> and VMs) which could
> direct access NIC and DMA engine inside the NIC using modified DPDK PMD
> are not trustable
> as they can potentially DAM to/from arbitrary memory regions using
> physical addresses, so IOMMU
> is needed to provide strict memory protection, at the cost of negative
> performance impact.
>
> So if you want to seek high performance, disable IOMMU in BIOS or OS. And
> if security is a major
> concern, tune it on and tradeoff between performance and security. But I
> do NOT think is comes with
> an extremely high performance costs according to our performance
> measurement, but it probably true
> for 100G NIC.
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shivapriya Hiremath
> > Sent: Wednesday, October 22, 2014 12:54 AM
> > To: Alex Markuze
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Why do we need iommu=pt?
> >
> > Hi,
> >
> > Thank you for all the replies.
> > I am trying to understand the impact of this on DPDK. What will be the
> > repercussions of disabling "iommu=pt" on the DPDK performance?
> >
> >
> > On Tue, Oct 21, 2014 at 12:32 AM, Alex Markuze <alex@weka.io> wrote:
> >
> > > DPDK uses a 1:1 mapping and doesn't support IOMMU.  IOMMU allows for
> > > simpler VM physical address translation.
> > > The second role of IOMMU is to allow protection from unwanted memory
> > > access by an unsafe devise that has DMA privileges. Unfortunately this
> > > protection comes with an extremely high performance costs for high
> speed
> > > nics.
> > >
> > > To your question iommu=pt disables IOMMU support for the hypervisor.
> > >
> > > On Tue, Oct 21, 2014 at 1:39 AM, Xie, Huawei <huawei.xie@intel.com>
> wrote:
> > >
> > >>
> > >>
> > >> > -----Original Message-----
> > >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Shivapriya
> > >> Hiremath
> > >> > Sent: Monday, October 20, 2014 2:59 PM
> > >> > To: dev@dpdk.org
> > >> > Subject: [dpdk-dev] Why do we need iommu=pt?
> > >> >
> > >> > Hi,
> > >> >
> > >> > My question is that if the Poll mode  driver used the DMA kernel
> > >> interface
> > >> > to set up its mappings appropriately, would it still require that
> > >> iommu=pt
> > >> > be set?
> > >> > What is the purpose of setting iommu=pt ?
> > >> PMD allocates memory though hugetlb file system, and fills the
> physical
> > >> address
> > >> into the descriptor.
> > >> pt is used to pass through iotlb translation. Refer to the below link.
> > >> http://lkml.iu.edu/hypermail/linux/kernel/0906.2/02129.html
> > >> >
> > >> > Thank you.
> > >>
> > >
> > >
>