DPDK patches and discussions
 help / color / mirror / Atom feed
From: Alexander Duyck <alexander.duyck@gmail.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Avi Kivity <avi@scylladb.com>
Subject: Re: [dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance
Date: Fri, 2 Oct 2015 09:57:18 -0700	[thread overview]
Message-ID: <560EB76E.2090301@gmail.com> (raw)
In-Reply-To: <20151002165033-mutt-send-email-mst@redhat.com>

On 10/02/2015 07:00 AM, Michael S. Tsirkin wrote:
> On Thu, Oct 01, 2015 at 02:02:24PM -0700, Alexander Duyck wrote:
>> validation and translation would add 10s if not 100s of nanoseconds to the
>> time needed to process each packet.  In addition we are talking about doing
>> this in kernel space which means we wouldn't really be able to take
>> advantage of things like SSE or AVX instructions.
> Yes. But the nice thing is that it's rearming so it can happen on
> a separate core, in parallel with packet processing.
> It does not need to add to latency.

Moving it to another core is automatically going to add extra latency.  
You will have to evict the data out of the L1 cache for one core and 
into the L1 cache for another when you update it, and then reading it 
will force it to have to transition back out.  If you are lucky it is 
only evicted to L2, if not then to L3, or possibly even back to memory.  
Odds are that alone will add tens of nanoseconds to the process, and you 
would need three or more cores to do the same workload as running the 
process over multiple threads means having to add synchronization 
primitives to the whole mess. Then there is the NUMA factor on top of that.

> You will burn up more CPU, but again, all this for boxes/hypervisors
> without an IOMMU.

There are use cases this will completely make useless.  If for example 
you are running a workload that needs three cores with DPDK bumping it 
to nine or more will likely push you out of being able to do the 
workload on some systems.

> I'm sure people can come up with even better approaches, once enough
> people get it that kernel absolutely needs to be protected from
> userspace.

I don't see that happening.  Many people don't care about kernel 
security that much.  If they did something like DPDK wouldn't have 
gotten off of the ground.  Once someone has the ability to load kernel 
modules any protection of the kernel from userspace pretty much goes 
right out the window.  You are just as much at risk from a buggy driver 
in userspace as you are from one that can be added to the kernel.

> Long term, the right thing to do is to focus on IOMMU support. This
> gives you hardware-based memory protection without need to burn up CPU
> cycles.

We have a solution that makes use of IOMMU support with vfio.  The 
problem is there are multiple cases where that support is either not 
available, or using the IOMMU provides excess overhead.

- Alex

  parent reply	other threads:[~2015-10-02 16:57 UTC|newest]

Thread overview: 100+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-27  7:05 Vlad Zolotarov
2015-09-27  9:43 ` Michael S. Tsirkin
2015-09-27 10:50   ` Vladislav Zolotarov
2015-09-29 16:41   ` Vlad Zolotarov
2015-09-29 20:54     ` Michael S. Tsirkin
2015-09-29 21:46       ` Stephen Hemminger
2015-09-29 21:49         ` Michael S. Tsirkin
2015-09-30 10:37           ` Vlad Zolotarov
2015-09-30 10:58             ` Michael S. Tsirkin
2015-09-30 11:26               ` Vlad Zolotarov
     [not found]                 ` <20150930143927-mutt-send-email-mst@redhat.com>
2015-09-30 11:53                   ` Vlad Zolotarov
2015-09-30 12:03                     ` Michael S. Tsirkin
2015-09-30 12:16                       ` Vlad Zolotarov
2015-09-30 12:27                         ` Michael S. Tsirkin
2015-09-30 12:50                           ` Vlad Zolotarov
2015-09-30 15:26                             ` Michael S. Tsirkin
2015-09-30 18:15                               ` Vlad Zolotarov
2015-09-30 18:55                                 ` Michael S. Tsirkin
2015-09-30 19:06                                   ` Vlad Zolotarov
2015-09-30 19:10                                     ` Vlad Zolotarov
2015-09-30 19:11                                       ` Vlad Zolotarov
2015-09-30 19:39                                     ` Michael S. Tsirkin
2015-09-30 20:09                                       ` Vlad Zolotarov
2015-09-30 21:36                                         ` Stephen Hemminger
2015-09-30 21:53                                           ` Michael S. Tsirkin
2015-09-30 22:20                                           ` Vlad Zolotarov
2015-10-01  8:00                                           ` Vlad Zolotarov
2015-10-01 14:47                                             ` Stephen Hemminger
2015-10-01 15:03                                               ` Vlad Zolotarov
2015-09-30 13:05                           ` Avi Kivity
2015-09-30 14:39                             ` Michael S. Tsirkin
2015-09-30 14:53                               ` Avi Kivity
2015-09-30 15:21                                 ` Michael S. Tsirkin
2015-09-30 15:36                                   ` Avi Kivity
2015-09-30 20:40                                     ` Michael S. Tsirkin
2015-09-30 21:00                                       ` Avi Kivity
2015-10-01  8:44                                       ` Michael S. Tsirkin
2015-10-01  8:46                                         ` Vlad Zolotarov
2015-10-01  8:52                                         ` Avi Kivity
2015-10-01  9:15                                           ` Michael S. Tsirkin
2015-10-01  9:22                                             ` Avi Kivity
2015-10-01  9:42                                               ` Michael S. Tsirkin
2015-10-01  9:53                                                 ` Avi Kivity
2015-10-01 10:17                                                   ` Michael S. Tsirkin
2015-10-01 10:24                                                     ` Avi Kivity
2015-10-01 10:25                                                       ` Avi Kivity
2015-10-01 10:44                                                         ` Michael S. Tsirkin
2015-10-01 10:55                                                           ` Avi Kivity
2015-10-01 21:17                                                 ` Alexander Duyck
2015-10-02 13:50                                                   ` Michael S. Tsirkin
2015-10-01  9:42                                               ` Vincent JARDIN
2015-10-01  9:43                                                 ` Avi Kivity
2015-10-01  9:48                                                   ` Vincent JARDIN
2015-10-01  9:54                                                     ` Avi Kivity
2015-10-01 10:14                                                   ` Michael S. Tsirkin
2015-10-01 10:23                                                     ` Avi Kivity
2015-10-01 14:55                                                     ` Stephen Hemminger
2015-10-01 15:49                                                       ` Michael S. Tsirkin
2015-10-01 14:54                                                 ` Stephen Hemminger
2015-10-01  9:55                                               ` Michael S. Tsirkin
2015-10-01  9:59                                                 ` Avi Kivity
2015-10-01 10:38                                                   ` Michael S. Tsirkin
2015-10-01 10:50                                                     ` Avi Kivity
2015-10-01 11:09                                                       ` Michael S. Tsirkin
2015-10-01 11:20                                                         ` Avi Kivity
2015-10-01 11:27                                                           ` Michael S. Tsirkin
2015-10-01 11:32                                                             ` Avi Kivity
2015-10-01 15:01                                                               ` Stephen Hemminger
2015-10-01 15:08                                                                 ` Avi Kivity
2015-10-01 15:46                                                                 ` Michael S. Tsirkin
2015-10-01 15:11                                                               ` Michael S. Tsirkin
2015-10-01 15:19                                                                 ` Avi Kivity
2015-10-01 15:40                                                                   ` Michael S. Tsirkin
2015-10-01 11:31                                                           ` Michael S. Tsirkin
2015-10-01 11:34                                                             ` Avi Kivity
2015-10-01 11:08                                                     ` Bruce Richardson
2015-10-01 11:23                                                       ` Michael S. Tsirkin
2015-10-01 12:07                                                         ` Bruce Richardson
2015-10-01 13:14                                                           ` Michael S. Tsirkin
2015-10-01 16:04                                                             ` Michael S. Tsirkin
2015-10-01 21:02                                                             ` Alexander Duyck
2015-10-02 14:00                                                               ` Michael S. Tsirkin
2015-10-02 14:07                                                                 ` Bruce Richardson
2015-10-04  9:07                                                                   ` Michael S. Tsirkin
2015-10-02 15:56                                                                 ` Gleb Natapov
2015-10-02 16:57                                                                 ` Alexander Duyck [this message]
2015-10-01  9:15                                           ` Avi Kivity
2015-10-01  9:29                                             ` Michael S. Tsirkin
2015-10-01  9:38                                               ` Avi Kivity
2015-10-01 10:07                                                 ` Michael S. Tsirkin
2015-10-01 10:11                                                   ` Avi Kivity
2015-10-01  9:16                                         ` Michael S. Tsirkin
2015-09-30 17:28             ` Stephen Hemminger
2015-09-30 17:39               ` Michael S. Tsirkin
2015-09-30 17:43                 ` Stephen Hemminger
2015-09-30 18:50                   ` Michael S. Tsirkin
2015-09-30 20:00                     ` Gleb Natapov
2015-09-30 20:36                       ` Michael S. Tsirkin
2015-10-01  5:04                         ` Gleb Natapov
2015-09-30 17:44                 ` Gleb Natapov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=560EB76E.2090301@gmail.com \
    --to=alexander.duyck@gmail.com \
    --cc=avi@scylladb.com \
    --cc=dev@dpdk.org \
    --cc=mst@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).