From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id B03391396 for ; Sun, 4 Oct 2015 11:07:40 +0200 (CEST) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id A927191E96; Sun, 4 Oct 2015 09:07:39 +0000 (UTC) Received: from redhat.com (ovpn-116-40.ams2.redhat.com [10.36.116.40]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id t9497api017287; Sun, 4 Oct 2015 05:07:37 -0400 Date: Sun, 4 Oct 2015 12:07:35 +0300 From: "Michael S. Tsirkin" To: Bruce Richardson Message-ID: <20151004115445-mutt-send-email-mst@redhat.com> References: <20151001124211-mutt-send-email-mst@redhat.com> <560D0413.5080401@scylladb.com> <20151001131754-mutt-send-email-mst@redhat.com> <20151001110806.GA16248@bricha3-MOBL3> <20151001141124-mutt-send-email-mst@redhat.com> <20151001120713.GA11504@bricha3-MOBL3> <20151001155943-mutt-send-email-mst@redhat.com> <560D9F60.6040907@gmail.com> <20151002165033-mutt-send-email-mst@redhat.com> <20151002140724.GA15136@bricha3-MOBL3> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151002140724.GA15136@bricha3-MOBL3> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: "dev@dpdk.org" , Avi Kivity Subject: Re: [dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Oct 2015 09:07:41 -0000 On Fri, Oct 02, 2015 at 03:07:24PM +0100, Bruce Richardson wrote: > On Fri, Oct 02, 2015 at 05:00:14PM +0300, Michael S. Tsirkin wrote: > > On Thu, Oct 01, 2015 at 02:02:24PM -0700, Alexander Duyck wrote: > > > validation and translation would add 10s if not 100s of nanoseconds to the > > > time needed to process each packet. In addition we are talking about doing > > > this in kernel space which means we wouldn't really be able to take > > > advantage of things like SSE or AVX instructions. > > > > Yes. But the nice thing is that it's rearming so it can happen on > > a separate core, in parallel with packet processing. > > It does not need to add to latency. > > > > You will burn up more CPU, but again, all this for boxes/hypervisors > > without an IOMMU. > > > > I'm sure people can come up with even better approaches, once enough > > people get it that kernel absolutely needs to be protected from > > userspace. > > > > Long term, the right thing to do is to focus on IOMMU support. This > > gives you hardware-based memory protection without need to burn up CPU > > cycles. > > > > -- > > MST > > Running it on another will have it's own problems. The main one that springs to > mind for me is the performance impact of having all those cache lines shared > between the two cores. > > /Bruce The cache line is currently invalidated by the device write packet processing core -> device -> packet processing core We are adding another stage packet processing core -> rearming core -> device -> packet processing core but the amount of sharing per core isn't increased. This is something that can be tried immediately without kernel changes. Who knows, maybe you will actually be able to push more pps this way. Further, rearming is not doing a lot besides moving bits around in memory, and it's in kernel so using very limited resources - maybe we can efficiently use an HT logical core for this task. This remains to be seen. -- MST