From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id D33638D9F for ; Thu, 1 Oct 2015 18:04:33 +0200 (CEST) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 349C2C0B1B4D; Thu, 1 Oct 2015 16:04:33 +0000 (UTC) Received: from redhat.com (ovpn-116-83.ams2.redhat.com [10.36.116.83]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id t91G4TSD027956; Thu, 1 Oct 2015 12:04:30 -0400 Date: Thu, 1 Oct 2015 19:04:29 +0300 From: "Michael S. Tsirkin" To: Bruce Richardson Message-ID: <20151001190248-mutt-send-email-mst@redhat.com> References: <560CF44A.60102@scylladb.com> <20151001120027-mutt-send-email-mst@redhat.com> <560CFB66.5050904@scylladb.com> <20151001124211-mutt-send-email-mst@redhat.com> <560D0413.5080401@scylladb.com> <20151001131754-mutt-send-email-mst@redhat.com> <20151001110806.GA16248@bricha3-MOBL3> <20151001141124-mutt-send-email-mst@redhat.com> <20151001120713.GA11504@bricha3-MOBL3> <20151001155943-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151001155943-mutt-send-email-mst@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: "dev@dpdk.org" , Avi Kivity Subject: Re: [dpdk-dev] Having troubles binding an SR-IOV VF to uio_pci_generic on Amazon instance X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2015 16:04:34 -0000 On Thu, Oct 01, 2015 at 04:14:33PM +0300, Michael S. Tsirkin wrote: > On Thu, Oct 01, 2015 at 01:07:13PM +0100, Bruce Richardson wrote: > > > > This in itself is going to use up > > > > a good proportion of the processing time, as well as that we have to spend cycles > > > > copying the descriptors from one ring in memory to another. Given that right now > > > > with the vector ixgbe driver, the cycle cost per packet of RX is just a few dozen > > > > cycles on modern cores, every additional cycle (fraction of a nanosecond) has > > > > an impact. > > > > > > > > Regards, > > > > /Bruce > > > > > > See above. There is no need for that on data path. Only re-adding > > > buffers requires a system call. > > > > > > > Re-adding buffers is a key part of the data path! Ok, the fact that its only on > > descriptor rearm does allow somewhat bigger batches, > > That was the point, yes. > > > but the whole point of having > > the kernel do this extra work you propose is to allow the kernel to scan and > > sanitize the physical addresses - and that will take a lot of cycles, especially > > if it has to handle all the different descriptor formats of all the different NICs, > > as has already been pointed out. > > > > /Bruce > > Well the driver would be per NIC, so there's only need to support > specific formats supported by a given NIC. > > An alternative is to format the descriptors in kernel, based > on just the list of addresses. This seems cleaner, but I don't > know how efficient it would be. Additionally. rearming descriptors can happen on another core in parallel with processing packets on the first one. This will use more CPU but help you stay within your PPS limits. -- MST