From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 89C6D2716 for ; Mon, 15 Jun 2015 15:44:13 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP; 15 Jun 2015 06:44:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,618,1427785200"; d="scan'208";a="746959081" Received: from bricha3-mobl3.ger.corp.intel.com ([10.243.20.22]) by orsmga002.jf.intel.com with SMTP; 15 Jun 2015 06:44:10 -0700 Received: by (sSMTP sendmail emulation); Mon, 15 Jun 2015 14:44:09 +0025 Date: Mon, 15 Jun 2015 14:44:09 +0100 From: Bruce Richardson To: Olivier MATZ Message-ID: <20150615134409.GA7500@bricha3-MOBL3> References: <87110795-201A-4A1E-A4CC-A778AA7C8218@cisco.com> <557ED116.7040508@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <557ED116.7040508@6wind.com> Organization: Intel Shannon Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "dev@dpdk.org" , "Damjan Marion \(damarion\)" Subject: Re: [dpdk-dev] rte_mbuf.next in 2nd cacheline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 13:44:14 -0000 On Mon, Jun 15, 2015 at 03:20:22PM +0200, Olivier MATZ wrote: > Hi Damjan, > > On 06/10/2015 11:47 PM, Damjan Marion (damarion) wrote: > > > > Hi, > > > > We noticed 7% performance improvement by simply moving rte_mbuf.next field to the 1st cache line. > > > > Currently, it falls under /* second cache line - fields only used in slow path or on TX */ > > but it is actually used at several places in rx fast path. (e.g.: i40e_rx_alloc_bufs() is setting that field to NULL). > > > > Is there anything we can do here (stop using next field, or move it to 1st cache line)? > > Agree, this is also something I noticed, see: > http://dpdk.org/ml/archives/dev/2015-February/014400.html > > I did not have the time to do performance testing, but it's something > I'd like to do as soon as I can. I don't see any obvious reason not to > do it. > > It seems we currently just have enough room to do it (8 bytes are > remaining in the first cache line when compiled in 64 bits). This, to me, is the obvious reason not to do it! It prevents us from taking in any other offload fields in the RX fast-path into the mbuf. That being said, I can see why we might want to look to move it - but from the work done in the ixgbe driver, I'd be hopeful we can get as much performance with it on the second cache line for most cases, through judicious use of prefetching, or otherwise. It took a lot of work and investigation to get free space in the mbuf - especially in cache line 0, and I'd like to avoid just filling the cache line up again as long as we possibly can! /Bruce > > > Regards, > Olivier > > > > > > Thanks, > > > > Damjan > > > > > > > >