From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D810F5A53 for ; Mon, 15 Jun 2015 15:54:23 +0200 (CEST) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga102.fm.intel.com with ESMTP; 15 Jun 2015 06:54:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,618,1427785200"; d="scan'208";a="588093052" Received: from irsmsx102.ger.corp.intel.com ([163.33.3.155]) by orsmga003.jf.intel.com with ESMTP; 15 Jun 2015 06:54:21 -0700 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.73]) by IRSMSX102.ger.corp.intel.com ([169.254.2.36]) with mapi id 14.03.0224.002; Mon, 15 Jun 2015 14:54:20 +0100 From: "Ananyev, Konstantin" To: "Richardson, Bruce" , Olivier MATZ Thread-Topic: [dpdk-dev] rte_mbuf.next in 2nd cacheline Thread-Index: AQHQo8caRiJzxo5OE0esWvWQH/eOLp2tg74AgAAGpYCAABHI0A== Date: Mon, 15 Jun 2015 13:54:19 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725836A0A838@irsmsx105.ger.corp.intel.com> References: <87110795-201A-4A1E-A4CC-A778AA7C8218@cisco.com> <557ED116.7040508@6wind.com> <20150615134409.GA7500@bricha3-MOBL3> In-Reply-To: <20150615134409.GA7500@bricha3-MOBL3> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" , "Damjan Marion \(damarion\)" Subject: Re: [dpdk-dev] rte_mbuf.next in 2nd cacheline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 13:54:24 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > Sent: Monday, June 15, 2015 2:44 PM > To: Olivier MATZ > Cc: dev@dpdk.org; Damjan Marion (damarion) > Subject: Re: [dpdk-dev] rte_mbuf.next in 2nd cacheline >=20 > On Mon, Jun 15, 2015 at 03:20:22PM +0200, Olivier MATZ wrote: > > Hi Damjan, > > > > On 06/10/2015 11:47 PM, Damjan Marion (damarion) wrote: > > > > > > Hi, > > > > > > We noticed 7% performance improvement by simply moving rte_mbuf.next = field to the 1st cache line. > > > > > > Currently, it falls under /* second cache line - fields only used in = slow path or on TX */ > > > but it is actually used at several places in rx fast path. (e.g.: i40= e_rx_alloc_bufs() is setting that field to NULL). > > > > > > Is there anything we can do here (stop using next field, or move it t= o 1st cache line)? > > > > Agree, this is also something I noticed, see: > > http://dpdk.org/ml/archives/dev/2015-February/014400.html > > > > I did not have the time to do performance testing, but it's something > > I'd like to do as soon as I can. I don't see any obvious reason not to > > do it. > > > > It seems we currently just have enough room to do it (8 bytes are > > remaining in the first cache line when compiled in 64 bits). >=20 > This, to me, is the obvious reason not to do it! It prevents us from taki= ng in > any other offload fields in the RX fast-path into the mbuf. >=20 > That being said, I can see why we might want to look to move it - but fro= m the > work done in the ixgbe driver, I'd be hopeful we can get as much performa= nce with > it on the second cache line for most cases, through judicious use of pref= etching, > or otherwise. >=20 > It took a lot of work and investigation to get free space in the mbuf - e= specially > in cache line 0, and I'd like to avoid just filling the cache line up aga= in as > long as we possibly can! Yep, agree with Bruce here. Plus, with packet_type going to be 4B and vlan_tci_outer, we just don't have 8 free bytes at the first cache line any more. Konstantin >=20 > /Bruce >=20 > > > > > > Regards, > > Olivier > > > > > > > > > > Thanks, > > > > > > Damjan > > > > > > > > > > > >