From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) by dpdk.org (Postfix) with ESMTP id F2544C3AE for ; Wed, 17 Jun 2015 16:23:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2421; q=dns/txt; s=iport; t=1434551032; x=1435760632; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=Vd6fLgC+ILyB4N+S36Tsnen52TQZnx1Fq5lLr7FHZRg=; b=YYqdOyc/LpnnSO1W/8AkhXtiigFzL4xo8IuOAqzGYJMB7b3BkW6l+VgQ YDa46HZtuy9eqbqHJdFmt1vrjMnszB9B/nbh3AZPVMsEfQWtWQO30WCe5 EcuuJ/dVyGpwMN69HzCka17SQQQ6mNgaV5LP6FeNpV+X3E3zAlEpfmkMs g=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0AJBQBogoFV/5JdJa1WBoMQgTMGvV6HXwKBNjoSAQEBAQEBAYEKhCIBAQEDATo/BQsCAQgYHhAyJQIEDgWIJwjKQwEBAQEBAQEBAQEBAQEBAQEBAQEBAReLRIRDEDMHgxeBFAWTaAGLRoE0hxOFG4o+JoIIH4FSb4FGgQIBAQE X-IronPort-AV: E=Sophos;i="5.13,633,1427760000"; d="scan'208";a="160342221" Received: from rcdn-core-10.cisco.com ([173.37.93.146]) by alln-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 17 Jun 2015 14:23:50 +0000 Received: from xhc-aln-x02.cisco.com (xhc-aln-x02.cisco.com [173.36.12.76]) by rcdn-core-10.cisco.com (8.14.5/8.14.5) with ESMTP id t5HENoZY008411 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 17 Jun 2015 14:23:50 GMT Received: from xmb-aln-x01.cisco.com ([169.254.2.74]) by xhc-aln-x02.cisco.com ([173.36.12.76]) with mapi id 14.03.0195.001; Wed, 17 Jun 2015 09:23:50 -0500 From: "Damjan Marion (damarion)" To: Bruce Richardson Thread-Topic: [dpdk-dev] rte_mbuf.next in 2nd cacheline Thread-Index: AQHQo8caZr3uhWmxzEWZx6NE1pCbq52t6FMAgAAGpoCAAALXgIAAAwKAgAACNACAAx/mgIAAAwoAgAAEwAA= Date: Wed, 17 Jun 2015 14:23:49 +0000 Message-ID: <56928EA5-A3DB-44B3-B0ED-54E6FC0AE361@cisco.com> References: <87110795-201A-4A1E-A4CC-A778AA7C8218@cisco.com> <557ED116.7040508@6wind.com> <20150615134409.GA7500@bricha3-MOBL3> <2601191342CEEE43887BDE71AB97725836A0A838@irsmsx105.ger.corp.intel.com> <557EDB91.9010503@6wind.com> <20150615141258.GA580@bricha3-MOBL3> <68EBE73B-D251-4297-BFE2-E2D2A3AEFD33@cisco.com> <20150617140648.GC8208@bricha3-MOBL3> In-Reply-To: <20150617140648.GC8208@bricha3-MOBL3> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.55.15.243] Content-Type: text/plain; charset="us-ascii" Content-ID: <80872E29B92E1B4A98BF4B3EC0944D98@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] rte_mbuf.next in 2nd cacheline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 14:23:52 -0000 > On 17 Jun 2015, at 16:06, Bruce Richardson w= rote: >=20 > On Wed, Jun 17, 2015 at 01:55:57PM +0000, Damjan Marion (damarion) wrote: >>=20 >>> On 15 Jun 2015, at 16:12, Bruce Richardson = wrote: >>>=20 >>> The next pointers always start out as NULL when the mbuf pool is create= d. The >>> only time it is set to non-NULL is when we have chained mbufs. If we ne= ver have >>> any chained mbufs, we never need to touch the next field, or even read = it - since >>> we have the num-segments count in the first cache line. If we do have a= multi-segment >>> mbuf, it's likely to be a big packet, so we have more processing time a= vailable >>> and we can then take the hit of setting the next pointer. >>=20 >> There are applications which are not using rx offload, but they deal wit= h chained mbufs. >> Why they are less important than ones using rx offload? This is somethin= g people=20 >> should be able to configure on build time. >=20 > It's not that they are less important, it's that the packet processing cy= cle count > budget is going to be greater. A packet which is 64 bytes, or 128 bytes i= n size > can make use of a number of RX offloads to reduce it's processing time. H= owever, > a 64/128 packet is not going to be split across multiple buffers [unless = we > are dealing with a very unusual setup!]. >=20 > To handle 64 byte packets at 40G line rate, one has 50 cycles per core pe= r packet > when running at 3GHz. [3000000000 cycles / 59.5 mpps]. > If we assume that we are dealing with fairly small buffers > here, and that anything greater than 1k packets are chained, we still hav= e 626 > cycles per 3GHz core per packet to work with for that 1k packet. Given th= at > "normal" DPDK buffers are 2k in size, we have over a thousand cycles per = packet > for any packet that is split.=20 >=20 > In summary, packets spread across multiple buffers are large packets, and= so have > larger packet cycle count budgets and so can much better absorb the cost = of > touching a second cache line in the mbuf than a 64-byte packet can. There= fore, > we optimize for the 64B packet case. This makes sense if there is no other work to do on the same core. Otherwise it is better to spent those cycles on actual work instead of wait= ing for=20 2nd cache line... Thanks, Damjan=