From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD8AEA04E7; Sun, 1 Nov 2020 21:23:14 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1EF581D9E; Sun, 1 Nov 2020 21:23:12 +0100 (CET) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by dpdk.org (Postfix) with ESMTP id 4F4B0160 for ; Sun, 1 Nov 2020 21:23:10 +0100 (CET) Received: from [192.168.1.70] (unknown [188.170.76.110]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 8695C7F518; Sun, 1 Nov 2020 23:23:08 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru 8695C7F518 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1604262189; bh=nUIF2yIBgFbn3xVCg0yZDiAXgBndR5UE6V19Q5HjmVc=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=jZJ57zxiNPdcRBsjc9ylheCLsyGL6UNLb+Gvmy8+KoGlnF3EL71rCHuMEpQaFjmgV A8+RntmX8SdGAVignktpNC3ojAoPC0nij4jZ0BQ9mE3pqoPrZA6pHqU2lWNrnL/YLp /oRHYmm1dWvTMO0GlMIKp9ynuKnYxCzqRuqBOm7w= To: Thomas Monjalon , dev@dpdk.org Cc: ferruh.yigit@intel.com, david.marchand@redhat.com, bruce.richardson@intel.com, olivier.matz@6wind.com, akhil.goyal@nxp.com, jerinj@marvell.com, Ray Kinsella , Neil Horman , Konstantin Ananyev References: <20201026052105.1561859-1-thomas@monjalon.net> <20201030172940.1073558-1-thomas@monjalon.net> <20201030172940.1073558-16-thomas@monjalon.net> From: Andrew Rybchenko Message-ID: Date: Sun, 1 Nov 2020 23:23:07 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.3.1 MIME-Version: 1.0 In-Reply-To: <20201030172940.1073558-16-thomas@monjalon.net> Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v5 15/15] mbuf: move pool pointer in hotter first half X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/30/20 8:29 PM, Thomas Monjalon wrote: > The mempool pointer in the mbuf struct is moved > from the second to the first half. > It should increase performance on most systems having 64-byte cache line, > i.e. mbuf is split in two cache lines. > On such system, the first half (also called first cache line) is hotter > than the second one where the pool pointer was. > > Moving this field gives more space to dynfield1. > > This is how the mbuf layout looks like (pahole-style): > > word type name byte size > 0 void * buf_addr; /* 0 + 8 */ > 1 rte_iova_t buf_iova /* 8 + 8 */ > /* --- RTE_MARKER64 rearm_data; */ > 2 uint16_t data_off; /* 16 + 2 */ > uint16_t refcnt; /* 18 + 2 */ > uint16_t nb_segs; /* 20 + 2 */ > uint16_t port; /* 22 + 2 */ > 3 uint64_t ol_flags; /* 24 + 8 */ > /* --- RTE_MARKER rx_descriptor_fields1; */ > 4 uint32_t union packet_type; /* 32 + 4 */ > uint32_t pkt_len; /* 36 + 4 */ > 5 uint16_t data_len; /* 40 + 2 */ > uint16_t vlan_tci; /* 42 + 2 */ > 5.5 uint64_t union hash; /* 44 + 8 */ > 6.5 uint16_t vlan_tci_outer; /* 52 + 2 */ > uint16_t buf_len; /* 54 + 2 */ > 7 struct rte_mempool * pool; /* 56 + 8 */ > /* --- RTE_MARKER cacheline1; */ > 8 struct rte_mbuf * next; /* 64 + 8 */ > 9 uint64_t union tx_offload; /* 72 + 8 */ > 10 uint16_t priv_size; /* 80 + 2 */ > uint16_t timesync; /* 82 + 2 */ > uint32_t seqn; /* 84 + 4 */ As I understand rebase is required since seqn is already removed (or at least fix here). > 11 struct rte_mbuf_ext_shared_info * shinfo; /* 88 + 8 */ > 12 uint64_t dynfield1[4]; /* 96 + 32 */ > 16 /* --- END 128 */ > > Signed-off-by: Thomas Monjalon Taking Konstantin reply into account ('next' is used on free in any case together with 'pool', so the second cache line is accessed in any case), I think that 'next' is a better candidate. Also 'tx_offload' is a better candidate than 'pool'. I think 'next' is better since it works for both Rx and Tx, but 'tx_offload' is Tx only.