From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 3D8D01B324 for ; Fri, 19 Jan 2018 18:31:24 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Jan 2018 09:31:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,382,1511856000"; d="scan'208";a="196970884" Received: from irsmsx107.ger.corp.intel.com ([163.33.3.99]) by fmsmga005.fm.intel.com with ESMTP; 19 Jan 2018 09:31:21 -0800 Received: from irsmsx102.ger.corp.intel.com ([169.254.2.180]) by IRSMSX107.ger.corp.intel.com ([169.254.10.239]) with mapi id 14.03.0319.002; Fri, 19 Jan 2018 17:31:20 +0000 From: "Van Haaren, Harry" To: Andriy Berestovskyy , "dev@dpdk.org" CC: "Horton, Remy" Thread-Topic: [dpdk-dev] [PATCH] keepalive: fix keepalive state alignment Thread-Index: AQHTkTSZUOcjpeWiA0S/RM3M0HElpqN7cAhA Date: Fri, 19 Jan 2018 17:31:20 +0000 Message-ID: References: <16d055c21f6c9e3788fff1a9ecef12d1449d7305.1516373253.git.aber@semihalf.com> In-Reply-To: <16d055c21f6c9e3788fff1a9ecef12d1449d7305.1516373253.git.aber@semihalf.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiN2UyZmYzMjktOWEwMC00NzA4LTgyMGYtMTA1ZWRmZjdjMWNlIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJ4Y0h0Um9ncHk2SDZuREV2dDM0alk2WTNuaWRLeGIxa3BPdDVWS0p5ZW92VnNtNUkwUCtLTmFGekdrZjNEK3lGIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] keepalive: fix keepalive state alignment X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2018 17:31:24 -0000 > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Andriy Berestovskyy > Sent: Friday, January 19, 2018 2:48 PM > To: dev@dpdk.org > Cc: Horton, Remy > Subject: [dpdk-dev] [PATCH] keepalive: fix keepalive state alignment >=20 > The __rte_cache_aligned was applied to the whole array, > not the array elements. This leads to a false sharing between > the monitored cores. >=20 > Fixes: e70a61ad50ab ("keepalive: export states") > Cc: remy.horton@intel.com > Signed-off-by: Andriy Berestovskyy > --- > lib/librte_eal/common/rte_keepalive.c | 25 +++++++++++++++---------- > 1 file changed, 15 insertions(+), 10 deletions(-) >=20 > diff --git a/lib/librte_eal/common/rte_keepalive.c > b/lib/librte_eal/common/rte_keepalive.c > index 7ddf201..a586e03 100644 > --- a/lib/librte_eal/common/rte_keepalive.c > +++ b/lib/librte_eal/common/rte_keepalive.c > @@ -13,8 +13,13 @@ >=20 > struct rte_keepalive { > /** Core Liveness. */ > - enum rte_keepalive_state __rte_cache_aligned state_flags[ > - RTE_KEEPALIVE_MAXCORES]; > + struct { > + /* > + * Each element of the state_flags table must be cache aligned > + * to prevent false sharing. > + */ > + enum rte_keepalive_state s __rte_cache_aligned; > + } state_flags[RTE_KEEPALIVE_MAXCORES]; By aligning each item in the array, we do reduce false-sharing of the cache= lines for all cores, however we pay the cost of increasing the footprint o= f the rte_keepalive struct. Note that the code iterates the full MAX_CORES = in various loops in the monitoring core. Before (gdb) p sizeof(struct rte_keepalive) $1 =3D 1728 # 27 cache lines After (gdb) p sizeof(struct rte_keepalive) $1 =3D 9408 # 147 cache lines These changes do reduce false-sharing however is there actually a performan= ce benefit? A lot of cache space will be taken up if each core requires its= own cache line, which will reduce performance again.. it's a tradeoff. Little fix for a v2: "s" is not a good variable name for the rte_keepalive_= state, please use something more descriptive.