From: "Van Haaren, Harry" <harry.van.haaren@intel.com>
To: Andriy Berestovskyy <aber@semihalf.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "Horton, Remy" <remy.horton@intel.com>
Subject: Re: [dpdk-dev] [PATCH] keepalive: fix keepalive state alignment
Date: Fri, 19 Jan 2018 17:31:20 +0000 [thread overview]
Message-ID: <E923DB57A917B54B9182A2E928D00FA650FF2A1C@IRSMSX102.ger.corp.intel.com> (raw)
In-Reply-To: <16d055c21f6c9e3788fff1a9ecef12d1449d7305.1516373253.git.aber@semihalf.com>
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Andriy Berestovskyy
> Sent: Friday, January 19, 2018 2:48 PM
> To: dev@dpdk.org
> Cc: Horton, Remy <remy.horton@intel.com>
> Subject: [dpdk-dev] [PATCH] keepalive: fix keepalive state alignment
>
> The __rte_cache_aligned was applied to the whole array,
> not the array elements. This leads to a false sharing between
> the monitored cores.
>
> Fixes: e70a61ad50ab ("keepalive: export states")
> Cc: remy.horton@intel.com
> Signed-off-by: Andriy Berestovskyy <aber@semihalf.com>
> ---
> lib/librte_eal/common/rte_keepalive.c | 25 +++++++++++++++----------
> 1 file changed, 15 insertions(+), 10 deletions(-)
>
> diff --git a/lib/librte_eal/common/rte_keepalive.c
> b/lib/librte_eal/common/rte_keepalive.c
> index 7ddf201..a586e03 100644
> --- a/lib/librte_eal/common/rte_keepalive.c
> +++ b/lib/librte_eal/common/rte_keepalive.c
> @@ -13,8 +13,13 @@
>
> struct rte_keepalive {
> /** Core Liveness. */
> - enum rte_keepalive_state __rte_cache_aligned state_flags[
> - RTE_KEEPALIVE_MAXCORES];
> + struct {
> + /*
> + * Each element of the state_flags table must be cache aligned
> + * to prevent false sharing.
> + */
> + enum rte_keepalive_state s __rte_cache_aligned;
> + } state_flags[RTE_KEEPALIVE_MAXCORES];
By aligning each item in the array, we do reduce false-sharing of the cache lines for all cores, however we pay the cost of increasing the footprint of the rte_keepalive struct. Note that the code iterates the full MAX_CORES in various loops in the monitoring core.
Before
(gdb) p sizeof(struct rte_keepalive)
$1 = 1728 # 27 cache lines
After
(gdb) p sizeof(struct rte_keepalive)
$1 = 9408 # 147 cache lines
These changes do reduce false-sharing however is there actually a performance benefit? A lot of cache space will be taken up if each core requires its own cache line, which will reduce performance again.. it's a tradeoff.
Little fix for a v2: "s" is not a good variable name for the rte_keepalive_state, please use something more descriptive.
<snip>
next prev parent reply other threads:[~2018-01-19 17:31 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-19 14:47 Andriy Berestovskyy
2018-01-19 17:31 ` Van Haaren, Harry [this message]
2018-01-22 18:20 ` Andriy Berestovskyy
2018-01-23 10:16 ` Remy Horton
2018-01-23 10:27 ` Van Haaren, Harry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E923DB57A917B54B9182A2E928D00FA650FF2A1C@IRSMSX102.ger.corp.intel.com \
--to=harry.van.haaren@intel.com \
--cc=aber@semihalf.com \
--cc=dev@dpdk.org \
--cc=remy.horton@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).