From: "Mattias Rönnblom" <hofors@lysator.liu.se>
To: "Morten Brørup" <mb@smartsharesystems.com>,
"Jakub Grajciar" <jgrajcia@cisco.com>
Cc: dev@dpdk.org
Subject: Re: memif insufficient padding
Date: Thu, 29 Aug 2024 09:54:35 +0200 [thread overview]
Message-ID: <7f424cfc-5fa3-4bce-8ea7-4e0c4caa23d1@lysator.liu.se> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F674@smartserver.smartshare.dk>
On 2024-08-28 23:04, Morten Brørup wrote:
> Jakub,
>
> While browsing virtual interfaces in DPDK, I noticed a possible performance issue in the memif driver:
>
> If "head" and "tail" are accessed by different lcores, they are not sufficiently far away from each other (and other hot fields) to prevent false sharing-like effects on systems with a next-N-lines hardware prefetcher, which will prefetch "tail" when fetching "head", and prefetch "head" when fetching "flags".
>
> I suggest updating the structure somewhat like this:
>
> -#define MEMIF_CACHELINE_ALIGN_MARK(mark) \
> - alignas(RTE_CACHE_LINE_SIZE) RTE_MARKER mark;
> -
> -typedef struct {
> - MEMIF_CACHELINE_ALIGN_MARK(cacheline0);
> +typedef struct __rte_cache_aligned {
> uint32_t cookie; /**< MEMIF_COOKIE */
> uint16_t flags; /**< flags */
> #define MEMIF_RING_FLAG_MASK_INT 1 /**< disable interrupt mode */
> + RTE_CACHE_GUARD; /* isolate head from flags */
Wouldn't it be better to cache align the 'head' (or cache-aligned 'head'
*and* add a RTE_CACHE_GUARD)? In other words, isn't the purpose of
RTE_CACHE_GUARD to provide zero or more cache line of extra padding,
rather than a mechanism to avoid same-cache line false sharing?
> RTE_ATOMIC(uint16_t) head; /**< pointer to ring buffer head */
> - MEMIF_CACHELINE_ALIGN_MARK(cacheline1);
> + RTE_CACHE_GUARD; /* isolate tail from head */
> RTE_ATOMIC(uint16_t) tail; /**< pointer to ring buffer tail */
> - MEMIF_CACHELINE_ALIGN_MARK(cacheline2);
> + RTE_CACHE_GUARD; /* isolate descriptors from tail */
> - memif_desc_t desc[0]; /**< buffer descriptors */
> + memif_desc_t desc[]; /**< buffer descriptors */
> } memif_ring_t;
>
>
> Med venlig hilsen / Kind regards,
> -Morten Brørup
>
next prev parent reply other threads:[~2024-08-29 7:54 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-28 21:04 Morten Brørup
2024-08-29 7:54 ` Mattias Rönnblom [this message]
2024-08-29 8:55 ` Morten Brørup
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7f424cfc-5fa3-4bce-8ea7-4e0c4caa23d1@lysator.liu.se \
--to=hofors@lysator.liu.se \
--cc=dev@dpdk.org \
--cc=jgrajcia@cisco.com \
--cc=mb@smartsharesystems.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).