From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C95945894; Thu, 29 Aug 2024 09:54:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2C1C0427AC; Thu, 29 Aug 2024 09:54:41 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 7FEF1402BE for ; Thu, 29 Aug 2024 09:54:39 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id C1D3D6953 for ; Thu, 29 Aug 2024 09:54:38 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id A6227698C; Thu, 29 Aug 2024 09:54:38 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,AWL, T_SCC_BODY_TEXT_LINE autolearn=disabled version=4.0.0 X-Spam-Score: -1.2 Received: from [192.168.1.86] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 764AD698B; Thu, 29 Aug 2024 09:54:36 +0200 (CEST) Message-ID: <7f424cfc-5fa3-4bce-8ea7-4e0c4caa23d1@lysator.liu.se> Date: Thu, 29 Aug 2024 09:54:35 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: memif insufficient padding To: =?UTF-8?Q?Morten_Br=C3=B8rup?= , Jakub Grajciar Cc: dev@dpdk.org References: <98CBD80474FA8B44BF855DF32C47DC35E9F674@smartserver.smartshare.dk> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F674@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-08-28 23:04, Morten Brørup wrote: > Jakub, > > While browsing virtual interfaces in DPDK, I noticed a possible performance issue in the memif driver: > > If "head" and "tail" are accessed by different lcores, they are not sufficiently far away from each other (and other hot fields) to prevent false sharing-like effects on systems with a next-N-lines hardware prefetcher, which will prefetch "tail" when fetching "head", and prefetch "head" when fetching "flags". > > I suggest updating the structure somewhat like this: > > -#define MEMIF_CACHELINE_ALIGN_MARK(mark) \ > - alignas(RTE_CACHE_LINE_SIZE) RTE_MARKER mark; > - > -typedef struct { > - MEMIF_CACHELINE_ALIGN_MARK(cacheline0); > +typedef struct __rte_cache_aligned { > uint32_t cookie; /**< MEMIF_COOKIE */ > uint16_t flags; /**< flags */ > #define MEMIF_RING_FLAG_MASK_INT 1 /**< disable interrupt mode */ > + RTE_CACHE_GUARD; /* isolate head from flags */ Wouldn't it be better to cache align the 'head' (or cache-aligned 'head' *and* add a RTE_CACHE_GUARD)? In other words, isn't the purpose of RTE_CACHE_GUARD to provide zero or more cache line of extra padding, rather than a mechanism to avoid same-cache line false sharing? > RTE_ATOMIC(uint16_t) head; /**< pointer to ring buffer head */ > - MEMIF_CACHELINE_ALIGN_MARK(cacheline1); > + RTE_CACHE_GUARD; /* isolate tail from head */ > RTE_ATOMIC(uint16_t) tail; /**< pointer to ring buffer tail */ > - MEMIF_CACHELINE_ALIGN_MARK(cacheline2); > + RTE_CACHE_GUARD; /* isolate descriptors from tail */ > - memif_desc_t desc[0]; /**< buffer descriptors */ > + memif_desc_t desc[]; /**< buffer descriptors */ > } memif_ring_t; > > > Med venlig hilsen / Kind regards, > -Morten Brørup >