From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7FDD143EEE; Tue, 23 Apr 2024 22:56:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 148D942EE8; Tue, 23 Apr 2024 22:56:57 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 2966140272 for ; Tue, 23 Apr 2024 22:56:55 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 5531A6FEE for ; Tue, 23 Apr 2024 22:56:54 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 2EB4B6FED; Tue, 23 Apr 2024 22:56:54 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=ALL_TRUSTED,AWL, T_SCC_BODY_TEXT_LINE autolearn=disabled version=4.0.0 X-Spam-Score: -1.3 Received: from [192.168.1.59] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id CAD0A7103; Tue, 23 Apr 2024 22:56:51 +0200 (CEST) Message-ID: <63dbb564-61f6-4d9f-9c13-4a21f5e97dc9@lysator.liu.se> Date: Tue, 23 Apr 2024 22:56:51 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] net/af_packet: cache align Rx/Tx structs To: Ferruh Yigit , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , "John W . Linville" Cc: dev@dpdk.org, Tyler Retzlaff References: <20240423090813.94110-1-mattias.ronnblom@ericsson.com> <6f7aabcb-2c12-4cfe-ae9d-73b42bfd4977@amd.com> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <6f7aabcb-2c12-4cfe-ae9d-73b42bfd4977@amd.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-04-23 13:15, Ferruh Yigit wrote: > On 4/23/2024 10:08 AM, Mattias Rönnblom wrote: >> Cache align Rx and Tx queue struct to avoid false sharing. >> >> RX struct happens to be 64 bytes on x86_64 already, so cache alignment >> makes no change there, but it does on 32-bit ISAs. >> >> TX struct is 56 bytes on x86_64. >> > > Hi Mattias, > > No objection to the patch. Is the improvement theoretical or do you > measure any improvement practically, if so how much is the improvement? > I didn't run any benchmarks. Two cores storing to a (falsely) shared cache line on a per-packet basis is going to be very expensive, at least for "light touch" applications. >> Both structs keep counters, and in the RX case they are updated even >> for empty polls. >> > > Do you think does it help if move 'rx_pkts' & 'rx_bytes' update within > the loop? > No, why? Wouldn't that be worse? Especially since rx_pkts and rx_bytes are declared volatile, so you are forcing a load-modify-store cycle for every increment. I would drop "volatile", or replace it with an atomic (although *not* use an atomic add for incrementing, but rather atomic load + non-atomic adds + atomic store). >> Signed-off-by: Mattias Rönnblom >> --- >> drivers/net/af_packet/rte_eth_af_packet.c | 5 +++-- >> 1 file changed, 3 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c >> index 397a32db58..28aeb7d08e 100644 >> --- a/drivers/net/af_packet/rte_eth_af_packet.c >> +++ b/drivers/net/af_packet/rte_eth_af_packet.c >> @@ -6,6 +6,7 @@ >> * All rights reserved. >> */ >> >> +#include >> #include >> #include >> #include >> @@ -53,7 +54,7 @@ struct pkt_rx_queue { >> >> volatile unsigned long rx_pkts; >> volatile unsigned long rx_bytes; >> -}; >> +} __rte_cache_aligned; >> > > Latest location for '__rte_cache_aligned' tag is at the beginning of the > struct [1], so something like: > `struct __rte_cache_aligned pkt_rx_queue {` > > [1] > https://patchwork.dpdk.org/project/dpdk/list/?series=31746&state=%2A&archive=both