From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7599343EF7; Wed, 24 Apr 2024 12:28:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 42ACE400D6; Wed, 24 Apr 2024 12:28:30 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) by mails.dpdk.org (Postfix) with ESMTP id AF67640042 for ; Wed, 24 Apr 2024 12:28:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1713954508; x=1745490508; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=oCw5VDAzitPy2+z7FiLzHZW7mOCKRRVk77CCMRSK94c=; b=V7kMDs0Euf/Y666SK+n522RKSWlZsO/J/w0R9qjTLUMyl2N9LGI7apQR fGpsLkqz9mp85quQX/Ol9q0oVFrpUbAGRQEEmdRI9jgjLzgOFEwQkdjo3 jWX7fWFFgcvjGunaTYgH/Oik3LSR9dI+qcOenhNl7QbwudP97QbWzwva8 vDGtsBKxmxzsWUu3K66qu3D/nz8q/orbbZ4eOyyRivi6iqr0Weksm7Dz+ XeK1+PzXvAkygBzpRlGFFVXY+Euc/GbTUDw7gI9/epoj5bh+lY+O/jCKl iGKS60LRa9IIl7wk0LtNrcVoT/ip64O3OsJ5CeNPd94EmdOdPAI84JamO w==; X-CSE-ConnectionGUID: 8At010jIQRahfjHde/AYyg== X-CSE-MsgGUID: IbGoUIc/Tp2FdbVqGmdj0Q== X-IronPort-AV: E=McAfee;i="6600,9927,11053"; a="9429552" X-IronPort-AV: E=Sophos;i="6.07,225,1708416000"; d="scan'208";a="9429552" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Apr 2024 03:28:26 -0700 X-CSE-ConnectionGUID: ruvmqBtpRBmKWnihJz4RHg== X-CSE-MsgGUID: 2ncDxlXnS1WoniMFVgchrg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,225,1708416000"; d="scan'208";a="24711778" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmviesa007.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 24 Apr 2024 03:28:26 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 24 Apr 2024 03:28:26 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Wed, 24 Apr 2024 03:28:26 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.101) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 24 Apr 2024 03:28:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YcmEXpqpqOyBHBGlbyBEiPIQVCVjxjZG1KQTdfUQT/KZyOhBIPaYOwiGYYZ0UgEKJ5+SicUtGoZ7b0ymOYyRZQRtGlbtlOcOkcAqLrY26Ovb2zvjO1gaYrXsuUPIMkDVs86aBj/YDXQKgBc8yo5pE3YJ0+tmwbHpueslpzBK/phEGs3S9TJP35sNHIGpJPMmAF3lCem7IWv9Ge9H4p8FwKn1inh421pVfAPnFcN6S8PoTnv0VTDWdxT8Fslmq/12P8KIZp4xM78nQ3L+/n5T+kEtM/Fq7AfLmyvi4FBuZhPqpR5PEEY+Fb6/rJmNL1Theq5VziJqsMsI58h0ZOEy6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QuPR75EJmbBeVsFTrKtP+UE2HNKUNgigxzhRz0nIrjM=; b=SbfoXaIasI43W6CoKxpcwNLAD9CbB4GOooNWUrzfqCdwUwSDEx/V1sZTmUJpSvslR02sCYqpVcWTcnWKuwwlzlGXkGsSF6iAMPQ95lIGzK8ILl58wsYCXfRkUzzOdeiLCJRdz82NrLpLC/OLLWA1Bt/rb/aaBDwK00orYsGJVbZ9IwfW1gp+SUqCPTYH9nEtJBK4b/N+Lr1VAJ59omZobrEmve2GvLThURxY0VpC8KplrB5GpTJFl31yTJreKG/09pokZw7jqpthEiTUtz/a3SY/T3NCrlBfNzzDQxE8k0IpKdKmXQ1qwEOfaDILkr9Dpop22BaXHwU5gv6+9mk6ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by IA1PR11MB6147.namprd11.prod.outlook.com (2603:10b6:208:3ed::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7519.19; Wed, 24 Apr 2024 10:28:24 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f%7]) with mapi id 15.20.7519.021; Wed, 24 Apr 2024 10:28:24 +0000 Date: Wed, 24 Apr 2024 11:28:18 +0100 From: Bruce Richardson To: Ferruh Yigit CC: Mattias =?iso-8859-1?Q?R=F6nnblom?= , Mattias =?iso-8859-1?Q?R=F6nnblom?= , "John W . Linville" , , Tyler Retzlaff , Honnappa Nagarahalli Subject: Re: [PATCH] net/af_packet: cache align Rx/Tx structs Message-ID: References: <20240423090813.94110-1-mattias.ronnblom@ericsson.com> <6f7aabcb-2c12-4cfe-ae9d-73b42bfd4977@amd.com> <63dbb564-61f6-4d9f-9c13-4a21f5e97dc9@lysator.liu.se> <5d2a0887-527a-4948-943c-65f1dfda9328@amd.com> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5d2a0887-527a-4948-943c-65f1dfda9328@amd.com> X-ClientProxiedBy: DUZPR01CA0246.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b5::12) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|IA1PR11MB6147:EE_ X-MS-Office365-Filtering-Correlation-Id: 15b6675e-106b-41ce-3b03-08dc64494563 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|366007; X-Microsoft-Antispam-Message-Info: =?iso-8859-1?Q?i/lYZi5AGO8OUwTQJ9EfB3R9tnpsj/l639jEYg8gi4p7dWUmYjotxWzIPL?= =?iso-8859-1?Q?muKv+alU7btwbm56pP0qqx8qgrql9I9FQKUIHKDfsca7mlDBbSaCFLp2d1?= =?iso-8859-1?Q?3i+kbtmEy8n1UGSvPqw1ZCTaffutwLko7WiJi+CvFHS1dJi5M36jY8HTJg?= =?iso-8859-1?Q?PFJRAewHs/wFM1oheDXWnheLLRYi4Wr7HwF9FxqrNynRw6DrpyT8pReYoF?= =?iso-8859-1?Q?b1I+rSggFVsfKSctvOWCRNbqxEn5SaEzJ+0PFY3sRUqk0mRSz+wY6DQ6en?= =?iso-8859-1?Q?Hvt1nnQZZYqwvcwk/XUHXUMbxfp5kvVJDKguHvy+YfzkAXjnT7mz5Ne+OL?= =?iso-8859-1?Q?mhYkQn1V6u86vzrDJGoThO8lpigbdTn0YqW6fFzRY7iTvABqvREkSRSl+N?= =?iso-8859-1?Q?rZjDOQa/IseGaP4R1cfeW90a7EMs5cmv/RtFMRUeeYZtG2pPwOsXutAwrL?= =?iso-8859-1?Q?57YWhHcJZzfch1DHKFlcm1v4RoT08uX4WxE92otorvHb576MIEp6dx1JoK?= =?iso-8859-1?Q?aMgtQFMxgXiiaVMtVQ3wrLyTQIZm/SjSFJnaTgxgBqt9CuX2MrOqZVhaMV?= =?iso-8859-1?Q?rwB2xeAOGl2RHoXrXdPdLa3z6PjkS+aGUXGou3QLovk9SR2PkgEnlrNNWB?= =?iso-8859-1?Q?GfqYrMGZ7ooOPLg+BYlZi3Bz6M/ho2NkUSZxdp9MtgZmZg5gREZCIXhsyV?= =?iso-8859-1?Q?to3GDKHgBPJ/eqCSijn9GL8fMJd1lI2nbBPPP5ZjJL6LrfFrPT28JHwdI5?= =?iso-8859-1?Q?UjLokV06/XiT6TdLTx35Wpr/TV5rYLMyIy5xtjEpuCqu5DaS2nxTIcrZn0?= =?iso-8859-1?Q?GTeeVyKVD22M2I5sPVTy4IU8RzUIs2LgC1jtGZZ/lYyJs09vh+skMTq6Jk?= =?iso-8859-1?Q?CxSBKjPPpGAXPOZ9vHVE7p076aq6KXlc2DpRN4rInYwmoj4YtGSKxCVB8x?= =?iso-8859-1?Q?gzcRSw83ISLabI0bqn3f3geTIYRa3YXrswpmt/OywN9xLmcfO48V//tDpy?= =?iso-8859-1?Q?NxKA3+4/imQ0ky8dO5bn7MjdCgI/vAOUOxJXFq7Smtk5UClTWCs/wzsccU?= =?iso-8859-1?Q?IKW4+9ogqxXIFZkwddiKRU/GD/youXhw2kv8ofYNeqU4t4Zk9JFFZ1A4M1?= =?iso-8859-1?Q?gaT6uRCNqFLg0RtG+TyAINXu6ZzGq46t6vvBTO+DBhnVgOV4ifr9hW4wFF?= =?iso-8859-1?Q?dd+RVi/AMNzveeK4VOU+6pSRDzZ3ieMn3gmDhWKv2VhJOaj70zQkdnaTqW?= =?iso-8859-1?Q?s5lT7SUP/cjS/OqQrrOrzhivYBJLgr5KuotZ3J/ykSDSyfS63MxP8cHhFq?= =?iso-8859-1?Q?YeoNJwRCAlzB9v+Yy3D5tIp7Zg=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(1800799015)(366007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?GeztMqkiBGPVayiP6/icf7SUZTlYLDEBkxZ8l43XMfOGZ+bKW/lyMbG4vX?= =?iso-8859-1?Q?IQ5B1mYBVl6MRgAKtjdEAIFmb1gejjs0TkSI62Inhva19vgBaf9j01F9Y9?= =?iso-8859-1?Q?mn9iwFNiDlYKhNmA1hqVVf0JDjnIhg9DASeztpTjuOXBeHnwv1x+i3aB8q?= =?iso-8859-1?Q?Zq8SiSASMGv+njIloXRvdku5EPpiqTRYc9jy9N1mDEPr3oW+NKH1M82ZH3?= =?iso-8859-1?Q?umRXhFjZRai0nvaVxmFEtFxTPcZlWRRGVTRAW0o3SEkn5wSJrW89SCVmDx?= =?iso-8859-1?Q?0S080AvxktrY30JUkUWMZ/Fo/EKuZfCU1quimdM+7UcrC1Km37Imd6OwdG?= =?iso-8859-1?Q?7th91sU722x66u2uz6AjfSqLHAa5QicvASOK6PksA5cyCwLAsKcxbJrV1g?= =?iso-8859-1?Q?oMAdorD6VRqOQa9PpfYvGHw1+TWYkAg/KwGjbTo5kb1rxw8DbhTdtnX7H3?= =?iso-8859-1?Q?t0sKvS8RtBiLc6w9/m5B8RsBnFDSmH9L3t+KFFGeaQaGSvgCm+sE2jjq2q?= =?iso-8859-1?Q?kJWOJMa5E/H/7GhPQRxfj6j++qioXBkR9aimT7viL5cXSDhGpBjO/0J2lf?= =?iso-8859-1?Q?V0KInUDceFz2tEnG08N7KklCc3Ub1PAiO8+oQh8dcFfgbkNUbrkSMV+aeU?= =?iso-8859-1?Q?8ShFb8zP5dU5iZr9KO2Eh9at63aofGEKj5XTQcsyaPi5+sz2tnyEOqndzD?= =?iso-8859-1?Q?5CKAcnYEa+wTfNqD0yXEKwVVjARiewDi9YOJU0dBkSbO0XLSq3n9/N6Pps?= =?iso-8859-1?Q?Ztc96tsFDDgrYtNdtK/HZjOITF92oQGQ8i9BWM4w6OHEdtkIa26oWWJYwq?= =?iso-8859-1?Q?SyOZu3hcijIUGzwNkE+WAxCbX6DlDR30mRL7cHsPaOq8yBahmLd4Wqq4mt?= =?iso-8859-1?Q?kcXb80M/qqQbrA69bMfZTpoCYBebnvRVrKrHt+srGmr7yvtuAcCAs3rsyo?= =?iso-8859-1?Q?Xvc7whA1Vs4WbvJ3v3V7P9UDlKiUeHqRKqy+y+MUg5N69ydJMti5er6ZAP?= =?iso-8859-1?Q?DTsAm5f+09ZxwZBp8dP+L03xZucJUs7z5TuAWHKfL4aOkfsPIMmNobTKwZ?= =?iso-8859-1?Q?tbDT2+1jyfWU+4V0urfYdUni8BH5z/c1EPxsHFwRO5n/SA7ng2OzgVGfIu?= =?iso-8859-1?Q?peVnEFYLInekFO66RUMl629meYIjetw2IQUBxxfZH9h08i6PpuRCwpqOZe?= =?iso-8859-1?Q?4cI2eVBDs+gZPZ0M5hEZ+5C7xbVfjZ2g4nqfJb9LmXi6BoepeJGbxBE6QF?= =?iso-8859-1?Q?EmnuVkQKK64VDjWeTCI91zlKN8VQkPmF5gllw0dS1+ifTjSWlesoFwrFtn?= =?iso-8859-1?Q?6kyV6gUiMRDRjefWf7oFAUPj/aV3pduiVvdEjFUfslaq29jr8hlVbMdpW9?= =?iso-8859-1?Q?PoQwvDHn7ywVLeK8o1bGEgrxnR7RXhTBXwzUcDBoeRU7v/4G6mArBhun2I?= =?iso-8859-1?Q?wOtHp0Y5XBz0O43eXkpVqX8B2VqxFPm5EBMsmJPhGQdUSwUoiaADIdNSpl?= =?iso-8859-1?Q?eZa3m4v7CB5enwaDZmNGQAOu5SwyD/3v9Q4XsVpNWkvM6gN3mhuBYTLPP/?= =?iso-8859-1?Q?4Tvji/M1/Sne8x2JUbocRI/ObOve7wdo/8scBkfJOi2JZhxcg+dWWxKR4/?= =?iso-8859-1?Q?1ZBxvJI8WsliKrNf86YDaUrJ/UdOhBM1Pv+M7fkpU0O1k6kleFaN1bYg?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 15b6675e-106b-41ce-3b03-08dc64494563 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2024 10:28:24.1279 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TH7snt+pnGz+qcSR5OUYLb4ywfBDMSw4hsPriUMpnGI2fFwCPnqsCWnEUpNjlZS+apFnI++AGCK2KEfWoBG+IwnPvuEDM8SfK2BQoWFqiJw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB6147 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Apr 24, 2024 at 11:21:52AM +0100, Ferruh Yigit wrote: > On 4/23/2024 9:56 PM, Mattias Rönnblom wrote: > > On 2024-04-23 13:15, Ferruh Yigit wrote: > >> On 4/23/2024 10:08 AM, Mattias Rönnblom wrote: > >>> Cache align Rx and Tx queue struct to avoid false sharing. > >>> > >>> RX struct happens to be 64 bytes on x86_64 already, so cache > >>> alignment makes no change there, but it does on 32-bit ISAs. > >>> > >>> TX struct is 56 bytes on x86_64. > >>> > >> > >> Hi Mattias, > >> > >> No objection to the patch. Is the improvement theoretical or do you > >> measure any improvement practically, if so how much is the > >> improvement? > >> > > > > I didn't run any benchmarks. > > > > Two cores storing to a (falsely) shared cache line on a per-packet > > basis is going to be very expensive, at least for "light touch" > > applications. > > > > ack I expect for af_packet bottleneck is the kernel side, so I don't > expect any visible improvement practically, but OK to fix this > theoretical issue. > > >>> Both structs keep counters, and in the RX case they are updated even > >>> for empty polls. > >>> > >> > >> Do you think does it help if move 'rx_pkts' & 'rx_bytes' update within > >> the loop? > >> > > > > No, why? Wouldn't that be worse? Especially since rx_pkts and rx_bytes > > are declared volatile, so you are forcing a load-modify-store cycle for > > every increment. > > > > My intention was to prevent updating stats in empty polls, I thought > stats will be hot in the cache but won't work with volatile. > Yes, it will. Volatile only prevents caching in registers, it does not affect the storing of data within the cache hierarchy. Reads/writes of stats on empty polls should indeed hit the L1 as expected. However, that is still less efficient than just doing a register increment which could theoretically be the result without the volatile. /Bruce