From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1923F41B26; Mon, 28 Aug 2023 09:57:27 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A08D54026D; Mon, 28 Aug 2023 09:57:26 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 29F3F4021E for ; Mon, 28 Aug 2023 09:57:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1693209444; x=1724745444; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=pS8IofwaWDqOuNLV7tq8UnBfeLOXFPmqgFUdKF94x5Y=; b=TXe+JeQeGMXCD6vqp6ht4IenLHv8ajhHJV89j+5N/46eIz/X53nfVqZw dgrzPaHLBSHifKi2+5oQLQBrsI5ulupNts1BD5057tK8fmgJ6TQ3qI4BT 2yMenp81EaEHbz9QwCUjX9z/sQIxyaKmJMw2KHkPmsLwD/xtsBprLuTe7 3GiDepDzpULgJ/heFQabxR+qk4m/C2VfZvGMOPXcQZOisF9E0Wt+xGOSC 5YoRDk+ZqBFavYSF9D1YDPhJC+SjEfH3fxT9Nv+0JqPa5jnzjvco4eTAe Y6qp6hnhS8WmtKSHH9OSSz4uHzK46+Vi8jlB+oeBZvJleI98sQyPHX95Z g==; X-IronPort-AV: E=McAfee;i="6600,9927,10815"; a="355368958" X-IronPort-AV: E=Sophos;i="6.02,207,1688454000"; d="scan'208";a="355368958" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2023 00:57:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10815"; a="688006989" X-IronPort-AV: E=Sophos;i="6.02,207,1688454000"; d="scan'208";a="688006989" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orsmga003.jf.intel.com with ESMTP; 28 Aug 2023 00:57:16 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Mon, 28 Aug 2023 00:57:16 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Mon, 28 Aug 2023 00:57:16 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.176) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Mon, 28 Aug 2023 00:57:15 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n/mWjWf2OIF6M65qgDJR42Emqa3rJ6DytYtDHTZsS4lUAV1M7hyboQu9SeEEIkNxxuBDOq5niHaYCnHA3BKt1kXxLZ0N+/TK1OIuOY3Y1i/ErOkNMZ5G9/VbTAbTmFmXCRReS4b5ppQOp3axrR7Yka6xjXVbY2WgG1sFHEA9rkxgaxn9/gu7VFJQ5/k++X+E/3+s/TMce0N6dIFxQgRGgU6FttnjBcx5mrYkpdKWteuz8RU03a8HG38OQgDcY9KayeQObwOv111Dgtl+3cpKJz+Ycz6UoRTDAT1RXFsT8XTeBwOSiCdZFalYjxxS7133cdBxdqseI49GkAYehsEn1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l7yONufT6tyuC7yxY03f7jdcoYLPSQ/c+IRV5LuMIZo=; b=YiPbi+NX0dFJsNXAFbv/+ewVbsxQmQ40xcqldLpniLrEBHWK4cldvHkJqpm37zlIhyNS5rUCfLqwbOAZIW9jWArnNn+Sgw1m/uS+jrnxg+2YUytnKdG6cdF04v9LpXCUtb9TZ65qkxEksycXFHzSTgljupO1O19ZIFStabn+lMaqJM+38mqsoadoq3RQTjKRZFcKtcJGBCja1ju0xehX5iDhuNRxVzU9o8gWpjvAxz3d+OGrPaE+JXqMYqTmCsUv2Esy/ZaqFZAkSBB+uzd+omCtZsHd2sV7msBwji1ijX5FCiB46lYfdd5t67Jh2bdb1RMvtO47vdGArXCX1HOVIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by LV2PR11MB6072.namprd11.prod.outlook.com (2603:10b6:408:176::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Mon, 28 Aug 2023 07:57:14 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347%5]) with mapi id 15.20.6699.034; Mon, 28 Aug 2023 07:57:14 +0000 Date: Mon, 28 Aug 2023 08:57:07 +0100 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Mattias =?iso-8859-1?Q?R=F6nnblom?= , , , , , , Subject: Re: [RFC] cache guard Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D87B39@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D87B3A@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D87B47@smartserver.smartshare.dk> <5a56c531-5d5c-777b-c1ea-dbcc25009220@lysator.liu.se> <98CBD80474FA8B44BF855DF32C47DC35D87B49@smartserver.smartshare.dk> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87B49@smartserver.smartshare.dk> X-ClientProxiedBy: DUZPR01CA0116.eurprd01.prod.exchangelabs.com (2603:10a6:10:4bc::15) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|LV2PR11MB6072:EE_ X-MS-Office365-Filtering-Correlation-Id: e5fdd4a0-eadf-49e8-db43-08dba79c63ef X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hm+Pb07HNezpro9+UqZrU9A5xXXFBqjIOY3o2MuxLvDSPrA/R23OyoxG3y2TyrftOUc7Qlxq05odZfjlzQzJVR7RyAqAEMsMaceCQor1HKkQKmSsEfhET5aLYXl57rU+vHSJ4jPcWvP3Tvhx1UA69mQjAcLYlCoeoLAkdsnJBS70c9dK+ZjAlGw4iODFBba0RsujQSYOdpWo20gfBHui2cBhTiW+678G2kEYePiZqCGTT9k2ONpl8d5OKjpys6+Gv/nZRIIE8We/5wOgWtISZHQhT0wwlc3A6Zxlth74Qkd+QcQgFem0G+8eMLLoV+MzI1zIDMc4MVETnhmVoMpVxc5VkbQih/bPDhRezZMSC3BKLAen6UOgAC1AaFIIpIksI7E0T8b5Q0QTaPu2TYC35vzWUvfPfwqzaOT1/mnWtcgoUX4kPuqm4L1jOmrE+1zwyDR3oM9F8cfrAFro95jFZYRg7ggOB6uY2PDzhAEq0HsW51h+6pcyVhXKE7qitTL8sm+yGHmvcK/vaMd13u8M4OkNHwNauo/VLfDNLCsBGWo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(39860400002)(376002)(366004)(396003)(346002)(451199024)(1800799009)(186009)(5660300002)(44832011)(2906002)(316002)(66476007)(66556008)(6916009)(66946007)(8676002)(8936002)(4326008)(66899024)(6486002)(53546011)(6506007)(26005)(6512007)(41300700001)(38100700002)(6666004)(966005)(83380400001)(86362001)(82960400001)(66574015)(478600001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?XcrNYih/Hi0d9H52Dj3YHezyrCUWnEH7CpFkBNAe1rqDA+pYzgUGXANd3V?= =?iso-8859-1?Q?hLZLBy98gty096WpELf8UK8J3sdH+Jt6QOiu0WJFsjALNwgnz4PXy0ZAY8?= =?iso-8859-1?Q?hB3PicmStV4Yei4DhMfZGDQKBw3FuttZho2MlghpaDM5sr6hZC+Ho2yNYd?= =?iso-8859-1?Q?YhEJc49NP7F3f1aM4jVDthteYgdiSgHJ1C8nUdAdXeXELLykMI77Gm1b1k?= =?iso-8859-1?Q?K2F+lSbW/HMbdCdQCJfeKNCnrJUB4FO3E7FEXioU+EWzppidbBSEFT3zWl?= =?iso-8859-1?Q?ZE1nME6vD+luEu84CIFd/+b3DPDwzGB8/MgE5NsV/dA6qSTBWztPzFtSx8?= =?iso-8859-1?Q?CMURWg1f33YO+CI1vLWno9X/2aC3jGFXoUurztRyePHGG9gfWx86LEVbLH?= =?iso-8859-1?Q?JQ992o6t7jvtPIBrGS1dPioktAHvSa2qpfNAS9aij2YtYfwWMxmdKG+JAT?= =?iso-8859-1?Q?JD+2L8WemyINXegFyWMEEbzN0mbTCBBEuaIpQ0YimhMUQJzvhvhWXvA8b6?= =?iso-8859-1?Q?W2HVWfzVTATG69NxeKQ5d/4kwx5vpEcCnIsDYz3T8yiQH9ol+VStGzOWhY?= =?iso-8859-1?Q?j4FC7ZAQ9A3deZARodaH6ZyuqHrOtfxDxixZATtBnjRE9uz8D2OJAH/3Iy?= =?iso-8859-1?Q?w14wm+muhLqIjudIgSXoEcjrtqm88L5GDArPLEjMqXfiioowthZRxJ3AlC?= =?iso-8859-1?Q?8dI6oAXoM4WNWpcKg5Vq3yXGwDNfIu2WTJkxUOdqkF3gNv3uSrGHIvy0Ql?= =?iso-8859-1?Q?BTwWV2HB7qApo+pttlGU3q2pFSpCN2hk81G7OZ1xK8pPz3yHZo8xdRnjZo?= =?iso-8859-1?Q?gA3dLHVVvzw3UOO1PjrLogHjtbJfKYQpNcpdsbkjGkgIB71bv94I+RAxnR?= =?iso-8859-1?Q?3KT2EanbZUQMfq394vAxPA09WFVr3YWOiVmmmC9gisqo8b4ET58JoETlWg?= =?iso-8859-1?Q?tdAezjRmakldXVc2dcDFj9WDbcXoUwf/ib4vvznz91J51TL26QaUiT+YDU?= =?iso-8859-1?Q?33cnBiiMzWXSi+9JphK57ULeawzthpP4GH0GvTRhnhBRgGghe7nCZD5uWl?= =?iso-8859-1?Q?8FgXvdgvvACMncbe/sEO7NP3Kb13oI1DEUXenRz06UNkxAt0PvgexOB59i?= =?iso-8859-1?Q?Ro/t81SI6gohiQ8PIv0Ib3r6jgOkkh8WKKGNWFMFsWNkqXnD4UF7K883bt?= =?iso-8859-1?Q?W0ELrhofGinzu/h2SA2ZlCiegCsMtllz4XxQZh0ki2jPtt3qz63rQZPP4Q?= =?iso-8859-1?Q?wm8UM4vdE6K6DlckpEqpy8v9cIGvrq29GFx9DjqBDvPdHl0aWpGj01UI+3?= =?iso-8859-1?Q?O+tF/TKxQLXJ0UXcjDQxfcjwVSOld6CB6gzWIuhlCS5BCkr0P2Kpj+qYQN?= =?iso-8859-1?Q?CW6SD/W9u5N4S7KcyryIRlFoxPsyi8lX2mzf6iNE0CqpNyhT52euMFUkOU?= =?iso-8859-1?Q?Ji7W91JALrEOwFVb+MCvLuAXobSSlkhjD00BZiojZ9uJSHFRuTnpLO68gJ?= =?iso-8859-1?Q?rgcZ4St6yzgqr2juUteBYCbY5NNSwcjnv8S7AX5CPEADZ28dpuEndsXm+q?= =?iso-8859-1?Q?viWBk1lW7YpidCdQuRoUt9eAKlzQX4Bsuytr8gv+BtYjChgNvkf4sn4LAG?= =?iso-8859-1?Q?LDE2TWbSDRq6zJsdl1w3nTKe7NERCmRJsB+PSg2YgRHjI9iOT1UBAqoQ?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: e5fdd4a0-eadf-49e8-db43-08dba79c63ef X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2023 07:57:13.9041 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SzVY6tOlPIqmoFK3oSfYPxTFgbZo6dWKQ1+KKd/ku5FUqVlxQPW8lgiE139czQEHOnfYB8RJt97BnV2MMqE26XErs79p6P4G8RKFpaIC4Lw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR11MB6072 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, Aug 27, 2023 at 05:40:33PM +0200, Morten Brørup wrote: > > From: Mattias Rönnblom [mailto:hofors@lysator.liu.se] > > Sent: Sunday, 27 August 2023 15.55 > > > > On 2023-08-27 10:34, Morten Brørup wrote: > > > +CC Honnappa and Konstantin, Ring lib maintainers > > > +CC Mattias, PRNG lib maintainer > > > > > >> From: Bruce Richardson [mailto:bruce.richardson@intel.com] > > >> Sent: Friday, 25 August 2023 11.24 > > >> > > >> On Fri, Aug 25, 2023 at 11:06:01AM +0200, Morten Brørup wrote: > > >>> +CC mempool maintainers > > >>> > > >>>> From: Bruce Richardson [mailto:bruce.richardson@intel.com] > > >>>> Sent: Friday, 25 August 2023 10.23 > > >>>> > > >>>> On Fri, Aug 25, 2023 at 08:45:12AM +0200, Morten Brørup wrote: > > >>>>> Bruce, > > >>>>> > > >>>>> With this patch [1], it is noted that the ring producer and > > >> consumer data > > >>>> should not be on adjacent cache lines, for performance reasons. > > >>>>> > > >>>>> [1]: > > >>>> > > >> > > https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring.h?id=d9f0d3a1f > > >> fd4b66 > > >>>> e75485cc8b63b9aedfbdfe8b0 > > >>>>> > > >>>>> (It's obvious that they cannot share the same cache line, because > > >> they are > > >>>> accessed by two different threads.) > > >>>>> > > >>>>> Intuitively, I would think that having them on different cache > > >> lines would > > >>>> suffice. Why does having an empty cache line between them make a > > >> difference? > > >>>>> > > >>>>> And does it need to be an empty cache line? Or does it suffice > > >> having the > > >>>> second structure start at two cache lines after the start of the > > >> first > > >>>> structure (e.g. if the size of the first structure is two cache > > >> lines)? > > >>>>> > > >>>>> I'm asking because the same principle might apply to other code > > >> too. > > >>>>> > > >>>> Hi Morten, > > >>>> > > >>>> this was something we discovered when working on the distributor > > >> library. > > >>>> If we have cachelines per core where there is heavy access, having > > >> some > > >>>> cachelines as a gap between the content cachelines can help > > >> performance. We > > >>>> believe this helps due to avoiding issues with the HW prefetchers > > >> (e.g. > > >>>> adjacent cacheline prefetcher) bringing in the second cacheline > > >>>> speculatively when an operation is done on the first line. > > >>> > > >>> I guessed that it had something to do with speculative prefetching, > > >> but wasn't sure. Good to get confirmation, and that it has a > > measureable > > >> effect somewhere. Very interesting! > > >>> > > >>> NB: More comments in the ring lib about stuff like this would be > > nice. > > >>> > > >>> So, for the mempool lib, what do you think about applying the same > > >> technique to the rte_mempool_debug_stats structure (which is an array > > >> indexed per lcore)... Two adjacent lcores heavily accessing their > > local > > >> mempool caches seems likely to me. But how heavy does the access need > > to > > >> be for this technique to be relevant? > > >>> > > >> > > >> No idea how heavy the accesses need to be for this to have a > > noticable > > >> effect. For things like debug stats, I wonder how worthwhile making > > such > > >> a > > >> change would be, but then again, any change would have very low > > impact > > >> too > > >> in that case. > > > > > > I just tried adding padding to some of the hot structures in our own > > application, and observed a significant performance improvement for > > those. > > > > > > So I think this technique should have higher visibility in DPDK by > > adding a new cache macro to rte_common.h: > > > > > > /** > > > * Empty cache line, to guard against speculative prefetching. > > > * > > > > "to guard against false sharing-like effects on systems with a > > next-N-lines hardware prefetcher" > > > > > * Use as spacing between data accessed by different lcores, > > > * to prevent cache thrashing on CPUs with speculative prefetching. > > > */ > > > #define RTE_CACHE_GUARD(name) char > > cache_guard_##name[RTE_CACHE_LINE_SIZE] __rte_cache_aligned; > > > > > > > You could have a macro which specified how much guarding there needs to > > be, ideally defined on a per-CPU basis. (These things has nothing to do > > with the ISA, but everything to do with the implementation.) > > > > I'm not sure N is always 1. > > > > So the guard padding should be RTE_CACHE_LINE_SIZE * > > RTE_CACHE_GUARD_LINES bytes, and wrap the whole thing in > > #if RTE_CACHE_GUARD_LINES > 0 > > #endif > > > > ...so you can disable this (cute!) hack (on custom DPDK builds) in case > > you have disabled hardware prefetching, which seems generally to be a > > good idea for packet processing type applications. > > > > ...which leads me to another suggestions: add a note on disabling > > hardware prefetching in the optimization guide. > > > > Seems like a very good idea to have this in , and > > otherwise make this issue visible and known. > > Good points, Mattias! > > I also prefer the name-less macro you suggested below. > > So, this gets added to rte_common.h: > > /** > * Empty cache lines, to guard against false sharing-like effects > * on systems with a next-N-lines hardware prefetcher. > * > * Use as spacing between data accessed by different lcores, > * to prevent cache thrashing on hardware with speculative prefetching. > */ > #if RTE_CACHE_GUARD_LINES > 0 > #define _RTE_CACHE_GUARD_HELPER2(unique) \ > char cache_guard_ ## unique[RTE_CACHE_LINE_SIZE * RTE_CACHE_GUARD_LINES] \ > __rte_cache_aligned; > #define _RTE_CACHE_GUARD_HELPER1(unique) _RTE_CACHE_GUARD_HELPER2(unique) > #define RTE_CACHE_GUARD _RTE_CACHE_GUARD_HELPER1(__COUNTER__) > #else > #define RTE_CACHE_GUARD > #endif > > And a line in /config/x86/meson.build for x86 architecture: > > dpdk_conf.set('RTE_CACHE_LINE_SIZE', 64) > + dpdk_conf.set('RTE_CACHE_GUARD_LINES', 1) > > I don't know about various architectures and implementations, so we should probably use a default of 1, matching the existing guard size in the ring lib. > > @Bruce, I hope you can help with the configuration part of this. > This all seems a good idea. For the config, I'm not sure what is best because I can't see many folks wanting to change the default very often. I'd probably tend towards a value in rte_config.h file, but putting a per-architecture default in meson.build is probably ok too, if we see different archs wanting different defaults. A third alternative is maybe just to put the #define in rte_common.h alongside the macro definition. I don't think we want an actual meson config option for this, I see it being too rarely used to make it worth expanding out that list. /Bruce