From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 50A60432B9; Mon, 6 Nov 2023 10:45:17 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5C52402A9; Mon, 6 Nov 2023 10:45:16 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 678364027F for ; Mon, 6 Nov 2023 10:45:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699263915; x=1730799915; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=EMkW2VFT6lBZwAzr8B47ToPwpNlDYepM07C5CclKsq4=; b=MjBZaf11lKucU4N1qBT+xHKHQPjRpZV8u3ZgnDzB2wABLUokLC1Dlb39 DIVyTsvM9CnzYmI3PAIw1tSgsIZHuTr0u5gz2oOvDUbGWtSpt5l157b2c nyLjTQffXMihbfgMy50CYrMPFgYeDMQrJQIwMUJgwo5jANloPboLpr0bV L08Kz7xUsgzX6fWWm3INnL48vgM4l0d4Ky7nVj6umaq7sLx9YHHYYK2xo yp4q1m799bg8uwqcuEAs7AU1jIesOu9TdJkIpf97A/bx2P1P3xIIg0xiB VtqXwnwnXCi07LSmjvSRWHOdorrZK19EAab2hBoNL/tkWz0LjoLPTROaV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10885"; a="392110055" X-IronPort-AV: E=Sophos;i="6.03,281,1694761200"; d="scan'208";a="392110055" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Nov 2023 01:45:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10885"; a="935749725" X-IronPort-AV: E=Sophos;i="6.03,281,1694761200"; d="scan'208";a="935749725" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga005.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 06 Nov 2023 01:45:13 -0800 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 6 Nov 2023 01:45:13 -0800 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 6 Nov 2023 01:45:12 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34 via Frontend Transport; Mon, 6 Nov 2023 01:45:12 -0800 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.34; Mon, 6 Nov 2023 01:45:12 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KXBrUtiU9E3s0vPPVO+Bu/QlAfGsRlM17wpp76bE2L6wtQdGPxfKMtLiFhNZHSI8MyMsoqyHYFBSQ/WZROekWOzq5QQ+0L8eeRGEvGQ135pDZ7h0wskF+yXuTuWHnKcl9VRDq1CLBeUHzh6NFZvmFthTpU1IsH7aZX54cS07BUUQ8fN8fMBZ87D+a0AtN04lIKdYbCaOI05eSApG6rCQ5Zvp+5cPa6fvxVjz+1yTTbaJAzelt2gPgfwUmLyg9ckaeB16dB+kLfVS4Pbau3iItenR8iMuckLXzVZw2xQ7/IJhzqAPwB88iy0CTWHtrQpETCMXtTlZIyJbClLIzrV/1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XI+Dz3Ck39+R9Nq6QjPeUngZ+2tdMwxW4EnQ3Ip8hRU=; b=OUWDD1cIv6nzNZB+s0xUMeZn8+NsyUlvfaJAnqk/nTY9YgZQa6pQhs2QedptJHDZLrQc9dPbd+ec06zbGdouEfFHimI39l+OWBMo/A+JvPdJl2bsI9oj8J7QMTAjtRFXnLw0JCI8NleSA7pwmTWmPxBr0K9nPEWIDF8uFaLNYijskC0CcBFPGpNyzuX9iKBakY+nSluSQlfKDCANkJyWISkuiaYAPeJQ5KZnCW/g78yW+marxpihFTTdPGmD4FWwPFolViK7cYSooO8kZUPpQNfLvT5YzQutxMyOHwq40OUo0dUy6PyqMVjXtKFty/VB6juWMUqZ352H3Wtk2FILwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA3PR11MB7416.namprd11.prod.outlook.com (2603:10b6:806:316::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.27; Mon, 6 Nov 2023 09:45:11 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::8645:d921:ce8a:12ba]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::8645:d921:ce8a:12ba%5]) with mapi id 15.20.6954.028; Mon, 6 Nov 2023 09:45:10 +0000 Date: Mon, 6 Nov 2023 09:45:05 +0000 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Subject: Re: [RFC] mempool: CPU cache aligning mempool driver accesses Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35E9EFD4@smartserver.smartshare.dk> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9EFD4@smartserver.smartshare.dk> X-ClientProxiedBy: DUZPR01CA0028.eurprd01.prod.exchangelabs.com (2603:10a6:10:46b::7) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA3PR11MB7416:EE_ X-MS-Office365-Filtering-Correlation-Id: 03e8741d-f20d-4eb4-5cb3-08dbdead112b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: STrCxzIm1toX3Zoq+qE2SKhifPX1hyLdvYgg6i2kaNpiYXNRc0M7+5CnlJrdtoMKy0LWeTiasuyVsv73o/BIdp2eLva4reZm08sRtfad+NJJMzSwPCLYGXC44JogjMvUxK8hndLjZR7vzouXckZziIh8POhGptVttSYMvoW3NpMKuZ5fiojCOf3aYZVPcNJKzWbjQno9zoMRArq5CG/BQzV64dkX9l9t64sQk+eu6qEfv4YdDAKTMDOQNqo4ItvswLLae0VDzraFsMIaredPJJ8ds+blcVOBtfFXBQdP5QNfl2hnVGne537wDW3HlC0Li3va0zheyMFL/tpVzFWb7nGDASD0V+9BptlZHFL610OgZXdjtewai/fWmigns3gFE0Hmgk7Tdo6iWwAT5GIVF+MTO0SL78/Jv9ER7BgiTdNX+3V2IetxZfxC8U2HN/xtouQ0d9Lyl9neVkRXNNf8Ec+Hl6MgoYtGqtb2J86g4J+lgHyu4lZLpg4ZnyWRvnhHIzS8t62lFkjhalMw6tBMsVHCaKSLatYPJMr0OjTRlNHFgM2DX6WIk8L+SQ6C2QbS X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(346002)(396003)(136003)(376002)(39860400002)(230922051799003)(64100799003)(186009)(1800799009)(451199024)(8936002)(6666004)(66574015)(26005)(6506007)(6512007)(41300700001)(83380400001)(8676002)(4326008)(2906002)(5660300002)(44832011)(6486002)(66556008)(478600001)(6916009)(66946007)(66476007)(316002)(38100700002)(82960400001)(86362001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?XFI9UdtawXOziNkzqJNx2cpS3xONnJWui6kg57ifnfiWXb2X30zF8XsIVn?= =?iso-8859-1?Q?PTLDSOJfg8kIkb3D2imFwzZmYJJtqUKiVYvB4G7brPAjmkIOrnG6dEU+IO?= =?iso-8859-1?Q?totbwDEHd5i+5hpzXOZn6ZcMpmRh3j8B42FkFZq3UrfuiIPX/BfQnNH5m4?= =?iso-8859-1?Q?w4lOiKUWdBJR1MwWysYMLCdFYWxLtQIBsDKsuP908gHhERCiMuY6sFoOzi?= =?iso-8859-1?Q?H/dFtWXrNOeCpvSu499o/a6Rbfrtk/yS/64Q6YTTaathKHztXJhNaXvIY5?= =?iso-8859-1?Q?AUDfr7iQvlBAvmsTPZ5d0Ey5d4h30rb5Sxs+hfzUHjERlDzSgvukafCD2R?= =?iso-8859-1?Q?8gLQOxoq9Cfh6niJAsKCDopR0LfbeLnjGBUzZmYXgTY0YxPJTm2EuIwtvs?= =?iso-8859-1?Q?vDEV51rAirAMYlX8olN3jwW3fPHIJEa0Qg+2EBvZV0ihfqbSdobQRjpFPw?= =?iso-8859-1?Q?sTBUT/DlGRe1MvN6CJ8SIdY+yjeD4NsP6tjFAuKl4NQ+j4m27ZSUj1fP5A?= =?iso-8859-1?Q?JaXSARBq1XZKHF/UZPQLAdhXkXNGCrf4j1vb3YaXLwfIwxdjWMUivr7zX9?= =?iso-8859-1?Q?/5sjq9nK0MlIC+vhx/W2k4rsMWpdh29PfdPqHcMcyn/XZfs4yghU2gVlcY?= =?iso-8859-1?Q?hPQdfgoIPgF+g1MLm/YyCODnf3IVszMDIEvsXgwtotg4nq3IzhYvLhDbRE?= =?iso-8859-1?Q?fBDrDyta0wXaQqEDhc/wrKjizgk8hSmaB33PYmD8ra7OGt7NN/sRXzhSCR?= =?iso-8859-1?Q?V9AxPlsDV2SfNKU52TMCUMoK431a2ON9qrPSL/y3k6LiZJfd5dAcUW96sK?= =?iso-8859-1?Q?6719SpsCMvx36p+E9XRP5LOKRZb/2QF3WlNMCQh1dNqvx+kfJctaOSlZWI?= =?iso-8859-1?Q?HjEApGn1pKk4H6avAp1XEe/0ObGQIvkt7ahg/HkVFbcNWN0JQkVIdHG+H6?= =?iso-8859-1?Q?Op/FvxT/78Be/af7PKiyeACV/E0FvFsM6RXYHpKfboDMbotlQIt3h9Pym8?= =?iso-8859-1?Q?KR036OF0R7fqVhWNrSDF8ZHgLbvhoGhufaMX3wpbZezROvrfkXt2eheww8?= =?iso-8859-1?Q?X36tHl8ywLmVIuYGkNK79SQhKuxEF1weeSDoOhqmOAsbtHSpDNyHfwgYkn?= =?iso-8859-1?Q?aoIh9CqzKqD7pJGQNRingrQ/8yoqH0Asw5GlJdl2KXVjXdb2rpmoBQC0sz?= =?iso-8859-1?Q?zr5aW6N6MmunjNM70h31BeNxX4f8tW1E8+x9kPFDLxZs+xKEFgCPHcB0F1?= =?iso-8859-1?Q?jEbfDr0aemqIOMDwo1LuUBv/e7LhGn795ba997KQJ8tTG/6CWEF4k1pZq5?= =?iso-8859-1?Q?pou0NBwxnd9IsOiqQ2IL3TcT4akQPuvPPCThyWdyZyBM0K/o/wAADqSIVN?= =?iso-8859-1?Q?0i6dI8vn65dNGbwTPJnBd+Mwt82r5CJMIld3IMSktspTWm6mFTebDCdcrz?= =?iso-8859-1?Q?KUCk35JkRAvV5LiFeRrjSJ6MUlReadFLEauZTV7IBvmFBOHhhJZKf9MQbz?= =?iso-8859-1?Q?escSqLk3phRIUGY1lozEVHom2JYDCw3QJZQek2L4Xo3+0buHmBphNMilxX?= =?iso-8859-1?Q?QXsm/LfokHosnV7hQJRmAQ5X1s829h/p7LtR1unRy3eD/bj1p/KbdqgBWD?= =?iso-8859-1?Q?ahUI7cW2HnV6W4O14Lqylr5fy9dYSLFeIsqz0dG+gkUnJBEKxKDKTlWA?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 03e8741d-f20d-4eb4-5cb3-08dbdead112b X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2023 09:45:10.6597 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7ZdK3U5CgFvNqGkDSWzyjBd5ruFlX8SiRYB7MeYXtIu0359gRY9OSHWs2Xml/rVqZPESR6+O5CYCULKeAQZe/+sE5jSW3fX/MGH6oB/61eU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR11MB7416 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, Nov 04, 2023 at 06:29:40PM +0100, Morten Brørup wrote: > I tried a little experiment, which gave a 25 % improvement in mempool > perf tests for long bursts (n_get_bulk=32 n_put_bulk=32 n_keep=512 > constant_n=0) on a Xeon E5-2620 v4 based system. > > This is the concept: > > If all accesses to the mempool driver goes through the mempool cache, > we can ensure that these bulk load/stores are always CPU cache aligned, > by using cache->size when loading/storing to the mempool driver. > > Furthermore, it is rumored that most applications use the default > mempool cache size, so if the driver tests for that specific value, > it can use rte_memcpy(src,dst,N) with N known at build time, allowing > optimal performance for copying the array of objects. > > Unfortunately, I need to change the flush threshold from 1.5 to 2 to > be able to always use cache->size when loading/storing to the mempool > driver. > > What do you think? > > PS: If we can't get rid of the mempool cache size threshold factor, > we really need to expose it through public APIs. A job for another day. > > Signed-off-by: Morten Brørup > --- Interesting, thanks. Out of interest, is there any different in performance you observe if using regular libc memcpy vs rte_memcpy for the ring copies? Since the copy amount is constant, a regular memcpy call should be expanded by the compiler itself, and so should be pretty efficient. /Bruce