From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2027BA0093; Thu, 13 Jan 2022 11:37:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A37B24271B; Thu, 13 Jan 2022 11:37:35 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id B7B2740150 for ; Thu, 13 Jan 2022 11:37:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1642070253; x=1673606253; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=ie76AMd5FySeT9glRCOir8YmQrupmjF6scFQmU5VlJk=; b=dSAVo4Ga1yuK+Pc/Trkuv6jo4xorAnYE+y/3MKg+NkNH3w3N2Zl29gKb yt6duia6i6kW6YAhc8j1cIQteM0Y17RYxnSYKbzynuYzQ3wCofqyxQLfp +PTWGCt4OuEaqyLjOEosNNv4d4ySVEYbavAWDvI25iAYv1pBAbjhYSbqq v3O7Zie3DeBelX8xMDYb6MWT9wJGyWMwtp2mTOxKKO9OABCvM0CGL+XkF gFlqVpNLUlem0R7AuX38pMmzIRsxtYFbZ1ILL5hbTe3EPOn/DQ+XzVBZo rVq83wPG/9SlzUZ3jeyUTIejssqo71qGQE54QxeeIzss2AfF6iuJr4ajP A==; X-IronPort-AV: E=McAfee;i="6200,9189,10225"; a="244186461" X-IronPort-AV: E=Sophos;i="5.88,284,1635231600"; d="scan'208";a="244186461" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jan 2022 02:37:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,284,1635231600"; d="scan'208";a="559063192" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga001.jf.intel.com with ESMTP; 13 Jan 2022 02:37:32 -0800 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 13 Jan 2022 02:37:31 -0800 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 13 Jan 2022 02:37:31 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Thu, 13 Jan 2022 02:37:31 -0800 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (104.47.73.177) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Thu, 13 Jan 2022 02:37:31 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZaTUNrg4Y+vE+rKoYVDP5C1A87fF2pmPUfDGQ5dW9tKcROfU6Z3Ss/OPz8zaxQGBYHX39ffzjW5wTGO5XmHiAm0lS8CpHXiSHgF9Dw7B0BGyrp8AXSPuJgrMGf0HfwubZ6nDMwHedKQkdriNaTqzViLCDA84e08ZUVS+YZ49oEPEIXcVVNP2RwEasnwiBkhK7LqQasQZN2N+h5gJaKNsVRUWn9oBA+vhBQ2zcs9zslnkUFLx0XcCsWD8OEQmUm4FsWkzbkmBi7lo7G5wU5Yb/MjJlg1OTMnLv0ZkbqMxim0ox020L0z5hzwFIgz2Rp63XhnFtU86PFLLTyyCPp8c2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y8H/qdfW7XONQFMjrIetq7PQcXXxBpt9hyt0zgSVHDg=; b=CuvhrdLrQnYMO2yIYhs7BeSCkRZh+ZnFIbUUuBTeFRKZqgoPdH2c2zKj6U6iwyP9kSpetnJuUCHsKFZsW7xJUbRtwhifZpiug/t+P9hgSGhRmtPD+vNRm3abncfoXMFw3c8/OzTxfvOgxDdr9++HUT6X06DlDvEH+nCX3ALyq+63X5g/ZvcStQjMoA8yunXT1geCYezZVbPNrWAxtFHXtFPHt0o3hhgVEIBS+9TUZtRMUPplCI4eI5x31DKyDmEcSTRWVQn556DYUksfNDzDE4b97PeaOZLuHQMGehGFmrQDLco6KQ621CZge0/YNPdJDzOghVfsmhyyVm7OFUZEUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19) by BN9PR11MB5547.namprd11.prod.outlook.com (2603:10b6:408:104::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.11; Thu, 13 Jan 2022 10:37:29 +0000 Received: from DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::7de4:731c:cee2:49c2]) by DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::7de4:731c:cee2:49c2%3]) with mapi id 15.20.4888.011; Thu, 13 Jan 2022 10:37:29 +0000 From: "Ananyev, Konstantin" To: Dharmik Thakkar CC: Olivier Matz , Andrew Rybchenko , "dev@dpdk.org" , nd , Honnappa Nagarahalli , "Ruifeng Wang" Subject: RE: [PATCH 1/1] mempool: implement index-based per core cache Thread-Topic: [PATCH 1/1] mempool: implement index-based per core cache Thread-Index: AQHX+RoX5eYid0d4TkS1YP4buw/aYqxctE0QgAPTL4CAAE7DgA== Date: Thu, 13 Jan 2022 10:37:29 +0000 Message-ID: References: <20210930172735.2675627-1-dharmik.thakkar@arm.com> <20211224225923.806498-1-dharmik.thakkar@arm.com> <20211224225923.806498-2-dharmik.thakkar@arm.com> <8F6CF7E6-BD3D-424B-A7E1-DB6E53276DFE@arm.com> In-Reply-To: <8F6CF7E6-BD3D-424B-A7E1-DB6E53276DFE@arm.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 790f6541-dc6c-4d09-33a7-08d9d680b2dc x-ms-traffictypediagnostic: BN9PR11MB5547:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: GEAKxq6CAovGe/9pFhaJsCR4ujBjlBo3ZEhmgBpjD7LOJMG5KnO/LHgvju9ocYMFkeBKw64C2jsn1WYAI6wLbk0DOxu3qz7vlOi2ilbYPiHKVlzi6ECvG7oOIaqbFRmT5rt5M2VU67A4jF3QXeoLCgajnUP5bRQ3k8Isq5rsepuQ2SHAVx1c4teIq91ElEysdA4aBWSuVFX4MRrzf4cB0zJcAnpKjBjkN4jBaMEKjOJrCQRDllcLQDZlqXCGdpynefyA5YRBtxORMaippwJA6lyRS+e9hN8R4rmBBJcF/7C5S/daI7kQ6Bjtw6J8/WRHImk4h8FYeAokdnU2OpoXM2R1d1ZaldC3q4oIPuwlJ2fGIlKfPJGncjTy7nwModG3QZXCHV2y9KcfzzTnicksYrT6agH3vu33nnslplDMBWf/BjyWsczOmyK8i9SLxVlV1pYsBqjGGaWSe7SEHJdyUwjRQxBlfOM2St9I5HD0aOEeOkG73vSvgYD4GK0z1ki/icbkVvJA+dfee1oUfrz+zwm1Q2fVtEiNYnbb575IqginOzNqC/kBIR7c83ZNWIQzh+bvm1VkzLKFGZ/MrMVC5uHYyiKZXIdNcYYI89nRgCkGZ/zGspwLSXhU9EpO054ehe6VYnT8llpYZUbGZ4SUI47n04xFGOWa2x78VFuQoIDuzZip1zirGGnstqXi5SD7H5aQ/k8aRKx6Q4EfGI2Flw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(508600001)(5660300002)(76116006)(9686003)(4326008)(64756008)(66946007)(66556008)(66476007)(66446008)(52536014)(55016003)(54906003)(122000001)(71200400001)(186003)(26005)(83380400001)(2906002)(38100700002)(8936002)(8676002)(82960400001)(33656002)(7696005)(86362001)(6506007)(38070700005)(316002)(6916009); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?RgdGloLSDxQojQXJ4i5oEzVRdS2Pfoi7Qs1KlK/LAuTvTlUYTt9K33YusG7O?= =?us-ascii?Q?Q5nGZvNTa8Wyn2LC8GglvQPxUA8y+d5J6BmFYHKudBG+CoWbSgVPD8Y6C2VA?= =?us-ascii?Q?xQdiYXqZ1i2QNzapO12+zuqNKm0DmjYEKCh0Xf/fOYwRDEUsk0kAGMsaLGUo?= =?us-ascii?Q?fzLL/HIJR57USyKetuQ0j0xY961ZRpJsrfdjBQWpPrWuoCAdpyZldXJhr2A4?= =?us-ascii?Q?4p1uFExiO8bAGpdCNxPGFahjyXlxLo8Hl6R4ZEKqoZXCY6+pjkaBbWnfknwh?= =?us-ascii?Q?pZQT9TRc5TOnvlC6FmlyOzmCjJuJAU7VSy7esJvWSkdMjSq0NTzgotyrJw9j?= =?us-ascii?Q?yLXL52h2k0K3EsDX5aKCHTIeukLpMbG1GyykocDbIpikKtkYlraJqG36BuCp?= =?us-ascii?Q?3OlEKHOreVsdeEqwZTrMVfbG+tM8udZOdaMdLNrNf1lB0hq2+Cd4x7x8F9km?= =?us-ascii?Q?zkskC2ZDDedWeladGfGScXgeEnJC5OW3Lmz/FEFaRpZP3S021sr4v7S9cOXz?= =?us-ascii?Q?f7EE4zn1FJmLzww5GOsaDrJykuqRUEgEmQTAYcddFFvRjeNAWW9uEDZCgbo9?= =?us-ascii?Q?MHOg1zXRbkubCe11fMSU9On8GWdEvsIxYVQ3S173Ys/J/SAV5Jjrd/87Gqdv?= =?us-ascii?Q?FsJyDXFAlIFW98mUKRIOlPqgYzqxucANxbVhIoSnEtspWzpNmgj+V7k2d6St?= =?us-ascii?Q?DpIvXN7EieF7WkP2HJru/40K9C6HHun12g6eKUV5FeAF6roipaF2KKDw+Thy?= =?us-ascii?Q?HR6YsvJbNZq7JFxOy6eUJw3EXJvbTFKKdTpeavTMyqWDGeIapI9x5rxFQzGP?= =?us-ascii?Q?dupak3IX2gVfn7dRnpd89c2VufY5fomagOIwRUPBjOu8d2Qn7SCbkD6D9Vu2?= =?us-ascii?Q?/gUx+yM2spbIjYl96ZbuJNBDp8Xel9IxNeQ+hzkSngl5YecdWo4XfJK7xrxF?= =?us-ascii?Q?zK9kNUBEZrY9Rv39HR4p1h8C8T5c1isHZkq4ThafWFfLrgkiIDszbNurcJuh?= =?us-ascii?Q?gUwGQwBDiMyI+rp4bL2Pz6L66pTwa0X7c0GR+aid2UVpF0dj3lH4npNaI+/h?= =?us-ascii?Q?8q07epohP3238k3U8rqfrPyrYWydZHAulC354qp1Ab1ZBCrRupx/js4ywnbC?= =?us-ascii?Q?vETodJOodehkaKxSi9OfkmMQ+6pKD+SqHcZ5X7iEvWZh/2xWCCWBVby5whuQ?= =?us-ascii?Q?QLBZlAbYOxvvU3ocI1Dwd80+54TNTlnzVSUM88z/qBboMXIyI23vUXZHJeK7?= =?us-ascii?Q?X5GO8zbzDW0HzsF5t2EfsMn3oOc9T19W1itq6SPtMw49DugH2is2GXS9w7q8?= =?us-ascii?Q?GjnTW3NOle6oehz2FMBbMHvG1NwJk14ctak/pBfhOYSLU6ASRqNy8in4lwHE?= =?us-ascii?Q?W35NKaGj+CYrBQ59a5PEMP3MYz+dy813IDh+fy+O/TzibKN3ChH70YVqGOJD?= =?us-ascii?Q?i/jPuV7ldMJBQwPlBoPh0Vu9yKAnWDwO5gq4gDTI8SIP8COaeLxUob94BJdx?= =?us-ascii?Q?q8PxVNYvxQ7BQZKyJ4noaXXeg1QSrvaWPMWwfbT3MimI7CEFJKYl3Mv5Hls0?= =?us-ascii?Q?GLA/VVeKCKAccPmu/KodBUigUxSF/MHRCM60GbmTxf070sYdGADbgbrgirbj?= =?us-ascii?Q?0OALCzB3mQbU3zRhyzEKRA2Pe/GnApGuU3qbrQ40r0UGleNw24nLf7qaCXEl?= =?us-ascii?Q?i4q5Qg=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 790f6541-dc6c-4d09-33a7-08d9d680b2dc X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Jan 2022 10:37:29.2850 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IF1Lewv+95dku68GaYTZK+D29R0Tzb1zLHN9q5ywH77gD3D5VbRy8qEan4K+QQNyjEBG8O3v3MGD+z8wtS8E2h8BA2NRddP/2FSyHr6O8Xs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5547 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Dharmik, > > > >> Current mempool per core cache implementation stores pointers to mbufs > >> On 64b architectures, each pointer consumes 8B > >> This patch replaces it with index-based implementation, > >> where in each buffer is addressed by (pool base address + index) > >> It reduces the amount of memory/cache required for per core cache > >> > >> L3Fwd performance testing reveals minor improvements in the cache > >> performance (L1 and L2 misses reduced by 0.60%) > >> with no change in throughput > > > > I feel really sceptical about that patch and the whole idea in general: > > - From what I read above there is no real performance improvement obser= ved. > > (In fact on my IA boxes mempool_perf_autotest reports ~20% slowdown, > > see below for more details). >=20 > Currently, the optimizations (loop unroll and vectorization) are only imp= lemented for ARM64. > Similar optimizations can be implemented for x86 platforms which should c= lose the performance gap > and in my understanding should give better performance for a bulk size of= 32. Might be, but I still don't see the reason for such effort. As you mentioned there is no performance improvement in 'real' apps: l3fwd,= etc. on ARM64 even with vectorized version of the code. > > - Space utilization difference looks neglectable too. >=20 > Sorry, I did not understand this point. As I understand one of the expectations from that patch was: reduce memory/cache required, which should improve cache utilization (less misses, etc.). Though I think such improvements would be neglectable and wouldn't cause any real performance gain.=20 > > - The change introduces a new build time config option with a major lim= itation: > > All memzones in a pool have to be within the same 4GB boundary. > > To address it properly, extra changes will be required in init(/popul= ate) part of the code. >=20 > I agree to the above mentioned challenges and I am currently working on r= esolving these issues. I still think that to justify such changes some really noticeable performan= ce improvement needs to be demonstrated: double-digit speedup for l3fwd/ipsec-= secgw/... =20 Otherwise it just not worth the hassle.=20 =20 > > All that will complicate mempool code, will make it more error prone > > and harder to maintain. > > But, as there is no real gain in return - no point to add such extra co= mplexity at all. > > > > Konstantin > >