From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 95784A0542 for ; Mon, 4 Jul 2022 18:20:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8F519427F0; Mon, 4 Jul 2022 18:20:06 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by mails.dpdk.org (Postfix) with ESMTP id CAE614021D; Mon, 4 Jul 2022 18:20:03 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TPVBS+DaGFdo5eh0fexf5mCdTPoO15IIdkK+iVjUnKx1aJKiYfc7vR6dG3nC570c8UciGjgaM5z0+bepjujeC0Rus5vrhgOvU9EYfF/k2Sg50AEstKGak0Cp7WxGghdMm31WiQfuofnojuxWZKMU1DjgXuP/MZ7I8qxEOefIvRA1AdqVO1hKq/RC4iSck7OIeHlZ1CLDc7UmmiEMTwV/ZnpiDMj2jOfXrcP7gegX3BaclJPYtLux+LVAvc8f7n/k2O2Cdw9X/7WiC39R2PBw3umpTCGSg7YXtmfEMTUF3z+UfiIlH67MO12LOUBJGOH0OEjNIdPD0RvZveA/aSsSVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L9UcJNNlggBgOYFCB3ZkacBmb2LkNdIlr4678CTDQvg=; b=UYojB4hRwL3TKi6X2V9FtkNgZXijniWGY81Vbo5XiHAaZrDJWBleaVy/0LSXtssyclQEYHXh2z7MXwc4dI4XzPvZgxhj2Kg1ebFH2LoNZeWf+p1SQnS6Nocsx97GFzEgtgEqQeqQirzhk7UqoBH4IAKfo1DqChDKW1Ahay3Vis4xcPxkKD5bWkoNVPQ70qyqFfpLT9vsM/cHA49zo8XEUbaL2MIFV7V2WdrcODKwaxT5joH7eRsVADJMuV1gYNFWYJ+u6zD1YKK+ugDUsC6MjNSOScgJhhnvee9JtLHV8keZfEyI+EXqIeJ7oiD9JD8et1anXszGmh7l7E8HUm2/Dg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L9UcJNNlggBgOYFCB3ZkacBmb2LkNdIlr4678CTDQvg=; b=JrqHW84RoOhirh6srYA1YVuL/goyxjN5B3sR+QZe5rKKeLIFZ8v4OEABTXF5Np6K2UMDboeQj7tkbHD6WTWB1QylRCI0yq9U8m+b5zkgLN11jEOwvPkxmbXO1MeSazMNBgS90JwkusE3xNstHCI5vHcCbpz6/0a+KhuzmNsyfhEgqn55G4LBsZmWRxSlZnefDk+fVkAyYHkWuexGazs2XC6KzP1SuO0/+ibFjVgZ5k/8nAroSeKtrqOITPPh5BjpWnvNBf+ZFtVaoTIPoPtT6Sk2ZsgMtuWcYmG/CYsXeLhxaEfH59+hHn+eDk6KJtNL36qD/l8odTDONGxNHmHEIw== Received: from BYAPR12MB3078.namprd12.prod.outlook.com (2603:10b6:a03:a8::31) by BY5PR12MB3841.namprd12.prod.outlook.com (2603:10b6:a03:194::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Mon, 4 Jul 2022 16:20:02 +0000 Received: from BYAPR12MB3078.namprd12.prod.outlook.com ([fe80::bd2a:f22f:f435:7b36]) by BYAPR12MB3078.namprd12.prod.outlook.com ([fe80::bd2a:f22f:f435:7b36%5]) with mapi id 15.20.5395.020; Mon, 4 Jul 2022 16:20:02 +0000 From: Raslan Darawsheh To: Dmitry Kozlyuk , "dev@dpdk.org" CC: "stable@dpdk.org" , Matan Azrad , Slava Ovsiienko Subject: RE: [PATCH v2] common/mlx5: fix non-expandable global MR cache Thread-Topic: [PATCH v2] common/mlx5: fix non-expandable global MR cache Thread-Index: AQHYjAS78Ljvv2NsOkmHD+eq6I0mLK1ua69w Date: Mon, 4 Jul 2022 16:20:01 +0000 Message-ID: References: <20220624203505.86435-1-dkozlyuk@nvidia.com> <20220629220800.751719-1-dkozlyuk@nvidia.com> In-Reply-To: <20220629220800.751719-1-dkozlyuk@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 11cc55fe-ba01-4e4e-9ef5-08da5dd90c3c x-ms-traffictypediagnostic: BY5PR12MB3841:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pMYsQQ5Pb9U79Koj8CMaM76CG9IAvjxWfnfTuGg8g/qNWbRnlC0SV6QWnOuYa8l90rVnGl+AfvXmYQp0mT5HUlMMl0qhmdjDsgjVKvubDiDd/jhIrZdMZntObTPjFfrS/pvZ+tNhQnwMCmUyvVdGuW4o5kOly/Iljrv5wG9ERemAyQIEgRTTSLHo0KxY9djqQJFbzRcH8Zr8Z5RilO7qio88asRWpPWa6JIjBJhjBWj+QlqcvkshvfARW6spG1AmrIs+RyX8qARDQ7oK9ebmRbpPBqj+4dIXQAWCMkjjeLSs5x9FXNsdfiiPfrDh/aU5xvPyGkjig7RbLNH+dU2IwMxeGoHIePG+pri74p2yIRwWJpjXbJAgdeg7BaesyvyBqNRaVLbUKdm9NdxEtG86TiCcBEDeKioBNkVO9AGYcL9uzYaqa6Kz97RAmCpMYsrPT3n1QgCAl5On1guL7q11/tYHaxeyOYpF3FlIAHssYw7MhebaP0tNLtVTfJKbK48fcxYsH6Qf4e9hiOf2dQS6BNygpwXzVG/ayBQdA/jzPVzLEvgQjNv8JBr68OiJR9CCpyWYHc81Kh5QV5ejHKckV5x8AQS/D9vSgX2gG9gn5W5P+Ewc8HcSvkU8EYGLqVzlxl8Rq8lRGpu/Rs81YrXrrhlWlHEzw5P029W41GNAYFPsdL1bgNTZN3smMw8Jl+b4v1WvJY/rlDM2DvNSvY68ZJQSwZR+amrJpB9RhCnUS7YvUGR+wff2v8uV3Xsi8SOZQXke9qHvf8e+a7g1xUcqdnIRY7fZeW7hCGOjThh8gxQhnl49YAUMv1X9h1/uxggp x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR12MB3078.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(366004)(136003)(396003)(39860400002)(346002)(376002)(107886003)(186003)(38100700002)(83380400001)(122000001)(38070700005)(66476007)(64756008)(41300700001)(66446008)(8936002)(5660300002)(4326008)(8676002)(55016003)(2906002)(66556008)(66946007)(478600001)(52536014)(26005)(9686003)(6506007)(53546011)(7696005)(316002)(76116006)(71200400001)(450100002)(110136005)(86362001)(54906003)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XJvKSpMBS40PpmyEGusp3X9q7vb62ACEan7rvTZxrQ+atLmYzr1sy0L9T9g4?= =?us-ascii?Q?pNpvfU8VDj7aCSknrrLDVF4BL8ZNSzgr54G/dcihUCaSTOYw3XOkQOjRRevD?= =?us-ascii?Q?6u5sc8ZVOEDS4YMcmnXTScJZNSBTo10lE2nFqvpb9QLO8FXXSGeROBmFybXQ?= =?us-ascii?Q?SYBLrFBGk/FtorKTtgIYGbXcNTAQ3kGoZ6Gz3vQjPBqeXfA9gRpBFaODxLyG?= =?us-ascii?Q?ivQ+ihWcr8x683haPac1oa8Je1SClAWYn2juqGmtkHV1DboZMUi7lfgvvkeV?= =?us-ascii?Q?LYe1emwsHVkXWpwJon54BzANytJ1yvz63NN/j8zO1NZ6CIVJZcMsq/341FEY?= =?us-ascii?Q?hS6+LuL+Iw/lCPcqR2cOzP3pknoa2pMl23OJtZzxTAh1ByOFMRud3ZMfUI4b?= =?us-ascii?Q?JmBUZz88Bo2BnlC3P+Ou0eSajDE9vlqobHmOQ/RiwN+9fV2fHw/WzG1OhmNn?= =?us-ascii?Q?pnjAcFz8/WUYfpVPR/HDyrJLw4D07ETa8BM5xfApL6WBcxF1LEhi6uKYG7H0?= =?us-ascii?Q?02T4iISrAqoNW7PG6cw+QRzu58WcMxWLKSD9DMHDG8IOdljiV1I8Nw4FCA05?= =?us-ascii?Q?tASYTh7yCjtd+1r2IJ8YhAHJT0qVvMFsVHom/4FNIaP7KijndddbvAERUBeg?= =?us-ascii?Q?Ny435PskW95sGol3HL3JWtYJAvwZkJb0l44vwG/soA8vXSGTqsnuAlMKGq4r?= =?us-ascii?Q?3k1Wy5QuO89Kf0wig/X6Ci8LUI6j3AqliMtuoe/12D5SlOQF0MK737OowUs6?= =?us-ascii?Q?UwKOJ8YN2+cBh+oQ4bGgZGVqKIsXC4k0FoePQ8P6ge9CHEpom97oAwnmU9Mb?= =?us-ascii?Q?uxKdLRVtM6l6IpWV+Ph8qL5MR2png9ol2JMp8XTI1xRqX4ktgcKDIRnNpL+s?= =?us-ascii?Q?gBL8xW/euIOI3E5vTmE07SPObIFCxLQT+c6p/Le+4Pgmwe+hAf1CHD33VaaN?= =?us-ascii?Q?23XvFLli5UmYcCNvzJk6C8NlAMGjtB1Pi++W4hUnWK4NdP7ctmlXOLBnTUXm?= =?us-ascii?Q?l3/3vVnwZrEkzBoKN3kzKXaTa2sIDkCK9cmyhTeyZ6zJYnWGpr9aXGh/9EJa?= =?us-ascii?Q?szDbBpDHr9IW08HaXWCK7lozDNYtu6mTttqiLoCj2sQayBpZ5d3hVlwhYGEA?= =?us-ascii?Q?9j2kuV9T0qcXk0rna4fMZgAsRjmKX2Cv/ZRM0G4uFVQyGp2uQ4certCr0BI2?= =?us-ascii?Q?yzYWx2ExPCDiK4xtGl2o3M/zOFJB6yJWsf3PG29UaC1of2H0zlbaX42Y0cSc?= =?us-ascii?Q?jKyqj7+/16CmBvQc1Ef3TYoRnYDREZmpMTMXsbH/uqEvS++vUWx3U1Z7LDIc?= =?us-ascii?Q?Abxy/nAYePwAdmUx44eRXu8oSyHeyFWBvHY7/w0KPXEQOo5IZX2nMCeOH2S6?= =?us-ascii?Q?12IuQFXOnxvunUSC51ohPSjNmVLVJYjVt/xcVgBnP5wd59fIggyNHfy9VdGy?= =?us-ascii?Q?f6fC1kLwXM/4seOL9vBPhosa/twEZxKs/99cO7F9uHwsW8XxjgbSTxoNtgka?= =?us-ascii?Q?ixSVLMIUVYY/Wsy3jBjxU7KoRf0K1GtcMoNrdT4cjIF2t+wmRGbrcB0ul9kx?= =?us-ascii?Q?Ij7MFluvffiLZXATbmublfEsl0kkiqdBwLoIXxhj?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3078.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 11cc55fe-ba01-4e4e-9ef5-08da5dd90c3c X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jul 2022 16:20:02.0307 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XwjQ26LjGY7E5ngoSBOZ+0V38ox+8W4QeUXn1Y2Z/Iupsj/2MC2MmYeD+nPNtye71SJa7v55rCY9SEVlre6L4w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3841 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, > -----Original Message----- > From: Dmitry Kozlyuk > Sent: Thursday, June 30, 2022 1:08 AM > To: dev@dpdk.org > Cc: Raslan Darawsheh ; stable@dpdk.org; Matan > Azrad ; Slava Ovsiienko > Subject: [PATCH v2] common/mlx5: fix non-expandable global MR cache >=20 > The number of memory regions (MR) that MLX5 PMD can use > was limited by 512 per IB device, the size of the global MR cache > that was fixed at compile time. > The cache allows to search MR LKey by address efficiently, > therefore it is the last place searched on data path > (skipped is the global MR database which would be slow). > If the application logic caused the PMD to create more than 512 MRs, > which can be the case with external memory, > those MRs would never be found on data path > and later cause a HW failure. >=20 > The cache size was fixed because at the time of overflow > the EAL memory hotplug lock may be held, > prohibiting to allocate a larger cache > (it must reside in DPDK memory for multi-process support). > This patch adds logic to release the necessary locks, > extend the cache, and repeat the attempt to insert new entries. >=20 > `mlx5_mr_btree` structure had `overflow` field > that was set when a cache (not only the global one) > could not accept new entries. > However, it was only checked for the global cache, > because caches of upper layers were dynamically expandable. > With the global cache size limitation removed, this field is not needed. > Cache size was previously limited by 16-bit indices. > Use the space in the structure previously fileld by `overflow` field > to extend indices to 32 bits. > With this patch, it is the HW and RAM that limit the number of MRs. >=20 > Fixes: 974f1e7ef146 ("net/mlx5: add new memory region support") > Cc: stable@dpdk.org >=20 > Signed-off-by: Dmitry Kozlyuk > --- > v2: fix warnings in debug mode and with assertions enabled (Raslan). >=20 Patch applied to next-net-mlx, Kindest regards, Raslan Darawsheh