From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 986C641FC8;
	Wed, 30 Aug 2023 04:18:24 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 5ECD940A76;
	Wed, 30 Aug 2023 04:16:07 +0200 (CEST)
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2121.outbound.protection.outlook.com [40.107.93.121])
 by mails.dpdk.org (Postfix) with ESMTP id 5C2DC4064E
 for <dev@dpdk.org>; Wed, 30 Aug 2023 04:16:04 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=ChFTAu7BB5rxHHPNYrYTXbmGprLrQPicK2NrcGepz+vD84VYvV5CpZ945JKS02u/B1hzgjV5js+sWXUR/mjqG1AE3JB2Tb5Jil5b5ZqHkXNQvA2nyJi3M1HM2EBTKK7qQ/VctzBo91D2Yv1RA2nXYIRF0JfYri9YEArmrOps6nencndQhS9xisfoYQFgQgCp48rbh4gyj0CH/E//8PYX7CX5Q9k5pF4B4JDvOC63wo5FLMjirdmJCD1rXaaK5xcYI6vFgZQKz6bIutTnwjiF1OlwyiG6jPaQzl8fh+BeTY5wRStpMHUu/t444oLdh4ge23TGVBg2vKpUUryzmLOKVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=PW2xHWL6QNyk3Ff7FWSfXySy6+lIgvRhxzehARxsxgc=;
 b=nAWdido8QVihu2/xDsFgwXmzWrAnZA7t/YfSKWUh6J0G0uYJq/Zle9oQES7Qpo9TBMHAnJ0NlMhrjXZAe2u/MrB/oSn+OHrGQfmtlg1TtVTZZ1ymv/TXI/QEvQ3+1+OLLEGtkJS1zOgP9ycbYGwH2btyL9qIpMaexNGMB+OQZECqU9S1inl/jpoPbvklSJHCafzK9pSviVPDMaqjKu+u0fk9PDX+eDAA+Ogu7mUKtBI4xqGQIHTN47SyfFbhxX/6NSzed+Jh6mVDOtoYNm7ScVuoI5d/8JC7LrJ4/kysIY/X4rR5f0kh7ACDGz8rrLwuH8Un9eIqr5NyjXP62HNttg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com;
 dkim=pass header.d=corigine.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PW2xHWL6QNyk3Ff7FWSfXySy6+lIgvRhxzehARxsxgc=;
 b=C0ITUEPNexH72FuiLF2cgPKG2c0UDocX9uKEo55qLngyDfKv9C32eGvJGlLY0PuK5eHs0dwgvFXgg49KtdG38ry8OmpmlPgFtWmvdt8WTfL/C1MJHWUIoqEL4zWxXQWGYKApLpDNJXI/ffJGBF4l4w4uiyZ0u8dt7zJKJYT6JeE=
Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=corigine.com;
Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5)
 by DM6PR13MB3786.namprd13.prod.outlook.com (2603:10b6:5:229::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.18; Wed, 30 Aug
 2023 02:16:02 +0000
Received: from SJ0PR13MB5545.namprd13.prod.outlook.com
 ([fe80::c0f3:c2cc:b5bb:4192]) by SJ0PR13MB5545.namprd13.prod.outlook.com
 ([fe80::c0f3:c2cc:b5bb:4192%4]) with mapi id 15.20.6699.034; Wed, 30 Aug 2023
 02:16:02 +0000
From: Chaoyong He <chaoyong.he@corigine.com>
To: dev@dpdk.org
Cc: oss-drivers@corigine.com, niklas.soderlund@corigine.com,
 Chaoyong He <chaoyong.he@corigine.com>
Subject: [PATCH v2 20/27] net/nfp: refact the mutex module
Date: Wed, 30 Aug 2023 10:14:50 +0800
Message-Id: <20230830021457.2064750-21-chaoyong.he@corigine.com>
X-Mailer: git-send-email 2.39.1
In-Reply-To: <20230830021457.2064750-1-chaoyong.he@corigine.com>
References: <20230824110956.1943559-1-chaoyong.he@corigine.com>
 <20230830021457.2064750-1-chaoyong.he@corigine.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-ClientProxiedBy: PH0PR07CA0113.namprd07.prod.outlook.com
 (2603:10b6:510:4::28) To SJ0PR13MB5545.namprd13.prod.outlook.com
 (2603:10b6:a03:424::5)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|DM6PR13MB3786:EE_
X-MS-Office365-Filtering-Correlation-Id: 2a82a9ea-d404-42c8-30bb-08dba8ff0ef3
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: BTannSoWbzdp4VipzR0ztAFc7nG/TmnsnPsV5WwrFP7izGL28mYVZo7gRex3uG1BC9jUybXlfTcJZT2bB9AuFemPzqw4yoBSprVcw6M9LXZCc3Ot5Aj1xubqV/5ye0VeYK1ffhYzLr3sUb6gzQ6uTYNofrc7k6kfU2iozHZhUcLieN8KeAQDN/X+MYBy2ScsWGtvXaep0RbZ+owS2ssyfaBEbWpsGf16VbAyABMBJOGg4QRfxLk3r2EIDVvX5A+Y/xH7qOkrC4if6hdKvvBiwMCcHttKbhyDio3dDnphVGF1DyxJ2FPiap8AHQgVuaeHfC/ElVu8NhJFuOs9SxwM0vtoXf2Np/1mDQSqAQD3i9D4wi1zg87b9icIuM14GOegPBZRmH+YNldrOyYgPjvWJpCaKxU/zp96WOhOl/8qJZiMIwyqVbbA/DtSIvmShpkUrjOYakL4X+K1omLGUPwOkdfLFBkUuk40u4Hl5X4yJwTO5ramRfrsP9J8sBx/Kv4LASve6C60ZzGvmsjyqwUKdzWPe07fvMFleCeR1j4usVeUZlNyetnL9Pt+7jGJFXn6id2uwTynq1VC0oHQorvuy+6ji7e4Mm/N0nUP4hFKuroOrak/P+mfp9OsY7ONsIosUlJGUXlNJDkxQyAXadSNy0BDJd4OTlbff6U7vTRKNKM=
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(13230031)(136003)(366004)(376002)(346002)(396003)(39830400003)(451199024)(186009)(1800799009)(6512007)(6916009)(316002)(38100700002)(41300700001)(38350700002)(2906002)(4326008)(66574015)(30864003)(83380400001)(2616005)(86362001)(44832011)(26005)(36756003)(1076003)(8676002)(5660300002)(107886003)(8936002)(6666004)(6506007)(6486002)(66556008)(66946007)(66476007)(52116002)(478600001);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cU91ZW1zTjdLZXZjYk5BT000SkdGMlVaMTdrYm5XOWQ5ajl0TGw4T050d2tR?=
 =?utf-8?B?Wk8yM2ZEUUlIWFRsVWk5cXgrVVdxemtrSnRIYngxb3NnZ0hwQ0Y1SkFGSk9D?=
 =?utf-8?B?KzVIYUZwYk1lc0d0UmZKOUR3SEhrQU8vcC9KU28vWGxuUU1XV0cxUWNhMHpi?=
 =?utf-8?B?c0g4TkNkbTlWOW5nTEhDV09lOUdWRGNobkROVm8vYjhNOGErRkJpZ0loM1Q5?=
 =?utf-8?B?aWtqN2dFeVhjWWg0UWFlNjR4YTFZc3lnSGtaRXZSQUdqY1kzN2ZHUUMxS0JN?=
 =?utf-8?B?eWZBY2VtcVdqSnBUMS9GaVRmaE5rQWMxVjdibUxZd0RiK1JzMFk5aFh6UXh0?=
 =?utf-8?B?VWR0YTJ3b3VpN1ZCUUlScTBEeC9nWUhIMGFORnplZ1R0YzU1RXMxMmlRMlJh?=
 =?utf-8?B?VlZRQ01WRDhuWkxYd0IxTGlEM0tFMFVGQTJEazk5SHo2NjIwaFJUWG03UklQ?=
 =?utf-8?B?clhFbTVBK2phdXBKcENvZ2IyaUZzVkdJUnY5UW5iV0UzZStySmEvc291aU5T?=
 =?utf-8?B?YVpEWTV1NW5OYWF2OG56QW1DdTN1T21zQTN1NVVTY3pNK3JNdnhOOFFDakls?=
 =?utf-8?B?MFNRUXhYSVFvYVVHOElqYVRLUHZMaXFZT0xCMDR2eHF2MmVScHlEbVgxYUdV?=
 =?utf-8?B?N1I1UjV1Ym56WkxxQ01aME83RHZkeEtRUHhQZWRheVZIRFkrV3VBUDJLbTRC?=
 =?utf-8?B?U1ZjTlJTSlpoKytBdlM4MG94ZldOKzNkTFREbFlrbnU4WDZCS2NQQVJKYlRG?=
 =?utf-8?B?M0NWVitQZk0rSGhXTDJlYWh6Kzl2MmQ1T3dWSnZFOXk2Q1JFd0sreWdOaW91?=
 =?utf-8?B?RzM3dngrOWt0SUxEbmloOWZCTGJaR3ZiTzNjandHNmRCemE4OHBtVEZ5RDN1?=
 =?utf-8?B?TWdjTHFNRDE1YW1ZMzJ6MWtoYUcwckkrUFNKMUJmaU5USXlpMURXNzZqSmIw?=
 =?utf-8?B?eFczYVlMckdzaGxHOEVidys4RlU0cDM3RWk2cU9ibHYxeTc4L3JEbWI5RWFP?=
 =?utf-8?B?K2F1UjM3M3ZqWWwzY0FUNk80VFI2aEh1YjBGb25ZcUF4bXF6MGNObGNxeVBY?=
 =?utf-8?B?YTYxUERyWmc0QnVad25hUklXZjVPUkJocDZIbktSdHU0WFVTTUFldkhoNHkx?=
 =?utf-8?B?S2RTWjZZb0pkUTlTLzRJSTVDNXJrS0hJRGFualBQcStQWXBwUC9iN0hDOUlL?=
 =?utf-8?B?ekxENlRFVjBtUFRraUhiTDBIRzMya0lEUnlkb2hoNmVwQ1lmTDIxam44cEJr?=
 =?utf-8?B?bDdiRnZtaEpZTzhiT3ZBRnlkc3I1WnM0WHp3bDFFWXEzZXJhMHJHNzhnbDV5?=
 =?utf-8?B?ODVnTis2ZDR1ZE96aEdTSHhvakJmSTQ0MkIrUkZtcmlUWUp4SlRKL2ExcGhs?=
 =?utf-8?B?M1ZoZEU2YUNJNEhCMW9MUnJyd1ozbVozbmxnSkJOYTVLZnRkdTdoU0Z4Tlpu?=
 =?utf-8?B?cEtUaWMyd3lydDFEVGdyYjFDUHBOZjNBQ0JyVFNaekNDOXBCQ2ZmSmJSVHkx?=
 =?utf-8?B?OXdxRndXbUYzKzVnTi9NekJoSU8vc3FZczhLQkEvRnRjbWVSaFhET0cyeGpp?=
 =?utf-8?B?cWdWOWxKSDBLWG8wVmhGMWJEVU1zK3F5TVE5eW90TVo3aUdXeXNLc3h6WmNz?=
 =?utf-8?B?OEJlb0NmeEVBdW9hakZieUlmbjY2VjVoTTQ4Z25OR3B6UC9pd3pzTG53N1Rz?=
 =?utf-8?B?U1JtTGwxNzk3czdHaGNqRTJHT0plMW1ScEhJaTFhOE5VM2NUS3VRTXhQYnRj?=
 =?utf-8?B?ekJ5U3VNckJ1RlRycnFzcGFSdEtkYzJDNTNWY2pJTGw0Z3VQZjM2aTF2TXRQ?=
 =?utf-8?B?OFN3MVJhMjRYQTlsb0JoZ1hBS0oxQngvR3BQVnBtRjdvdW54Z3RrdEdQcFRk?=
 =?utf-8?B?RkpIdWUvMG1lRVVrMVlMNFV6OW9oSlVwSGxPRGc4bi9VMWxLYzZoOGt4OVpZ?=
 =?utf-8?B?WFNWcWIwdG5FOVozbFlWY3ZiOXpiV0d3eGwwc0lVTE9nc0dVWDZsRmJPZlg4?=
 =?utf-8?B?UDZEOC91R25wSFF5VTRRRVY1Z0REN21OTk9xUTJIOEZKdWJKLzNvcWVtY01L?=
 =?utf-8?B?cHRNWWhObWw3NUtMV1cyTW5KMkVqbGlZRUZMYlBnRFFacnRjalExVUJwN3A2?=
 =?utf-8?B?UnRIMjFBeGloUnF4RmFDcGxsdHN2YjRDUERlazF2ZGMwNlFtdVNkNVJlSXlL?=
 =?utf-8?B?a1E9PQ==?=
X-OriginatorOrg: corigine.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2a82a9ea-d404-42c8-30bb-08dba8ff0ef3
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2023 02:16:02.7543 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ytvlFI579d7RnD53R2AxwUiXadBFCDl9B/BJ4i8s4E/n5zcOckVKHPXrOsuSUye0j2Jm8RQkTERhc8/fRqQG2+Buu37yxIum7N6/HyAkc1Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB3786
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Add a header file to holds the API declarations of this module.
Also sync the logic from kernel driver and remove the unneeded header
file include statements.

Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
---
 drivers/net/nfp/nfpcore/nfp_cpp.h      |  16 --
 drivers/net/nfp/nfpcore/nfp_mutex.c    | 205 ++++++++++++++-----------
 drivers/net/nfp/nfpcore/nfp_mutex.h    |  25 +++
 drivers/net/nfp/nfpcore/nfp_resource.c |   1 +
 4 files changed, 138 insertions(+), 109 deletions(-)
 create mode 100644 drivers/net/nfp/nfpcore/nfp_mutex.h

diff --git a/drivers/net/nfp/nfpcore/nfp_cpp.h b/drivers/net/nfp/nfpcore/nfp_cpp.h
index be7ae1d919..42c4df5fdd 100644
--- a/drivers/net/nfp/nfpcore/nfp_cpp.h
+++ b/drivers/net/nfp/nfpcore/nfp_cpp.h
@@ -8,8 +8,6 @@
 
 #include <ethdev_pci.h>
 
-struct nfp_cpp_mutex;
-
 /* NFP CPP handle */
 struct nfp_cpp {
 	uint32_t model;
@@ -426,20 +424,6 @@ int nfp_cpp_readq(struct nfp_cpp *cpp, uint32_t cpp_id,
 int nfp_cpp_writeq(struct nfp_cpp *cpp, uint32_t cpp_id,
 		uint64_t address, uint64_t value);
 
-int nfp_cpp_mutex_init(struct nfp_cpp *cpp, int target,
-		uint64_t address, uint32_t key_id);
-
-struct nfp_cpp_mutex *nfp_cpp_mutex_alloc(struct nfp_cpp *cpp, int target,
-		uint64_t address, uint32_t key_id);
-
-void nfp_cpp_mutex_free(struct nfp_cpp_mutex *mutex);
-
-int nfp_cpp_mutex_lock(struct nfp_cpp_mutex *mutex);
-
-int nfp_cpp_mutex_unlock(struct nfp_cpp_mutex *mutex);
-
-int nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex);
-
 uint32_t nfp_cpp_mu_locality_lsb(struct nfp_cpp *cpp);
 
 #endif /* __NFP_CPP_H__ */
diff --git a/drivers/net/nfp/nfpcore/nfp_mutex.c b/drivers/net/nfp/nfpcore/nfp_mutex.c
index 87a9351ff9..9363543422 100644
--- a/drivers/net/nfp/nfpcore/nfp_mutex.c
+++ b/drivers/net/nfp/nfpcore/nfp_mutex.c
@@ -3,21 +3,13 @@
  * All rights reserved.
  */
 
-#include <malloc.h>
-#include <time.h>
+#include "nfp_mutex.h"
+
 #include <sched.h>
 
-#include "nfp_cpp.h"
 #include "nfp_logs.h"
 #include "nfp_target.h"
 
-#define MUTEX_LOCKED(interface)  ((((uint32_t)(interface)) << 16) | 0x000f)
-#define MUTEX_UNLOCK(interface)  (0                               | 0x0000)
-
-#define MUTEX_IS_LOCKED(value)   (((value) & 0xffff) == 0x000f)
-#define MUTEX_IS_UNLOCKED(value) (((value) & 0xffff) == 0x0000)
-#define MUTEX_INTERFACE(value)   (((value) >> 16) & 0xffff)
-
 /*
  * If you need more than 65536 recursive locks, please
  * rethink your code.
@@ -34,21 +26,51 @@ struct nfp_cpp_mutex {
 	struct nfp_cpp_mutex *prev, *next;
 };
 
+static inline uint32_t
+nfp_mutex_locked(uint16_t interface)
+{
+	return (uint32_t)interface << 16 | 0x000f;
+}
+
+static inline uint32_t
+nfp_mutex_unlocked(uint16_t interface)
+{
+	return (uint32_t)interface << 16 | 0x0000;
+}
+
+static inline uint16_t
+nfp_mutex_owner(uint32_t val)
+{
+	return (val >> 16) & 0xffff;
+}
+
+static inline bool
+nfp_mutex_is_locked(uint32_t val)
+{
+	return (val & 0xffff) == 0x000f;
+}
+
+static inline bool
+nfp_mutex_is_unlocked(uint32_t val)
+{
+	return (val & 0xffff) == 0;
+}
+
 static int
-nfp_cpp_mutex_validate(uint32_t model,
+nfp_cpp_mutex_validate(uint16_t interface,
 		int *target,
 		uint64_t address)
 {
+	/* Not permitted on invalid interfaces */
+	if (NFP_CPP_INTERFACE_TYPE_of(interface) == NFP_CPP_INTERFACE_TYPE_INVALID)
+		return -EINVAL;
+
 	/* Address must be 64-bit aligned */
 	if ((address & 7) != 0)
 		return -EINVAL;
 
-	if (NFP_CPP_MODEL_IS_6000(model)) {
-		if (*target != NFP_CPP_TARGET_MU)
-			return -EINVAL;
-	} else {
+	if (*target != NFP_CPP_TARGET_MU)
 		return -EINVAL;
-	}
 
 	return 0;
 }
@@ -84,10 +106,10 @@ nfp_cpp_mutex_init(struct nfp_cpp *cpp,
 		uint32_t key)
 {
 	int err;
-	uint32_t model = nfp_cpp_model(cpp);
 	uint32_t muw = NFP_CPP_ID(target, 4, 0);    /* atomic_write */
+	uint16_t interface = nfp_cpp_interface(cpp);
 
-	err = nfp_cpp_mutex_validate(model, &target, address);
+	err = nfp_cpp_mutex_validate(interface, &target, address);
 	if (err < 0)
 		return err;
 
@@ -95,8 +117,7 @@ nfp_cpp_mutex_init(struct nfp_cpp *cpp,
 	if (err < 0)
 		return err;
 
-	err = nfp_cpp_writel(cpp, muw, address + 0,
-			MUTEX_LOCKED(nfp_cpp_interface(cpp)));
+	err = nfp_cpp_writel(cpp, muw, address, nfp_mutex_locked(interface));
 	if (err < 0)
 		return err;
 
@@ -133,26 +154,10 @@ nfp_cpp_mutex_alloc(struct nfp_cpp *cpp,
 	int err;
 	uint32_t tmp;
 	struct nfp_cpp_mutex *mutex;
-	uint32_t model = nfp_cpp_model(cpp);
 	uint32_t mur = NFP_CPP_ID(target, 3, 0);    /* atomic_read */
+	uint16_t interface = nfp_cpp_interface(cpp);
 
-	/* Look for cached mutex */
-	for (mutex = cpp->mutex_cache; mutex; mutex = mutex->next) {
-		if (mutex->target == target && mutex->address == address)
-			break;
-	}
-
-	if (mutex) {
-		if (mutex->key == key) {
-			mutex->usage++;
-			return mutex;
-		}
-
-		/* If the key doesn't match... */
-		return NULL;
-	}
-
-	err = nfp_cpp_mutex_validate(model, &target, address);
+	err = nfp_cpp_mutex_validate(interface, &target, address);
 	if (err < 0)
 		return NULL;
 
@@ -172,16 +177,6 @@ nfp_cpp_mutex_alloc(struct nfp_cpp *cpp,
 	mutex->address = address;
 	mutex->key = key;
 	mutex->depth = 0;
-	mutex->usage = 1;
-
-	/* Add mutex to the cache */
-	if (cpp->mutex_cache) {
-		cpp->mutex_cache->prev = mutex;
-		mutex->next = cpp->mutex_cache;
-		cpp->mutex_cache = mutex;
-	} else {
-		cpp->mutex_cache = mutex;
-	}
 
 	return mutex;
 }
@@ -195,20 +190,6 @@ nfp_cpp_mutex_alloc(struct nfp_cpp *cpp,
 void
 nfp_cpp_mutex_free(struct nfp_cpp_mutex *mutex)
 {
-	mutex->usage--;
-	if (mutex->usage > 0)
-		return;
-
-	/* Remove mutex from the cache */
-	if (mutex->next)
-		mutex->next->prev = mutex->prev;
-	if (mutex->prev)
-		mutex->prev->next = mutex->next;
-
-	/* If mutex->cpp == NULL, something broke */
-	if (mutex->cpp && mutex == mutex->cpp->mutex_cache)
-		mutex->cpp->mutex_cache = mutex->next;
-
 	rte_free(mutex);
 }
 
@@ -268,32 +249,28 @@ nfp_cpp_mutex_unlock(struct nfp_cpp_mutex *mutex)
 		return 0;
 	}
 
-	err = nfp_cpp_readl(mutex->cpp, mur, mutex->address, &value);
-	if (err < 0)
-		goto exit;
-
 	err = nfp_cpp_readl(mutex->cpp, mur, mutex->address + 4, &key);
 	if (err < 0)
-		goto exit;
+		return err;
 
-	if (key != mutex->key) {
-		err = -EPERM;
-		goto exit;
-	}
+	if (key != mutex->key)
+		return -EPERM;
 
-	if (value != MUTEX_LOCKED(interface)) {
-		err = -EACCES;
-		goto exit;
-	}
+	err = nfp_cpp_readl(mutex->cpp, mur, mutex->address, &value);
+	if (err < 0)
+		return err;
+
+	if (value != nfp_mutex_locked(interface))
+		return -EACCES;
 
-	err = nfp_cpp_writel(cpp, muw, mutex->address, MUTEX_UNLOCK(interface));
+	err = nfp_cpp_writel(cpp, muw, mutex->address,
+			nfp_mutex_unlocked(interface));
 	if (err < 0)
-		goto exit;
+		return err;
 
 	mutex->depth = 0;
 
-exit:
-	return err;
+	return 0;
 }
 
 /**
@@ -332,19 +309,17 @@ nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex)
 	/* Verify that the lock marker is not damaged */
 	err = nfp_cpp_readl(cpp, mur, mutex->address + 4, &key);
 	if (err < 0)
-		goto exit;
+		return err;
 
-	if (key != mutex->key) {
-		err = -EPERM;
-		goto exit;
-	}
+	if (key != mutex->key)
+		return -EPERM;
 
 	/*
 	 * Compare against the unlocked state, and if true,
 	 * write the interface id into the top 16 bits, and
 	 * mark as locked.
 	 */
-	value = MUTEX_LOCKED(nfp_cpp_interface(cpp));
+	value = nfp_mutex_locked(nfp_cpp_interface(cpp));
 
 	/*
 	 * We use test_set_imm here, as it implies a read
@@ -361,10 +336,10 @@ nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex)
 	 */
 	err = nfp_cpp_readl(cpp, mus, mutex->address, &tmp);
 	if (err < 0)
-		goto exit;
+		return err;
 
 	/* Was it unlocked? */
-	if (MUTEX_IS_UNLOCKED(tmp)) {
+	if (nfp_mutex_is_unlocked(tmp)) {
 		/*
 		 * The read value can only be 0x....0000 in the unlocked state.
 		 * If there was another contending for this lock, then
@@ -376,20 +351,64 @@ nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex)
 		 */
 		err = nfp_cpp_writel(cpp, muw, mutex->address, value);
 		if (err < 0)
-			goto exit;
+			return err;
 
 		mutex->depth = 1;
-		goto exit;
+		return 0;
 	}
 
 	/* Already locked by us? Success! */
 	if (tmp == value) {
 		mutex->depth = 1;
-		goto exit;
+		return 0;
 	}
 
-	err = MUTEX_IS_LOCKED(tmp) ? -EBUSY : -EINVAL;
+	return nfp_mutex_is_locked(tmp) ? -EBUSY : -EINVAL;
+}
+
+/**
+ * Release lock if held by local system.
+ * Extreme care is advised, call only when no local lock users can exist.
+ *
+ * @param cpp
+ *   NFP CPP handle
+ * @param target
+ *   NFP CPP target ID (ie NFP_CPP_TARGET_CLS or NFP_CPP_TARGET_MU)
+ * @param address
+ *   Offset into the address space of the NFP CPP target ID
+ *
+ * @return
+ *   - (0) if the lock was OK
+ *   - (1) if locked by us
+ *   - (-errno) on invalid mutex
+ */
+int
+nfp_cpp_mutex_reclaim(struct nfp_cpp *cpp,
+		int target,
+		uint64_t address)
+{
+	int err;
+	uint32_t tmp;
+	uint16_t interface = nfp_cpp_interface(cpp);
+	const uint32_t mur = NFP_CPP_ID(target, 3, 0);    /* atomic_read */
+	const uint32_t muw = NFP_CPP_ID(target, 4, 0);    /* atomic_write */
+
+	err = nfp_cpp_mutex_validate(interface, &target, address);
+	if (err != 0)
+		return err;
+
+	/* Check lock */
+	err = nfp_cpp_readl(cpp, mur, address, &tmp);
+	if (err < 0)
+		return err;
+
+	if (nfp_mutex_is_unlocked(tmp) || nfp_mutex_owner(tmp) != interface)
+		return 0;
+
+	/* Bust the lock */
+	err = nfp_cpp_writel(cpp, muw, address, nfp_mutex_unlocked(interface));
+	if (err < 0)
+		return err;
 
-exit:
-	return err;
+	return 1;
 }
diff --git a/drivers/net/nfp/nfpcore/nfp_mutex.h b/drivers/net/nfp/nfpcore/nfp_mutex.h
new file mode 100644
index 0000000000..a79490b4d6
--- /dev/null
+++ b/drivers/net/nfp/nfpcore/nfp_mutex.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2023 Corigine, Inc.
+ * All rights reserved.
+ */
+
+#ifndef __NFP_MUTEX_H__
+#define __NFP_MUTEX_H__
+
+#include "nfp_cpp.h"
+
+struct nfp_cpp_mutex;
+
+int nfp_cpp_mutex_init(struct nfp_cpp *cpp, int target,
+		uint64_t address, uint32_t key_id);
+
+struct nfp_cpp_mutex *nfp_cpp_mutex_alloc(struct nfp_cpp *cpp, int target,
+		uint64_t address, uint32_t key_id);
+
+void nfp_cpp_mutex_free(struct nfp_cpp_mutex *mutex);
+int nfp_cpp_mutex_lock(struct nfp_cpp_mutex *mutex);
+int nfp_cpp_mutex_unlock(struct nfp_cpp_mutex *mutex);
+int nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex);
+int nfp_cpp_mutex_reclaim(struct nfp_cpp *cpp, int target, uint64_t address);
+
+#endif /* __NFP_MUTEX_H__ */
diff --git a/drivers/net/nfp/nfpcore/nfp_resource.c b/drivers/net/nfp/nfpcore/nfp_resource.c
index ee987c4438..c4cc7f42f6 100644
--- a/drivers/net/nfp/nfpcore/nfp_resource.c
+++ b/drivers/net/nfp/nfpcore/nfp_resource.c
@@ -7,6 +7,7 @@
 
 #include "nfp_crc.h"
 #include "nfp_logs.h"
+#include "nfp_mutex.h"
 #include "nfp_target.h"
 
 #define NFP_RESOURCE_TBL_TARGET         NFP_CPP_TARGET_MU
-- 
2.39.1