From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 50BA845BCC;
	Wed, 30 Oct 2024 22:43:59 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 2345B43497;
	Wed, 30 Oct 2024 22:41:35 +0100 (CET)
Received: from egress-ip42a.ess.de.barracuda.com
 (egress-ip42a.ess.de.barracuda.com [18.185.115.201])
 by mails.dpdk.org (Postfix) with ESMTP id 4F1C5434C3
 for <dev@dpdk.org>; Wed, 30 Oct 2024 22:40:31 +0100 (CET)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com
 (mail-vi1eur05lp2170.outbound.protection.outlook.com [104.47.17.170]) by
 mx-outbound47-144.eu-central-1c.ess.aws.cudaops.com (version=TLSv1.2
 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Wed, 30 Oct 2024 21:40:28 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;
 b=tcUiGtrntXjN4tdOkJo6iDIx2KPKR+PTQvpZqjWYgUWjqeyjzTG9wROss/0u+As2V0/202r/VTWQszii81LZRfeWxHYzxh+sb2c2jypQPXhGbPu3cgCMA5QmMqb1++AF6OsS+Z7ob55n64ypAvqETotINdS4bphP8CT6n+IGxDhKvGLPLfkSFg0aNIdRYxXWmMpQT5h2SfW7EzJ5cTfa7lM1W9G08J1JgiTHDDalJcdp9/rAYp4gvKerfT5bna528oDrm4oqtKTI+J9RLBDdQQzX/mU/qTTTFN3Pp4PNX3b1j2belUrbQJUTCMvYtYTmKAZobnPc43Ujyk+ltiG5hQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector10001;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=eSGSqpWzzBqAAABZsryUNyWn49dgT5Otstrz9LdbS/w=;
 b=JN/E7w93zHWl1Yyrs0lM6xOeZ/wQS/XDAZeNzYOMhrz535w+IwB0JCk03p+WaRf0DuJWGJYRIMcB/p6Zp2zZm3VXyhyjWrYX4/imDHbOrg4sgEsXoPEWZs89uOHfTL7Nt/df1sw01yqJ91KXUCCU28VyutJ0cUTlA6t/C9NUj6p95OE5lP6wRggQqfrzwHtQqW0chif/e6kdRuKKdCmcB4jtZigbwNrO7USXPbdcVpK5h932AfB9jPZyiDI+iplCW2f13Fx1Pfjb+NQkpzw3ZcqTFW0iQCbULMP4DaUmEWvJjn7WmlDeAMww6q2OSDkp1aHi5uExGZ8aUeaOwbZHVw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=fail (sender ip is
 178.72.21.4) smtp.rcpttodomain=dpdk.org smtp.mailfrom=napatech.com;
 dmarc=fail (p=reject sp=reject pct=100) action=oreject
 header.from=napatech.com; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=napatech.com;
 s=selector1;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=eSGSqpWzzBqAAABZsryUNyWn49dgT5Otstrz9LdbS/w=;
 b=m4B9uWUlY5o8R88+J+f87iu/UEsYJ58TEoAEVd+r3MIL5O8AZjpJIFPVSBRjgv1cjp8OlEWNPQBz8movIaKeriVAVRElJKl8GezGELX9uc1uWEersWlML8DcXxU+a5KX5snB8TsLp5xlh7dmsCvGsoqJhcN1MSASIAtLUSFWi+g=
Received: from DUZPR01CA0014.eurprd01.prod.exchangelabs.com
 (2603:10a6:10:3c3::7) by AM7P190MB0727.EURP190.PROD.OUTLOOK.COM
 (2603:10a6:20b:114::9) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20; Wed, 30 Oct
 2024 21:40:25 +0000
Received: from DU2PEPF00028D13.eurprd03.prod.outlook.com
 (2603:10a6:10:3c3:cafe::6b) by DUZPR01CA0014.outlook.office365.com
 (2603:10a6:10:3c3::7) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8114.20 via Frontend
 Transport; Wed, 30 Oct 2024 21:40:25 +0000
X-MS-Exchange-Authentication-Results: spf=fail (sender IP is 178.72.21.4)
 smtp.mailfrom=napatech.com; dkim=none (message not signed)
 header.d=none;dmarc=fail action=oreject header.from=napatech.com;
Received-SPF: Fail (protection.outlook.com: domain of napatech.com does not
 designate 178.72.21.4 as permitted sender) receiver=protection.outlook.com;
 client-ip=178.72.21.4; helo=localhost.localdomain;
Received: from localhost.localdomain (178.72.21.4) by
 DU2PEPF00028D13.mail.protection.outlook.com (10.167.242.27) with Microsoft
 SMTP Server id 15.20.8114.16 via Frontend Transport; Wed, 30 Oct 2024
 21:40:25 +0000
From: Serhii Iliushyk <sil-plv@napatech.com>
To: dev@dpdk.org
Cc: mko-plv@napatech.com, sil-plv@napatech.com, ckm@napatech.com,
 andrew.rybchenko@oktetlabs.ru, ferruh.yigit@amd.com,
 stephen@networkplumber.org, Danylo Vodopianov <dvo-plv@napatech.com>
Subject: [PATCH v5 35/80] net/ntnic: add learn flow queue handling
Date: Wed, 30 Oct 2024 22:38:42 +0100
Message-ID: <20241030213940.3470062-36-sil-plv@napatech.com>
X-Mailer: git-send-email 2.45.0
In-Reply-To: <20241030213940.3470062-1-sil-plv@napatech.com>
References: <20241021210527.2075431-1-sil-plv@napatech.com>
 <20241030213940.3470062-1-sil-plv@napatech.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: DU2PEPF00028D13:EE_|AM7P190MB0727:EE_
Content-Type: text/plain
X-MS-Office365-Filtering-Correlation-Id: 10b5b495-5560-4691-29cb-08dcf92b770a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
 ARA:13230040|82310400026|376014|1800799024|36860700013; 
X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?2emdn7bLzzAgXXela5Qx+T34s4EstnqZ33yTpExHv37TASZAdMfstcJ7PqwE?=
 =?us-ascii?Q?fI6ONnUfpfypmCEvt3bY6RTPmQZSSgTnHB44+Pwox+5kWUuzFcacHEPnFN2v?=
 =?us-ascii?Q?iKqXqNiSD2vM7KFuOGVvPiNDOL7j3CD5a+NklUG9NrFJbiz+slJHpuayXzqm?=
 =?us-ascii?Q?la27h5ojvXK5c+pu/uoAxbpXehMcruDXXiPv9N/NADuMnK0gYd1GF1DA2rf6?=
 =?us-ascii?Q?pswOoPDUm83Gc9a9lWc8sB1INsa5zbnKHQt+PRU1YQbxQqwvPuqMtoJdYoCI?=
 =?us-ascii?Q?+WwFGaacVxDH08o82a7E447MY3cxa8k4IJN9YBIrMZ+NBqZcX0yaLLJlDKoi?=
 =?us-ascii?Q?cX1ixORgSVGFmAAw3hqIwqbkdw7d5VkM3PzvZsTvuts4loakDrIUuuTQPA+0?=
 =?us-ascii?Q?84XgK45dgg4yCgkXPCsntDZFoTrSS3axUhM3Yvw4WGLBKExmtwzGJOFCR6t6?=
 =?us-ascii?Q?r3tMhNNTCI8eg7YgT/UOjrEV9xm7Td7KHwwIn84S8ed5eTV8o9oYFL0w7epx?=
 =?us-ascii?Q?eAyceezDGr37bOYdqNfmeQT4P2Xi0h5PAOcDVsfAg5uEbgPwzVKnu6an6OY4?=
 =?us-ascii?Q?QA/VQjpmCUWvXlc/4crTPwIDeMi28wTFw/9LXtkNzsVSDugNtikAXTdNmmmZ?=
 =?us-ascii?Q?gZMFPLgc07zQpJ7EkEwmOkJ20wXAF4Dit4uCTzaumMNRQcYNRNmR/Fvt4aZo?=
 =?us-ascii?Q?FU5O88ST8u+vJ+xJ5wGHFpl/Gn1sfE1hhp89eoTvZyXy6DxTom0P8JpLeeM/?=
 =?us-ascii?Q?dMLY8vjBVGGXSqU2l67nT24gtKTuLkqJg9B5o4RfBUMqyXUQN0PSKWEfeig4?=
 =?us-ascii?Q?psfBnAgNP6gxcx+c6sVOQKk4/UEAMjxZwrSJFzKAr2ZWOR6bgJpKyN0WyBju?=
 =?us-ascii?Q?VxZM52M5ORQ/BYd0EJ3Iv+7+WHBwzJyafNsaXuzMp70+lZpfq4jd6hitZtF6?=
 =?us-ascii?Q?JP84H9CePTA/G7OQidc4GcBD5KhgnINFlvz4/9Y3rvfJvPiWTPNheOy1Zact?=
 =?us-ascii?Q?hY8jJ3ql4h1uHzE8kKBIckCb62Gs3tZD0c+iU2u1l+Iwun9r1om0zom7iXtI?=
 =?us-ascii?Q?LOXxFscWTxX2aD7sAtW04J6ZQ8pGgVv8cSRPJxWaf6U0i53Sx9i9hoRswIki?=
 =?us-ascii?Q?TfFDeq3WHqt8JhWJqarrIRQI7q0SP0HdcXPFvI3Bh9sChYeMEow+spGHxE1d?=
 =?us-ascii?Q?IGX75ZlMiHxuXaC9R4eMpNtGzdIa+0L9JWrnxYcZlM32ZzFxYbqEVR8wu5jU?=
 =?us-ascii?Q?kw5e9EBN40k9CMO9j7BbeMMj8s+hmFXS0msLPp/+dgOLd2ZnshpOXjNCSLSs?=
 =?us-ascii?Q?a/qMMufVVxnsWWV467MfzcetVB9gjSOLEI4kYFmH+8wGsq4dGnUyo2Tses1F?=
 =?us-ascii?Q?cEW53C8sgidHxQMnB6xXM5ZeNB/cAFK3jpZwwixyPTc+q7c7UA=3D=3D?=
X-Forefront-Antispam-Report: CIP:178.72.21.4; CTRY:DK; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:localhost.localdomain; PTR:InfoDomainNonexistent;
 CAT:NONE; SFS:(13230040)(82310400026)(376014)(1800799024)(36860700013);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 5TKh6im2/SP6gjK9G9mR0emGnG+3wehOsn/BCxUxjla1UO0G6jKVKcKm62kxLKWDRVds2aTIf8upHgLSqDPVlYYAI6A+pmRfFcNoIzSci3j8knksQOkGjaZBi8KQvBgpVeRJM4C6XlivbdFUrRkhEnrAOyvZaxq+A1lj1WIEdOhIlpIch8O0POjbDg+qZpg89zleMRs4uAvCyRNlMGHGtwPA6yxE6ECb6UKY6wDBURWOYtq6VxcjczztXjdETGHEcsVFwJnWkkWsAcuzbO/n6PvtbcF2MtWvdfAH0WsGxzwM80NyfUEGjnh8moUQhakQkTVJ2PL28ev4NCDg8wNg/IpQbMcxJtHasOgQWzxCjt/NczKmjSfxtKm3y4z37pOZR0hIm6EVh7XivNSOo+ODe56xf5iO8x++aTPF4ImXVzNCSgfchbfwUkk7tjpoRqZllJXa/hzoIAQafvo2d0Bsh0YKAroFVOEBrfxMCDPttXNf3JFKcXOJodxJC7PQKkyDLV0r7U3mOhevJyGmNXN+93uTdxQgjpgfY7OlQ9MlJo/rHDMhMzcAjyIA2HEBUhRIVcOQ+hDM1KEkjTjeaXnA0k0O8V3TNRosIHOU/liNoVkG3rN3/Z+j2fSJtjoKhqq9cuYniCMv9b31xv4geSo3OvSKUBl14HJqQx0NujjREL4=
X-OriginatorOrg: napatech.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2024 21:40:25.2863 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 10b5b495-5560-4691-29cb-08dcf92b770a
X-MS-Exchange-CrossTenant-Id: c4540d0b-728a-4233-9da5-9ea30c7ec3ed
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=c4540d0b-728a-4233-9da5-9ea30c7ec3ed; Ip=[178.72.21.4];
 Helo=[localhost.localdomain]
X-MS-Exchange-CrossTenant-AuthSource: DU2PEPF00028D13.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7P190MB0727
X-BESS-ID: 1730324427-312176-12709-40671-1
X-BESS-VER: 2019.1_20241018.1852
X-BESS-Apparent-Source-IP: 104.47.17.170
X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVoZGxiZGQGYGUNQgLdUgxSjJKN
 nIyNAixcjM0MDILNncwtzQ1DAp2TTJQqk2FgDTZGFCQgAAAA==
X-BESS-Outbound-Spam-Score: 0.00
X-BESS-Outbound-Spam-Report: Code version 3.2,
 rules version 3.2.2.260091 [from 
 cloudscan20-80.eu-central-1b.ess.aws.cudaops.com]
 Rule breakdown below
 pts rule name              description
 ---- ---------------------- --------------------------------
 0.00 BSF_BESS_OUTBOUND      META: BESS Outbound 
X-BESS-Outbound-Spam-Status: SCORE=0.00 using account:ESS113687 scores of
 KILL_LEVEL=7.0 tests=BSF_BESS_OUTBOUND
X-BESS-BRTS-Status: 1
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

From: Danylo Vodopianov <dvo-plv@napatech.com>

Implements thread for handling flow learn queue

Signed-off-by: Danylo Vodopianov <dvo-plv@napatech.com>
---
 drivers/net/ntnic/include/hw_mod_backend.h    |  5 +
 drivers/net/ntnic/include/ntdrv_4ga.h         |  1 +
 .../ntnic/nthw/flow_api/hw_mod/hw_mod_flm.c   | 33 +++++++
 .../flow_api/profile_inline/flm_lrn_queue.c   | 42 +++++++++
 .../flow_api/profile_inline/flm_lrn_queue.h   | 11 +++
 .../profile_inline/flow_api_profile_inline.c  | 48 ++++++++++
 drivers/net/ntnic/ntnic_ethdev.c              | 94 +++++++++++++++++++
 drivers/net/ntnic/ntnic_mod_reg.h             |  7 ++
 8 files changed, 241 insertions(+)

diff --git a/drivers/net/ntnic/include/hw_mod_backend.h b/drivers/net/ntnic/include/hw_mod_backend.h
index 13722c30a9..17d5755634 100644
--- a/drivers/net/ntnic/include/hw_mod_backend.h
+++ b/drivers/net/ntnic/include/hw_mod_backend.h
@@ -688,6 +688,11 @@ int hw_mod_flm_rcp_set_mask(struct flow_api_backend_s *be, enum hw_flm_e field,
 int hw_mod_flm_rcp_set(struct flow_api_backend_s *be, enum hw_flm_e field, int index,
 	uint32_t value);
 
+int hw_mod_flm_lrn_data_set_flush(struct flow_api_backend_s *be, enum hw_flm_e field,
+	const uint32_t *value, uint32_t records,
+	uint32_t *handled_records, uint32_t *inf_word_cnt,
+	uint32_t *sta_word_cnt);
+
 int hw_mod_flm_scrub_flush(struct flow_api_backend_s *be, int start_idx, int count);
 
 struct hsh_func_s {
diff --git a/drivers/net/ntnic/include/ntdrv_4ga.h b/drivers/net/ntnic/include/ntdrv_4ga.h
index 8017aa4fc3..8ebdd98db0 100644
--- a/drivers/net/ntnic/include/ntdrv_4ga.h
+++ b/drivers/net/ntnic/include/ntdrv_4ga.h
@@ -14,6 +14,7 @@ typedef struct ntdrv_4ga_s {
 	char *p_drv_name;
 
 	volatile bool b_shutdown;
+	rte_thread_t flm_thread;
 } ntdrv_4ga_t;
 
 #endif	/* __NTDRV_4GA_H__ */
diff --git a/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_flm.c b/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_flm.c
index 0a7e90c04f..f4c29b8bde 100644
--- a/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_flm.c
+++ b/drivers/net/ntnic/nthw/flow_api/hw_mod/hw_mod_flm.c
@@ -712,3 +712,36 @@ int hw_mod_flm_rcp_set(struct flow_api_backend_s *be, enum hw_flm_e field, int i
 
 	return hw_mod_flm_rcp_mod(be, field, index, &value, 0);
 }
+
+int hw_mod_flm_lrn_data_set_flush(struct flow_api_backend_s *be, enum hw_flm_e field,
+	const uint32_t *value, uint32_t records,
+	uint32_t *handled_records, uint32_t *inf_word_cnt,
+	uint32_t *sta_word_cnt)
+{
+	int ret = 0;
+
+	switch (_VER_) {
+	case 25:
+		switch (field) {
+		case HW_FLM_FLOW_LRN_DATA:
+			ret = be->iface->flm_lrn_data_flush(be->be_dev, &be->flm, value, records,
+					handled_records,
+					(sizeof(struct flm_v25_lrn_data_s) /
+						sizeof(uint32_t)),
+					inf_word_cnt, sta_word_cnt);
+			break;
+
+		default:
+			UNSUP_FIELD_LOG;
+			return UNSUP_FIELD;
+		}
+
+		break;
+
+	default:
+		UNSUP_VER_LOG;
+		return UNSUP_VER;
+	}
+
+	return ret;
+}
diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c
index ad7efafe08..6e77c28f93 100644
--- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c
+++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.c
@@ -13,8 +13,28 @@
 
 #include "flm_lrn_queue.h"
 
+#define QUEUE_SIZE (1 << 13)
+
 #define ELEM_SIZE sizeof(struct flm_v25_lrn_data_s)
 
+void *flm_lrn_queue_create(void)
+{
+	static_assert((ELEM_SIZE & ~(size_t)3) == ELEM_SIZE, "FLM LEARN struct size");
+	struct rte_ring *q = rte_ring_create_elem("RFQ",
+		ELEM_SIZE,
+		QUEUE_SIZE,
+		SOCKET_ID_ANY,
+		RING_F_MP_HTS_ENQ | RING_F_SC_DEQ);
+	assert(q != NULL);
+	return q;
+}
+
+void flm_lrn_queue_free(void *q)
+{
+	if (q)
+		rte_ring_free(q);
+}
+
 uint32_t *flm_lrn_queue_get_write_buffer(void *q)
 {
 	struct rte_ring_zc_data zcd;
@@ -26,3 +46,25 @@ void flm_lrn_queue_release_write_buffer(void *q)
 {
 	rte_ring_enqueue_zc_elem_finish(q, 1);
 }
+
+read_record flm_lrn_queue_get_read_buffer(void *q)
+{
+	struct rte_ring_zc_data zcd;
+	read_record rr;
+
+	if (rte_ring_dequeue_zc_burst_elem_start(q, ELEM_SIZE, QUEUE_SIZE, &zcd, NULL) != 0) {
+		rr.num = zcd.n1;
+		rr.p = zcd.ptr1;
+
+	} else {
+		rr.num = 0;
+		rr.p = NULL;
+	}
+
+	return rr;
+}
+
+void flm_lrn_queue_release_read_buffer(void *q, uint32_t num)
+{
+	rte_ring_dequeue_zc_elem_finish(q, num);
+}
diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h
index 8cee0c8e78..40558f4201 100644
--- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h
+++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flm_lrn_queue.h
@@ -8,7 +8,18 @@
 
 #include <stdint.h>
 
+typedef struct read_record {
+	uint32_t *p;
+	uint32_t num;
+} read_record;
+
+void *flm_lrn_queue_create(void);
+void flm_lrn_queue_free(void *q);
+
 uint32_t *flm_lrn_queue_get_write_buffer(void *q);
 void flm_lrn_queue_release_write_buffer(void *q);
 
+read_record flm_lrn_queue_get_read_buffer(void *q);
+void flm_lrn_queue_release_read_buffer(void *q, uint32_t num);
+
 #endif	/* _FLM_LRN_QUEUE_H_ */
diff --git a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c
index 7a0cb1f9c4..7487b5150e 100644
--- a/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c
+++ b/drivers/net/ntnic/nthw/flow_api/profile_inline/flow_api_profile_inline.c
@@ -39,6 +39,48 @@
 
 static void *flm_lrn_queue_arr;
 
+static void flm_setup_queues(void)
+{
+	flm_lrn_queue_arr = flm_lrn_queue_create();
+	assert(flm_lrn_queue_arr != NULL);
+}
+
+static void flm_free_queues(void)
+{
+	flm_lrn_queue_free(flm_lrn_queue_arr);
+}
+
+static uint32_t flm_lrn_update(struct flow_eth_dev *dev, uint32_t *inf_word_cnt,
+	uint32_t *sta_word_cnt)
+{
+	read_record r = flm_lrn_queue_get_read_buffer(flm_lrn_queue_arr);
+
+	if (r.num) {
+		uint32_t handled_records = 0;
+
+		if (hw_mod_flm_lrn_data_set_flush(&dev->ndev->be, HW_FLM_FLOW_LRN_DATA, r.p, r.num,
+			&handled_records, inf_word_cnt, sta_word_cnt)) {
+			NT_LOG(ERR, FILTER, "Flow programming failed");
+
+		} else if (handled_records > 0) {
+			flm_lrn_queue_release_read_buffer(flm_lrn_queue_arr, handled_records);
+		}
+	}
+
+	return r.num;
+}
+
+static uint32_t flm_update(struct flow_eth_dev *dev)
+{
+	static uint32_t inf_word_cnt;
+	static uint32_t sta_word_cnt;
+
+	if (flm_lrn_update(dev, &inf_word_cnt, &sta_word_cnt) != 0)
+		return 1;
+
+	return inf_word_cnt + sta_word_cnt;
+}
+
 static int rx_queue_idx_to_hw_id(const struct flow_eth_dev *dev, int id)
 {
 	for (int i = 0; i < dev->num_queues; ++i)
@@ -4214,6 +4256,12 @@ static const struct profile_inline_ops ops = {
 	.flow_create_profile_inline = flow_create_profile_inline,
 	.flow_destroy_profile_inline = flow_destroy_profile_inline,
 	.flow_nic_set_hasher_fields_inline = flow_nic_set_hasher_fields_inline,
+	/*
+	 * NT Flow FLM Meter API
+	 */
+	.flm_setup_queues = flm_setup_queues,
+	.flm_free_queues = flm_free_queues,
+	.flm_update = flm_update,
 };
 
 void profile_inline_init(void)
diff --git a/drivers/net/ntnic/ntnic_ethdev.c b/drivers/net/ntnic/ntnic_ethdev.c
index a509a8eb51..bfca8f28b1 100644
--- a/drivers/net/ntnic/ntnic_ethdev.c
+++ b/drivers/net/ntnic/ntnic_ethdev.c
@@ -24,6 +24,11 @@
 #include "ntnic_mod_reg.h"
 #include "nt_util.h"
 
+const rte_thread_attr_t thread_attr = { .priority = RTE_THREAD_PRIORITY_NORMAL };
+#define THREAD_CTRL_CREATE(a, b, c, d) rte_thread_create_internal_control(a, b, c, d)
+#define THREAD_JOIN(a) rte_thread_join(a, NULL)
+#define THREAD_FUNC static uint32_t
+#define THREAD_RETURN (0)
 #define HW_MAX_PKT_LEN (10000)
 #define MAX_MTU (HW_MAX_PKT_LEN - RTE_ETHER_HDR_LEN - RTE_ETHER_CRC_LEN)
 
@@ -120,6 +125,16 @@ store_pdrv(struct drv_s *p_drv)
 	rte_spinlock_unlock(&hwlock);
 }
 
+static void clear_pdrv(struct drv_s *p_drv)
+{
+	if (p_drv->adapter_no > NUM_ADAPTER_MAX)
+		return;
+
+	rte_spinlock_lock(&hwlock);
+	_g_p_drv[p_drv->adapter_no] = NULL;
+	rte_spinlock_unlock(&hwlock);
+}
+
 static struct drv_s *
 get_pdrv_from_pci(struct rte_pci_addr addr)
 {
@@ -1240,6 +1255,13 @@ eth_dev_set_link_down(struct rte_eth_dev *eth_dev)
 static void
 drv_deinit(struct drv_s *p_drv)
 {
+	const struct profile_inline_ops *profile_inline_ops = get_profile_inline_ops();
+
+	if (profile_inline_ops == NULL) {
+		NT_LOG_DBGX(ERR, NTNIC, "profile_inline module uninitialized");
+		return;
+	}
+
 	const struct adapter_ops *adapter_ops = get_adapter_ops();
 
 	if (adapter_ops == NULL) {
@@ -1251,6 +1273,22 @@ drv_deinit(struct drv_s *p_drv)
 		return;
 
 	ntdrv_4ga_t *p_nt_drv = &p_drv->ntdrv;
+	fpga_info_t *fpga_info = &p_nt_drv->adapter_info.fpga_info;
+
+	/*
+	 * Mark the global pdrv for cleared. Used by some threads to terminate.
+	 * 1 second to give the threads a chance to see the termonation.
+	 */
+	clear_pdrv(p_drv);
+	nt_os_wait_usec(1000000);
+
+	/* stop statistics threads */
+	p_drv->ntdrv.b_shutdown = true;
+
+	if (fpga_info->profile == FPGA_INFO_PROFILE_INLINE) {
+		THREAD_JOIN(p_nt_drv->flm_thread);
+		profile_inline_ops->flm_free_queues();
+	}
 
 	/* stop adapter */
 	adapter_ops->deinit(&p_nt_drv->adapter_info);
@@ -1359,6 +1397,43 @@ static const struct eth_dev_ops nthw_eth_dev_ops = {
 	.promiscuous_enable = promiscuous_enable,
 };
 
+/*
+ * Adapter flm stat thread
+ */
+THREAD_FUNC adapter_flm_update_thread_fn(void *context)
+{
+	const struct profile_inline_ops *profile_inline_ops = get_profile_inline_ops();
+
+	if (profile_inline_ops == NULL) {
+		NT_LOG(ERR, NTNIC, "%s: profile_inline module uninitialized", __func__);
+		return THREAD_RETURN;
+	}
+
+	struct drv_s *p_drv = context;
+
+	struct ntdrv_4ga_s *p_nt_drv = &p_drv->ntdrv;
+	struct adapter_info_s *p_adapter_info = &p_nt_drv->adapter_info;
+	struct nt4ga_filter_s *p_nt4ga_filter = &p_adapter_info->nt4ga_filter;
+	struct flow_nic_dev *p_flow_nic_dev = p_nt4ga_filter->mp_flow_device;
+
+	NT_LOG(DBG, NTNIC, "%s: %s: waiting for port configuration",
+		p_adapter_info->mp_adapter_id_str, __func__);
+
+	while (p_flow_nic_dev->eth_base == NULL)
+		nt_os_wait_usec(1 * 1000 * 1000);
+
+	struct flow_eth_dev *dev = p_flow_nic_dev->eth_base;
+
+	NT_LOG(DBG, NTNIC, "%s: %s: begin", p_adapter_info->mp_adapter_id_str, __func__);
+
+	while (!p_drv->ntdrv.b_shutdown)
+		if (profile_inline_ops->flm_update(dev) == 0)
+			nt_os_wait_usec(10);
+
+	NT_LOG(DBG, NTNIC, "%s: %s: end", p_adapter_info->mp_adapter_id_str, __func__);
+	return THREAD_RETURN;
+}
+
 static int
 nthw_pci_dev_init(struct rte_pci_device *pci_dev)
 {
@@ -1369,6 +1444,13 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev)
 		/* Return statement is not necessary here to allow traffic processing by SW  */
 	}
 
+	const struct profile_inline_ops *profile_inline_ops = get_profile_inline_ops();
+
+	if (profile_inline_ops == NULL) {
+		NT_LOG_DBGX(ERR, NTNIC, "profile_inline module uninitialized");
+		/* Return statement is not necessary here to allow traffic processing by SW  */
+	}
+
 	nt_vfio_init();
 	const struct port_ops *port_ops = get_port_ops();
 
@@ -1597,6 +1679,18 @@ nthw_pci_dev_init(struct rte_pci_device *pci_dev)
 		return -1;
 	}
 
+	if (profile_inline_ops != NULL && fpga_info->profile == FPGA_INFO_PROFILE_INLINE) {
+		profile_inline_ops->flm_setup_queues();
+		res = THREAD_CTRL_CREATE(&p_nt_drv->flm_thread, "ntnic-nt_flm_update_thr",
+			adapter_flm_update_thread_fn, (void *)p_drv);
+
+		if (res) {
+			NT_LOG_DBGX(ERR, NTNIC, "%s: error=%d",
+				(pci_dev->name[0] ? pci_dev->name : "NA"), res);
+			return -1;
+		}
+	}
+
 	n_phy_ports = fpga_info->n_phy_ports;
 
 	for (int n_intf_no = 0; n_intf_no < n_phy_ports; n_intf_no++) {
diff --git a/drivers/net/ntnic/ntnic_mod_reg.h b/drivers/net/ntnic/ntnic_mod_reg.h
index 1069be2f85..27d6cbef01 100644
--- a/drivers/net/ntnic/ntnic_mod_reg.h
+++ b/drivers/net/ntnic/ntnic_mod_reg.h
@@ -256,6 +256,13 @@ struct profile_inline_ops {
 	int (*flow_nic_set_hasher_fields_inline)(struct flow_nic_dev *ndev,
 		int hsh_idx,
 		struct nt_eth_rss_conf rss_conf);
+
+	/*
+	 * NT Flow FLM queue API
+	 */
+	void (*flm_setup_queues)(void);
+	void (*flm_free_queues)(void);
+	uint32_t (*flm_update)(struct flow_eth_dev *dev);
 };
 
 void register_profile_inline_ops(const struct profile_inline_ops *ops);
-- 
2.45.0