From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id CC31645B9E;
Tue, 22 Oct 2024 11:18:18 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 2F9554066C;
Tue, 22 Oct 2024 11:18:01 +0200 (CEST)
Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com
[67.231.156.173])
by mails.dpdk.org (Postfix) with ESMTP id 8768D4027C
for <dev@dpdk.org>; Tue, 22 Oct 2024 11:17:48 +0200 (CEST)
Received: from pps.filterd (m0431383.ppops.net [127.0.0.1])
by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49M6SSw1020805;
Tue, 22 Oct 2024 02:17:48 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=
cc:content-transfer-encoding:content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to; s=pfpt0220; bh=h
JpnYPwYJobFynS9xtXcbWk/+eggNExXeFQLgbrsVT0=; b=I28xoCABf0/ABeXJy
kP3ICm1ijoInys6E0qft+nZBASptqgmEEkkkXxACAKlUxEjAsdSmOkgpycolI5Uq
K5QEfpW2XyOk+SXlPQuNPE1kSOv3y1M2Ykp0S320XZ/qK9w07go+Vlb6/Z/CEJb7
AbkdD6H/fJbUHhlN72wbK+1xVVqEAv808nS56Ct2f6Yp+V1c2A1hKh3Nh8D2Xkab
t0JAfZdeZTKHOukvh/5KxkBAuEGnNd1bAsn/es64MJ0p6UZcuCJQtE4tmn3d8xrV
PygrMpgShUkciSUfC7nahijFKjR5rs8Zde9MgIVQ4DfNwof/lRPyTPkA+YqfIOHf
QflVA==
Received: from dc5-exch05.marvell.com ([199.233.59.128])
by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42e6tf0bky-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Tue, 22 Oct 2024 02:17:46 -0700 (PDT)
Received: from DC5-EXCH05.marvell.com (10.69.176.209) by
DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.2.1544.4; Tue, 22 Oct 2024 02:17:45 -0700
Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com
(10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend
Transport; Tue, 22 Oct 2024 02:17:45 -0700
Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com
[10.28.164.106])
by maili.marvell.com (Postfix) with ESMTP id 0C5945E6A48;
Tue, 22 Oct 2024 01:47:29 -0700 (PDT)
From: <pbhagavatula@marvell.com>
To: <jerinj@marvell.com>, <stephen@networkplumber.org>,
<mattias.ronnblom@ericsson.com>,
Pavan Nikhilesh <pbhagavatula@marvell.com>,
Shijith Thotton <sthotton@marvell.com>
CC: <dev@dpdk.org>
Subject: [PATCH v3 16/22] event/cnxk: support CN20K Rx adapter fast path
Date: Tue, 22 Oct 2024 14:16:34 +0530
Message-ID: <20241022084641.14497-16-pbhagavatula@marvell.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20241022084641.14497-1-pbhagavatula@marvell.com>
References: <20241021205745.7310-1-pbhagavatula@marvell.com>
<20241022084641.14497-1-pbhagavatula@marvell.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-Proofpoint-ORIG-GUID: G0EsaS4Ml1aWZ6RwM_vu1IuDR4MfmjVm
X-Proofpoint-GUID: G0EsaS4Ml1aWZ6RwM_vu1IuDR4MfmjVm
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475
definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
From: Pavan Nikhilesh <pbhagavatula@marvell.com>
Add support for event eth Rx adapter fastpath operations.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
drivers/event/cnxk/cn20k_eventdev.c | 122 ++++++++++++-
drivers/event/cnxk/cn20k_worker.c | 54 ------
drivers/event/cnxk/cn20k_worker.h | 165 +++++++++++++++++-
drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_0_15_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_112_127_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_112_127_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_112_127_tmo_burst.c | 22 +++
.../deq/cn20k/deq_112_127_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_16_31_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_16_31_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_16_31_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_32_47_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_32_47_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_32_47_tmo_burst.c | 23 +++
.../cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_48_63_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_48_63_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_48_63_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_64_79_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_64_79_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_64_79_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_80_95_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_80_95_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_80_95_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_96_111_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_96_111_seg_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_96_111_tmo_burst.c | 22 +++
.../cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c | 22 +++
.../event/cnxk/deq/cn20k/deq_all_offload.c | 65 +++++++
drivers/event/cnxk/meson.build | 43 +++++
37 files changed, 1085 insertions(+), 69 deletions(-)
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c
create mode 100644 drivers/event/cnxk/deq/cn20k/deq_all_offload.c
diff --git a/drivers/event/cnxk/cn20k_eventdev.c b/drivers/event/cnxk/cn20k_eventdev.c
index bb6a6f94e0..aa2f32e005 100644
--- a/drivers/event/cnxk/cn20k_eventdev.c
+++ b/drivers/event/cnxk/cn20k_eventdev.c
@@ -11,6 +11,9 @@
#include "cnxk_eventdev.h"
#include "cnxk_worker.h"
+#define CN20K_SET_EVDEV_DEQ_OP(dev, deq_op, deq_ops) \
+ deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)]
+
static void *
cn20k_sso_init_hws_mem(void *arg, uint8_t port_id)
{
@@ -164,21 +167,124 @@ cn20k_sso_rsrc_init(void *arg, uint8_t hws, uint8_t hwgrp)
return roc_sso_rsrc_init(&dev->sso, hws, hwgrp, nb_tim_lfs);
}
+#if defined(RTE_ARCH_ARM64)
+static inline void
+cn20k_sso_fp_tmplt_fns_set(struct rte_eventdev *event_dev)
+{
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+ struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev);
+
+ const event_dequeue_burst_t sso_hws_deq_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_deq_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_deq_tmo_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_deq_tmo_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_deq_seg_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_deq_seg_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_deq_tmo_seg_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_deq_tmo_seg_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_reas_deq_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_reas_deq_tmo_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_tmo_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_reas_deq_seg_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_seg_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ const event_dequeue_burst_t sso_hws_reas_deq_tmo_seg_burst[NIX_RX_OFFLOAD_MAX] = {
+#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_tmo_seg_burst_##name,
+ NIX_RX_FASTPATH_MODES
+#undef R
+ };
+
+ if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) {
+ if (dev->rx_offloads & NIX_RX_REAS_F) {
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_reas_deq_seg_burst);
+ if (dev->is_timeout_deq)
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_reas_deq_tmo_seg_burst);
+ } else {
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_deq_seg_burst);
+
+ if (dev->is_timeout_deq)
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_deq_tmo_seg_burst);
+ }
+ } else {
+ if (dev->rx_offloads & NIX_RX_REAS_F) {
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_reas_deq_burst);
+
+ if (dev->is_timeout_deq)
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_reas_deq_tmo_burst);
+ } else {
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, sso_hws_deq_burst);
+
+ if (dev->is_timeout_deq)
+ CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst,
+ sso_hws_deq_tmo_burst);
+ }
+ }
+
+#else
+ RTE_SET_USED(event_dev);
+#endif
+}
+
+static inline void
+cn20k_sso_fp_blk_fns_set(struct rte_eventdev *event_dev)
+{
+#if defined(CNXK_DIS_TMPLT_FUNC)
+ struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev);
+
+ event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload;
+ if (dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F)
+ event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload_tst;
+#else
+ RTE_SET_USED(event_dev);
+#endif
+}
+#endif
static void
cn20k_sso_fp_fns_set(struct rte_eventdev *event_dev)
{
#if defined(RTE_ARCH_ARM64)
- struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev);
+ cn20k_sso_fp_blk_fns_set(event_dev);
+ cn20k_sso_fp_tmplt_fns_set(event_dev);
event_dev->enqueue_burst = cn20k_sso_hws_enq_burst;
event_dev->enqueue_new_burst = cn20k_sso_hws_enq_new_burst;
event_dev->enqueue_forward_burst = cn20k_sso_hws_enq_fwd_burst;
- event_dev->dequeue_burst = cn20k_sso_hws_deq_burst;
- if (dev->deq_tmo_ns)
- event_dev->dequeue_burst = cn20k_sso_hws_tmo_deq_burst;
-
event_dev->profile_switch = cn20k_sso_hws_profile_switch;
event_dev->preschedule_modify = cn20k_sso_hws_preschedule_modify;
event_dev->preschedule = cn20k_sso_hws_preschedule;
@@ -285,7 +391,8 @@ cn20k_sso_port_quiesce(struct rte_eventdev *event_dev, void *port,
ptag = plt_read64(ws->base + SSOW_LF_GWS_PENDSTATE);
} while (ptag & (BIT_ULL(62) | BIT_ULL(58) | BIT_ULL(56) | BIT_ULL(54)));
- cn20k_sso_hws_get_work_empty(ws, &ev, 0);
+ cn20k_sso_hws_get_work_empty(ws, &ev,
+ (NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F | NIX_RX_MULTI_SEG_F);
if (is_pend && ev.u64)
if (flush_cb)
flush_cb(event_dev->data->dev_id, ev, args);
@@ -311,7 +418,8 @@ cn20k_sso_port_quiesce(struct rte_eventdev *event_dev, void *port,
if (CNXK_TT_FROM_TAG(plt_read64(ws->base + SSOW_LF_GWS_PRF_WQE0)) != SSO_TT_EMPTY) {
plt_write64(BIT_ULL(16) | 1, ws->base + SSOW_LF_GWS_OP_GET_WORK0);
- cn20k_sso_hws_get_work_empty(ws, &ev, 0);
+ cn20k_sso_hws_get_work_empty(
+ ws, &ev, (NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F | NIX_RX_MULTI_SEG_F);
if (ev.u64) {
if (flush_cb)
flush_cb(event_dev->data->dev_id, ev, args);
diff --git a/drivers/event/cnxk/cn20k_worker.c b/drivers/event/cnxk/cn20k_worker.c
index ebfe863bc5..53daf3b4b0 100644
--- a/drivers/event/cnxk/cn20k_worker.c
+++ b/drivers/event/cnxk/cn20k_worker.c
@@ -423,57 +423,3 @@ cn20k_sso_hws_preschedule(void *port, enum rte_event_dev_preschedule_type type)
RTE_SET_USED(type);
plt_write64(ws->gw_wdata, ws->base + SSOW_LF_GWS_OP_PRF_GETWORK);
}
-
-uint16_t __rte_hot
-cn20k_sso_hws_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks)
-{
- struct cn20k_sso_hws *ws = port;
-
- RTE_SET_USED(timeout_ticks);
-
- if (ws->swtag_req) {
- ws->swtag_req = 0;
- cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0);
- return 1;
- }
-
- return cn20k_sso_hws_get_work(ws, ev, 0);
-}
-
-uint16_t __rte_hot
-cn20k_sso_hws_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events,
- uint64_t timeout_ticks)
-{
- RTE_SET_USED(nb_events);
-
- return cn20k_sso_hws_deq(port, ev, timeout_ticks);
-}
-
-uint16_t __rte_hot
-cn20k_sso_hws_tmo_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks)
-{
- struct cn20k_sso_hws *ws = port;
- uint16_t ret = 1;
- uint64_t iter;
-
- if (ws->swtag_req) {
- ws->swtag_req = 0;
- cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0);
- return ret;
- }
-
- ret = cn20k_sso_hws_get_work(ws, ev, 0);
- for (iter = 1; iter < timeout_ticks && (ret == 0); iter++)
- ret = cn20k_sso_hws_get_work(ws, ev, 0);
-
- return ret;
-}
-
-uint16_t __rte_hot
-cn20k_sso_hws_tmo_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events,
- uint64_t timeout_ticks)
-{
- RTE_SET_USED(nb_events);
-
- return cn20k_sso_hws_tmo_deq(port, ev, timeout_ticks);
-}
diff --git a/drivers/event/cnxk/cn20k_worker.h b/drivers/event/cnxk/cn20k_worker.h
index dd8b72bc53..9075073fd2 100644
--- a/drivers/event/cnxk/cn20k_worker.h
+++ b/drivers/event/cnxk/cn20k_worker.h
@@ -8,16 +8,64 @@
#include <rte_eventdev.h>
#include "cn20k_eventdev.h"
+#include "cn20k_rx.h"
#include "cnxk_worker.h"
+/* CN20K Rx event fastpath */
+
+static __rte_always_inline void
+cn20k_wqe_to_mbuf(uint64_t wqe, const uint64_t __mbuf, uint8_t port_id, const uint32_t tag,
+ const uint32_t flags, const void *const lookup_mem, uintptr_t cpth,
+ uintptr_t sa_base)
+{
+ const uint64_t mbuf_init =
+ 0x100010000ULL | RTE_PKTMBUF_HEADROOM | (flags & NIX_RX_OFFLOAD_TSTAMP_F ? 8 : 0);
+ struct rte_mbuf *mbuf = (struct rte_mbuf *)__mbuf;
+
+ cn20k_nix_cqe_to_mbuf((struct nix_cqe_hdr_s *)wqe, tag, (struct rte_mbuf *)mbuf, lookup_mem,
+ mbuf_init | ((uint64_t)port_id) << 48, cpth, sa_base, flags);
+}
+
+static void
+cn20k_sso_process_tstamp(uint64_t u64, uint64_t mbuf, struct cnxk_timesync_info *tstamp)
+{
+ uint64_t tstamp_ptr;
+ uint8_t laptr;
+
+ laptr = (uint8_t)*(uint64_t *)(u64 + (CNXK_SSO_WQE_LAYR_PTR * sizeof(uint64_t)));
+ if (laptr == sizeof(uint64_t)) {
+ /* Extracting tstamp, if PTP enabled*/
+ tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *)u64) + CNXK_SSO_WQE_SG_PTR);
+ cn20k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, tstamp, true,
+ (uint64_t *)tstamp_ptr);
+ }
+}
+
static __rte_always_inline void
cn20k_sso_hws_post_process(struct cn20k_sso_hws *ws, uint64_t *u64, const uint32_t flags)
{
- RTE_SET_USED(ws);
- RTE_SET_USED(flags);
+ uintptr_t sa_base = 0;
u64[0] = (u64[0] & (0x3ull << 32)) << 6 | (u64[0] & (0x3FFull << 36)) << 4 |
(u64[0] & 0xffffffff);
+ if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_ETHDEV) {
+ uint8_t port = CNXK_SUB_EVENT_FROM_TAG(u64[0]);
+ uintptr_t cpth = 0;
+ uint64_t mbuf;
+
+ mbuf = u64[1] - sizeof(struct rte_mbuf);
+ rte_prefetch0((void *)mbuf);
+
+ /* Mark mempool obj as "get" as it is alloc'ed by NIX */
+ RTE_MEMPOOL_CHECK_COOKIES(((struct rte_mbuf *)mbuf)->pool, (void **)&mbuf, 1, 1);
+
+ u64[0] = CNXK_CLR_SUB_EVENT(u64[0]);
+ cn20k_wqe_to_mbuf(u64[1], mbuf, port, u64[0] & 0xFFFFF, flags, ws->lookup_mem, cpth,
+ sa_base);
+ if (flags & NIX_RX_OFFLOAD_TSTAMP_F)
+ cn20k_sso_process_tstamp(u64[1], mbuf, ws->tstamp[port]);
+ u64[1] = mbuf;
+ }
}
static __rte_always_inline uint16_t
@@ -150,11 +198,112 @@ int __rte_hot cn20k_sso_hws_preschedule_modify(void *port,
enum rte_event_dev_preschedule_type type);
void __rte_hot cn20k_sso_hws_preschedule(void *port, enum rte_event_dev_preschedule_type type);
-uint16_t __rte_hot cn20k_sso_hws_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks);
-uint16_t __rte_hot cn20k_sso_hws_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events,
- uint64_t timeout_ticks);
-uint16_t __rte_hot cn20k_sso_hws_tmo_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks);
-uint16_t __rte_hot cn20k_sso_hws_tmo_deq_burst(void *port, struct rte_event ev[],
- uint16_t nb_events, uint64_t timeout_ticks);
+#define R(name, flags) \
+ uint16_t __rte_hot cn20k_sso_hws_deq_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_tmo_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_ca_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_tmo_ca_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_tmo_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_ca_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_deq_tmo_ca_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_ca_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_ca_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_ca_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \
+ uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_ca_seg_burst_##name( \
+ void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks);
+
+NIX_RX_FASTPATH_MODES
+#undef R
+
+#define SSO_DEQ(fn, flags) \
+ static __rte_always_inline uint16_t fn(void *port, struct rte_event *ev, \
+ uint64_t timeout_ticks) \
+ { \
+ struct cn20k_sso_hws *ws = port; \
+ RTE_SET_USED(timeout_ticks); \
+ if (ws->swtag_req) { \
+ ws->swtag_req = 0; \
+ ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); \
+ return 1; \
+ } \
+ return cn20k_sso_hws_get_work(ws, ev, flags); \
+ }
+
+#define SSO_DEQ_SEG(fn, flags) SSO_DEQ(fn, flags | NIX_RX_MULTI_SEG_F)
+
+#define SSO_DEQ_TMO(fn, flags) \
+ static __rte_always_inline uint16_t fn(void *port, struct rte_event *ev, \
+ uint64_t timeout_ticks) \
+ { \
+ struct cn20k_sso_hws *ws = port; \
+ uint16_t ret = 1; \
+ uint64_t iter; \
+ if (ws->swtag_req) { \
+ ws->swtag_req = 0; \
+ ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); \
+ return ret; \
+ } \
+ ret = cn20k_sso_hws_get_work(ws, ev, flags); \
+ for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \
+ ret = cn20k_sso_hws_get_work(ws, ev, flags); \
+ return ret; \
+ }
+
+#define SSO_DEQ_TMO_SEG(fn, flags) SSO_DEQ_TMO(fn, flags | NIX_RX_MULTI_SEG_F)
+
+#define R(name, flags) \
+ SSO_DEQ(cn20k_sso_hws_deq_##name, flags) \
+ SSO_DEQ(cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) \
+ SSO_DEQ_SEG(cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_DEQ_SEG(cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) \
+ SSO_DEQ_TMO(cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_DEQ_TMO(cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) \
+ SSO_DEQ_TMO_SEG(cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_DEQ_TMO_SEG(cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES
+#undef R
+
+#define SSO_CMN_DEQ_BURST(fnb, fn, flags) \
+ uint16_t __rte_hot fnb(void *port, struct rte_event ev[], uint16_t nb_events, \
+ uint64_t timeout_ticks) \
+ { \
+ RTE_SET_USED(nb_events); \
+ return fn(port, ev, timeout_ticks); \
+ }
+
+#define SSO_CMN_DEQ_SEG_BURST(fnb, fn, flags) \
+ uint16_t __rte_hot fnb(void *port, struct rte_event ev[], uint16_t nb_events, \
+ uint64_t timeout_ticks) \
+ { \
+ RTE_SET_USED(nb_events); \
+ return fn(port, ev, timeout_ticks); \
+ }
+
+uint16_t __rte_hot cn20k_sso_hws_deq_burst_all_offload(void *port, struct rte_event ev[],
+ uint16_t nb_events, uint64_t timeout_ticks);
+uint16_t __rte_hot cn20k_sso_hws_deq_burst_all_offload_tst(void *port, struct rte_event ev[],
+ uint16_t nb_events,
+ uint64_t timeout_ticks);
#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c
new file mode 100644
index 0000000000..f7e0e8fe71
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_0_15
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c
new file mode 100644
index 0000000000..7d5d4823c3
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_0_15
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c
new file mode 100644
index 0000000000..1bdc4bc82d
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_0_15
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c
new file mode 100644
index 0000000000..d3ed5fcac0
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_0_15
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c
new file mode 100644
index 0000000000..29c21441cf
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_112_127
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c
new file mode 100644
index 0000000000..004b5ecb95
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_112_127
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c
new file mode 100644
index 0000000000..d544b39e9e
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_112_127
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c
new file mode 100644
index 0000000000..ba7a1207ad
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_112_127
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c
new file mode 100644
index 0000000000..eb7382e9d9
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F_)
+
+NIX_RX_FASTPATH_MODES_16_31
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c
new file mode 100644
index 0000000000..770b7221e6
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_16_31
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c
new file mode 100644
index 0000000000..1e71d22fc3
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_16_31
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c
new file mode 100644
index 0000000000..1a9e7efa0a
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_16_31
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c
new file mode 100644
index 0000000000..3d51bd6659
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F_)
+
+NIX_RX_FASTPATH_MODES_32_47
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c
new file mode 100644
index 0000000000..851b5b7d31
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_32_47
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c
new file mode 100644
index 0000000000..038ba726a0
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, \
+ flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_32_47
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c
new file mode 100644
index 0000000000..68fb3ff53d
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_32_47
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c
new file mode 100644
index 0000000000..84f3ccd39c
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_48_63
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c
new file mode 100644
index 0000000000..417f622412
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_48_63
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c
new file mode 100644
index 0000000000..7fbea69134
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_48_63
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c
new file mode 100644
index 0000000000..3bee216768
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_48_63
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c
new file mode 100644
index 0000000000..9b341a0df5
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_64_79
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c
new file mode 100644
index 0000000000..1f051f74a9
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_64_79
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c
new file mode 100644
index 0000000000..c134e27f25
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_64_79
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c
new file mode 100644
index 0000000000..849e8e12fc
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_64_79
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c
new file mode 100644
index 0000000000..9724caf5d6
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_80_95
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c
new file mode 100644
index 0000000000..997c208511
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_80_95
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c
new file mode 100644
index 0000000000..bcf32e646b
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_80_95
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c
new file mode 100644
index 0000000000..b24e73439a
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_80_95
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c
new file mode 100644
index 0000000000..c03d034b66
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \
+ cn20k_sso_hws_deq_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \
+ cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_96_111
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c
new file mode 100644
index 0000000000..b37ef7a998
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \
+ cn20k_sso_hws_deq_seg_##name, flags) \
+ SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_96_111
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c
new file mode 100644
index 0000000000..da76b589a0
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \
+ cn20k_sso_hws_deq_tmo_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_96_111
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c
new file mode 100644
index 0000000000..3a8c02e4d2
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if !defined(CNXK_DIS_TMPLT_FUNC)
+
+#define R(name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_deq_tmo_seg_##name, flags) \
+ SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \
+ cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F)
+
+NIX_RX_FASTPATH_MODES_96_111
+#undef R
+
+#endif
diff --git a/drivers/event/cnxk/deq/cn20k/deq_all_offload.c b/drivers/event/cnxk/deq/cn20k/deq_all_offload.c
new file mode 100644
index 0000000000..3983736b7e
--- /dev/null
+++ b/drivers/event/cnxk/deq/cn20k/deq_all_offload.c
@@ -0,0 +1,65 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2022 Marvell.
+ */
+
+#include "cn20k_worker.h"
+
+#ifdef _ROC_API_H_
+#error "roc_api.h is included"
+#endif
+
+#if defined(CNXK_DIS_TMPLT_FUNC)
+
+uint16_t __rte_hot
+cn20k_sso_hws_deq_burst_all_offload(void *port, struct rte_event ev[], uint16_t nb_events,
+ uint64_t timeout_ticks)
+{
+ const uint32_t flags = (NIX_RX_OFFLOAD_RSS_F | NIX_RX_OFFLOAD_PTYPE_F |
+ NIX_RX_OFFLOAD_CHECKSUM_F | NIX_RX_OFFLOAD_MARK_UPDATE_F |
+ NIX_RX_OFFLOAD_VLAN_STRIP_F |
+ NIX_RX_OFFLOAD_SECURITY_F | NIX_RX_MULTI_SEG_F | NIX_RX_REAS_F);
+ struct cn20k_sso_hws *ws = port;
+ uint16_t ret = 1;
+ uint64_t iter;
+
+ RTE_SET_USED(nb_events);
+ if (ws->swtag_req) {
+ ws->swtag_req = 0;
+ ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0);
+ return ret;
+ }
+
+ ret = cn20k_sso_hws_get_work(ws, ev, flags);
+ for (iter = 1; iter < timeout_ticks && (ret == 0); iter++)
+ ret = cn20k_sso_hws_get_work(ws, ev, flags);
+
+ return ret;
+}
+
+uint16_t __rte_hot
+cn20k_sso_hws_deq_burst_all_offload_tst(void *port, struct rte_event ev[], uint16_t nb_events,
+ uint64_t timeout_ticks)
+{
+ const uint32_t flags = (NIX_RX_OFFLOAD_RSS_F | NIX_RX_OFFLOAD_PTYPE_F |
+ NIX_RX_OFFLOAD_CHECKSUM_F | NIX_RX_OFFLOAD_MARK_UPDATE_F |
+ NIX_RX_OFFLOAD_TSTAMP_F | NIX_RX_OFFLOAD_VLAN_STRIP_F |
+ NIX_RX_OFFLOAD_SECURITY_F | NIX_RX_MULTI_SEG_F | NIX_RX_REAS_F);
+ struct cn20k_sso_hws *ws = port;
+ uint16_t ret = 1;
+ uint64_t iter;
+
+ RTE_SET_USED(nb_events);
+ if (ws->swtag_req) {
+ ws->swtag_req = 0;
+ ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0);
+ return ret;
+ }
+
+ ret = cn20k_sso_hws_get_work(ws, ev, flags);
+ for (iter = 1; iter < timeout_ticks && (ret == 0); iter++)
+ ret = cn20k_sso_hws_get_work(ws, ev, flags);
+
+ return ret;
+}
+
+#endif
diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build
index ee27d38115..2f7fe0dc47 100644
--- a/drivers/event/cnxk/meson.build
+++ b/drivers/event/cnxk/meson.build
@@ -330,6 +330,49 @@ sources += files(
'cn20k_eventdev.c',
'cn20k_worker.c',
)
+
+if host_machine.cpu_family().startswith('aarch') and not disable_template
+sources += files(
+ 'deq/cn20k/deq_0_15_burst.c',
+ 'deq/cn20k/deq_16_31_burst.c',
+ 'deq/cn20k/deq_32_47_burst.c',
+ 'deq/cn20k/deq_48_63_burst.c',
+ 'deq/cn20k/deq_64_79_burst.c',
+ 'deq/cn20k/deq_80_95_burst.c',
+ 'deq/cn20k/deq_96_111_burst.c',
+ 'deq/cn20k/deq_112_127_burst.c',
+ 'deq/cn20k/deq_0_15_seg_burst.c',
+ 'deq/cn20k/deq_16_31_seg_burst.c',
+ 'deq/cn20k/deq_32_47_seg_burst.c',
+ 'deq/cn20k/deq_48_63_seg_burst.c',
+ 'deq/cn20k/deq_64_79_seg_burst.c',
+ 'deq/cn20k/deq_80_95_seg_burst.c',
+ 'deq/cn20k/deq_96_111_seg_burst.c',
+ 'deq/cn20k/deq_112_127_seg_burst.c',
+ 'deq/cn20k/deq_0_15_tmo_burst.c',
+ 'deq/cn20k/deq_16_31_tmo_burst.c',
+ 'deq/cn20k/deq_32_47_tmo_burst.c',
+ 'deq/cn20k/deq_48_63_tmo_burst.c',
+ 'deq/cn20k/deq_64_79_tmo_burst.c',
+ 'deq/cn20k/deq_80_95_tmo_burst.c',
+ 'deq/cn20k/deq_96_111_tmo_burst.c',
+ 'deq/cn20k/deq_112_127_tmo_burst.c',
+ 'deq/cn20k/deq_0_15_tmo_seg_burst.c',
+ 'deq/cn20k/deq_16_31_tmo_seg_burst.c',
+ 'deq/cn20k/deq_32_47_tmo_seg_burst.c',
+ 'deq/cn20k/deq_48_63_tmo_seg_burst.c',
+ 'deq/cn20k/deq_64_79_tmo_seg_burst.c',
+ 'deq/cn20k/deq_80_95_tmo_seg_burst.c',
+ 'deq/cn20k/deq_96_111_tmo_seg_burst.c',
+ 'deq/cn20k/deq_112_127_tmo_seg_burst.c',
+ 'deq/cn20k/deq_all_offload.c',
+)
+
+else
+sources += files(
+ 'deq/cn20k/deq_all_offload.c',
+)
+endif
endif
extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing']
--
2.25.1