From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8908245B96; Mon, 21 Oct 2024 22:59:30 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD9164067C; Mon, 21 Oct 2024 22:58:52 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 148B440611 for ; Mon, 21 Oct 2024 22:58:26 +0200 (CEST) Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49LCQx8Y004165 for ; Mon, 21 Oct 2024 13:58:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=C 6QsD4lILSANIkPJrS8plmHtLgY8j+I9MibRIQQ7TRc=; b=VnnXILuwR/xCfdX/N qzOJ/d662Xobzgi9vgZr7QbqYO7y6gvp6tkMUhD5kX77O+I7XOu7CE/7g8WzJD8M LYZa1vCuR2JlGJ+GFnhj7DJgVTkHdzI0poMs8KUXC2rgW6txV4Ge7oxvyDGAmgqC pPvjzTkfB+fxCQJ3ykE1uZANW5qviFjWDxmKlJf7Iy0uT61DPWDASQtLjqOCS5ef PzQN6+4zhzKe44xVcwKzv+6qyU1T2xffktsQcPpNfMq7I/e8Ewn6IED9rUUNO9o5 1d47rLWbL0eHz+cgBw/GaDrEXpf5cUco3sVrHqrOjceCqxxAbzgtl0O7TDlTlQjj Jl04Q== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42dpxv17n0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 21 Oct 2024 13:58:26 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 21 Oct 2024 13:58:24 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 21 Oct 2024 13:58:24 -0700 Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com [10.28.164.106]) by maili.marvell.com (Postfix) with ESMTP id B66713F7041; Mon, 21 Oct 2024 13:58:22 -0700 (PDT) From: To: , Pavan Nikhilesh , "Shijith Thotton" CC: Subject: [PATCH v2 15/21] event/cnxk: support CN20K Rx adapter fast path Date: Tue, 22 Oct 2024 02:27:38 +0530 Message-ID: <20241021205745.7310-15-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241021205745.7310-1-pbhagavatula@marvell.com> References: <20241003132237.20193-1-pbhagavatula@marvell.com> <20241021205745.7310-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: HDkGzzSmhYox7Rixy73fWOWB1o3B5t2q X-Proofpoint-ORIG-GUID: HDkGzzSmhYox7Rixy73fWOWB1o3B5t2q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Add support for event eth Rx adapter fastpath operations. Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cn20k_eventdev.c | 122 ++++++++++++- drivers/event/cnxk/cn20k_worker.c | 54 ------ drivers/event/cnxk/cn20k_worker.h | 165 +++++++++++++++++- drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_0_15_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_112_127_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_112_127_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_112_127_tmo_burst.c | 22 +++ .../deq/cn20k/deq_112_127_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_16_31_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_16_31_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_16_31_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_32_47_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_32_47_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_32_47_tmo_burst.c | 23 +++ .../cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_48_63_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_48_63_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_48_63_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_64_79_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_64_79_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_64_79_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_80_95_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_80_95_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_80_95_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_96_111_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_96_111_seg_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_96_111_tmo_burst.c | 22 +++ .../cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c | 22 +++ .../event/cnxk/deq/cn20k/deq_all_offload.c | 65 +++++++ drivers/event/cnxk/meson.build | 43 +++++ 37 files changed, 1085 insertions(+), 69 deletions(-) create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn20k/deq_all_offload.c diff --git a/drivers/event/cnxk/cn20k_eventdev.c b/drivers/event/cnxk/cn20k_eventdev.c index ea2b218373..90c34b1d62 100644 --- a/drivers/event/cnxk/cn20k_eventdev.c +++ b/drivers/event/cnxk/cn20k_eventdev.c @@ -11,6 +11,9 @@ #include "cnxk_eventdev.h" #include "cnxk_worker.h" +#define CN20K_SET_EVDEV_DEQ_OP(dev, deq_op, deq_ops) \ + deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)] + static void * cn20k_sso_init_hws_mem(void *arg, uint8_t port_id) { @@ -163,21 +166,124 @@ cn20k_sso_rsrc_init(void *arg, uint8_t hws, uint8_t hwgrp) return roc_sso_rsrc_init(&dev->sso, hws, hwgrp, nb_tim_lfs); } +#if defined(RTE_ARCH_ARM64) +static inline void +cn20k_sso_fp_tmplt_fns_set(struct rte_eventdev *event_dev) +{ +#if !defined(CNXK_DIS_TMPLT_FUNC) + struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev); + + const event_dequeue_burst_t sso_hws_deq_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_deq_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_deq_tmo_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_deq_tmo_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_deq_seg_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_deq_seg_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_deq_tmo_seg_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_deq_tmo_seg_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_reas_deq_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_reas_deq_tmo_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_tmo_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_reas_deq_seg_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_seg_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + const event_dequeue_burst_t sso_hws_reas_deq_tmo_seg_burst[NIX_RX_OFFLOAD_MAX] = { +#define R(name, flags) [flags] = cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { + if (dev->rx_offloads & NIX_RX_REAS_F) { + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_reas_deq_seg_burst); + if (dev->is_timeout_deq) + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_reas_deq_tmo_seg_burst); + } else { + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_deq_seg_burst); + + if (dev->is_timeout_deq) + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_deq_tmo_seg_burst); + } + } else { + if (dev->rx_offloads & NIX_RX_REAS_F) { + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_reas_deq_burst); + + if (dev->is_timeout_deq) + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_reas_deq_tmo_burst); + } else { + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, sso_hws_deq_burst); + + if (dev->is_timeout_deq) + CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, + sso_hws_deq_tmo_burst); + } + } + +#else + RTE_SET_USED(event_dev); +#endif +} + +static inline void +cn20k_sso_fp_blk_fns_set(struct rte_eventdev *event_dev) +{ +#if defined(CNXK_DIS_TMPLT_FUNC) + struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev); + + event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload; + if (dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F) + event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload_tst; +#else + RTE_SET_USED(event_dev); +#endif +} +#endif static void cn20k_sso_fp_fns_set(struct rte_eventdev *event_dev) { #if defined(RTE_ARCH_ARM64) - struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(event_dev); + cn20k_sso_fp_blk_fns_set(event_dev); + cn20k_sso_fp_tmplt_fns_set(event_dev); event_dev->enqueue_burst = cn20k_sso_hws_enq_burst; event_dev->enqueue_new_burst = cn20k_sso_hws_enq_new_burst; event_dev->enqueue_forward_burst = cn20k_sso_hws_enq_fwd_burst; - event_dev->dequeue_burst = cn20k_sso_hws_deq_burst; - if (dev->deq_tmo_ns) - event_dev->dequeue_burst = cn20k_sso_hws_tmo_deq_burst; - event_dev->profile_switch = cn20k_sso_hws_profile_switch; event_dev->preschedule_modify = cn20k_sso_hws_preschedule_modify; event_dev->preschedule = cn20k_sso_hws_preschedule; @@ -284,7 +390,8 @@ cn20k_sso_port_quiesce(struct rte_eventdev *event_dev, void *port, ptag = plt_read64(ws->base + SSOW_LF_GWS_PENDSTATE); } while (ptag & (BIT_ULL(62) | BIT_ULL(58) | BIT_ULL(56) | BIT_ULL(54))); - cn20k_sso_hws_get_work_empty(ws, &ev, 0); + cn20k_sso_hws_get_work_empty(ws, &ev, + (NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F | NIX_RX_MULTI_SEG_F); if (is_pend && ev.u64) if (flush_cb) flush_cb(event_dev->data->dev_id, ev, args); @@ -310,7 +417,8 @@ cn20k_sso_port_quiesce(struct rte_eventdev *event_dev, void *port, if (CNXK_TT_FROM_TAG(plt_read64(ws->base + SSOW_LF_GWS_PRF_WQE0)) != SSO_TT_EMPTY) { plt_write64(BIT_ULL(16) | 1, ws->base + SSOW_LF_GWS_OP_GET_WORK0); - cn20k_sso_hws_get_work_empty(ws, &ev, 0); + cn20k_sso_hws_get_work_empty( + ws, &ev, (NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F | NIX_RX_MULTI_SEG_F); if (ev.u64) { if (flush_cb) flush_cb(event_dev->data->dev_id, ev, args); diff --git a/drivers/event/cnxk/cn20k_worker.c b/drivers/event/cnxk/cn20k_worker.c index b0196e9885..b67f921b67 100644 --- a/drivers/event/cnxk/cn20k_worker.c +++ b/drivers/event/cnxk/cn20k_worker.c @@ -419,57 +419,3 @@ cn20k_sso_hws_preschedule(void *port, enum rte_event_dev_preschedule_type type) RTE_SET_USED(type); plt_write64(ws->gw_wdata, ws->base + SSOW_LF_GWS_OP_PRF_GETWORK); } - -uint16_t __rte_hot -cn20k_sso_hws_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct cn20k_sso_hws *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); - return 1; - } - - return cn20k_sso_hws_get_work(ws, ev, 0); -} - -uint16_t __rte_hot -cn20k_sso_hws_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return cn20k_sso_hws_deq(port, ev, timeout_ticks); -} - -uint16_t __rte_hot -cn20k_sso_hws_tmo_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct cn20k_sso_hws *ws = port; - uint16_t ret = 1; - uint64_t iter; - - if (ws->swtag_req) { - ws->swtag_req = 0; - cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); - return ret; - } - - ret = cn20k_sso_hws_get_work(ws, ev, 0); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = cn20k_sso_hws_get_work(ws, ev, 0); - - return ret; -} - -uint16_t __rte_hot -cn20k_sso_hws_tmo_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return cn20k_sso_hws_tmo_deq(port, ev, timeout_ticks); -} diff --git a/drivers/event/cnxk/cn20k_worker.h b/drivers/event/cnxk/cn20k_worker.h index dd8b72bc53..9075073fd2 100644 --- a/drivers/event/cnxk/cn20k_worker.h +++ b/drivers/event/cnxk/cn20k_worker.h @@ -8,16 +8,64 @@ #include #include "cn20k_eventdev.h" +#include "cn20k_rx.h" #include "cnxk_worker.h" +/* CN20K Rx event fastpath */ + +static __rte_always_inline void +cn20k_wqe_to_mbuf(uint64_t wqe, const uint64_t __mbuf, uint8_t port_id, const uint32_t tag, + const uint32_t flags, const void *const lookup_mem, uintptr_t cpth, + uintptr_t sa_base) +{ + const uint64_t mbuf_init = + 0x100010000ULL | RTE_PKTMBUF_HEADROOM | (flags & NIX_RX_OFFLOAD_TSTAMP_F ? 8 : 0); + struct rte_mbuf *mbuf = (struct rte_mbuf *)__mbuf; + + cn20k_nix_cqe_to_mbuf((struct nix_cqe_hdr_s *)wqe, tag, (struct rte_mbuf *)mbuf, lookup_mem, + mbuf_init | ((uint64_t)port_id) << 48, cpth, sa_base, flags); +} + +static void +cn20k_sso_process_tstamp(uint64_t u64, uint64_t mbuf, struct cnxk_timesync_info *tstamp) +{ + uint64_t tstamp_ptr; + uint8_t laptr; + + laptr = (uint8_t)*(uint64_t *)(u64 + (CNXK_SSO_WQE_LAYR_PTR * sizeof(uint64_t))); + if (laptr == sizeof(uint64_t)) { + /* Extracting tstamp, if PTP enabled*/ + tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *)u64) + CNXK_SSO_WQE_SG_PTR); + cn20k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, tstamp, true, + (uint64_t *)tstamp_ptr); + } +} + static __rte_always_inline void cn20k_sso_hws_post_process(struct cn20k_sso_hws *ws, uint64_t *u64, const uint32_t flags) { - RTE_SET_USED(ws); - RTE_SET_USED(flags); + uintptr_t sa_base = 0; u64[0] = (u64[0] & (0x3ull << 32)) << 6 | (u64[0] & (0x3FFull << 36)) << 4 | (u64[0] & 0xffffffff); + if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_ETHDEV) { + uint8_t port = CNXK_SUB_EVENT_FROM_TAG(u64[0]); + uintptr_t cpth = 0; + uint64_t mbuf; + + mbuf = u64[1] - sizeof(struct rte_mbuf); + rte_prefetch0((void *)mbuf); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + RTE_MEMPOOL_CHECK_COOKIES(((struct rte_mbuf *)mbuf)->pool, (void **)&mbuf, 1, 1); + + u64[0] = CNXK_CLR_SUB_EVENT(u64[0]); + cn20k_wqe_to_mbuf(u64[1], mbuf, port, u64[0] & 0xFFFFF, flags, ws->lookup_mem, cpth, + sa_base); + if (flags & NIX_RX_OFFLOAD_TSTAMP_F) + cn20k_sso_process_tstamp(u64[1], mbuf, ws->tstamp[port]); + u64[1] = mbuf; + } } static __rte_always_inline uint16_t @@ -150,11 +198,112 @@ int __rte_hot cn20k_sso_hws_preschedule_modify(void *port, enum rte_event_dev_preschedule_type type); void __rte_hot cn20k_sso_hws_preschedule(void *port, enum rte_event_dev_preschedule_type type); -uint16_t __rte_hot cn20k_sso_hws_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks); -uint16_t __rte_hot cn20k_sso_hws_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks); -uint16_t __rte_hot cn20k_sso_hws_tmo_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks); -uint16_t __rte_hot cn20k_sso_hws_tmo_deq_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); +#define R(name, flags) \ + uint16_t __rte_hot cn20k_sso_hws_deq_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_tmo_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_ca_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_tmo_ca_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_tmo_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_ca_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_deq_tmo_ca_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_ca_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_ca_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_ca_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); \ + uint16_t __rte_hot cn20k_sso_hws_reas_deq_tmo_ca_seg_burst_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); + +NIX_RX_FASTPATH_MODES +#undef R + +#define SSO_DEQ(fn, flags) \ + static __rte_always_inline uint16_t fn(void *port, struct rte_event *ev, \ + uint64_t timeout_ticks) \ + { \ + struct cn20k_sso_hws *ws = port; \ + RTE_SET_USED(timeout_ticks); \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); \ + return 1; \ + } \ + return cn20k_sso_hws_get_work(ws, ev, flags); \ + } + +#define SSO_DEQ_SEG(fn, flags) SSO_DEQ(fn, flags | NIX_RX_MULTI_SEG_F) + +#define SSO_DEQ_TMO(fn, flags) \ + static __rte_always_inline uint16_t fn(void *port, struct rte_event *ev, \ + uint64_t timeout_ticks) \ + { \ + struct cn20k_sso_hws *ws = port; \ + uint16_t ret = 1; \ + uint64_t iter; \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); \ + return ret; \ + } \ + ret = cn20k_sso_hws_get_work(ws, ev, flags); \ + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \ + ret = cn20k_sso_hws_get_work(ws, ev, flags); \ + return ret; \ + } + +#define SSO_DEQ_TMO_SEG(fn, flags) SSO_DEQ_TMO(fn, flags | NIX_RX_MULTI_SEG_F) + +#define R(name, flags) \ + SSO_DEQ(cn20k_sso_hws_deq_##name, flags) \ + SSO_DEQ(cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) \ + SSO_DEQ_SEG(cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_DEQ_SEG(cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) \ + SSO_DEQ_TMO(cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_DEQ_TMO(cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) \ + SSO_DEQ_TMO_SEG(cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_DEQ_TMO_SEG(cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES +#undef R + +#define SSO_CMN_DEQ_BURST(fnb, fn, flags) \ + uint16_t __rte_hot fnb(void *port, struct rte_event ev[], uint16_t nb_events, \ + uint64_t timeout_ticks) \ + { \ + RTE_SET_USED(nb_events); \ + return fn(port, ev, timeout_ticks); \ + } + +#define SSO_CMN_DEQ_SEG_BURST(fnb, fn, flags) \ + uint16_t __rte_hot fnb(void *port, struct rte_event ev[], uint16_t nb_events, \ + uint64_t timeout_ticks) \ + { \ + RTE_SET_USED(nb_events); \ + return fn(port, ev, timeout_ticks); \ + } + +uint16_t __rte_hot cn20k_sso_hws_deq_burst_all_offload(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks); +uint16_t __rte_hot cn20k_sso_hws_deq_burst_all_offload_tst(void *port, struct rte_event ev[], + uint16_t nb_events, + uint64_t timeout_ticks); #endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c new file mode 100644 index 0000000000..f7e0e8fe71 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_0_15 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c new file mode 100644 index 0000000000..7d5d4823c3 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_0_15 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c new file mode 100644 index 0000000000..1bdc4bc82d --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_0_15 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c new file mode 100644 index 0000000000..d3ed5fcac0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_0_15_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_0_15 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c new file mode 100644 index 0000000000..29c21441cf --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_112_127 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c new file mode 100644 index 0000000000..004b5ecb95 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_112_127 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c new file mode 100644 index 0000000000..d544b39e9e --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_112_127 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c new file mode 100644 index 0000000000..ba7a1207ad --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_112_127_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_112_127 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c new file mode 100644 index 0000000000..eb7382e9d9 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F_) + +NIX_RX_FASTPATH_MODES_16_31 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c new file mode 100644 index 0000000000..770b7221e6 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_16_31 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c new file mode 100644 index 0000000000..1e71d22fc3 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_16_31 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c new file mode 100644 index 0000000000..1a9e7efa0a --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_16_31_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_16_31 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c new file mode 100644 index 0000000000..3d51bd6659 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F_) + +NIX_RX_FASTPATH_MODES_32_47 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c new file mode 100644 index 0000000000..851b5b7d31 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_32_47 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c new file mode 100644 index 0000000000..038ba726a0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_burst.c @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, \ + flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_32_47 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c new file mode 100644 index 0000000000..68fb3ff53d --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_32_47_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_32_47 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c new file mode 100644 index 0000000000..84f3ccd39c --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_48_63 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c new file mode 100644 index 0000000000..417f622412 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_48_63 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c new file mode 100644 index 0000000000..7fbea69134 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_48_63 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c new file mode 100644 index 0000000000..3bee216768 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_48_63_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_48_63 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c new file mode 100644 index 0000000000..9b341a0df5 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_64_79 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c new file mode 100644 index 0000000000..1f051f74a9 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_64_79 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c new file mode 100644 index 0000000000..c134e27f25 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_64_79 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c new file mode 100644 index 0000000000..849e8e12fc --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_64_79_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_64_79 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c new file mode 100644 index 0000000000..9724caf5d6 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_80_95 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c new file mode 100644 index 0000000000..997c208511 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_80_95 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c new file mode 100644 index 0000000000..bcf32e646b --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_80_95 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c new file mode 100644 index 0000000000..b24e73439a --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_80_95_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_80_95 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c new file mode 100644 index 0000000000..c03d034b66 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_burst_##name, \ + cn20k_sso_hws_deq_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_burst_##name, \ + cn20k_sso_hws_reas_deq_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_96_111 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c new file mode 100644 index 0000000000..b37ef7a998 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_deq_seg_burst_##name, \ + cn20k_sso_hws_deq_seg_##name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn20k_sso_hws_reas_deq_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_96_111 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c new file mode 100644 index 0000000000..da76b589a0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_burst_##name, \ + cn20k_sso_hws_deq_tmo_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_96_111 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c new file mode 100644 index 0000000000..3a8c02e4d2 --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_96_111_tmo_seg_burst.c @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_deq_tmo_seg_##name, flags) \ + SSO_CMN_DEQ_BURST(cn20k_sso_hws_reas_deq_tmo_seg_burst_##name, \ + cn20k_sso_hws_reas_deq_tmo_seg_##name, flags | NIX_RX_REAS_F) + +NIX_RX_FASTPATH_MODES_96_111 +#undef R + +#endif diff --git a/drivers/event/cnxk/deq/cn20k/deq_all_offload.c b/drivers/event/cnxk/deq/cn20k/deq_all_offload.c new file mode 100644 index 0000000000..3983736b7e --- /dev/null +++ b/drivers/event/cnxk/deq/cn20k/deq_all_offload.c @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if defined(CNXK_DIS_TMPLT_FUNC) + +uint16_t __rte_hot +cn20k_sso_hws_deq_burst_all_offload(void *port, struct rte_event ev[], uint16_t nb_events, + uint64_t timeout_ticks) +{ + const uint32_t flags = (NIX_RX_OFFLOAD_RSS_F | NIX_RX_OFFLOAD_PTYPE_F | + NIX_RX_OFFLOAD_CHECKSUM_F | NIX_RX_OFFLOAD_MARK_UPDATE_F | + NIX_RX_OFFLOAD_VLAN_STRIP_F | + NIX_RX_OFFLOAD_SECURITY_F | NIX_RX_MULTI_SEG_F | NIX_RX_REAS_F); + struct cn20k_sso_hws *ws = port; + uint16_t ret = 1; + uint64_t iter; + + RTE_SET_USED(nb_events); + if (ws->swtag_req) { + ws->swtag_req = 0; + ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); + return ret; + } + + ret = cn20k_sso_hws_get_work(ws, ev, flags); + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) + ret = cn20k_sso_hws_get_work(ws, ev, flags); + + return ret; +} + +uint16_t __rte_hot +cn20k_sso_hws_deq_burst_all_offload_tst(void *port, struct rte_event ev[], uint16_t nb_events, + uint64_t timeout_ticks) +{ + const uint32_t flags = (NIX_RX_OFFLOAD_RSS_F | NIX_RX_OFFLOAD_PTYPE_F | + NIX_RX_OFFLOAD_CHECKSUM_F | NIX_RX_OFFLOAD_MARK_UPDATE_F | + NIX_RX_OFFLOAD_TSTAMP_F | NIX_RX_OFFLOAD_VLAN_STRIP_F | + NIX_RX_OFFLOAD_SECURITY_F | NIX_RX_MULTI_SEG_F | NIX_RX_REAS_F); + struct cn20k_sso_hws *ws = port; + uint16_t ret = 1; + uint64_t iter; + + RTE_SET_USED(nb_events); + if (ws->swtag_req) { + ws->swtag_req = 0; + ws->gw_rdata = cnxk_sso_hws_swtag_wait(ws->base + SSOW_LF_GWS_WQE0); + return ret; + } + + ret = cn20k_sso_hws_get_work(ws, ev, flags); + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) + ret = cn20k_sso_hws_get_work(ws, ev, flags); + + return ret; +} + +#endif diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build index ee27d38115..2f7fe0dc47 100644 --- a/drivers/event/cnxk/meson.build +++ b/drivers/event/cnxk/meson.build @@ -330,6 +330,49 @@ sources += files( 'cn20k_eventdev.c', 'cn20k_worker.c', ) + +if host_machine.cpu_family().startswith('aarch') and not disable_template +sources += files( + 'deq/cn20k/deq_0_15_burst.c', + 'deq/cn20k/deq_16_31_burst.c', + 'deq/cn20k/deq_32_47_burst.c', + 'deq/cn20k/deq_48_63_burst.c', + 'deq/cn20k/deq_64_79_burst.c', + 'deq/cn20k/deq_80_95_burst.c', + 'deq/cn20k/deq_96_111_burst.c', + 'deq/cn20k/deq_112_127_burst.c', + 'deq/cn20k/deq_0_15_seg_burst.c', + 'deq/cn20k/deq_16_31_seg_burst.c', + 'deq/cn20k/deq_32_47_seg_burst.c', + 'deq/cn20k/deq_48_63_seg_burst.c', + 'deq/cn20k/deq_64_79_seg_burst.c', + 'deq/cn20k/deq_80_95_seg_burst.c', + 'deq/cn20k/deq_96_111_seg_burst.c', + 'deq/cn20k/deq_112_127_seg_burst.c', + 'deq/cn20k/deq_0_15_tmo_burst.c', + 'deq/cn20k/deq_16_31_tmo_burst.c', + 'deq/cn20k/deq_32_47_tmo_burst.c', + 'deq/cn20k/deq_48_63_tmo_burst.c', + 'deq/cn20k/deq_64_79_tmo_burst.c', + 'deq/cn20k/deq_80_95_tmo_burst.c', + 'deq/cn20k/deq_96_111_tmo_burst.c', + 'deq/cn20k/deq_112_127_tmo_burst.c', + 'deq/cn20k/deq_0_15_tmo_seg_burst.c', + 'deq/cn20k/deq_16_31_tmo_seg_burst.c', + 'deq/cn20k/deq_32_47_tmo_seg_burst.c', + 'deq/cn20k/deq_48_63_tmo_seg_burst.c', + 'deq/cn20k/deq_64_79_tmo_seg_burst.c', + 'deq/cn20k/deq_80_95_tmo_seg_burst.c', + 'deq/cn20k/deq_96_111_tmo_seg_burst.c', + 'deq/cn20k/deq_112_127_tmo_seg_burst.c', + 'deq/cn20k/deq_all_offload.c', +) + +else +sources += files( + 'deq/cn20k/deq_all_offload.c', +) +endif endif extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] -- 2.25.1