From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84653A04FF; Wed, 27 Apr 2022 12:58:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 751FC427F5; Wed, 27 Apr 2022 12:58:37 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 636E241145 for ; Wed, 27 Apr 2022 12:58:36 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 23R8RAi3029656 for ; Wed, 27 Apr 2022 03:58:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=hKnC9tVpbQx5GW7ZMEmDGSfRNEx54w77FJRW+czDn6M=; b=dFpNeE9mLED2qQ0G95iYhH5mFuG+OR6KwqEbcatBRQysGvdqEZ7Lf5rTFlRaPNiTEJgY P4p6X9ulHOEqAsdE+h9abZykO4VARx0gvF5v1Bgk5CxTmxPGEcOjo/chJ9HE4hFy5dwk sOuOD5y8CkkVYOEN6FWElNSJGIIQkoFrsvTunIrxtqjIkWmBqV5wtY1FJkexONaM6587 aJSLmRbv/N7ISagpN2xnwVsox7ynRnyudDHyb7L4wzbbwc+iHGOZ5ysF3u7vZC6rSl/4 2MqoBjOHd+0ZaWcRXtTrSy+eIIeXmRaoXCQNYVDpmt2t5nPll0MERtHEpLiov4AbhoDe aQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3fprsqtcmh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Wed, 27 Apr 2022 03:58:35 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 27 Apr 2022 03:58:33 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 27 Apr 2022 03:58:33 -0700 Received: from MININT-80QBFE8.corp.innovium.com (unknown [10.193.69.50]) by maili.marvell.com (Postfix) with ESMTP id 984115B6932; Wed, 27 Apr 2022 03:58:31 -0700 (PDT) From: Pavan Nikhilesh To: , Pavan Nikhilesh , "Shijith Thotton" CC: Subject: [PATCH v2 2/2] event/cnxk: move post-processing to separate function Date: Wed, 27 Apr 2022 16:28:28 +0530 Message-ID: <20220427105828.12753-2-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220427105828.12753-1-pbhagavatula@marvell.com> References: <20220426214723.6537-1-pbhagavatula@marvell.com> <20220427105828.12753-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: 7WFZbowZT4NMVSckuoZ9-ya2upyAnOF2 X-Proofpoint-GUID: 7WFZbowZT4NMVSckuoZ9-ya2upyAnOF2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-27_03,2022-04-27_01,2022-02-23_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Move event post-processing to a separate function. Do complete event post-processing in tear-down functions to prevent incorrect memory free. Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cn10k_eventdev.c | 5 +- drivers/event/cnxk/cn10k_worker.h | 189 +++++++++++++--------------- drivers/event/cnxk/cn9k_eventdev.c | 9 +- drivers/event/cnxk/cn9k_worker.h | 114 ++++++----------- 4 files changed, 137 insertions(+), 180 deletions(-) diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c index 2fa2cd31c2..94829e789c 100644 --- a/drivers/event/cnxk/cn10k_eventdev.c +++ b/drivers/event/cnxk/cn10k_eventdev.c @@ -133,7 +133,10 @@ cn10k_sso_hws_flush_events(void *hws, uint8_t queue_id, uintptr_t base, while (aq_cnt || cq_ds_cnt || ds_cnt) { plt_write64(req, ws->base + SSOW_LF_GWS_OP_GET_WORK0); - cn10k_sso_hws_get_work_empty(ws, &ev); + cn10k_sso_hws_get_work_empty( + ws, &ev, + (NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F | + NIX_RX_MULTI_SEG_F | CPT_RX_WQE_F); if (fn != NULL && ev.u64 != 0) fn(arg, ev); if (ev.sched_type != SSO_TT_EMPTY) diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index c96048f47d..034f508dd8 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -196,15 +196,87 @@ cn10k_process_vwqe(uintptr_t vwqe, uint16_t port_id, const uint32_t flags, } } +static __rte_always_inline void +cn10k_sso_hws_post_process(struct cn10k_sso_hws *ws, uint64_t *u64, + const uint32_t flags) +{ + uint64_t tstamp_ptr; + + u64[0] = (u64[0] & (0x3ull << 32)) << 6 | + (u64[0] & (0x3FFull << 36)) << 4 | (u64[0] & 0xffffffff); + if ((flags & CPT_RX_WQE_F) && + (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_CRYPTODEV)) { + u64[1] = cn10k_cpt_crypto_adapter_dequeue(u64[1]); + } else if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_ETHDEV) { + uint8_t port = CNXK_SUB_EVENT_FROM_TAG(u64[0]); + uint64_t mbuf; + + mbuf = u64[1] - sizeof(struct rte_mbuf); + rte_prefetch0((void *)mbuf); + if (flags & NIX_RX_OFFLOAD_SECURITY_F) { + const uint64_t mbuf_init = + 0x100010000ULL | RTE_PKTMBUF_HEADROOM | + (flags & NIX_RX_OFFLOAD_TSTAMP_F ? 8 : 0); + struct rte_mbuf *m; + uintptr_t sa_base; + uint64_t iova = 0; + uint8_t loff = 0; + uint16_t d_off; + uint64_t cq_w1; + uint64_t cq_w5; + + m = (struct rte_mbuf *)mbuf; + d_off = (uintptr_t)(m->buf_addr) - (uintptr_t)m; + d_off += RTE_PKTMBUF_HEADROOM; + + cq_w1 = *(uint64_t *)(u64[1] + 8); + cq_w5 = *(uint64_t *)(u64[1] + 40); + + sa_base = cnxk_nix_sa_base_get(port, ws->lookup_mem); + sa_base &= ~(ROC_NIX_INL_SA_BASE_ALIGN - 1); + + mbuf = (uint64_t)nix_sec_meta_to_mbuf_sc( + cq_w1, cq_w5, sa_base, (uintptr_t)&iova, &loff, + (struct rte_mbuf *)mbuf, d_off, flags, + mbuf_init | ((uint64_t)port) << 48); + if (loff) + roc_npa_aura_op_free(m->pool->pool_id, 0, iova); + } + + u64[0] = CNXK_CLR_SUB_EVENT(u64[0]); + cn10k_wqe_to_mbuf(u64[1], mbuf, port, u64[0] & 0xFFFFF, flags, + ws->lookup_mem); + /* Extracting tstamp, if PTP enabled*/ + tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *)u64[1]) + + CNXK_SSO_WQE_SG_PTR); + cn10k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, ws->tstamp, + flags & NIX_RX_OFFLOAD_TSTAMP_F, + (uint64_t *)tstamp_ptr); + u64[1] = mbuf; + } else if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == + RTE_EVENT_TYPE_ETHDEV_VECTOR) { + uint8_t port = CNXK_SUB_EVENT_FROM_TAG(u64[0]); + __uint128_t vwqe_hdr = *(__uint128_t *)u64[1]; + + vwqe_hdr = ((vwqe_hdr >> 64) & 0xFFF) | BIT_ULL(31) | + ((vwqe_hdr & 0xFFFF) << 48) | ((uint64_t)port << 32); + *(uint64_t *)u64[1] = (uint64_t)vwqe_hdr; + cn10k_process_vwqe(u64[1], port, flags, ws->lookup_mem, + ws->tstamp, ws->lmt_base); + /* Mark vector mempool object as get */ + RTE_MEMPOOL_CHECK_COOKIES(rte_mempool_from_obj((void *)u64[1]), + (void **)&u64[1], 1, 1); + } +} + static __rte_always_inline uint16_t cn10k_sso_hws_get_work(struct cn10k_sso_hws *ws, struct rte_event *ev, - const uint32_t flags, void *lookup_mem) + const uint32_t flags) { union { __uint128_t get_work; uint64_t u64[2]; } gw; - uint64_t tstamp_ptr; gw.get_work = ws->gw_wdata; #if defined(RTE_ARCH_ARM64) && !defined(__clang__) @@ -222,83 +294,8 @@ cn10k_sso_hws_get_work(struct cn10k_sso_hws *ws, struct rte_event *ev, } while (gw.u64[0] & BIT_ULL(63)); #endif ws->gw_rdata = gw.u64[0]; - if (gw.u64[1]) { - gw.u64[0] = (gw.u64[0] & (0x3ull << 32)) << 6 | - (gw.u64[0] & (0x3FFull << 36)) << 4 | - (gw.u64[0] & 0xffffffff); - if ((flags & CPT_RX_WQE_F) && - (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_CRYPTODEV)) { - gw.u64[1] = cn10k_cpt_crypto_adapter_dequeue(gw.u64[1]); - } else if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - uint64_t mbuf; - - mbuf = gw.u64[1] - sizeof(struct rte_mbuf); - rte_prefetch0((void *)mbuf); - if (flags & NIX_RX_OFFLOAD_SECURITY_F) { - const uint64_t mbuf_init = 0x100010000ULL | - RTE_PKTMBUF_HEADROOM | - (flags & NIX_RX_OFFLOAD_TSTAMP_F ? 8 : 0); - struct rte_mbuf *m; - uintptr_t sa_base; - uint64_t iova = 0; - uint8_t loff = 0; - uint16_t d_off; - uint64_t cq_w1; - uint64_t cq_w5; - - m = (struct rte_mbuf *)mbuf; - d_off = (uintptr_t)(m->buf_addr) - (uintptr_t)m; - d_off += RTE_PKTMBUF_HEADROOM; - - cq_w1 = *(uint64_t *)(gw.u64[1] + 8); - cq_w5 = *(uint64_t *)(gw.u64[1] + 40); - - sa_base = - cnxk_nix_sa_base_get(port, lookup_mem); - sa_base &= ~(ROC_NIX_INL_SA_BASE_ALIGN - 1); - - mbuf = (uint64_t)nix_sec_meta_to_mbuf_sc( - cq_w1, cq_w5, sa_base, (uintptr_t)&iova, &loff, - (struct rte_mbuf *)mbuf, d_off, flags, - mbuf_init | ((uint64_t)port) << 48); - if (loff) - roc_npa_aura_op_free(m->pool->pool_id, - 0, iova); - } - - gw.u64[0] = CNXK_CLR_SUB_EVENT(gw.u64[0]); - cn10k_wqe_to_mbuf(gw.u64[1], mbuf, port, - gw.u64[0] & 0xFFFFF, flags, - lookup_mem); - /* Extracting tstamp, if PTP enabled*/ - tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *) - gw.u64[1]) + - CNXK_SSO_WQE_SG_PTR); - cn10k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, - ws->tstamp, - flags & NIX_RX_OFFLOAD_TSTAMP_F, - (uint64_t *)tstamp_ptr); - gw.u64[1] = mbuf; - } else if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV_VECTOR) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - __uint128_t vwqe_hdr = *(__uint128_t *)gw.u64[1]; - - vwqe_hdr = ((vwqe_hdr >> 64) & 0xFFF) | BIT_ULL(31) | - ((vwqe_hdr & 0xFFFF) << 48) | - ((uint64_t)port << 32); - *(uint64_t *)gw.u64[1] = (uint64_t)vwqe_hdr; - cn10k_process_vwqe(gw.u64[1], port, flags, lookup_mem, - ws->tstamp, ws->lmt_base); - /* Mark vector mempool object as get */ - RTE_MEMPOOL_CHECK_COOKIES( - rte_mempool_from_obj((void *)gw.u64[1]), - (void **)&gw.u64[1], 1, 1); - } - } + if (gw.u64[1]) + cn10k_sso_hws_post_process(ws, gw.u64, flags); ev->event = gw.u64[0]; ev->u64 = gw.u64[1]; @@ -308,13 +305,13 @@ cn10k_sso_hws_get_work(struct cn10k_sso_hws *ws, struct rte_event *ev, /* Used in cleaning up workslot. */ static __rte_always_inline uint16_t -cn10k_sso_hws_get_work_empty(struct cn10k_sso_hws *ws, struct rte_event *ev) +cn10k_sso_hws_get_work_empty(struct cn10k_sso_hws *ws, struct rte_event *ev, + const uint32_t flags) { union { __uint128_t get_work; uint64_t u64[2]; } gw; - uint64_t mbuf; #ifdef RTE_ARCH_ARM64 asm volatile(PLT_CPU_FEATURE_PREAMBLE @@ -325,9 +322,7 @@ cn10k_sso_hws_get_work_empty(struct cn10k_sso_hws *ws, struct rte_event *ev) " ldp %[tag], %[wqp], [%[tag_loc]] \n" " tbnz %[tag], 63, rty%= \n" "done%=: dmb ld \n" - " sub %[mbuf], %[wqp], #0x80 \n" - : [tag] "=&r"(gw.u64[0]), [wqp] "=&r"(gw.u64[1]), - [mbuf] "=&r"(mbuf) + : [tag] "=&r"(gw.u64[0]), [wqp] "=&r"(gw.u64[1]) : [tag_loc] "r"(ws->base + SSOW_LF_GWS_WQE0) : "memory"); #else @@ -335,24 +330,11 @@ cn10k_sso_hws_get_work_empty(struct cn10k_sso_hws *ws, struct rte_event *ev) roc_load_pair(gw.u64[0], gw.u64[1], ws->base + SSOW_LF_GWS_WQE0); } while (gw.u64[0] & BIT_ULL(63)); - mbuf = (uint64_t)((char *)gw.u64[1] - sizeof(struct rte_mbuf)); #endif - gw.u64[0] = (gw.u64[0] & (0x3ull << 32)) << 6 | - (gw.u64[0] & (0x3FFull << 36)) << 4 | - (gw.u64[0] & 0xffffffff); - - if (CNXK_TT_FROM_EVENT(gw.u64[0]) != SSO_TT_EMPTY) { - if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - - gw.u64[0] = CNXK_CLR_SUB_EVENT(gw.u64[0]); - cn10k_wqe_to_mbuf(gw.u64[1], mbuf, port, - gw.u64[0] & 0xFFFFF, 0, NULL); - gw.u64[1] = mbuf; - } - } + ws->gw_rdata = gw.u64[0]; + if (gw.u64[1]) + cn10k_sso_hws_post_process(ws, gw.u64, flags); ev->event = gw.u64[0]; ev->u64 = gw.u64[1]; @@ -471,7 +453,7 @@ NIX_RX_FASTPATH_MODES ws->base + SSOW_LF_GWS_WQE0); \ return 1; \ } \ - return cn10k_sso_hws_get_work(ws, ev, flags, ws->lookup_mem); \ + return cn10k_sso_hws_get_work(ws, ev, flags); \ } #define SSO_DEQ_SEG(fn, flags) SSO_DEQ(fn, flags | NIX_RX_MULTI_SEG_F) @@ -491,10 +473,9 @@ NIX_RX_FASTPATH_MODES ws->base + SSOW_LF_GWS_WQE0); \ return ret; \ } \ - ret = cn10k_sso_hws_get_work(ws, ev, flags, ws->lookup_mem); \ + ret = cn10k_sso_hws_get_work(ws, ev, flags); \ for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \ - ret = cn10k_sso_hws_get_work(ws, ev, flags, \ - ws->lookup_mem); \ + ret = cn10k_sso_hws_get_work(ws, ev, flags); \ return ret; \ } diff --git a/drivers/event/cnxk/cn9k_eventdev.c b/drivers/event/cnxk/cn9k_eventdev.c index 41bbe3cb22..987888d3db 100644 --- a/drivers/event/cnxk/cn9k_eventdev.c +++ b/drivers/event/cnxk/cn9k_eventdev.c @@ -122,6 +122,7 @@ cn9k_sso_hws_flush_events(void *hws, uint8_t queue_id, uintptr_t base, cnxk_handle_event_t fn, void *arg) { struct cnxk_sso_evdev *dev = cnxk_sso_pmd_priv(arg); + struct cnxk_timesync_info *tstamp; struct cn9k_sso_hws_dual *dws; struct cn9k_sso_hws *ws; uint64_t cq_ds_cnt = 1; @@ -130,6 +131,7 @@ cn9k_sso_hws_flush_events(void *hws, uint8_t queue_id, uintptr_t base, struct rte_event ev; uintptr_t ws_base; uint64_t val, req; + void *lookup_mem; plt_write64(0, base + SSO_LF_GGRP_QCTL); @@ -145,14 +147,19 @@ cn9k_sso_hws_flush_events(void *hws, uint8_t queue_id, uintptr_t base, if (dev->dual_ws) { dws = hws; ws_base = dws->base[0]; + lookup_mem = dws->lookup_mem; + tstamp = dws->tstamp; } else { ws = hws; ws_base = ws->base; + lookup_mem = ws->lookup_mem; + tstamp = ws->tstamp; } while (aq_cnt || cq_ds_cnt || ds_cnt) { plt_write64(req, ws_base + SSOW_LF_GWS_OP_GET_WORK0); - cn9k_sso_hws_get_work_empty(ws_base, &ev); + cn9k_sso_hws_get_work_empty(ws_base, &ev, dev->rx_offloads, + lookup_mem, tstamp); if (fn != NULL && ev.u64 != 0) fn(arg, ev); if (ev.sched_type != SSO_TT_EMPTY) diff --git a/drivers/event/cnxk/cn9k_worker.h b/drivers/event/cnxk/cn9k_worker.h index 88eb4e9cf9..64e97e321a 100644 --- a/drivers/event/cnxk/cn9k_worker.h +++ b/drivers/event/cnxk/cn9k_worker.h @@ -169,6 +169,34 @@ cn9k_wqe_to_mbuf(uint64_t wqe, const uint64_t mbuf, uint8_t port_id, mbuf_init | ((uint64_t)port_id) << 48, flags); } +static __rte_always_inline void +cn9k_sso_hws_post_process(uint64_t *u64, uint64_t mbuf, const uint32_t flags, + const void *const lookup_mem, + struct cnxk_timesync_info *tstamp) +{ + uint64_t tstamp_ptr; + + u64[0] = (u64[0] & (0x3ull << 32)) << 6 | + (u64[0] & (0x3FFull << 36)) << 4 | (u64[0] & 0xffffffff); + if ((flags & CPT_RX_WQE_F) && + (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_CRYPTODEV)) { + u64[1] = cn9k_cpt_crypto_adapter_dequeue(u64[1]); + } else if (CNXK_EVENT_TYPE_FROM_TAG(u64[0]) == RTE_EVENT_TYPE_ETHDEV) { + uint8_t port = CNXK_SUB_EVENT_FROM_TAG(u64[0]); + + u64[0] = CNXK_CLR_SUB_EVENT(u64[0]); + cn9k_wqe_to_mbuf(u64[1], mbuf, port, u64[0] & 0xFFFFF, flags, + lookup_mem); + /* Extracting tstamp, if PTP enabled*/ + tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *)u64[1]) + + CNXK_SSO_WQE_SG_PTR); + cn9k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, tstamp, + flags & NIX_RX_OFFLOAD_TSTAMP_F, + (uint64_t *)tstamp_ptr); + u64[1] = mbuf; + } +} + static __rte_always_inline uint16_t cn9k_sso_hws_dual_get_work(uint64_t base, uint64_t pair_base, struct rte_event *ev, const uint32_t flags, @@ -178,7 +206,6 @@ cn9k_sso_hws_dual_get_work(uint64_t base, uint64_t pair_base, __uint128_t get_work; uint64_t u64[2]; } gw; - uint64_t tstamp_ptr; uint64_t mbuf; if (flags & NIX_RX_OFFLOAD_PTYPE_F) @@ -207,34 +234,9 @@ cn9k_sso_hws_dual_get_work(uint64_t base, uint64_t pair_base, mbuf = (uint64_t)((char *)gw.u64[1] - sizeof(struct rte_mbuf)); #endif - gw.u64[0] = (gw.u64[0] & (0x3ull << 32)) << 6 | - (gw.u64[0] & (0x3FFull << 36)) << 4 | - (gw.u64[0] & 0xffffffff); - - if (CNXK_TT_FROM_EVENT(gw.u64[0]) != SSO_TT_EMPTY) { - if ((flags & CPT_RX_WQE_F) && - (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_CRYPTODEV)) { - gw.u64[1] = cn9k_cpt_crypto_adapter_dequeue(gw.u64[1]); - } else if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - - gw.u64[0] = CNXK_CLR_SUB_EVENT(gw.u64[0]); - cn9k_wqe_to_mbuf(gw.u64[1], mbuf, port, - gw.u64[0] & 0xFFFFF, flags, - dws->lookup_mem); - /* Extracting tstamp, if PTP enabled*/ - tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *) - gw.u64[1]) + - CNXK_SSO_WQE_SG_PTR); - cn9k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, - dws->tstamp, - flags & NIX_RX_OFFLOAD_TSTAMP_F, - (uint64_t *)tstamp_ptr); - gw.u64[1] = mbuf; - } - } + if (gw.u64[1]) + cn9k_sso_hws_post_process(gw.u64, mbuf, flags, dws->lookup_mem, + dws->tstamp); ev->event = gw.u64[0]; ev->u64 = gw.u64[1]; @@ -250,7 +252,6 @@ cn9k_sso_hws_get_work(struct cn9k_sso_hws *ws, struct rte_event *ev, __uint128_t get_work; uint64_t u64[2]; } gw; - uint64_t tstamp_ptr; uint64_t mbuf; plt_write64(ws->gw_wdata, ws->base + SSOW_LF_GWS_OP_GET_WORK0); @@ -283,34 +284,9 @@ cn9k_sso_hws_get_work(struct cn9k_sso_hws *ws, struct rte_event *ev, mbuf = (uint64_t)((char *)gw.u64[1] - sizeof(struct rte_mbuf)); #endif - gw.u64[0] = (gw.u64[0] & (0x3ull << 32)) << 6 | - (gw.u64[0] & (0x3FFull << 36)) << 4 | - (gw.u64[0] & 0xffffffff); - - if (CNXK_TT_FROM_EVENT(gw.u64[0]) != SSO_TT_EMPTY) { - if ((flags & CPT_RX_WQE_F) && - (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_CRYPTODEV)) { - gw.u64[1] = cn9k_cpt_crypto_adapter_dequeue(gw.u64[1]); - } else if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - - gw.u64[0] = CNXK_CLR_SUB_EVENT(gw.u64[0]); - cn9k_wqe_to_mbuf(gw.u64[1], mbuf, port, - gw.u64[0] & 0xFFFFF, flags, - lookup_mem); - /* Extracting tstamp, if PTP enabled*/ - tstamp_ptr = *(uint64_t *)(((struct nix_wqe_hdr_s *) - gw.u64[1]) + - CNXK_SSO_WQE_SG_PTR); - cn9k_nix_mbuf_to_tstamp((struct rte_mbuf *)mbuf, - ws->tstamp, - flags & NIX_RX_OFFLOAD_TSTAMP_F, - (uint64_t *)tstamp_ptr); - gw.u64[1] = mbuf; - } - } + if (gw.u64[1]) + cn9k_sso_hws_post_process(gw.u64, mbuf, flags, lookup_mem, + ws->tstamp); ev->event = gw.u64[0]; ev->u64 = gw.u64[1]; @@ -320,7 +296,9 @@ cn9k_sso_hws_get_work(struct cn9k_sso_hws *ws, struct rte_event *ev, /* Used in cleaning up workslot. */ static __rte_always_inline uint16_t -cn9k_sso_hws_get_work_empty(uint64_t base, struct rte_event *ev) +cn9k_sso_hws_get_work_empty(uint64_t base, struct rte_event *ev, + const uint32_t flags, void *lookup_mem, + struct cnxk_timesync_info *tstamp) { union { __uint128_t get_work; @@ -353,21 +331,9 @@ cn9k_sso_hws_get_work_empty(uint64_t base, struct rte_event *ev) mbuf = (uint64_t)((char *)gw.u64[1] - sizeof(struct rte_mbuf)); #endif - gw.u64[0] = (gw.u64[0] & (0x3ull << 32)) << 6 | - (gw.u64[0] & (0x3FFull << 36)) << 4 | - (gw.u64[0] & 0xffffffff); - - if (CNXK_TT_FROM_EVENT(gw.u64[0]) != SSO_TT_EMPTY) { - if (CNXK_EVENT_TYPE_FROM_TAG(gw.u64[0]) == - RTE_EVENT_TYPE_ETHDEV) { - uint8_t port = CNXK_SUB_EVENT_FROM_TAG(gw.u64[0]); - - gw.u64[0] = CNXK_CLR_SUB_EVENT(gw.u64[0]); - cn9k_wqe_to_mbuf(gw.u64[1], mbuf, port, - gw.u64[0] & 0xFFFFF, 0, NULL); - gw.u64[1] = mbuf; - } - } + if (gw.u64[1]) + cn9k_sso_hws_post_process(gw.u64, mbuf, flags, lookup_mem, + tstamp); ev->event = gw.u64[0]; ev->u64 = gw.u64[1]; -- 2.25.1