From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45B8DA034C; Wed, 23 Feb 2022 13:28:56 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14F73426ED; Wed, 23 Feb 2022 13:28:52 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 6A933426F5 for ; Wed, 23 Feb 2022 13:28:49 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 21NC87Br023579; Wed, 23 Feb 2022 04:28:49 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=kJrNJCTyjdutghhPPWNyUU0oYzVRwTzw7gFgMi8KXCk=; b=C9S2AsILTq8gGRWwfNGHl3H9YesgDZn1ETh4aKLzESi3/GJ25fS1gGsNxAdzM0kIiO+F Wovc5d3ukHLoOYUIYu0/KPQD/xyIAtB/9VuUGDP630xrwLpOK8T0SQM3nL70bMNuWUDZ o885n36QwaYdfOuVBxoS7Vw3+6UW2yd/OSTCx1GNcn0LvScPe2pRdGBQrheKYoVsBNVB 1A4nWt3vN5hYF/CLW6e0LSI43gPalrGofV5+ypSu2yuDmtjknO/TNYyNOGc/bWkt/AjI 3UX29c3MCkHkIfs3qtlKUfiBQAj713WqaCxbpm5C4feQ+WrLv58iN7HKG150uE55SMJv 6g== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3edjergmuj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 23 Feb 2022 04:28:48 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 23 Feb 2022 04:28:46 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Wed, 23 Feb 2022 04:28:46 -0800 Received: from localhost.localdomain (unknown [10.28.48.55]) by maili.marvell.com (Postfix) with ESMTP id AE4813F708A; Wed, 23 Feb 2022 04:28:42 -0800 (PST) From: Akhil Goyal To: CC: , , , , , , , Akhil Goyal Subject: [PATCH v3 2/2] net/cnxk: support IP reassembly Date: Wed, 23 Feb 2022 17:58:34 +0530 Message-ID: <20220223122834.2571860-3-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220223122834.2571860-1-gakhil@marvell.com> References: <20220120165310.4165567-1-gakhil@marvell.com> <20220223122834.2571860-1-gakhil@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: P0R0BzRbdPZbb_pQ7uc5fWTpjJeT0SAL X-Proofpoint-GUID: P0R0BzRbdPZbb_pQ7uc5fWTpjJeT0SAL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-23_05,2022-02-23_01,2022-02-23_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add capability and support for inbound reassembly in cnxk driver. Register the dynamic field for IPsec reassembly. Attach the fragments using the dynamic field in case of incomplete reassembly Signed-off-by: Vidya Sagar Velumuri Signed-off-by: Akhil Goyal Signed-off-by: Nithin Dabilpuram --- drivers/event/cnxk/cn10k_worker.h | 16 +- drivers/event/cnxk/deq/cn10k/deq_192_207.c | 12 + .../event/cnxk/deq/cn10k/deq_192_207_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_192_207_ca.c | 12 + .../cnxk/deq/cn10k/deq_192_207_ca_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_192_207_ca_seg.c | 12 + .../cnxk/deq/cn10k/deq_192_207_ca_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_192_207_ca_tmo.c | 12 + .../cnxk/deq/cn10k/deq_192_207_ca_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_192_207_ca_tmo_seg.c | 13 + .../deq/cn10k/deq_192_207_ca_tmo_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_192_207_dual.c | 12 + .../event/cnxk/deq/cn10k/deq_192_207_seg.c | 12 + .../cnxk/deq/cn10k/deq_192_207_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_192_207_tmo.c | 12 + .../cnxk/deq/cn10k/deq_192_207_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_192_207_tmo_seg.c | 12 + .../deq/cn10k/deq_192_207_tmo_seg_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_208_223.c | 12 + .../event/cnxk/deq/cn10k/deq_208_223_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_208_223_ca.c | 12 + .../cnxk/deq/cn10k/deq_208_223_ca_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_208_223_ca_seg.c | 12 + .../cnxk/deq/cn10k/deq_208_223_ca_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_208_223_ca_tmo.c | 12 + .../cnxk/deq/cn10k/deq_208_223_ca_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_208_223_ca_tmo_seg.c | 13 + .../deq/cn10k/deq_208_223_ca_tmo_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_208_223_dual.c | 12 + .../event/cnxk/deq/cn10k/deq_208_223_seg.c | 12 + .../cnxk/deq/cn10k/deq_208_223_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_208_223_tmo.c | 12 + .../cnxk/deq/cn10k/deq_208_223_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_208_223_tmo_seg.c | 12 + .../deq/cn10k/deq_208_223_tmo_seg_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_224_239.c | 12 + .../event/cnxk/deq/cn10k/deq_224_239_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_224_239_ca.c | 12 + .../cnxk/deq/cn10k/deq_224_239_ca_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_224_239_ca_seg.c | 12 + .../cnxk/deq/cn10k/deq_224_239_ca_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_224_239_ca_tmo.c | 12 + .../cnxk/deq/cn10k/deq_224_239_ca_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_224_239_ca_tmo_seg.c | 13 + .../deq/cn10k/deq_224_239_ca_tmo_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_224_239_dual.c | 12 + .../event/cnxk/deq/cn10k/deq_224_239_seg.c | 12 + .../cnxk/deq/cn10k/deq_224_239_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_224_239_tmo.c | 12 + .../cnxk/deq/cn10k/deq_224_239_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_224_239_tmo_seg.c | 12 + .../deq/cn10k/deq_224_239_tmo_seg_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_240_255.c | 12 + .../event/cnxk/deq/cn10k/deq_240_255_burst.c | 14 + drivers/event/cnxk/deq/cn10k/deq_240_255_ca.c | 12 + .../cnxk/deq/cn10k/deq_240_255_ca_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_240_255_ca_seg.c | 12 + .../cnxk/deq/cn10k/deq_240_255_ca_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_240_255_ca_tmo.c | 12 + .../cnxk/deq/cn10k/deq_240_255_ca_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_240_255_ca_tmo_seg.c | 13 + .../deq/cn10k/deq_240_255_ca_tmo_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_240_255_dual.c | 12 + .../event/cnxk/deq/cn10k/deq_240_255_seg.c | 12 + .../cnxk/deq/cn10k/deq_240_255_seg_burst.c | 14 + .../event/cnxk/deq/cn10k/deq_240_255_tmo.c | 12 + .../cnxk/deq/cn10k/deq_240_255_tmo_burst.c | 14 + .../cnxk/deq/cn10k/deq_240_255_tmo_seg.c | 12 + .../deq/cn10k/deq_240_255_tmo_seg_burst.c | 14 + drivers/event/cnxk/meson.build | 64 ++ drivers/net/cnxk/cn10k_ethdev.c | 51 ++ drivers/net/cnxk/cn10k_ethdev.h | 2 + drivers/net/cnxk/cn10k_ethdev_sec.c | 14 + drivers/net/cnxk/cn10k_rx.c | 35 +- drivers/net/cnxk/cn10k_rx.h | 672 ++++++++++++++++-- drivers/net/cnxk/cnxk_ethdev.h | 4 + drivers/net/cnxk/meson.build | 16 + drivers/net/cnxk/rx/cn10k/rx_192_207.c | 11 + drivers/net/cnxk/rx/cn10k/rx_192_207_mseg.c | 11 + drivers/net/cnxk/rx/cn10k/rx_192_207_vec.c | 11 + .../net/cnxk/rx/cn10k/rx_192_207_vec_mseg.c | 12 + drivers/net/cnxk/rx/cn10k/rx_208_223.c | 11 + drivers/net/cnxk/rx/cn10k/rx_208_223_mseg.c | 11 + drivers/net/cnxk/rx/cn10k/rx_208_223_vec.c | 11 + .../net/cnxk/rx/cn10k/rx_208_223_vec_mseg.c | 12 + drivers/net/cnxk/rx/cn10k/rx_224_239.c | 11 + drivers/net/cnxk/rx/cn10k/rx_224_239_mseg.c | 11 + drivers/net/cnxk/rx/cn10k/rx_224_239_vec.c | 11 + .../net/cnxk/rx/cn10k/rx_224_239_vec_mseg.c | 12 + drivers/net/cnxk/rx/cn10k/rx_240_255.c | 11 + drivers/net/cnxk/rx/cn10k/rx_240_255_mseg.c | 11 + drivers/net/cnxk/rx/cn10k/rx_240_255_vec.c | 11 + .../net/cnxk/rx/cn10k/rx_240_255_vec_mseg.c | 12 + 93 files changed, 1864 insertions(+), 74 deletions(-) create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_dual.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_dual.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_dual.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_dual.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_seg_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_tmo.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_burst.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg.c create mode 100644 drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg_burst.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_192_207.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_192_207_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_192_207_vec.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_192_207_vec_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_208_223.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_208_223_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_208_223_vec.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_208_223_vec_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_224_239.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_224_239_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_224_239_vec.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_224_239_vec_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_240_255.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_240_255_mseg.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_240_255_vec.c create mode 100644 drivers/net/cnxk/rx/cn10k/rx_240_255_vec_mseg.c diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h index 08a3953643..4c166c5193 100644 --- a/drivers/event/cnxk/cn10k_worker.h +++ b/drivers/event/cnxk/cn10k_worker.h @@ -169,9 +169,11 @@ cn10k_process_vwqe(uintptr_t vwqe, uint16_t port_id, const uint32_t flags, /* Translate meta to mbuf */ if (flags & NIX_RX_OFFLOAD_SECURITY_F) { const uint64_t cq_w1 = *((const uint64_t *)cqe + 1); + const uint64_t cq_w5 = *((const uint64_t *)cqe + 5); - mbuf = nix_sec_meta_to_mbuf_sc(cq_w1, sa_base, laddr, - &loff, mbuf, d_off); + mbuf = nix_sec_meta_to_mbuf_sc(cq_w1, cq_w5, sa_base, laddr, + &loff, mbuf, d_off, + flags, mbuf_init); } cn10k_nix_cqe_to_mbuf(cqe, cqe->tag, mbuf, lookup_mem, @@ -237,26 +239,32 @@ cn10k_sso_hws_get_work(struct cn10k_sso_hws *ws, struct rte_event *ev, mbuf = gw.u64[1] - sizeof(struct rte_mbuf); rte_prefetch0((void *)mbuf); if (flags & NIX_RX_OFFLOAD_SECURITY_F) { + const uint64_t mbuf_init = 0x100010000ULL | + RTE_PKTMBUF_HEADROOM | + (flags & NIX_RX_OFFLOAD_TSTAMP_F ? 8 : 0); struct rte_mbuf *m; uintptr_t sa_base; uint64_t iova = 0; uint8_t loff = 0; uint16_t d_off; uint64_t cq_w1; + uint64_t cq_w5; m = (struct rte_mbuf *)mbuf; d_off = (uintptr_t)(m->buf_addr) - (uintptr_t)m; d_off += RTE_PKTMBUF_HEADROOM; cq_w1 = *(uint64_t *)(gw.u64[1] + 8); + cq_w5 = *(uint64_t *)(gw.u64[1] + 40); sa_base = cnxk_nix_sa_base_get(port, lookup_mem); sa_base &= ~(ROC_NIX_INL_SA_BASE_ALIGN - 1); mbuf = (uint64_t)nix_sec_meta_to_mbuf_sc( - cq_w1, sa_base, (uintptr_t)&iova, &loff, - (struct rte_mbuf *)mbuf, d_off); + cq_w1, cq_w5, sa_base, (uintptr_t)&iova, &loff, + (struct rte_mbuf *)mbuf, d_off, flags, + mbuf_init | ((uint64_t)port) << 48); if (loff) roc_npa_aura_op_free(m->pool->pool_id, 0, iova); diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207.c b/drivers/event/cnxk/deq/cn10k/deq_192_207.c new file mode 100644 index 0000000000..6f53dbce2a --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ(cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_burst.c new file mode 100644 index 0000000000..f9789efbb2 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_burst_##name, \ + cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca.c new file mode 100644 index 0000000000..56ed0cae38 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA(cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_burst.c new file mode 100644 index 0000000000..70ed05e6e9 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_burst_##name, \ + cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg.c new file mode 100644 index 0000000000..806b9e4ef0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA_SEG(cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg_burst.c new file mode 100644 index 0000000000..93360814e1 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo.c new file mode 100644 index 0000000000..43c735d123 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_CA(cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_burst.c new file mode 100644 index 0000000000..5f7da5500d --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg.c new file mode 100644 index 0000000000..4c6126f13a --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_DEQ_TMO_CA_SEG(cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg_burst.c new file mode 100644 index 0000000000..81ae021b51 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_ca_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_dual.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_dual.c new file mode 100644 index 0000000000..2adcdaf59c --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_dual.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DUAL_DEQ(cn10k_sso_hws_dual_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_seg.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_seg.c new file mode 100644 index 0000000000..5d59f7fe23 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_SEG(cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_seg_burst.c new file mode 100644 index 0000000000..ddebe2aa40 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn10k_sso_hws_deq_seg_burst_##name, \ + cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo.c new file mode 100644 index 0000000000..e1c5beb41d --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO(cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_burst.c new file mode 100644 index 0000000000..5d733766ee --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_burst_##name, \ + cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg.c new file mode 100644 index 0000000000..12bff5902c --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_SEG(cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg_burst.c new file mode 100644 index 0000000000..2d10cc6e68 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_192_207_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223.c b/drivers/event/cnxk/deq/cn10k/deq_208_223.c new file mode 100644 index 0000000000..965a9405f0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ(cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_burst.c new file mode 100644 index 0000000000..de9a048a0e --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_burst_##name, \ + cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca.c new file mode 100644 index 0000000000..f9148acef3 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA(cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_burst.c new file mode 100644 index 0000000000..12a927a872 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_burst_##name, \ + cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg.c new file mode 100644 index 0000000000..1b020fbf98 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA_SEG(cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg_burst.c new file mode 100644 index 0000000000..2c826f2284 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo.c new file mode 100644 index 0000000000..48c23b910d --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_CA(cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_burst.c new file mode 100644 index 0000000000..5381f1ce17 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg.c new file mode 100644 index 0000000000..116d9efc2d --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_DEQ_TMO_CA_SEG(cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg_burst.c new file mode 100644 index 0000000000..86571c7027 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_ca_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_dual.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_dual.c new file mode 100644 index 0000000000..0fb35b9c83 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_dual.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DUAL_DEQ(cn10k_sso_hws_dual_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_seg.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_seg.c new file mode 100644 index 0000000000..df48426ff1 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_SEG(cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_seg_burst.c new file mode 100644 index 0000000000..f1342e3232 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn10k_sso_hws_deq_seg_burst_##name, \ + cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo.c new file mode 100644 index 0000000000..6f2d909806 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO(cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_burst.c new file mode 100644 index 0000000000..3c6f226c4b --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_burst_##name, \ + cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg.c new file mode 100644 index 0000000000..44a794233f --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_SEG(cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg_burst.c new file mode 100644 index 0000000000..aa89930d3e --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_208_223_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239.c b/drivers/event/cnxk/deq/cn10k/deq_224_239.c new file mode 100644 index 0000000000..03f5bd3588 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ(cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_burst.c new file mode 100644 index 0000000000..091419eb9b --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_burst_##name, \ + cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca.c new file mode 100644 index 0000000000..8f4693e057 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA(cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_burst.c new file mode 100644 index 0000000000..474c9b1eee --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_burst_##name, \ + cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg.c new file mode 100644 index 0000000000..478df24934 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA_SEG(cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg_burst.c new file mode 100644 index 0000000000..3b78a0f91e --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo.c new file mode 100644 index 0000000000..366ae27814 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_CA(cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_burst.c new file mode 100644 index 0000000000..db0751fece --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg.c new file mode 100644 index 0000000000..6dc3d2f870 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_DEQ_TMO_CA_SEG(cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg_burst.c new file mode 100644 index 0000000000..11b1784868 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_ca_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_dual.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_dual.c new file mode 100644 index 0000000000..35429014da --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_dual.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DUAL_DEQ(cn10k_sso_hws_dual_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_seg.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_seg.c new file mode 100644 index 0000000000..19b5d2eedd --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_SEG(cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_seg_burst.c new file mode 100644 index 0000000000..fe8f8099b0 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn10k_sso_hws_deq_seg_burst_##name, \ + cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo.c new file mode 100644 index 0000000000..4f196ddaaf --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO(cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_burst.c new file mode 100644 index 0000000000..252cf4f44d --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_burst_##name, \ + cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg.c new file mode 100644 index 0000000000..675149ec3a --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_SEG(cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg_burst.c new file mode 100644 index 0000000000..9347485e05 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_224_239_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255.c b/drivers/event/cnxk/deq/cn10k/deq_240_255.c new file mode 100644 index 0000000000..7890137548 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ(cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_burst.c new file mode 100644 index 0000000000..b1241915c8 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_burst_##name, \ + cn10k_sso_hws_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca.c new file mode 100644 index 0000000000..1f9abae4bc --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA(cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_burst.c new file mode 100644 index 0000000000..a518f5e285 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_burst_##name, \ + cn10k_sso_hws_deq_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg.c new file mode 100644 index 0000000000..f6408fbdd3 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_CA_SEG(cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg_burst.c new file mode 100644 index 0000000000..76bfa4ac08 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo.c new file mode 100644 index 0000000000..f28b909b66 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_CA(cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_burst.c new file mode 100644 index 0000000000..5c76f78dca --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg.c new file mode 100644 index 0000000000..35af75874b --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_DEQ_TMO_CA_SEG(cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg_burst.c new file mode 100644 index 0000000000..f8c6deb8ef --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_ca_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_ca_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_ca_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_dual.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_dual.c new file mode 100644 index 0000000000..0b227c2f99 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_dual.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DUAL_DEQ(cn10k_sso_hws_dual_deq_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_seg.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_seg.c new file mode 100644 index 0000000000..6af63491da --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_SEG(cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_seg_burst.c new file mode 100644 index 0000000000..6b20efd787 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_SEG_BURST(cn10k_sso_hws_deq_seg_burst_##name, \ + cn10k_sso_hws_deq_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo.c new file mode 100644 index 0000000000..c074e55d02 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO(cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_burst.c new file mode 100644 index 0000000000..88bf1541cb --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_burst_##name, \ + cn10k_sso_hws_deq_tmo_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg.c new file mode 100644 index 0000000000..560fc74f64 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) SSO_DEQ_TMO_SEG(cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg_burst.c b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg_burst.c new file mode 100644 index 0000000000..92017de4e2 --- /dev/null +++ b/drivers/event/cnxk/deq/cn10k/deq_240_255_tmo_seg_burst.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_worker.h" +#include "cnxk_eventdev.h" +#include "cnxk_worker.h" + +#define R(name, flags) \ + SSO_CMN_DEQ_BURST(cn10k_sso_hws_deq_tmo_seg_burst_##name, \ + cn10k_sso_hws_deq_tmo_seg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build index b27bae7b12..ae538f394d 100644 --- a/drivers/event/cnxk/meson.build +++ b/drivers/event/cnxk/meson.build @@ -330,6 +330,10 @@ sources += files( 'deq/cn10k/deq_80_95_burst.c', 'deq/cn10k/deq_96_111_burst.c', 'deq/cn10k/deq_112_127_burst.c', + 'deq/cn10k/deq_192_207_burst.c', + 'deq/cn10k/deq_208_223_burst.c', + 'deq/cn10k/deq_224_239_burst.c', + 'deq/cn10k/deq_240_255_burst.c', 'deq/cn10k/deq_0_15_seg_burst.c', 'deq/cn10k/deq_16_31_seg_burst.c', 'deq/cn10k/deq_32_47_seg_burst.c', @@ -338,6 +342,10 @@ sources += files( 'deq/cn10k/deq_80_95_seg_burst.c', 'deq/cn10k/deq_96_111_seg_burst.c', 'deq/cn10k/deq_112_127_seg_burst.c', + 'deq/cn10k/deq_192_207_seg_burst.c', + 'deq/cn10k/deq_208_223_seg_burst.c', + 'deq/cn10k/deq_224_239_seg_burst.c', + 'deq/cn10k/deq_240_255_seg_burst.c', 'deq/cn10k/deq_0_15.c', 'deq/cn10k/deq_16_31.c', 'deq/cn10k/deq_32_47.c', @@ -346,6 +354,10 @@ sources += files( 'deq/cn10k/deq_80_95.c', 'deq/cn10k/deq_96_111.c', 'deq/cn10k/deq_112_127.c', + 'deq/cn10k/deq_192_207.c', + 'deq/cn10k/deq_208_223.c', + 'deq/cn10k/deq_224_239.c', + 'deq/cn10k/deq_240_255.c', 'deq/cn10k/deq_0_15_seg.c', 'deq/cn10k/deq_16_31_seg.c', 'deq/cn10k/deq_32_47_seg.c', @@ -354,6 +366,10 @@ sources += files( 'deq/cn10k/deq_80_95_seg.c', 'deq/cn10k/deq_96_111_seg.c', 'deq/cn10k/deq_112_127_seg.c', + 'deq/cn10k/deq_192_207_seg.c', + 'deq/cn10k/deq_208_223_seg.c', + 'deq/cn10k/deq_224_239_seg.c', + 'deq/cn10k/deq_240_255_seg.c', 'deq/cn10k/deq_0_15_tmo.c', 'deq/cn10k/deq_16_31_tmo.c', 'deq/cn10k/deq_32_47_tmo.c', @@ -362,6 +378,10 @@ sources += files( 'deq/cn10k/deq_80_95_tmo.c', 'deq/cn10k/deq_96_111_tmo.c', 'deq/cn10k/deq_112_127_tmo.c', + 'deq/cn10k/deq_192_207_tmo.c', + 'deq/cn10k/deq_208_223_tmo.c', + 'deq/cn10k/deq_224_239_tmo.c', + 'deq/cn10k/deq_240_255_tmo.c', 'deq/cn10k/deq_0_15_tmo_burst.c', 'deq/cn10k/deq_16_31_tmo_burst.c', 'deq/cn10k/deq_32_47_tmo_burst.c', @@ -370,6 +390,10 @@ sources += files( 'deq/cn10k/deq_80_95_tmo_burst.c', 'deq/cn10k/deq_96_111_tmo_burst.c', 'deq/cn10k/deq_112_127_tmo_burst.c', + 'deq/cn10k/deq_192_207_tmo_burst.c', + 'deq/cn10k/deq_208_223_tmo_burst.c', + 'deq/cn10k/deq_224_239_tmo_burst.c', + 'deq/cn10k/deq_240_255_tmo_burst.c', 'deq/cn10k/deq_0_15_tmo_seg.c', 'deq/cn10k/deq_16_31_tmo_seg.c', 'deq/cn10k/deq_32_47_tmo_seg.c', @@ -378,6 +402,10 @@ sources += files( 'deq/cn10k/deq_80_95_tmo_seg.c', 'deq/cn10k/deq_96_111_tmo_seg.c', 'deq/cn10k/deq_112_127_tmo_seg.c', + 'deq/cn10k/deq_192_207_tmo_seg.c', + 'deq/cn10k/deq_208_223_tmo_seg.c', + 'deq/cn10k/deq_224_239_tmo_seg.c', + 'deq/cn10k/deq_240_255_tmo_seg.c', 'deq/cn10k/deq_0_15_tmo_seg_burst.c', 'deq/cn10k/deq_16_31_tmo_seg_burst.c', 'deq/cn10k/deq_32_47_tmo_seg_burst.c', @@ -386,6 +414,10 @@ sources += files( 'deq/cn10k/deq_80_95_tmo_seg_burst.c', 'deq/cn10k/deq_96_111_tmo_seg_burst.c', 'deq/cn10k/deq_112_127_tmo_seg_burst.c', + 'deq/cn10k/deq_192_207_tmo_seg_burst.c', + 'deq/cn10k/deq_208_223_tmo_seg_burst.c', + 'deq/cn10k/deq_224_239_tmo_seg_burst.c', + 'deq/cn10k/deq_240_255_tmo_seg_burst.c', 'deq/cn10k/deq_0_15_ca.c', 'deq/cn10k/deq_16_31_ca.c', 'deq/cn10k/deq_32_47_ca.c', @@ -394,6 +426,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca.c', 'deq/cn10k/deq_96_111_ca.c', 'deq/cn10k/deq_112_127_ca.c', + 'deq/cn10k/deq_192_207_ca.c', + 'deq/cn10k/deq_208_223_ca.c', + 'deq/cn10k/deq_224_239_ca.c', + 'deq/cn10k/deq_240_255_ca.c', 'deq/cn10k/deq_0_15_ca_burst.c', 'deq/cn10k/deq_16_31_ca_burst.c', 'deq/cn10k/deq_32_47_ca_burst.c', @@ -402,6 +438,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_burst.c', 'deq/cn10k/deq_96_111_ca_burst.c', 'deq/cn10k/deq_112_127_ca_burst.c', + 'deq/cn10k/deq_192_207_ca_burst.c', + 'deq/cn10k/deq_208_223_ca_burst.c', + 'deq/cn10k/deq_224_239_ca_burst.c', + 'deq/cn10k/deq_240_255_ca_burst.c', 'deq/cn10k/deq_0_15_ca_seg.c', 'deq/cn10k/deq_16_31_ca_seg.c', 'deq/cn10k/deq_32_47_ca_seg.c', @@ -410,6 +450,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_seg.c', 'deq/cn10k/deq_96_111_ca_seg.c', 'deq/cn10k/deq_112_127_ca_seg.c', + 'deq/cn10k/deq_192_207_ca_seg.c', + 'deq/cn10k/deq_208_223_ca_seg.c', + 'deq/cn10k/deq_224_239_ca_seg.c', + 'deq/cn10k/deq_240_255_ca_seg.c', 'deq/cn10k/deq_0_15_ca_seg_burst.c', 'deq/cn10k/deq_16_31_ca_seg_burst.c', 'deq/cn10k/deq_32_47_ca_seg_burst.c', @@ -418,6 +462,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_seg_burst.c', 'deq/cn10k/deq_96_111_ca_seg_burst.c', 'deq/cn10k/deq_112_127_ca_seg_burst.c', + 'deq/cn10k/deq_192_207_ca_seg_burst.c', + 'deq/cn10k/deq_208_223_ca_seg_burst.c', + 'deq/cn10k/deq_224_239_ca_seg_burst.c', + 'deq/cn10k/deq_240_255_ca_seg_burst.c', 'deq/cn10k/deq_0_15_ca_tmo.c', 'deq/cn10k/deq_16_31_ca_tmo.c', 'deq/cn10k/deq_32_47_ca_tmo.c', @@ -426,6 +474,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_tmo.c', 'deq/cn10k/deq_96_111_ca_tmo.c', 'deq/cn10k/deq_112_127_ca_tmo.c', + 'deq/cn10k/deq_192_207_ca_tmo.c', + 'deq/cn10k/deq_208_223_ca_tmo.c', + 'deq/cn10k/deq_224_239_ca_tmo.c', + 'deq/cn10k/deq_240_255_ca_tmo.c', 'deq/cn10k/deq_0_15_ca_tmo_burst.c', 'deq/cn10k/deq_16_31_ca_tmo_burst.c', 'deq/cn10k/deq_32_47_ca_tmo_burst.c', @@ -434,6 +486,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_tmo_burst.c', 'deq/cn10k/deq_96_111_ca_tmo_burst.c', 'deq/cn10k/deq_112_127_ca_tmo_burst.c', + 'deq/cn10k/deq_192_207_ca_tmo_burst.c', + 'deq/cn10k/deq_208_223_ca_tmo_burst.c', + 'deq/cn10k/deq_224_239_ca_tmo_burst.c', + 'deq/cn10k/deq_240_255_ca_tmo_burst.c', 'deq/cn10k/deq_0_15_ca_tmo_seg.c', 'deq/cn10k/deq_16_31_ca_tmo_seg.c', 'deq/cn10k/deq_32_47_ca_tmo_seg.c', @@ -442,6 +498,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_tmo_seg.c', 'deq/cn10k/deq_96_111_ca_tmo_seg.c', 'deq/cn10k/deq_112_127_ca_tmo_seg.c', + 'deq/cn10k/deq_192_207_ca_tmo_seg.c', + 'deq/cn10k/deq_208_223_ca_tmo_seg.c', + 'deq/cn10k/deq_224_239_ca_tmo_seg.c', + 'deq/cn10k/deq_240_255_ca_tmo_seg.c', 'deq/cn10k/deq_0_15_ca_tmo_seg_burst.c', 'deq/cn10k/deq_16_31_ca_tmo_seg_burst.c', 'deq/cn10k/deq_32_47_ca_tmo_seg_burst.c', @@ -450,6 +510,10 @@ sources += files( 'deq/cn10k/deq_80_95_ca_tmo_seg_burst.c', 'deq/cn10k/deq_96_111_ca_tmo_seg_burst.c', 'deq/cn10k/deq_112_127_ca_tmo_seg_burst.c', + 'deq/cn10k/deq_192_207_ca_tmo_seg_burst.c', + 'deq/cn10k/deq_208_223_ca_tmo_seg_burst.c', + 'deq/cn10k/deq_224_239_ca_tmo_seg_burst.c', + 'deq/cn10k/deq_240_255_ca_tmo_seg_burst.c', ) sources += files( diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index e47e04d922..218334eefc 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -304,6 +304,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + /* reset reassembly dynfield/flag offset */ + dev->reass_dynfield_off = -1; + dev->reass_dynflag_bit = -1; + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -478,6 +482,49 @@ cn10k_nix_rx_metadata_negotiate(struct rte_eth_dev *eth_dev, uint64_t *features) return 0; } +static int +cn10k_nix_reassembly_capability_get(struct rte_eth_dev *eth_dev, + struct rte_eth_ip_reassembly_params *reassembly_capa) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc = -ENOTSUP; + RTE_SET_USED(eth_dev); + + if (dev->rx_offloads & RTE_ETH_RX_OFFLOAD_SECURITY) { + reassembly_capa->timeout_ms = 60 * 1000; + reassembly_capa->max_frags = 4; + reassembly_capa->flags = RTE_ETH_DEV_REASSEMBLY_F_IPV4 | + RTE_ETH_DEV_REASSEMBLY_F_IPV6; + rc = 0; + } + + return rc; +} + +static int +cn10k_nix_reassembly_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_ip_reassembly_params *conf) +{ + RTE_SET_USED(eth_dev); + RTE_SET_USED(conf); + return -ENOTSUP; +} + +static int +cn10k_nix_reassembly_conf_set(struct rte_eth_dev *eth_dev, + const struct rte_eth_ip_reassembly_params *conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc = 0; + + rc = roc_nix_reassembly_configure(conf->timeout_ms, + conf->max_frags); + if (!rc && dev->rx_offloads & RTE_ETH_RX_OFFLOAD_SECURITY) + dev->rx_offload_flags |= NIX_RX_OFFLOAD_REASSEMBLY_F; + + return rc; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -499,6 +546,10 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.timesync_disable = cn10k_nix_timesync_disable; cnxk_eth_dev_ops.rx_metadata_negotiate = cn10k_nix_rx_metadata_negotiate; + cnxk_eth_dev_ops.ip_reassembly_capability_get = + cn10k_nix_reassembly_capability_get; + cnxk_eth_dev_ops.ip_reassembly_conf_get = cn10k_nix_reassembly_conf_get; + cnxk_eth_dev_ops.ip_reassembly_conf_set = cn10k_nix_reassembly_conf_set; } static void diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index fd7273078a..ab51eafcb1 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -44,6 +44,8 @@ struct cn10k_eth_rxq { /* Private data in sw rsvd area of struct roc_ot_ipsec_inb_sa */ struct cn10k_inb_priv_data { void *userdata; + int reass_dynfield_off; + int reass_dynflag_bit; struct cnxk_eth_sec_sess *eth_sec; }; diff --git a/drivers/net/cnxk/cn10k_ethdev_sec.c b/drivers/net/cnxk/cn10k_ethdev_sec.c index a514345034..7ae926c792 100644 --- a/drivers/net/cnxk/cn10k_ethdev_sec.c +++ b/drivers/net/cnxk/cn10k_ethdev_sec.c @@ -336,6 +336,14 @@ cn10k_eth_sec_session_create(void *device, if (rte_security_dynfield_register() < 0) return -ENOTSUP; + if (conf->ipsec.options.ip_reassembly_en && + dev->reass_dynfield_off < 0) { + if (rte_eth_ip_reassembly_dynfield_register( + &dev->reass_dynfield_off, + &dev->reass_dynflag_bit) < 0) + return -rte_errno; + } + ipsec = &conf->ipsec; crypto = conf->crypto_xform; inbound = !!(ipsec->direction == RTE_SECURITY_IPSEC_SA_DIR_INGRESS); @@ -442,6 +450,12 @@ cn10k_eth_sec_session_create(void *device, sizeof(struct roc_ot_ipsec_inb_sa)); if (rc) goto mempool_put; + + if (conf->ipsec.options.ip_reassembly_en) { + inb_priv->reass_dynfield_off = dev->reass_dynfield_off; + inb_priv->reass_dynflag_bit = dev->reass_dynflag_bit; + } + } else { struct roc_ot_ipsec_outb_sa *outb_sa, *outb_sa_dptr; struct cn10k_outb_priv_data *outb_priv; diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 5d603514c0..028abe8f39 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ +#define R(name, f7, f6, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); /* [VLAN] [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_REASSEMBLY_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_SECURITY_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] @@ -39,42 +40,42 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2][2] = { -#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2][2][2] = { +#define R(name, f7, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2][2] = { -#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2][2][2] = { +#define R(name, f7, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2][2] = { -#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2][2][2] = { +#define R(name, f7, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst_mseg[2][2][2][2][2][2][2] = { -#define R(name, f6, f5, f4, f3, f2, f1, f0, flags) \ - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_mseg_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst_mseg[2][2][2][2][2][2][2][2] = { +#define R(name, f7, f6, f5, f4, f3, f2, f1, f0, flags) \ + [f7][f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_mseg_##name, - NIX_RX_FASTPATH_MODES + NIX_RX_FASTPATH_MODES #undef R - }; + }; /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0][0][0]; if (dev->scalar_ena) { if (dev->rx_offloads & RTE_ETH_RX_OFFLOAD_SCATTER) diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 6697e391ac..48f993119f 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -17,7 +17,8 @@ #define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) #define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(5) #define NIX_RX_OFFLOAD_SECURITY_F BIT(6) -#define NIX_RX_OFFLOAD_MAX (NIX_RX_OFFLOAD_SECURITY_F << 1) +#define NIX_RX_OFFLOAD_REASSEMBLY_F BIT(7) +#define NIX_RX_OFFLOAD_MAX (NIX_RX_OFFLOAD_REASSEMBLY_F << 1) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -37,6 +38,17 @@ (uint64_t *)(((uintptr_t)((uint64_t *)(b))[i]) + (o)) : \ (uint64_t *)(((uintptr_t)(b)) + CQE_SZ(i) + (o))) +#define NIX_RX_SEC_REASSEMBLY_F \ + (NIX_RX_OFFLOAD_REASSEMBLY_F | NIX_RX_OFFLOAD_SECURITY_F) + +static inline rte_eth_ip_reassembly_dynfield_t * +cnxk_ip_reassembly_dynfield(struct rte_mbuf *mbuf, + int ip_reassembly_dynfield_offset) +{ + return RTE_MBUF_DYNFIELD(mbuf, ip_reassembly_dynfield_offset, + rte_eth_ip_reassembly_dynfield_t *); +} + union mbuf_initializer { struct { uint16_t data_off; @@ -104,19 +116,348 @@ nix_sec_flush_meta(uintptr_t laddr, uint16_t lmt_id, uint8_t loff, roc_lmt_submit_steorl(lmt_id, pa); } +static struct rte_mbuf * +nix_sec_attach_frags(const struct cpt_parse_hdr_s *hdr, + struct cn10k_inb_priv_data *inb_priv, + const uint64_t mbuf_init) +{ + struct rte_mbuf *head, *mbuf, *mbuf_prev; + uint32_t offset = hdr->w2.fi_offset; + union nix_rx_parse_u *frag_rx; + struct cpt_frag_info_s *finfo; + uint64_t *frag_ptr, ol_flags; + uint16_t frag_size; + uint16_t rlen; + uint64_t *wqe; + int off; + + off = inb_priv->reass_dynfield_off; + ol_flags = BIT_ULL(inb_priv->reass_dynflag_bit); + ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + + /* offset of 0 implies 256B, otherwise it implies offset*8B */ + offset = (((offset - 1) & 0x1f) + 1) * 8; + finfo = RTE_PTR_ADD(hdr, offset + hdr->w2.fi_pad); + + /* Frag-0: */ + wqe = (uint64_t *)(rte_be_to_cpu_64(hdr->wqe_ptr)); + rlen = ((*(wqe + 10)) >> 16) & 0xFFFF; + + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + frag_size = rlen + frag_rx->lcptr - frag_rx->laptr; + frag_rx->pkt_lenm1 = frag_size - 1; + + mbuf = (struct rte_mbuf *)((uintptr_t)wqe - sizeof(struct rte_mbuf)); + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init; + mbuf->data_len = frag_size; + mbuf->pkt_len = frag_size; + mbuf->ol_flags = ol_flags; + mbuf->next = NULL; + head = mbuf; + mbuf_prev = mbuf; + /* Update dynamic field with userdata */ + *rte_security_dynfield(mbuf) = (uint64_t)inb_priv->userdata; + + cnxk_ip_reassembly_dynfield(head, off)->nb_frags = hdr->w0.num_frags - 1; + cnxk_ip_reassembly_dynfield(head, off)->next_frag = NULL; + + /* Frag-1: */ + if (hdr->w0.num_frags > 1) { + wqe = (uint64_t *)(rte_be_to_cpu_64(hdr->frag1_wqe_ptr)); + rlen = ((*(wqe + 10)) >> 16) & 0xFFFF; + + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + frag_size = rlen + frag_rx->lcptr - frag_rx->laptr; + frag_rx->pkt_lenm1 = frag_size - 1; + + mbuf = (struct rte_mbuf *)((uintptr_t)wqe - + sizeof(struct rte_mbuf)); + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init; + mbuf->data_len = frag_size; + mbuf->pkt_len = frag_size; + mbuf->ol_flags = ol_flags; + mbuf->next = NULL; + + /* Update dynamic field with userdata */ + *rte_security_dynfield(mbuf) = (uint64_t)inb_priv->userdata; + + cnxk_ip_reassembly_dynfield(mbuf, off)->nb_frags = + hdr->w0.num_frags - 2; + cnxk_ip_reassembly_dynfield(mbuf, off)->next_frag = NULL; + cnxk_ip_reassembly_dynfield(mbuf_prev, off)->next_frag = mbuf; + mbuf_prev = mbuf; + } + + /* Frag-2: */ + if (hdr->w0.num_frags > 2) { + frag_ptr = (uint64_t *)(finfo + 1); + wqe = (uint64_t *)(rte_be_to_cpu_64(*frag_ptr)); + rlen = ((*(wqe + 10)) >> 16) & 0xFFFF; + + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + frag_size = rlen + frag_rx->lcptr - frag_rx->laptr; + frag_rx->pkt_lenm1 = frag_size - 1; + + mbuf = (struct rte_mbuf *)((uintptr_t)wqe - + sizeof(struct rte_mbuf)); + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init; + mbuf->data_len = frag_size; + mbuf->pkt_len = frag_size; + mbuf->ol_flags = ol_flags; + mbuf->next = NULL; + + /* Update dynamic field with userdata */ + *rte_security_dynfield(mbuf) = (uint64_t)inb_priv->userdata; + + cnxk_ip_reassembly_dynfield(mbuf, off)->nb_frags = + hdr->w0.num_frags - 3; + cnxk_ip_reassembly_dynfield(mbuf, off)->next_frag = NULL; + cnxk_ip_reassembly_dynfield(mbuf_prev, off)->next_frag = mbuf; + mbuf_prev = mbuf; + } + + /* Frag-3: */ + if (hdr->w0.num_frags > 3) { + wqe = (uint64_t *)(rte_be_to_cpu_64(*(frag_ptr + 1))); + rlen = ((*(wqe + 10)) >> 16) & 0xFFFF; + + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + frag_size = rlen + frag_rx->lcptr - frag_rx->laptr; + frag_rx->pkt_lenm1 = frag_size - 1; + + mbuf = (struct rte_mbuf *)((uintptr_t)wqe - + sizeof(struct rte_mbuf)); + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init; + mbuf->data_len = frag_size; + mbuf->pkt_len = frag_size; + mbuf->ol_flags = ol_flags; + mbuf->next = NULL; + + /* Update dynamic field with userdata */ + *rte_security_dynfield(mbuf) = (uint64_t)inb_priv->userdata; + + cnxk_ip_reassembly_dynfield(mbuf, off)->nb_frags = + hdr->w0.num_frags - 4; + cnxk_ip_reassembly_dynfield(mbuf, off)->next_frag = NULL; + cnxk_ip_reassembly_dynfield(mbuf_prev, off)->next_frag = mbuf; + } + return head; +} + +static struct rte_mbuf * +nix_sec_reassemble_frags(const struct cpt_parse_hdr_s *hdr, uint64_t cq_w1, + uint64_t cq_w5, uint64_t mbuf_init) +{ + uint32_t fragx_sum, pkt_hdr_len, l3_hdr_size; + uint32_t offset = hdr->w2.fi_offset; + union nix_rx_parse_u *inner_rx; + uint16_t rlen, data_off, b_off; + union nix_rx_parse_u *frag_rx; + struct cpt_frag_info_s *finfo; + struct rte_mbuf *head, *mbuf; + rte_iova_t *inner_iova; + uint64_t *frag_ptr; + uint16_t frag_size; + uint64_t *wqe; + + /* Base data offset */ + b_off = mbuf_init & 0xFFFFUL; + mbuf_init &= ~0xFFFFUL; + + /* offset of 0 implies 256B, otherwise it implies offset*8B */ + offset = (((offset - 1) & 0x1f) + 1) * 8; + finfo = RTE_PTR_ADD(hdr, offset + hdr->w2.fi_pad); + + /* Frag-0: */ + wqe = (uint64_t *)rte_be_to_cpu_64(hdr->wqe_ptr); + inner_rx = (union nix_rx_parse_u *)(wqe + 1); + inner_iova = (rte_iova_t *)*(wqe + 9); + + /* Update only the upper 28-bits from meta pkt parse info */ + *((uint64_t *)inner_rx) = ((*((uint64_t *)inner_rx) & ((1ULL << 36) - 1)) | + (cq_w1 & ~((1ULL << 36) - 1))); + + rlen = ((*(wqe + 10)) >> 16) & 0xFFFF; + frag_size = rlen + ((cq_w5 >> 16) & 0xFF) - (cq_w5 & 0xFF); + fragx_sum = rte_be_to_cpu_16(finfo->w1.frag_size0); + pkt_hdr_len = frag_size - fragx_sum; + + mbuf = (struct rte_mbuf *)((uintptr_t)wqe - sizeof(struct rte_mbuf)); + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | b_off; + mbuf->data_len = frag_size; + head = mbuf; + + if (inner_rx->lctype == NPC_LT_LC_IP) { + struct rte_ipv4_hdr *hdr = (struct rte_ipv4_hdr *) + RTE_PTR_ADD(inner_iova, inner_rx->lcptr); + + l3_hdr_size = (hdr->version_ihl & 0xf) << 2; + } else { + struct rte_ipv6_hdr *hdr = (struct rte_ipv6_hdr *) + RTE_PTR_ADD(inner_iova, inner_rx->lcptr); + size_t ext_len = sizeof(struct rte_ipv6_hdr); + uint8_t *nxt_hdr = (uint8_t *)hdr; + int nh = hdr->proto; + + l3_hdr_size = 0; + while (nh != -EINVAL) { + nxt_hdr += ext_len; + l3_hdr_size += ext_len; + nh = rte_ipv6_get_next_ext(nxt_hdr, nh, &ext_len); + } + } + + /* Frag-1: */ + wqe = (uint64_t *)(rte_be_to_cpu_64(hdr->frag1_wqe_ptr)); + frag_size = rte_be_to_cpu_16(finfo->w1.frag_size1); + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + + mbuf->next = (struct rte_mbuf *)((uintptr_t)wqe - sizeof(struct rte_mbuf)); + mbuf = mbuf->next; + data_off = b_off + frag_rx->lcptr + l3_hdr_size; + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + mbuf->data_len = frag_size; + fragx_sum += frag_size; + + /* Frag-2: */ + if (hdr->w0.num_frags > 2) { + frag_ptr = (uint64_t *)(finfo + 1); + wqe = (uint64_t *)(rte_be_to_cpu_64(*frag_ptr)); + frag_size = rte_be_to_cpu_16(finfo->w1.frag_size2); + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + + mbuf->next = (struct rte_mbuf *)((uintptr_t)wqe - sizeof(struct rte_mbuf)); + mbuf = mbuf->next; + data_off = b_off + frag_rx->lcptr + l3_hdr_size; + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + mbuf->data_len = frag_size; + fragx_sum += frag_size; + } + + /* Frag-3: */ + if (hdr->w0.num_frags > 3) { + wqe = (uint64_t *)(rte_be_to_cpu_64(*(frag_ptr + 1))); + frag_size = rte_be_to_cpu_16(finfo->w1.frag_size3); + frag_rx = (union nix_rx_parse_u *)(wqe + 1); + + mbuf->next = (struct rte_mbuf *)((uintptr_t)wqe - sizeof(struct rte_mbuf)); + mbuf = mbuf->next; + data_off = b_off + frag_rx->lcptr + l3_hdr_size; + *(uint64_t *)(&mbuf->rearm_data) = mbuf_init | data_off; + mbuf->data_len = frag_size; + fragx_sum += frag_size; + } + + if (inner_rx->lctype == NPC_LT_LC_IP) { + struct rte_ipv4_hdr *hdr = (struct rte_ipv4_hdr *) + RTE_PTR_ADD(inner_iova, inner_rx->lcptr); + + hdr->fragment_offset = 0; + hdr->total_length = rte_cpu_to_be_16(fragx_sum + l3_hdr_size); + hdr->hdr_checksum = 0; + hdr->hdr_checksum = rte_ipv4_cksum(hdr); + + inner_rx->pkt_lenm1 = pkt_hdr_len + fragx_sum - 1; + } else { + /* Remove the frag header by moving header 8 bytes forward */ + struct rte_ipv6_hdr *hdr = (struct rte_ipv6_hdr *) + RTE_PTR_ADD(inner_iova, inner_rx->lcptr); + + hdr->payload_len = rte_cpu_to_be_16(fragx_sum + l3_hdr_size - + 8 - sizeof(struct rte_ipv6_hdr)); + + rte_memcpy(rte_pktmbuf_mtod_offset(head, void *, 8), + rte_pktmbuf_mtod(head, void *), + inner_rx->lcptr + sizeof(struct rte_ipv6_hdr)); + + inner_rx->pkt_lenm1 = pkt_hdr_len + fragx_sum - 8 - 1; + head->data_len -= 8; + head->data_off += 8; + } + mbuf->next = NULL; + head->pkt_len = inner_rx->pkt_lenm1 + 1; + head->nb_segs = hdr->w0.num_frags; + + return head; +} + static __rte_always_inline struct rte_mbuf * -nix_sec_meta_to_mbuf_sc(uint64_t cq_w1, const uint64_t sa_base, uintptr_t laddr, - uint8_t *loff, struct rte_mbuf *mbuf, uint16_t data_off) +nix_sec_meta_to_mbuf_sc(uint64_t cq_w1, uint64_t cq_w5, const uint64_t sa_base, + uintptr_t laddr, uint8_t *loff, struct rte_mbuf *mbuf, + uint16_t data_off, const uint16_t flags, + const uint64_t mbuf_init) { const void *__p = (void *)((uintptr_t)mbuf + (uint16_t)data_off); const struct cpt_parse_hdr_s *hdr = (const struct cpt_parse_hdr_s *)__p; struct cn10k_inb_priv_data *inb_priv; - struct rte_mbuf *inner; + struct rte_mbuf *inner = NULL; + uint64_t res_w1; uint32_t sa_idx; + uint16_t uc_cc; + uint32_t len; void *inb_sa; uint64_t w0; - if (cq_w1 & BIT(11)) { + if ((flags & NIX_RX_OFFLOAD_REASSEMBLY_F) && (cq_w1 & BIT(11))) { + /* Get SPI from CPT_PARSE_S's cookie(already swapped) */ + w0 = hdr->w0.u64; + sa_idx = w0 >> 32; + + inb_sa = roc_nix_inl_ot_ipsec_inb_sa(sa_base, sa_idx); + inb_priv = roc_nix_inl_ot_ipsec_inb_sa_sw_rsvd(inb_sa); + + if (!hdr->w0.num_frags) { + /* No Reassembly or inbound error */ + inner = (struct rte_mbuf *) + (rte_be_to_cpu_64(hdr->wqe_ptr) - + sizeof(struct rte_mbuf)); + + /* Update dynamic field with userdata */ + *rte_security_dynfield(inner) = + (uint64_t)inb_priv->userdata; + + /* CPT result(struct cpt_cn10k_res_s) is at + * after first IOVA in meta + */ + res_w1 = *((uint64_t *)(&inner[1]) + 10); + uc_cc = res_w1 & 0xFF; + + /* Calculate inner packet length */ + len = ((res_w1 >> 16) & 0xFFFF) + hdr->w2.il3_off - + sizeof(struct cpt_parse_hdr_s) - (w0 & 0x7); + inner->pkt_len = len; + inner->data_len = len; + *(uint64_t *)(&inner->rearm_data) = mbuf_init; + + inner->ol_flags = ((uc_cc == CPT_COMP_WARN) ? + RTE_MBUF_F_RX_SEC_OFFLOAD : + (RTE_MBUF_F_RX_SEC_OFFLOAD | + RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)); + inner->next = NULL; + } else if (!(hdr->w0.err_sum) && !(hdr->w0.reas_sts)) { + /* Reassembly success */ + inner = nix_sec_reassemble_frags(hdr, cq_w1, cq_w5, + mbuf_init); + + /* Update dynamic field with userdata */ + *rte_security_dynfield(inner) = + (uint64_t)inb_priv->userdata; + + /* Assume success */ + inner->ol_flags = RTE_MBUF_F_RX_SEC_OFFLOAD; + } else { + /* Reassembly failure */ + inner = nix_sec_attach_frags(hdr, inb_priv, mbuf_init); + } + + /* Store meta in lmtline to free + * Assume all meta's from same aura. + */ + *(uint64_t *)(laddr + (*loff << 3)) = (uint64_t)mbuf; + *loff = *loff + 1; + + return inner; + } else if (cq_w1 & BIT(11)) { inner = (struct rte_mbuf *)(rte_be_to_cpu_64(hdr->wqe_ptr) - sizeof(struct rte_mbuf)); @@ -131,8 +472,25 @@ nix_sec_meta_to_mbuf_sc(uint64_t cq_w1, const uint64_t sa_base, uintptr_t laddr, *rte_security_dynfield(inner) = (uint64_t)inb_priv->userdata; /* Update l2 hdr length first */ - inner->pkt_len = (hdr->w2.il3_off - - sizeof(struct cpt_parse_hdr_s) - (w0 & 0x7)); + + /* CPT result(struct cpt_cn10k_res_s) is at + * after first IOVA in meta + */ + res_w1 = *((uint64_t *)(&inner[1]) + 10); + uc_cc = res_w1 & 0xFF; + + /* Calculate inner packet length */ + len = ((res_w1 >> 16) & 0xFFFF) + hdr->w2.il3_off - + sizeof(struct cpt_parse_hdr_s) - (w0 & 0x7); + inner->pkt_len = len; + inner->data_len = len; + *(uint64_t *)(&inner->rearm_data) = mbuf_init; + + inner->ol_flags = ((uc_cc == CPT_COMP_WARN) ? + RTE_MBUF_F_RX_SEC_OFFLOAD : + (RTE_MBUF_F_RX_SEC_OFFLOAD | + RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)); + inner->next = NULL; /* Store meta in lmtline to free * Assume all meta's from same aura. @@ -148,18 +506,23 @@ nix_sec_meta_to_mbuf_sc(uint64_t cq_w1, const uint64_t sa_base, uintptr_t laddr, return inner; } + + mbuf->next = NULL; return mbuf; } #if defined(RTE_ARCH_ARM64) static __rte_always_inline struct rte_mbuf * -nix_sec_meta_to_mbuf(uint64_t cq_w1, uintptr_t sa_base, uintptr_t laddr, - uint8_t *loff, struct rte_mbuf *mbuf, uint16_t data_off, - uint8x16_t *rx_desc_field1, uint64_t *ol_flags) +nix_sec_meta_to_mbuf(uint64_t cq_w1, uint64_t cq_w5, uintptr_t sa_base, + uintptr_t laddr, uint8_t *loff, struct rte_mbuf *mbuf, + uint16_t data_off, uint8x16_t *rx_desc_field1, + uint64_t *ol_flags, const uint16_t flags, + uint64x2_t *rearm) { const void *__p = (void *)((uintptr_t)mbuf + (uint16_t)data_off); const struct cpt_parse_hdr_s *hdr = (const struct cpt_parse_hdr_s *)__p; + uint64_t mbuf_init = vgetq_lane_u64(*rearm, 0); struct cn10k_inb_priv_data *inb_priv; struct rte_mbuf *inner; uint64_t *sg, res_w1; @@ -168,7 +531,102 @@ nix_sec_meta_to_mbuf(uint64_t cq_w1, uintptr_t sa_base, uintptr_t laddr, uint16_t len; uint64_t w0; - if (cq_w1 & BIT(11)) { + if ((flags & NIX_RX_OFFLOAD_REASSEMBLY_F) && (cq_w1 & BIT(11))) { + w0 = hdr->w0.u64; + sa_idx = w0 >> 32; + + /* Get SPI from CPT_PARSE_S's cookie(already swapped) */ + w0 = hdr->w0.u64; + sa_idx = w0 >> 32; + + inb_sa = roc_nix_inl_ot_ipsec_inb_sa(sa_base, sa_idx); + inb_priv = roc_nix_inl_ot_ipsec_inb_sa_sw_rsvd(inb_sa); + + /* Clear checksum flags */ + *ol_flags &= ~(RTE_MBUF_F_RX_L4_CKSUM_MASK | + RTE_MBUF_F_RX_IP_CKSUM_MASK); + + if (!hdr->w0.num_frags) { + /* No Reassembly or inbound error */ + inner = (struct rte_mbuf *) + (rte_be_to_cpu_64(hdr->wqe_ptr) - + sizeof(struct rte_mbuf)); + /* Update dynamic field with userdata */ + *rte_security_dynfield(inner) = + (uint64_t)inb_priv->userdata; + + /* CPT result(struct cpt_cn10k_res_s) is at + * after first IOVA in meta + */ + sg = (uint64_t *)(inner + 1); + res_w1 = sg[10]; + + /* Clear checksum flags and update security flag + */ + *ol_flags &= ~(RTE_MBUF_F_RX_L4_CKSUM_MASK | + RTE_MBUF_F_RX_IP_CKSUM_MASK); + *ol_flags |= + (((res_w1 & 0xFF) == CPT_COMP_WARN) ? + RTE_MBUF_F_RX_SEC_OFFLOAD : + (RTE_MBUF_F_RX_SEC_OFFLOAD | + RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)); + /* Calculate inner packet length */ + len = ((res_w1 >> 16) & 0xFFFF) + + hdr->w2.il3_off - + sizeof(struct cpt_parse_hdr_s) - + (w0 & 0x7); + /* Update pkt_len and data_len */ + *rx_desc_field1 = + vsetq_lane_u16(len, *rx_desc_field1, 2); + *rx_desc_field1 = + vsetq_lane_u16(len, *rx_desc_field1, 4); + + inner->next = NULL; + } else if (!(hdr->w0.err_sum) && !(hdr->w0.reas_sts)) { + /* Reassembly success */ + inner = nix_sec_reassemble_frags(hdr, cq_w1, cq_w5, + mbuf_init); + sg = (uint64_t *)(inner + 1); + res_w1 = sg[10]; + + /* Update dynamic field with userdata */ + *rte_security_dynfield(inner) = + (uint64_t)inb_priv->userdata; + + /* Assume success */ + *ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + + /* Update pkt_len and data_len */ + *rx_desc_field1 = vsetq_lane_u16(inner->pkt_len, + *rx_desc_field1, 2); + *rx_desc_field1 = vsetq_lane_u16(inner->data_len, + *rx_desc_field1, 4); + + /* Data offset might be updated */ + mbuf_init = *(uint64_t *)(&inner->rearm_data); + *rearm = vsetq_lane_u64(mbuf_init, *rearm, 0); + } else { + /* Reassembly failure */ + inner = nix_sec_attach_frags(hdr, inb_priv, mbuf_init); + *ol_flags |= inner->ol_flags; + + /* Update pkt_len and data_len */ + *rx_desc_field1 = vsetq_lane_u16(inner->pkt_len, + *rx_desc_field1, 2); + *rx_desc_field1 = vsetq_lane_u16(inner->data_len, + *rx_desc_field1, 4); + } + + /* Store meta in lmtline to free + * Assume all meta's from same aura. + */ + *(uint64_t *)(laddr + (*loff << 3)) = (uint64_t)mbuf; + *loff = *loff + 1; + + /* Return inner mbuf */ + return inner; + + } else if (cq_w1 & BIT(11)) { inner = (struct rte_mbuf *)(rte_be_to_cpu_64(hdr->wqe_ptr) - sizeof(struct rte_mbuf)); /* Get SPI from CPT_PARSE_S's cookie(already swapped) */ @@ -211,10 +669,12 @@ nix_sec_meta_to_mbuf(uint64_t cq_w1, uintptr_t sa_base, uintptr_t laddr, /* Mark inner mbuf as get */ RTE_MEMPOOL_CHECK_COOKIES(inner->pool, (void **)&inner, 1, 1); + inner->next = NULL; /* Return inner mbuf */ return inner; } + mbuf->next = NULL; /* Return same mbuf as it is not a decrypted pkt */ return mbuf; } @@ -283,7 +743,7 @@ nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, sg = *(const uint64_t *)(rx + 1); nb_segs = (sg >> 48) & 0x3; - if (nb_segs == 1) { + if (nb_segs == 1 && !(flags & NIX_RX_SEC_REASSEMBLY_F)) { mbuf->next = NULL; return; } @@ -346,30 +806,10 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, ol_flags |= RTE_MBUF_F_RX_RSS_HASH; } - /* Process Security packets */ - if (flag & NIX_RX_OFFLOAD_SECURITY_F) { - if (w1 & BIT(11)) { - /* CPT result(struct cpt_cn10k_res_s) is at - * after first IOVA in meta - */ - const uint64_t *sg = (const uint64_t *)(mbuf + 1); - const uint64_t res_w1 = sg[10]; - const uint16_t uc_cc = res_w1 & 0xFF; - - /* Rlen */ - len = ((res_w1 >> 16) & 0xFFFF) + mbuf->pkt_len; - ol_flags |= ((uc_cc == CPT_COMP_WARN) ? - RTE_MBUF_F_RX_SEC_OFFLOAD : - (RTE_MBUF_F_RX_SEC_OFFLOAD | - RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)); - } else { - if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) - ol_flags |= nix_rx_olflags_get(lookup_mem, w1); - } - } else { - if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) - ol_flags |= nix_rx_olflags_get(lookup_mem, w1); - } + /* Skip rx ol flags extraction for Security packets */ + if ((!(flag & NIX_RX_SEC_REASSEMBLY_F) || !(w1 & BIT(11))) && + flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); if (flag & NIX_RX_OFFLOAD_VLAN_STRIP_F) { if (rx->vtag0_gone) { @@ -385,14 +825,19 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); - mbuf->ol_flags = ol_flags; - mbuf->pkt_len = len; - mbuf->data_len = len; - *(uint64_t *)(&mbuf->rearm_data) = val; + /* Packet data length and ol flags is already updated for sec */ + if (flag & NIX_RX_SEC_REASSEMBLY_F && w1 & BIT_ULL(11)) { + mbuf->ol_flags |= ol_flags; + } else { + mbuf->ol_flags = ol_flags; + mbuf->pkt_len = len; + mbuf->data_len = len; + *(uint64_t *)(&mbuf->rearm_data) = val; + } if (flag & NIX_RX_MULTI_SEG_F) nix_cqe_xtract_mseg(rx, mbuf, val, flag); - else + else if (!(flag & NIX_RX_SEC_REASSEMBLY_F)) mbuf->next = NULL; } @@ -473,9 +918,11 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, /* Translate meta to mbuf */ if (flags & NIX_RX_OFFLOAD_SECURITY_F) { const uint64_t cq_w1 = *((const uint64_t *)cq + 1); + const uint64_t cq_w5 = *((const uint64_t *)cq + 5); - mbuf = nix_sec_meta_to_mbuf_sc(cq_w1, sa_base, laddr, - &loff, mbuf, data_off); + mbuf = nix_sec_meta_to_mbuf_sc(cq_w1, cq_w5, sa_base, laddr, + &loff, mbuf, data_off, + flags, mbuf_init); } cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, @@ -747,25 +1194,40 @@ cn10k_nix_recv_pkts_vector(void *args, struct rte_mbuf **mbufs, uint16_t pkts, /* Translate meta to mbuf */ if (flags & NIX_RX_OFFLOAD_SECURITY_F) { + uint64_t cq0_w5 = *(uint64_t *)(cq0 + CQE_SZ(0) + 40); + uint64_t cq1_w5 = *(uint64_t *)(cq0 + CQE_SZ(1) + 40); + uint64_t cq2_w5 = *(uint64_t *)(cq0 + CQE_SZ(2) + 40); + uint64_t cq3_w5 = *(uint64_t *)(cq0 + CQE_SZ(3) + 40); + + /* Initialize rearm data when reassembly is enabled as + * data offset might change. + */ + if (flags & NIX_RX_OFFLOAD_REASSEMBLY_F) { + rearm0 = vdupq_n_u64(mbuf_initializer); + rearm1 = vdupq_n_u64(mbuf_initializer); + rearm2 = vdupq_n_u64(mbuf_initializer); + rearm3 = vdupq_n_u64(mbuf_initializer); + } + /* Checksum ol_flags will be cleared if mbuf is meta */ - mbuf0 = nix_sec_meta_to_mbuf(cq0_w1, sa_base, laddr, + mbuf0 = nix_sec_meta_to_mbuf(cq0_w1, cq0_w5, sa_base, laddr, &loff, mbuf0, d_off, &f0, - &ol_flags0); + &ol_flags0, flags, &rearm0); mbuf01 = vsetq_lane_u64((uint64_t)mbuf0, mbuf01, 0); - mbuf1 = nix_sec_meta_to_mbuf(cq1_w1, sa_base, laddr, + mbuf1 = nix_sec_meta_to_mbuf(cq1_w1, cq1_w5, sa_base, laddr, &loff, mbuf1, d_off, &f1, - &ol_flags1); + &ol_flags1, flags, &rearm1); mbuf01 = vsetq_lane_u64((uint64_t)mbuf1, mbuf01, 1); - mbuf2 = nix_sec_meta_to_mbuf(cq2_w1, sa_base, laddr, + mbuf2 = nix_sec_meta_to_mbuf(cq2_w1, cq2_w5, sa_base, laddr, &loff, mbuf2, d_off, &f2, - &ol_flags2); + &ol_flags2, flags, &rearm2); mbuf23 = vsetq_lane_u64((uint64_t)mbuf2, mbuf23, 0); - mbuf3 = nix_sec_meta_to_mbuf(cq3_w1, sa_base, laddr, + mbuf3 = nix_sec_meta_to_mbuf(cq3_w1, cq3_w5, sa_base, laddr, &loff, mbuf3, d_off, &f3, - &ol_flags3); + &ol_flags3, flags, &rearm3); mbuf23 = vsetq_lane_u64((uint64_t)mbuf3, mbuf23, 1); } @@ -927,7 +1389,7 @@ cn10k_nix_recv_pkts_vector(void *args, struct rte_mbuf **mbufs, uint16_t pkts, nix_cqe_xtract_mseg((union nix_rx_parse_u *) (CQE_PTR_OFF(cq0, 3, 8, flags)), mbuf3, mbuf_initializer, flags); - } else { + } else if (!(flags & NIX_RX_SEC_REASSEMBLY_F)) { /* Update that no more segments */ mbuf0->next = NULL; mbuf1->next = NULL; @@ -1060,6 +1522,7 @@ cn10k_nix_recv_pkts_vector(void *args, struct rte_mbuf **mbufs, uint16_t pkts, #define TS_F NIX_RX_OFFLOAD_TSTAMP_F #define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F #define R_SEC_F NIX_RX_OFFLOAD_SECURITY_F +#define R_REAS_F NIX_RX_OFFLOAD_REASSEMBLY_F /* [R_SEC_F] [RX_VLAN_F] [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES_0_15 \ @@ -1226,6 +1689,101 @@ cn10k_nix_recv_pkts_vector(void *args, struct rte_mbuf **mbufs, uint16_t pkts, R(sec_vlan_ts_mark_cksum_ptype_rss, \ R_SEC_F | RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* R_REAS_F cannot work without R_SEC_F */ +#define NIX_RX_FASTPATH_MODES_128_143 +#define NIX_RX_FASTPATH_MODES_144_159 +#define NIX_RX_FASTPATH_MODES_160_175 +#define NIX_RX_FASTPATH_MODES_176_191 + +#define NIX_RX_FASTPATH_MODES_192_207 \ + R(reas_sec, R_REAS_F | R_SEC_F) \ + R(reas_sec_rss, R_REAS_F | R_SEC_F | RSS_F) \ + R(reas_sec_ptype, R_REAS_F | R_SEC_F | PTYPE_F) \ + R(reas_sec_ptype_rss, R_REAS_F | R_SEC_F | PTYPE_F | RSS_F) \ + R(reas_sec_cksum, R_REAS_F | R_SEC_F | CKSUM_F) \ + R(reas_sec_cksum_rss, R_REAS_F | R_SEC_F | CKSUM_F | RSS_F) \ + R(reas_sec_cksum_ptype, R_REAS_F | R_SEC_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_cksum_ptype_rss, R_REAS_F | R_SEC_F | CKSUM_F | PTYPE_F | RSS_F) \ + R(reas_sec_mark, R_REAS_F | R_SEC_F | MARK_F) \ + R(reas_sec_mark_rss, R_REAS_F | R_SEC_F | MARK_F | RSS_F) \ + R(reas_sec_mark_ptype, R_REAS_F | R_SEC_F | MARK_F | PTYPE_F) \ + R(reas_sec_mark_ptype_rss, R_REAS_F | R_SEC_F | MARK_F | PTYPE_F | RSS_F) \ + R(reas_sec_mark_cksum, R_REAS_F | R_SEC_F | MARK_F | CKSUM_F) \ + R(reas_sec_mark_cksum_rss, R_REAS_F | R_SEC_F | MARK_F | CKSUM_F | RSS_F) \ + R(reas_sec_mark_cksum_ptype, R_REAS_F | R_SEC_F | MARK_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_mark_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define NIX_RX_FASTPATH_MODES_208_223 \ + R(reas_sec_ts, R_REAS_F | R_SEC_F | TS_F) \ + R(reas_sec_ts_rss, R_REAS_F | R_SEC_F | TS_F | RSS_F) \ + R(reas_sec_ts_ptype, R_REAS_F | R_SEC_F | TS_F | PTYPE_F) \ + R(reas_sec_ts_ptype_rss, R_REAS_F | R_SEC_F | TS_F | PTYPE_F | RSS_F) \ + R(reas_sec_ts_cksum, R_REAS_F | R_SEC_F | TS_F | CKSUM_F) \ + R(reas_sec_ts_cksum_rss, R_REAS_F | R_SEC_F | TS_F | CKSUM_F | RSS_F) \ + R(reas_sec_ts_cksum_ptype, R_REAS_F | R_SEC_F | TS_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_ts_cksum_ptype_rss, R_REAS_F | R_SEC_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ + R(reas_sec_ts_mark, R_REAS_F | R_SEC_F | TS_F | MARK_F) \ + R(reas_sec_ts_mark_rss, R_REAS_F | R_SEC_F | TS_F | MARK_F | RSS_F) \ + R(reas_sec_ts_mark_ptype, R_REAS_F | R_SEC_F | TS_F | MARK_F | PTYPE_F) \ + R(reas_sec_ts_mark_ptype_rss, R_REAS_F | R_SEC_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ + R(reas_sec_ts_mark_cksum, R_REAS_F | R_SEC_F | TS_F | MARK_F | CKSUM_F) \ + R(reas_sec_ts_mark_cksum_rss, R_REAS_F | R_SEC_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ + R(reas_sec_ts_mark_cksum_ptype, \ + R_REAS_F | R_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_ts_mark_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define NIX_RX_FASTPATH_MODES_224_239 \ + R(reas_sec_vlan, R_REAS_F | R_SEC_F | RX_VLAN_F) \ + R(reas_sec_vlan_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | RSS_F) \ + R(reas_sec_vlan_ptype, R_REAS_F | R_SEC_F | RX_VLAN_F | PTYPE_F) \ + R(reas_sec_vlan_ptype_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_cksum, R_REAS_F | R_SEC_F | RX_VLAN_F | CKSUM_F) \ + R(reas_sec_vlan_cksum_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | CKSUM_F | RSS_F) \ + R(reas_sec_vlan_cksum_ptype, R_REAS_F | R_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_vlan_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_mark, R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F) \ + R(reas_sec_vlan_mark_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | RSS_F) \ + R(reas_sec_vlan_mark_ptype, R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | PTYPE_F) \ + R(reas_sec_vlan_mark_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_mark_cksum, R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | CKSUM_F) \ + R(reas_sec_vlan_mark_cksum_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | CKSUM_F | RSS_F) \ + R(reas_sec_vlan_mark_cksum_ptype, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_vlan_mark_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define NIX_RX_FASTPATH_MODES_240_255 \ + R(reas_sec_vlan_ts, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F) \ + R(reas_sec_vlan_ts_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | RSS_F) \ + R(reas_sec_vlan_ts_ptype, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | PTYPE_F) \ + R(reas_sec_vlan_ts_ptype_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_ts_cksum, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | CKSUM_F) \ + R(reas_sec_vlan_ts_cksum_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | CKSUM_F | RSS_F) \ + R(reas_sec_vlan_ts_cksum_ptype, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_vlan_ts_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_ts_mark, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F) \ + R(reas_sec_vlan_ts_mark_rss, R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | RSS_F) \ + R(reas_sec_vlan_ts_mark_ptype, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | PTYPE_F) \ + R(reas_sec_vlan_ts_mark_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ + R(reas_sec_vlan_ts_mark_cksum, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | CKSUM_F) \ + R(reas_sec_vlan_ts_mark_cksum_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ + R(reas_sec_vlan_ts_mark_cksum_ptype, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ + R(reas_sec_vlan_ts_mark_cksum_ptype_rss, \ + R_REAS_F | R_SEC_F | RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) + + #define NIX_RX_FASTPATH_MODES \ NIX_RX_FASTPATH_MODES_0_15 \ NIX_RX_FASTPATH_MODES_16_31 \ @@ -1234,7 +1792,15 @@ cn10k_nix_recv_pkts_vector(void *args, struct rte_mbuf **mbufs, uint16_t pkts, NIX_RX_FASTPATH_MODES_64_79 \ NIX_RX_FASTPATH_MODES_80_95 \ NIX_RX_FASTPATH_MODES_96_111 \ - NIX_RX_FASTPATH_MODES_112_127 + NIX_RX_FASTPATH_MODES_112_127 \ + NIX_RX_FASTPATH_MODES_128_143 \ + NIX_RX_FASTPATH_MODES_144_159 \ + NIX_RX_FASTPATH_MODES_160_175 \ + NIX_RX_FASTPATH_MODES_176_191 \ + NIX_RX_FASTPATH_MODES_192_207 \ + NIX_RX_FASTPATH_MODES_208_223 \ + NIX_RX_FASTPATH_MODES_224_239 \ + NIX_RX_FASTPATH_MODES_240_255 #define R(name, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index f1363af70f..75f33ff36d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -405,6 +405,10 @@ struct cnxk_eth_dev { /* Security data */ struct cnxk_eth_dev_sec_inb inb; struct cnxk_eth_dev_sec_outb outb; + + /* Reassembly dynfield/flag offsets */ + int reass_dynfield_off; + int reass_dynflag_bit; }; struct cnxk_eth_rxq_sp { diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index cd8c13bd1c..16db47c519 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -120,6 +120,10 @@ sources += files( 'rx/cn10k/rx_80_95.c', 'rx/cn10k/rx_96_111.c', 'rx/cn10k/rx_112_127.c', + 'rx/cn10k/rx_192_207.c', + 'rx/cn10k/rx_208_223.c', + 'rx/cn10k/rx_224_239.c', + 'rx/cn10k/rx_240_255.c', 'rx/cn10k/rx_0_15_mseg.c', 'rx/cn10k/rx_16_31_mseg.c', 'rx/cn10k/rx_32_47_mseg.c', @@ -128,6 +132,10 @@ sources += files( 'rx/cn10k/rx_80_95_mseg.c', 'rx/cn10k/rx_96_111_mseg.c', 'rx/cn10k/rx_112_127_mseg.c', + 'rx/cn10k/rx_192_207_mseg.c', + 'rx/cn10k/rx_208_223_mseg.c', + 'rx/cn10k/rx_224_239_mseg.c', + 'rx/cn10k/rx_240_255_mseg.c', 'rx/cn10k/rx_0_15_vec.c', 'rx/cn10k/rx_16_31_vec.c', 'rx/cn10k/rx_32_47_vec.c', @@ -136,6 +144,10 @@ sources += files( 'rx/cn10k/rx_80_95_vec.c', 'rx/cn10k/rx_96_111_vec.c', 'rx/cn10k/rx_112_127_vec.c', + 'rx/cn10k/rx_192_207_vec.c', + 'rx/cn10k/rx_208_223_vec.c', + 'rx/cn10k/rx_224_239_vec.c', + 'rx/cn10k/rx_240_255_vec.c', 'rx/cn10k/rx_0_15_vec_mseg.c', 'rx/cn10k/rx_16_31_vec_mseg.c', 'rx/cn10k/rx_32_47_vec_mseg.c', @@ -144,6 +156,10 @@ sources += files( 'rx/cn10k/rx_80_95_vec_mseg.c', 'rx/cn10k/rx_96_111_vec_mseg.c', 'rx/cn10k/rx_112_127_vec_mseg.c', + 'rx/cn10k/rx_192_207_vec_mseg.c', + 'rx/cn10k/rx_208_223_vec_mseg.c', + 'rx/cn10k/rx_224_239_vec_mseg.c', + 'rx/cn10k/rx_240_255_vec_mseg.c', ) sources += files( diff --git a/drivers/net/cnxk/rx/cn10k/rx_192_207.c b/drivers/net/cnxk/rx/cn10k/rx_192_207.c new file mode 100644 index 0000000000..45da8e9997 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_192_207.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV(cn10k_nix_recv_pkts_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_192_207_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_192_207_mseg.c new file mode 100644 index 0000000000..869be355f9 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_192_207_mseg.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_MSEG(cn10k_nix_recv_pkts_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_192_207_vec.c b/drivers/net/cnxk/rx/cn10k/rx_192_207_vec.c new file mode 100644 index 0000000000..3d214158ec --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_192_207_vec.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_VEC(cn10k_nix_recv_pkts_vec_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_192_207_vec_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_192_207_vec_mseg.c new file mode 100644 index 0000000000..762e9f5512 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_192_207_vec_mseg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) \ + NIX_RX_RECV_VEC_MSEG(cn10k_nix_recv_pkts_vec_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_192_207 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_208_223.c b/drivers/net/cnxk/rx/cn10k/rx_208_223.c new file mode 100644 index 0000000000..ca64ab5fde --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_208_223.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV(cn10k_nix_recv_pkts_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_208_223_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_208_223_mseg.c new file mode 100644 index 0000000000..35e443a06e --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_208_223_mseg.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_MSEG(cn10k_nix_recv_pkts_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_208_223_vec.c b/drivers/net/cnxk/rx/cn10k/rx_208_223_vec.c new file mode 100644 index 0000000000..b7f9feece2 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_208_223_vec.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_VEC(cn10k_nix_recv_pkts_vec_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_208_223_vec_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_208_223_vec_mseg.c new file mode 100644 index 0000000000..b3da6ec11e --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_208_223_vec_mseg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) \ + NIX_RX_RECV_VEC_MSEG(cn10k_nix_recv_pkts_vec_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_208_223 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_224_239.c b/drivers/net/cnxk/rx/cn10k/rx_224_239.c new file mode 100644 index 0000000000..bd288f3330 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_224_239.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV(cn10k_nix_recv_pkts_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_224_239_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_224_239_mseg.c new file mode 100644 index 0000000000..bf5af065df --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_224_239_mseg.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_MSEG(cn10k_nix_recv_pkts_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_224_239_vec.c b/drivers/net/cnxk/rx/cn10k/rx_224_239_vec.c new file mode 100644 index 0000000000..c05bb99414 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_224_239_vec.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_VEC(cn10k_nix_recv_pkts_vec_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_224_239_vec_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_224_239_vec_mseg.c new file mode 100644 index 0000000000..5f7f8efdae --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_224_239_vec_mseg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) \ + NIX_RX_RECV_VEC_MSEG(cn10k_nix_recv_pkts_vec_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_224_239 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_240_255.c b/drivers/net/cnxk/rx/cn10k/rx_240_255.c new file mode 100644 index 0000000000..d72b2eec1c --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_240_255.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV(cn10k_nix_recv_pkts_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_240_255_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_240_255_mseg.c new file mode 100644 index 0000000000..f248ad8c77 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_240_255_mseg.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_MSEG(cn10k_nix_recv_pkts_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_240_255_vec.c b/drivers/net/cnxk/rx/cn10k/rx_240_255_vec.c new file mode 100644 index 0000000000..7e81ed1883 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_240_255_vec.c @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) NIX_RX_RECV_VEC(cn10k_nix_recv_pkts_vec_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R diff --git a/drivers/net/cnxk/rx/cn10k/rx_240_255_vec_mseg.c b/drivers/net/cnxk/rx/cn10k/rx_240_255_vec_mseg.c new file mode 100644 index 0000000000..db8aeca013 --- /dev/null +++ b/drivers/net/cnxk/rx/cn10k/rx_240_255_vec_mseg.c @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, flags) \ + NIX_RX_RECV_VEC_MSEG(cn10k_nix_recv_pkts_vec_mseg_##name, flags) + +NIX_RX_FASTPATH_MODES_240_255 +#undef R -- 2.25.1