From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 42E3645BD6; Fri, 25 Oct 2024 15:06:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C4547406B6; Fri, 25 Oct 2024 15:05:58 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 16D6B4068E for ; Fri, 25 Oct 2024 15:04:16 +0200 (CEST) Received: from pps.filterd (m0431383.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49PBs4rw014087; Fri, 25 Oct 2024 06:04:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=q 511z/hEqj6bFkW7RGPwcTZtMt7SA1tAudKvkbfHfI4=; b=iunIJPQJOTDksAcBM RJ5UR80l4T7sw6N9t+mcfG79yTqevgOVMXSxtsN6fqMmtqezEs6iGjNzF+EQ3zez JPL3QZl6imegnxddIW6PMhiZaDk+ONPYujXLXOeZoVx18a40ngWQj0ZhBkQfnEkP jxVifljqFX1v2gOuUbWqhPM3r20ukgIz/zK+lzrQaFiOxj12Oy74x6Gdt+anWp4I WUcDsP2PxODU1kIvE93qXVT6YsdgszFzAezof4QXtLhOIsBuVyroFqTR1P9E6fsO tq8mw2lRQzH1mx/PuGq7Nfb8ruu8lXuub2ioOLVkiOLhmlRh7T2aeY8HlnvC+zV2 ZNKDg== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 42gauyg48m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 25 Oct 2024 06:04:16 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 25 Oct 2024 06:04:14 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Fri, 25 Oct 2024 06:04:14 -0700 Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com [10.28.164.106]) by maili.marvell.com (Postfix) with ESMTP id 990395B6922; Fri, 25 Oct 2024 06:04:12 -0700 (PDT) From: To: , , , Pavan Nikhilesh , Shijith Thotton CC: Subject: [PATCH v7 18/22] event/cnxk: support CN20K Tx adapter fast path Date: Fri, 25 Oct 2024 18:33:17 +0530 Message-ID: <20241025130321.29105-18-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241025130321.29105-1-pbhagavatula@marvell.com> References: <20241025122944.27745-1-pbhagavatula@marvell.com> <20241025130321.29105-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: vfT5HiSOE_y5WbSHZZeQvuKhNY_YEyxM X-Proofpoint-ORIG-GUID: vfT5HiSOE_y5WbSHZZeQvuKhNY_YEyxM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Add support for event eth Tx adapter fastpath operations. Signed-off-by: Pavan Nikhilesh --- drivers/event/cnxk/cn20k_eventdev.c | 29 +++ drivers/event/cnxk/cn20k_tx_worker.h | 176 +++++++++++++++++++ drivers/event/cnxk/meson.build | 20 +++ drivers/event/cnxk/tx/cn20k/tx_0_15.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_0_15_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_112_127.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_112_127_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_16_31.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_16_31_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_32_47.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_32_47_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_48_63.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_48_63_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_64_79.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_64_79_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_80_95.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_80_95_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_96_111.c | 18 ++ drivers/event/cnxk/tx/cn20k/tx_96_111_seg.c | 19 ++ drivers/event/cnxk/tx/cn20k/tx_all_offload.c | 40 +++++ 20 files changed, 561 insertions(+) create mode 100644 drivers/event/cnxk/tx/cn20k/tx_0_15.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_0_15_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_112_127.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_112_127_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_16_31.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_16_31_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_32_47.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_32_47_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_48_63.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_48_63_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_64_79.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_64_79_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_80_95.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_80_95_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_96_111.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_96_111_seg.c create mode 100644 drivers/event/cnxk/tx/cn20k/tx_all_offload.c diff --git a/drivers/event/cnxk/cn20k_eventdev.c b/drivers/event/cnxk/cn20k_eventdev.c index 509c6ea630..5d49a5e5c6 100644 --- a/drivers/event/cnxk/cn20k_eventdev.c +++ b/drivers/event/cnxk/cn20k_eventdev.c @@ -15,6 +15,9 @@ #define CN20K_SET_EVDEV_DEQ_OP(dev, deq_op, deq_ops) \ deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)] +#define CN20K_SET_EVDEV_ENQ_OP(dev, enq_op, enq_ops) \ + enq_op = enq_ops[dev->tx_offloads & (NIX_TX_OFFLOAD_MAX - 1)] + static void * cn20k_sso_init_hws_mem(void *arg, uint8_t port_id) { @@ -253,6 +256,19 @@ cn20k_sso_fp_tmplt_fns_set(struct rte_eventdev *event_dev) #undef R }; + /* Tx modes */ + const event_tx_adapter_enqueue_t sso_hws_tx_adptr_enq[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_sso_hws_tx_adptr_enq_##name, + NIX_TX_FASTPATH_MODES +#undef T + }; + + const event_tx_adapter_enqueue_t sso_hws_tx_adptr_enq_seg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_sso_hws_tx_adptr_enq_seg_##name, + NIX_TX_FASTPATH_MODES +#undef T + }; + if (dev->rx_offloads & NIX_RX_MULTI_SEG_F) { if (dev->rx_offloads & NIX_RX_REAS_F) { CN20K_SET_EVDEV_DEQ_OP(dev, event_dev->dequeue_burst, @@ -285,6 +301,12 @@ cn20k_sso_fp_tmplt_fns_set(struct rte_eventdev *event_dev) } } + if (dev->tx_offloads & NIX_TX_MULTI_SEG_F) + CN20K_SET_EVDEV_ENQ_OP(dev, event_dev->txa_enqueue, sso_hws_tx_adptr_enq_seg); + else + CN20K_SET_EVDEV_ENQ_OP(dev, event_dev->txa_enqueue, sso_hws_tx_adptr_enq); + + event_dev->txa_enqueue_same_dest = event_dev->txa_enqueue; #else RTE_SET_USED(event_dev); #endif @@ -299,6 +321,13 @@ cn20k_sso_fp_blk_fns_set(struct rte_eventdev *event_dev) event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload; if (dev->rx_offloads & NIX_RX_OFFLOAD_TSTAMP_F) event_dev->dequeue_burst = cn20k_sso_hws_deq_burst_all_offload_tst; + event_dev->txa_enqueue = cn20k_sso_hws_tx_adptr_enq_seg_all_offload; + event_dev->txa_enqueue_same_dest = cn20k_sso_hws_tx_adptr_enq_seg_all_offload; + if (dev->tx_offloads & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_VLAN_QINQ_F | + NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_TSTAMP_F)) { + event_dev->txa_enqueue = cn20k_sso_hws_tx_adptr_enq_seg_all_offload_tst; + event_dev->txa_enqueue_same_dest = cn20k_sso_hws_tx_adptr_enq_seg_all_offload_tst; + } #else RTE_SET_USED(event_dev); #endif diff --git a/drivers/event/cnxk/cn20k_tx_worker.h b/drivers/event/cnxk/cn20k_tx_worker.h index 63fbdf5328..c8ab560b0e 100644 --- a/drivers/event/cnxk/cn20k_tx_worker.h +++ b/drivers/event/cnxk/cn20k_tx_worker.h @@ -13,4 +13,180 @@ #include "cnxk_eventdev_dp.h" #include +/* CN20K Tx event fastpath */ + +static __rte_always_inline struct cn20k_eth_txq * +cn20k_sso_hws_xtract_meta(struct rte_mbuf *m, const uint64_t *txq_data) +{ + return (struct cn20k_eth_txq *)(txq_data[(txq_data[m->port] >> 48) + + rte_event_eth_tx_adapter_txq_get(m)] & + (BIT_ULL(48) - 1)); +} + +static __rte_always_inline void +cn20k_sso_txq_fc_wait(const struct cn20k_eth_txq *txq) +{ + int64_t avail; + +#ifdef RTE_ARCH_ARM64 + int64_t val; + + asm volatile(PLT_CPU_FEATURE_PREAMBLE + " ldxr %[val], [%[addr]] \n" + " sub %[val], %[adj], %[val] \n" + " lsl %[refill], %[val], %[shft] \n" + " sub %[refill], %[refill], %[val] \n" + " cmp %[refill], #0x0 \n" + " b.gt .Ldne%= \n" + " sevl \n" + ".Lrty%=: wfe \n" + " ldxr %[val], [%[addr]] \n" + " sub %[val], %[adj], %[val] \n" + " lsl %[refill], %[val], %[shft] \n" + " sub %[refill], %[refill], %[val] \n" + " cmp %[refill], #0x0 \n" + " b.le .Lrty%= \n" + ".Ldne%=: \n" + : [refill] "=&r"(avail), [val] "=&r" (val) + : [addr] "r" (txq->fc_mem), [adj] "r" (txq->nb_sqb_bufs_adj), + [shft] "r" (txq->sqes_per_sqb_log2) + : "memory"); +#else + do { + avail = txq->nb_sqb_bufs_adj - + rte_atomic_load_explicit((uint64_t __rte_atomic *)txq->fc_mem, + rte_memory_order_relaxed); + } while (((avail << txq->sqes_per_sqb_log2) - avail) <= 0); +#endif +} + +static __rte_always_inline int32_t +cn20k_sso_sq_depth(const struct cn20k_eth_txq *txq) +{ + int32_t avail = (int32_t)txq->nb_sqb_bufs_adj - + (int32_t)rte_atomic_load_explicit((uint64_t __rte_atomic *)txq->fc_mem, + rte_memory_order_relaxed); + return (avail << txq->sqes_per_sqb_log2) - avail; +} + +static __rte_always_inline uint16_t +cn20k_sso_tx_one(struct cn20k_sso_hws *ws, struct rte_mbuf *m, uint64_t *cmd, uint16_t lmt_id, + uintptr_t lmt_addr, uint8_t sched_type, const uint64_t *txq_data, + const uint32_t flags) +{ + uint8_t lnum = 0, loff = 0, shft = 0; + struct rte_mbuf *extm = NULL; + struct cn20k_eth_txq *txq; + uintptr_t laddr; + uint16_t segdw; + uintptr_t pa; + bool sec; + + txq = cn20k_sso_hws_xtract_meta(m, txq_data); + if (cn20k_sso_sq_depth(txq) <= 0) + return 0; + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F && txq->tx_compl.ena) + handle_tx_completion_pkts(txq, 1); + + cn20k_nix_tx_skeleton(txq, cmd, flags, 0); + /* Perform header writes before barrier + * for TSO + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn20k_nix_xmit_prepare_tso(m, flags); + + cn20k_nix_xmit_prepare(txq, m, &extm, cmd, flags, txq->lso_tun_fmt, &sec, txq->mark_flag, + txq->mark_fmt); + + laddr = lmt_addr; + /* Prepare CPT instruction and get nixtx addr if + * it is for CPT on same lmtline. + */ + if (flags & NIX_TX_OFFLOAD_SECURITY_F && sec) + cn20k_nix_prep_sec(m, cmd, &laddr, lmt_addr, &lnum, &loff, &shft, txq->sa_base, + flags); + + /* Move NIX desc to LMT/NIXTX area */ + cn20k_nix_xmit_mv_lmt_base(laddr, cmd, flags); + + if (flags & NIX_TX_MULTI_SEG_F) + segdw = cn20k_nix_prepare_mseg(txq, m, &extm, (uint64_t *)laddr, flags); + else + segdw = cn20k_nix_tx_ext_subs(flags) + 2; + + cn20k_nix_xmit_prepare_tstamp(txq, laddr, m->ol_flags, segdw, flags); + if (flags & NIX_TX_OFFLOAD_SECURITY_F && sec) + pa = txq->cpt_io_addr | 3 << 4; + else + pa = txq->io_addr | ((segdw - 1) << 4); + + if (!CNXK_TAG_IS_HEAD(ws->gw_rdata) && !sched_type) + ws->gw_rdata = roc_sso_hws_head_wait(ws->base); + + cn20k_sso_txq_fc_wait(txq); + if (flags & NIX_TX_OFFLOAD_SECURITY_F && sec) + cn20k_nix_sec_fc_wait_one(txq); + + roc_lmt_submit_steorl(lmt_id, pa); + + /* Memory barrier to make sure lmtst store completes */ + rte_io_wmb(); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F && !txq->tx_compl.ena) + cn20k_nix_free_extmbuf(extm); + + return 1; +} + +static __rte_always_inline uint16_t +cn20k_sso_hws_event_tx(struct cn20k_sso_hws *ws, struct rte_event *ev, uint64_t *cmd, + const uint64_t *txq_data, const uint32_t flags) +{ + struct rte_mbuf *m; + uintptr_t lmt_addr; + uint16_t lmt_id; + + lmt_addr = ws->lmt_base; + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + + m = ev->mbuf; + return cn20k_sso_tx_one(ws, m, cmd, lmt_id, lmt_addr, ev->sched_type, txq_data, flags); +} + +#define T(name, sz, flags) \ + uint16_t __rte_hot cn20k_sso_hws_tx_adptr_enq_##name(void *port, struct rte_event ev[], \ + uint16_t nb_events); \ + uint16_t __rte_hot cn20k_sso_hws_tx_adptr_enq_seg_##name( \ + void *port, struct rte_event ev[], uint16_t nb_events); + +NIX_TX_FASTPATH_MODES +#undef T + +#define SSO_TX(fn, sz, flags) \ + uint16_t __rte_hot fn(void *port, struct rte_event ev[], uint16_t nb_events) \ + { \ + struct cn20k_sso_hws *ws = port; \ + uint64_t cmd[sz]; \ + RTE_SET_USED(nb_events); \ + return cn20k_sso_hws_event_tx(ws, &ev[0], cmd, \ + (const uint64_t *)ws->tx_adptr_data, flags); \ + } + +#define SSO_TX_SEG(fn, sz, flags) \ + uint16_t __rte_hot fn(void *port, struct rte_event ev[], uint16_t nb_events) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + struct cn20k_sso_hws *ws = port; \ + RTE_SET_USED(nb_events); \ + return cn20k_sso_hws_event_tx(ws, &ev[0], cmd, \ + (const uint64_t *)ws->tx_adptr_data, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +uint16_t __rte_hot cn20k_sso_hws_tx_adptr_enq_seg_all_offload(void *port, struct rte_event ev[], + uint16_t nb_events); +uint16_t __rte_hot cn20k_sso_hws_tx_adptr_enq_seg_all_offload_tst(void *port, struct rte_event ev[], + uint16_t nb_events); + #endif diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build index a2bafab268..8aaf8116f7 100644 --- a/drivers/event/cnxk/meson.build +++ b/drivers/event/cnxk/meson.build @@ -272,9 +272,29 @@ sources += files( 'deq/cn20k/deq_all_offload.c', ) +sources += files( + 'tx/cn20k/tx_0_15.c', + 'tx/cn20k/tx_16_31.c', + 'tx/cn20k/tx_32_47.c', + 'tx/cn20k/tx_48_63.c', + 'tx/cn20k/tx_64_79.c', + 'tx/cn20k/tx_80_95.c', + 'tx/cn20k/tx_96_111.c', + 'tx/cn20k/tx_112_127.c', + 'tx/cn20k/tx_0_15_seg.c', + 'tx/cn20k/tx_16_31_seg.c', + 'tx/cn20k/tx_32_47_seg.c', + 'tx/cn20k/tx_48_63_seg.c', + 'tx/cn20k/tx_64_79_seg.c', + 'tx/cn20k/tx_80_95_seg.c', + 'tx/cn20k/tx_96_111_seg.c', + 'tx/cn20k/tx_112_127_seg.c', + 'tx/cn20k/tx_all_offload.c', +) else sources += files( 'deq/cn20k/deq_all_offload.c', + 'tx/cn20k/tx_all_offload.c', ) endif endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_0_15.c b/drivers/event/cnxk/tx/cn20k/tx_0_15.c new file mode 100644 index 0000000000..b681bc8ab0 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_0_15.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_0_15_seg.c b/drivers/event/cnxk/tx/cn20k/tx_0_15_seg.c new file mode 100644 index 0000000000..1dacb63d4b --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_0_15_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_112_127.c b/drivers/event/cnxk/tx/cn20k/tx_112_127.c new file mode 100644 index 0000000000..abdb8b76a1 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_112_127.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_112_127_seg.c b/drivers/event/cnxk/tx/cn20k/tx_112_127_seg.c new file mode 100644 index 0000000000..c39d331b25 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_112_127_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_16_31.c b/drivers/event/cnxk/tx/cn20k/tx_16_31.c new file mode 100644 index 0000000000..5b88c47914 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_16_31.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_16_31_seg.c b/drivers/event/cnxk/tx/cn20k/tx_16_31_seg.c new file mode 100644 index 0000000000..13f00ac478 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_16_31_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_32_47.c b/drivers/event/cnxk/tx/cn20k/tx_32_47.c new file mode 100644 index 0000000000..1f6008c425 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_32_47.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_32_47_seg.c b/drivers/event/cnxk/tx/cn20k/tx_32_47_seg.c new file mode 100644 index 0000000000..587f22df3a --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_32_47_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_48_63.c b/drivers/event/cnxk/tx/cn20k/tx_48_63.c new file mode 100644 index 0000000000..c712825417 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_48_63.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_48_63_seg.c b/drivers/event/cnxk/tx/cn20k/tx_48_63_seg.c new file mode 100644 index 0000000000..1fc11ec904 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_48_63_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_64_79.c b/drivers/event/cnxk/tx/cn20k/tx_64_79.c new file mode 100644 index 0000000000..0e427f79d8 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_64_79.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_64_79_seg.c b/drivers/event/cnxk/tx/cn20k/tx_64_79_seg.c new file mode 100644 index 0000000000..6e1ae41b26 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_64_79_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_80_95.c b/drivers/event/cnxk/tx/cn20k/tx_80_95.c new file mode 100644 index 0000000000..8c87d2341d --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_80_95.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_80_95_seg.c b/drivers/event/cnxk/tx/cn20k/tx_80_95_seg.c new file mode 100644 index 0000000000..43a143f4bd --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_80_95_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_96_111.c b/drivers/event/cnxk/tx/cn20k/tx_96_111.c new file mode 100644 index 0000000000..1a43af8b02 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_96_111.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) SSO_TX(cn20k_sso_hws_tx_adptr_enq_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_96_111_seg.c b/drivers/event/cnxk/tx/cn20k/tx_96_111_seg.c new file mode 100644 index 0000000000..e0e1d8a4ef --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_96_111_seg.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) \ + SSO_TX_SEG(cn20k_sso_hws_tx_adptr_enq_seg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/event/cnxk/tx/cn20k/tx_all_offload.c b/drivers/event/cnxk/tx/cn20k/tx_all_offload.c new file mode 100644 index 0000000000..d2158a4256 --- /dev/null +++ b/drivers/event/cnxk/tx/cn20k/tx_all_offload.c @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include "cn20k_tx_worker.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if defined(CNXK_DIS_TMPLT_FUNC) + +uint16_t __rte_hot +cn20k_sso_hws_tx_adptr_enq_seg_all_offload(void *port, struct rte_event ev[], uint16_t nb_events) +{ + const uint32_t flags = (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_MBUF_NOFF_F | + NIX_TX_MULTI_SEG_F | NIX_TX_OFFLOAD_SECURITY_F); + uint64_t cmd[8 + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; + + struct cn20k_sso_hws *ws = port; + RTE_SET_USED(nb_events); + return cn20k_sso_hws_event_tx(ws, &ev[0], cmd, (const uint64_t *)ws->tx_adptr_data, flags); +} + +uint16_t __rte_hot +cn20k_sso_hws_tx_adptr_enq_seg_all_offload_tst(void *port, struct rte_event ev[], + uint16_t nb_events) +{ + const uint32_t flags = + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_MBUF_NOFF_F | NIX_TX_OFFLOAD_TSO_F | + NIX_TX_OFFLOAD_TSTAMP_F | NIX_TX_OFFLOAD_SECURITY_F | NIX_TX_MULTI_SEG_F); + uint64_t cmd[8 + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; + + struct cn20k_sso_hws *ws = port; + RTE_SET_USED(nb_events); + return cn20k_sso_hws_event_tx(ws, &ev[0], cmd, (const uint64_t *)ws->tx_adptr_data, flags); +} + +#endif -- 2.25.1