From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3CB9245954; Tue, 10 Sep 2024 11:02:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9570842ECD; Tue, 10 Sep 2024 11:00:47 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 0BDEC42E5C for ; Tue, 10 Sep 2024 11:00:44 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48A4qBes029385 for ; Tue, 10 Sep 2024 02:00:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=2 rCpWPMghl554qIutS740fASyQc2eJZ5YXdup/BpatU=; b=gmQsCbuxFmd79WAyL hPI0BePtvcmZig61okLZDwRw+RcK6KXMmz0inRfqbEoO+f+OSQazt2uW5Q4mqizZ QylU2aKjaZs5ovv8GHdJnyTcn4va7dgxd0psLwkMTPWNnDqGzLc/hmrDMiGYxxKG 4XX9FpUu5q/Ls3ndMjgokR7S/b3PPV8iS10J/ZRqTQmfoUgVmacUf5LKImJrvm6N 5L/EnQeUI7CTinscD9iV18JnqmmabIcI5pS8LZ2Iz0mbBGljpJRk1WxFV37WHNhQ T9w1fD89E37/wN1sfoTYcJlsvKrfSEagVFZ2b+njmKioNNkVRBdqg/i6/+g/xsEm f7ChQ== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 41gygtg1nv-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 10 Sep 2024 02:00:42 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 10 Sep 2024 02:00:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 10 Sep 2024 02:00:28 -0700 Received: from hyd1588t430.caveonetworks.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id DDEB33F709E; Tue, 10 Sep 2024 02:00:25 -0700 (PDT) From: Nithin Dabilpuram To: , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao , Harman Kalra CC: Subject: [PATCH 20/33] net/cnxk: support Tx function select for cn20k Date: Tue, 10 Sep 2024 14:28:56 +0530 Message-ID: <20240910085909.1514457-21-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240910085909.1514457-1-ndabilpuram@marvell.com> References: <20240910085909.1514457-1-ndabilpuram@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: MqFHhxQQgAgbK5j3Ip4POiLY08xnVAca X-Proofpoint-GUID: MqFHhxQQgAgbK5j3Ip4POiLY08xnVAca X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support to select Tx function based on offload flags for cn20k. Signed-off-by: Nithin Dabilpuram --- drivers/net/cnxk/cn20k_ethdev.c | 80 ++++++ drivers/net/cnxk/cn20k_ethdev.h | 1 + drivers/net/cnxk/cn20k_tx.h | 237 ++++++++++++++++++ drivers/net/cnxk/cn20k_tx_select.c | 122 +++++++++ drivers/net/cnxk/meson.build | 37 +++ drivers/net/cnxk/tx/cn20k/tx_0_15.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_0_15_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_0_15_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_0_15_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_112_127.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_112_127_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_112_127_vec.c | 18 ++ .../net/cnxk/tx/cn20k/tx_112_127_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_16_31.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_16_31_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_16_31_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_16_31_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_32_47.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_32_47_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_32_47_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_32_47_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_48_63.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_48_63_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_48_63_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_48_63_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_64_79.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_64_79_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_64_79_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_64_79_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_80_95.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_80_95_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_80_95_vec.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_80_95_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_96_111.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_96_111_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_96_111_vec.c | 18 ++ .../net/cnxk/tx/cn20k/tx_96_111_vec_mseg.c | 18 ++ drivers/net/cnxk/tx/cn20k/tx_all_offload.c | 39 +++ 38 files changed, 1092 insertions(+) create mode 100644 drivers/net/cnxk/cn20k_tx_select.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_0_15.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_0_15_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_0_15_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_0_15_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_112_127.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_112_127_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_112_127_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_112_127_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_16_31.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_16_31_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_16_31_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_16_31_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_32_47.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_32_47_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_32_47_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_32_47_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_48_63.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_48_63_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_48_63_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_48_63_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_64_79.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_64_79_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_64_79_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_64_79_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_80_95.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_80_95_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_80_95_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_80_95_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_96_111.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_96_111_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_96_111_vec.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_96_111_vec_mseg.c create mode 100644 drivers/net/cnxk/tx/cn20k/tx_all_offload.c diff --git a/drivers/net/cnxk/cn20k_ethdev.c b/drivers/net/cnxk/cn20k_ethdev.c index d1cb3a52bf..4b2f04ba31 100644 --- a/drivers/net/cnxk/cn20k_ethdev.c +++ b/drivers/net/cnxk/cn20k_ethdev.c @@ -40,6 +40,78 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) return flags; } +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + struct roc_nix *nix = &dev->nix; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(RTE_MBUF_F_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_addr) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_addr) + 24); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & RTE_ETH_TX_OFFLOAD_VLAN_INSERT || conf & RTE_ETH_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM || conf & RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM || conf & RTE_ETH_TX_OFFLOAD_TCP_CKSUM || + conf & RTE_ETH_TX_OFFLOAD_UDP_CKSUM || conf & RTE_ETH_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & RTE_ETH_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO | RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO | + RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + if ((dev->rx_offloads & RTE_ETH_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + if (conf & RTE_ETH_TX_OFFLOAD_SECURITY) + flags |= NIX_TX_OFFLOAD_SECURITY_F; + + if (dev->tx_mark) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (nix->tx_compl_ena) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + return flags; +} + static int cn20k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -226,6 +298,7 @@ cn20k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_ /* Update offload flags */ dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); } rq = &dev->rqs[qid]; @@ -286,6 +359,8 @@ cn20k_nix_configure(struct rte_eth_dev *eth_dev) /* Update offload flags */ dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + /* reset reassembly dynfield/flag offset */ dev->reass_dynfield_off = -1; dev->reass_dynflag_bit = -1; @@ -316,6 +391,7 @@ cn20k_nix_timesync_enable(struct rte_eth_dev *eth_dev) * in rx[tx]_offloads. */ cn20k_eth_set_rx_function(eth_dev); + cn20k_eth_set_tx_function(eth_dev); return 0; } @@ -339,6 +415,7 @@ cn20k_nix_timesync_disable(struct rte_eth_dev *eth_dev) * in rx[tx]_offloads. */ cn20k_eth_set_rx_function(eth_dev); + cn20k_eth_set_tx_function(eth_dev); return 0; } @@ -378,10 +455,12 @@ cn20k_nix_dev_start(struct rte_eth_dev *eth_dev) * in rx[tx]_offloads. */ dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); /* Set flags for Rx Inject feature */ if (roc_idev_nix_rx_inject_get(nix->port_id)) dev->rx_offload_flags |= NIX_RX_SEC_REASSEMBLY_F; + cn20k_eth_set_tx_function(eth_dev); cn20k_eth_set_rx_function(eth_dev); return 0; } @@ -581,6 +660,7 @@ cn20k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (rte_eal_process_type() != RTE_PROC_PRIMARY) { /* Setup callbacks for secondary process */ + cn20k_eth_set_tx_function(eth_dev); cn20k_eth_set_rx_function(eth_dev); return 0; } diff --git a/drivers/net/cnxk/cn20k_ethdev.h b/drivers/net/cnxk/cn20k_ethdev.h index 2049ee7fa4..cb46044d60 100644 --- a/drivers/net/cnxk/cn20k_ethdev.h +++ b/drivers/net/cnxk/cn20k_ethdev.h @@ -10,5 +10,6 @@ /* Rx and Tx routines */ void cn20k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn20k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN20K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn20k_tx.h b/drivers/net/cnxk/cn20k_tx.h index a00c9d5776..9fd925ac34 100644 --- a/drivers/net/cnxk/cn20k_tx.h +++ b/drivers/net/cnxk/cn20k_tx.h @@ -32,4 +32,241 @@ #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | NIX_TX_OFFLOAD_TSO_F) +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F +#define T_SEC_F NIX_TX_OFFLOAD_SECURITY_F + +/* [T_SEC_F] [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES_0_15 \ + T(no_offload, 6, NIX_TX_OFFLOAD_NONE) \ + T(l3l4csum, 6, L3L4CSUM_F) \ + T(ol3ol4csum, 6, OL3OL4CSUM_F) \ + T(ol3ol4csum_l3l4csum, 6, OL3OL4CSUM_F | L3L4CSUM_F) \ + T(vlan, 6, VLAN_F) \ + T(vlan_l3l4csum, 6, VLAN_F | L3L4CSUM_F) \ + T(vlan_ol3ol4csum, 6, VLAN_F | OL3OL4CSUM_F) \ + T(vlan_ol3ol4csum_l3l4csum, 6, VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(noff, 6, NOFF_F) \ + T(noff_l3l4csum, 6, NOFF_F | L3L4CSUM_F) \ + T(noff_ol3ol4csum, 6, NOFF_F | OL3OL4CSUM_F) \ + T(noff_ol3ol4csum_l3l4csum, 6, NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(noff_vlan, 6, NOFF_F | VLAN_F) \ + T(noff_vlan_l3l4csum, 6, NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(noff_vlan_ol3ol4csum, 6, NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(noff_vlan_ol3ol4csum_l3l4csum, 6, NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_16_31 \ + T(tso, 6, TSO_F) \ + T(tso_l3l4csum, 6, TSO_F | L3L4CSUM_F) \ + T(tso_ol3ol4csum, 6, TSO_F | OL3OL4CSUM_F) \ + T(tso_ol3ol4csum_l3l4csum, 6, TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_vlan, 6, TSO_F | VLAN_F) \ + T(tso_vlan_l3l4csum, 6, TSO_F | VLAN_F | L3L4CSUM_F) \ + T(tso_vlan_ol3ol4csum, 6, TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(tso_vlan_ol3ol4csum_l3l4csum, 6, TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_noff, 6, TSO_F | NOFF_F) \ + T(tso_noff_l3l4csum, 6, TSO_F | NOFF_F | L3L4CSUM_F) \ + T(tso_noff_ol3ol4csum, 6, TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(tso_noff_ol3ol4csum_l3l4csum, 6, TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(tso_noff_vlan, 6, TSO_F | NOFF_F | VLAN_F) \ + T(tso_noff_vlan_l3l4csum, 6, TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(tso_noff_vlan_ol3ol4csum, 6, TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(tso_noff_vlan_ol3ol4csum_l3l4csum, 6, TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_32_47 \ + T(ts, 8, TSP_F) \ + T(ts_l3l4csum, 8, TSP_F | L3L4CSUM_F) \ + T(ts_ol3ol4csum, 8, TSP_F | OL3OL4CSUM_F) \ + T(ts_ol3ol4csum_l3l4csum, 8, TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_vlan, 8, TSP_F | VLAN_F) \ + T(ts_vlan_l3l4csum, 8, TSP_F | VLAN_F | L3L4CSUM_F) \ + T(ts_vlan_ol3ol4csum, 8, TSP_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_vlan_ol3ol4csum_l3l4csum, 8, TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_noff, 8, TSP_F | NOFF_F) \ + T(ts_noff_l3l4csum, 8, TSP_F | NOFF_F | L3L4CSUM_F) \ + T(ts_noff_ol3ol4csum, 8, TSP_F | NOFF_F | OL3OL4CSUM_F) \ + T(ts_noff_ol3ol4csum_l3l4csum, 8, TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_noff_vlan, 8, TSP_F | NOFF_F | VLAN_F) \ + T(ts_noff_vlan_l3l4csum, 8, TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(ts_noff_vlan_ol3ol4csum, 8, TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_noff_vlan_ol3ol4csum_l3l4csum, 8, TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_48_63 \ + T(ts_tso, 8, TSP_F | TSO_F) \ + T(ts_tso_l3l4csum, 8, TSP_F | TSO_F | L3L4CSUM_F) \ + T(ts_tso_ol3ol4csum, 8, TSP_F | TSO_F | OL3OL4CSUM_F) \ + T(ts_tso_ol3ol4csum_l3l4csum, 8, TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_vlan, 8, TSP_F | TSO_F | VLAN_F) \ + T(ts_tso_vlan_l3l4csum, 8, TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(ts_tso_vlan_ol3ol4csum, 8, TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_tso_vlan_ol3ol4csum_l3l4csum, 8, TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_noff, 8, TSP_F | TSO_F | NOFF_F) \ + T(ts_tso_noff_l3l4csum, 8, TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(ts_tso_noff_ol3ol4csum, 8, TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(ts_tso_noff_ol3ol4csum_l3l4csum, 8, TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(ts_tso_noff_vlan, 8, TSP_F | TSO_F | NOFF_F | VLAN_F) \ + T(ts_tso_noff_vlan_l3l4csum, 8, TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(ts_tso_noff_vlan_ol3ol4csum, 8, TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_64_79 \ + T(sec, 6, T_SEC_F) \ + T(sec_l3l4csum, 6, T_SEC_F | L3L4CSUM_F) \ + T(sec_ol3ol4csum, 6, T_SEC_F | OL3OL4CSUM_F) \ + T(sec_ol3ol4csum_l3l4csum, 6, T_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_vlan, 6, T_SEC_F | VLAN_F) \ + T(sec_vlan_l3l4csum, 6, T_SEC_F | VLAN_F | L3L4CSUM_F) \ + T(sec_vlan_ol3ol4csum, 6, T_SEC_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_vlan_ol3ol4csum_l3l4csum, 6, T_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_noff, 6, T_SEC_F | NOFF_F) \ + T(sec_noff_l3l4csum, 6, T_SEC_F | NOFF_F | L3L4CSUM_F) \ + T(sec_noff_ol3ol4csum, 6, T_SEC_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_noff_ol3ol4csum_l3l4csum, 6, T_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_noff_vlan, 6, T_SEC_F | NOFF_F | VLAN_F) \ + T(sec_noff_vlan_l3l4csum, 6, T_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_noff_vlan_ol3ol4csum, 6, T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_noff_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_80_95 \ + T(sec_tso, 6, T_SEC_F | TSO_F) \ + T(sec_tso_l3l4csum, 6, T_SEC_F | TSO_F | L3L4CSUM_F) \ + T(sec_tso_ol3ol4csum, 6, T_SEC_F | TSO_F | OL3OL4CSUM_F) \ + T(sec_tso_ol3ol4csum_l3l4csum, 6, T_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_vlan, 6, T_SEC_F | TSO_F | VLAN_F) \ + T(sec_tso_vlan_l3l4csum, 6, T_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(sec_tso_vlan_ol3ol4csum, 6, T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_tso_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_noff, 6, T_SEC_F | TSO_F | NOFF_F) \ + T(sec_tso_noff_l3l4csum, 6, T_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(sec_tso_noff_ol3ol4csum, 6, T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_tso_noff_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_tso_noff_vlan, 6, T_SEC_F | TSO_F | NOFF_F | VLAN_F) \ + T(sec_tso_noff_vlan_l3l4csum, 6, T_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_tso_noff_vlan_ol3ol4csum, 6, T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, 6, \ + T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_96_111 \ + T(sec_ts, 8, T_SEC_F | TSP_F) \ + T(sec_ts_l3l4csum, 8, T_SEC_F | TSP_F | L3L4CSUM_F) \ + T(sec_ts_ol3ol4csum, 8, T_SEC_F | TSP_F | OL3OL4CSUM_F) \ + T(sec_ts_ol3ol4csum_l3l4csum, 8, T_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_vlan, 8, T_SEC_F | TSP_F | VLAN_F) \ + T(sec_ts_vlan_l3l4csum, 8, T_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_vlan_ol3ol4csum, 8, T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_noff, 8, T_SEC_F | TSP_F | NOFF_F) \ + T(sec_ts_noff_l3l4csum, 8, T_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ + T(sec_ts_noff_ol3ol4csum, 8, T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_ts_noff_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_noff_vlan, 8, T_SEC_F | TSP_F | NOFF_F | VLAN_F) \ + T(sec_ts_noff_vlan_l3l4csum, 8, T_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_noff_vlan_ol3ol4csum, 8, T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES_112_127 \ + T(sec_ts_tso, 8, T_SEC_F | TSP_F | TSO_F) \ + T(sec_ts_tso_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | L3L4CSUM_F) \ + T(sec_ts_tso_ol3ol4csum, 8, T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_ol3ol4csum_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_vlan, 8, T_SEC_F | TSP_F | TSO_F | VLAN_F) \ + T(sec_ts_tso_vlan_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_tso_vlan_ol3ol4csum, 8, T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F) \ + T(sec_ts_tso_noff_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_ol3ol4csum, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_noff_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_vlan, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F) \ + T(sec_ts_tso_noff_vlan_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ + T(sec_ts_tso_noff_vlan_ol3ol4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ + T(sec_ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define NIX_TX_FASTPATH_MODES \ + NIX_TX_FASTPATH_MODES_0_15 \ + NIX_TX_FASTPATH_MODES_16_31 \ + NIX_TX_FASTPATH_MODES_32_47 \ + NIX_TX_FASTPATH_MODES_48_63 \ + NIX_TX_FASTPATH_MODES_64_79 \ + NIX_TX_FASTPATH_MODES_80_95 \ + NIX_TX_FASTPATH_MODES_96_111 \ + NIX_TX_FASTPATH_MODES_112_127 + +#define T(name, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_vec_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_vec_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + +#define NIX_TX_XMIT(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn(void *tx_queue, struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + RTE_SET_USED(tx_queue); \ + RTE_SET_USED(tx_pkts); \ + RTE_SET_USED(pkts); \ + return 0; \ + } + +#define NIX_TX_XMIT_MSEG(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn(void *tx_queue, struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + RTE_SET_USED(tx_queue); \ + RTE_SET_USED(tx_pkts); \ + RTE_SET_USED(pkts); \ + return 0; \ + } + +#define NIX_TX_XMIT_VEC(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn(void *tx_queue, struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + RTE_SET_USED(tx_queue); \ + RTE_SET_USED(tx_pkts); \ + RTE_SET_USED(pkts); \ + return 0; \ + } + +#define NIX_TX_XMIT_VEC_MSEG(fn, sz, flags) \ + uint16_t __rte_noinline __rte_hot fn(void *tx_queue, struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + RTE_SET_USED(tx_queue); \ + RTE_SET_USED(tx_pkts); \ + RTE_SET_USED(pkts); \ + return 0; \ + } + +uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_all_offload(void *tx_queue, + struct rte_mbuf **tx_pkts, + uint16_t pkts); + +uint16_t __rte_noinline __rte_hot cn20k_nix_xmit_pkts_vec_all_offload(void *tx_queue, + struct rte_mbuf **tx_pkts, + uint16_t pkts); + #endif /* __CN20K_TX_H__ */ diff --git a/drivers/net/cnxk/cn20k_tx_select.c b/drivers/net/cnxk/cn20k_tx_select.c new file mode 100644 index 0000000000..fb62b54a5f --- /dev/null +++ b/drivers/net/cnxk/cn20k_tx_select.c @@ -0,0 +1,122 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_ethdev.h" +#include "cn20k_tx.h" + +static __rte_used inline void +pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[NIX_TX_OFFLOAD_MAX]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [SEC] [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst[dev->tx_offload_flags & (NIX_TX_OFFLOAD_MAX - 1)]; + + if (eth_dev->data->dev_started) + rte_eth_fp_ops[eth_dev->data->port_id].tx_pkt_burst = eth_dev->tx_pkt_burst; +} + +#if defined(RTE_ARCH_ARM64) +static int +cn20k_nix_tx_queue_count(void *tx_queue) +{ + struct cn20k_eth_txq *txq = (struct cn20k_eth_txq *)tx_queue; + + return cnxk_nix_tx_queue_count(txq->fc_mem, txq->sqes_per_sqb_log2); +} + +static int +cn20k_nix_tx_queue_sec_count(void *tx_queue) +{ + struct cn20k_eth_txq *txq = (struct cn20k_eth_txq *)tx_queue; + + return cnxk_nix_tx_queue_sec_count(txq->fc_mem, txq->sqes_per_sqb_log2, txq->cpt_fc); +} + +static void +cn20k_eth_set_tx_tmplt_func(struct rte_eth_dev *eth_dev) +{ +#if !defined(CNXK_DIS_TMPLT_FUNC) + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + const eth_tx_burst_t nix_eth_tx_burst[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_burst_mseg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_vec_burst[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + const eth_tx_burst_t nix_eth_tx_vec_burst_mseg[NIX_TX_OFFLOAD_MAX] = { +#define T(name, sz, flags) [flags] = cn20k_nix_xmit_pkts_vec_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || dev->tx_mark) { + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + } else { + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); + if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_vec_burst_mseg); + } +#else + RTE_SET_USED(eth_dev); +#endif +} + +static void +cn20k_eth_set_tx_blk_func(struct rte_eth_dev *eth_dev) +{ +#if defined(CNXK_DIS_TMPLT_FUNC) + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (dev->scalar_ena || dev->tx_mark) + eth_dev->tx_pkt_burst = cn20k_nix_xmit_pkts_all_offload; + else + eth_dev->tx_pkt_burst = cn20k_nix_xmit_pkts_vec_all_offload; + + if (eth_dev->data->dev_started) + rte_eth_fp_ops[eth_dev->data->port_id].tx_pkt_burst = eth_dev->tx_pkt_burst; +#else + RTE_SET_USED(eth_dev); +#endif +} +#endif + +void +cn20k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ +#if defined(RTE_ARCH_ARM64) + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + cn20k_eth_set_tx_blk_func(eth_dev); + cn20k_eth_set_tx_tmplt_func(eth_dev); + + if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_SECURITY) + eth_dev->tx_queue_count = cn20k_nix_tx_queue_sec_count; + else + eth_dev->tx_queue_count = cn20k_nix_tx_queue_count; + + rte_atomic_thread_fence(rte_memory_order_release); +#else + RTE_SET_USED(eth_dev); +#endif +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index f41238be9c..fcf48f600a 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -237,6 +237,7 @@ if soc_type == 'cn20k' or soc_type == 'all' sources += files( 'cn20k_ethdev.c', 'cn20k_rx_select.c', + 'cn20k_tx_select.c', ) if host_machine.cpu_family().startswith('aarch') and not disable_template @@ -276,9 +277,45 @@ sources += files( 'rx/cn20k/rx_all_offload.c', ) +sources += files( + 'tx/cn20k/tx_0_15.c', + 'tx/cn20k/tx_16_31.c', + 'tx/cn20k/tx_32_47.c', + 'tx/cn20k/tx_48_63.c', + 'tx/cn20k/tx_64_79.c', + 'tx/cn20k/tx_80_95.c', + 'tx/cn20k/tx_96_111.c', + 'tx/cn20k/tx_112_127.c', + 'tx/cn20k/tx_0_15_mseg.c', + 'tx/cn20k/tx_16_31_mseg.c', + 'tx/cn20k/tx_32_47_mseg.c', + 'tx/cn20k/tx_48_63_mseg.c', + 'tx/cn20k/tx_64_79_mseg.c', + 'tx/cn20k/tx_80_95_mseg.c', + 'tx/cn20k/tx_96_111_mseg.c', + 'tx/cn20k/tx_112_127_mseg.c', + 'tx/cn20k/tx_0_15_vec.c', + 'tx/cn20k/tx_16_31_vec.c', + 'tx/cn20k/tx_32_47_vec.c', + 'tx/cn20k/tx_48_63_vec.c', + 'tx/cn20k/tx_64_79_vec.c', + 'tx/cn20k/tx_80_95_vec.c', + 'tx/cn20k/tx_96_111_vec.c', + 'tx/cn20k/tx_112_127_vec.c', + 'tx/cn20k/tx_0_15_vec_mseg.c', + 'tx/cn20k/tx_16_31_vec_mseg.c', + 'tx/cn20k/tx_32_47_vec_mseg.c', + 'tx/cn20k/tx_48_63_vec_mseg.c', + 'tx/cn20k/tx_64_79_vec_mseg.c', + 'tx/cn20k/tx_80_95_vec_mseg.c', + 'tx/cn20k/tx_96_111_vec_mseg.c', + 'tx/cn20k/tx_112_127_vec_mseg.c', + 'tx/cn20k/tx_all_offload.c', +) else sources += files( 'rx/cn20k/rx_all_offload.c', + 'tx/cn20k/tx_all_offload.c', ) endif endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_0_15.c b/drivers/net/cnxk/tx/cn20k/tx_0_15.c new file mode 100644 index 0000000000..2de434ccb4 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_0_15.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_0_15_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_0_15_mseg.c new file mode 100644 index 0000000000..c928902b02 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_0_15_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_0_15_vec.c b/drivers/net/cnxk/tx/cn20k/tx_0_15_vec.c new file mode 100644 index 0000000000..0e82451c7e --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_0_15_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_0_15_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_0_15_vec_mseg.c new file mode 100644 index 0000000000..b0cd33f781 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_0_15_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_0_15 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_112_127.c b/drivers/net/cnxk/tx/cn20k/tx_112_127.c new file mode 100644 index 0000000000..c116c48763 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_112_127.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_112_127_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_112_127_mseg.c new file mode 100644 index 0000000000..5d67426f2b --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_112_127_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_112_127_vec.c b/drivers/net/cnxk/tx/cn20k/tx_112_127_vec.c new file mode 100644 index 0000000000..5a3e5c660d --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_112_127_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_112_127_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_112_127_vec_mseg.c new file mode 100644 index 0000000000..c6918de6df --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_112_127_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_112_127 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_16_31.c b/drivers/net/cnxk/tx/cn20k/tx_16_31.c new file mode 100644 index 0000000000..953f63b192 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_16_31.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_16_31_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_16_31_mseg.c new file mode 100644 index 0000000000..cdfd6bf69c --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_16_31_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_16_31_vec.c b/drivers/net/cnxk/tx/cn20k/tx_16_31_vec.c new file mode 100644 index 0000000000..6e6ad7c968 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_16_31_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_16_31_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_16_31_vec_mseg.c new file mode 100644 index 0000000000..a3a0fcace3 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_16_31_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_16_31 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_32_47.c b/drivers/net/cnxk/tx/cn20k/tx_32_47.c new file mode 100644 index 0000000000..50295fcd16 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_32_47.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_32_47_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_32_47_mseg.c new file mode 100644 index 0000000000..8b4da505ad --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_32_47_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_32_47_vec.c b/drivers/net/cnxk/tx/cn20k/tx_32_47_vec.c new file mode 100644 index 0000000000..3a3298ffa6 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_32_47_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_32_47_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_32_47_vec_mseg.c new file mode 100644 index 0000000000..93168990a8 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_32_47_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_32_47 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_48_63.c b/drivers/net/cnxk/tx/cn20k/tx_48_63.c new file mode 100644 index 0000000000..5765b1fe57 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_48_63.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_48_63_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_48_63_mseg.c new file mode 100644 index 0000000000..5f591eee68 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_48_63_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_48_63_vec.c b/drivers/net/cnxk/tx/cn20k/tx_48_63_vec.c new file mode 100644 index 0000000000..06eec15976 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_48_63_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_48_63_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_48_63_vec_mseg.c new file mode 100644 index 0000000000..220f117c47 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_48_63_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_48_63 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_64_79.c b/drivers/net/cnxk/tx/cn20k/tx_64_79.c new file mode 100644 index 0000000000..c05ef2a238 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_64_79.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_64_79_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_64_79_mseg.c new file mode 100644 index 0000000000..79d40a09ed --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_64_79_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_64_79_vec.c b/drivers/net/cnxk/tx/cn20k/tx_64_79_vec.c new file mode 100644 index 0000000000..a4fac7e73e --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_64_79_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_64_79_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_64_79_vec_mseg.c new file mode 100644 index 0000000000..90d6b4f2f9 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_64_79_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_64_79 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_80_95.c b/drivers/net/cnxk/tx/cn20k/tx_80_95.c new file mode 100644 index 0000000000..8a09ff842b --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_80_95.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_80_95_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_80_95_mseg.c new file mode 100644 index 0000000000..59f959b29f --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_80_95_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_80_95_vec.c b/drivers/net/cnxk/tx/cn20k/tx_80_95_vec.c new file mode 100644 index 0000000000..ca78d42344 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_80_95_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_80_95_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_80_95_vec_mseg.c new file mode 100644 index 0000000000..a3a9856783 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_80_95_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_80_95 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_96_111.c b/drivers/net/cnxk/tx/cn20k/tx_96_111.c new file mode 100644 index 0000000000..fab39f8fcc --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_96_111.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT(cn20k_nix_xmit_pkts_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_96_111_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_96_111_mseg.c new file mode 100644 index 0000000000..11b6814223 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_96_111_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_MSEG(cn20k_nix_xmit_pkts_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_96_111_vec.c b/drivers/net/cnxk/tx/cn20k/tx_96_111_vec.c new file mode 100644 index 0000000000..e1e3b1bca3 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_96_111_vec.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC(cn20k_nix_xmit_pkts_vec_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_96_111_vec_mseg.c b/drivers/net/cnxk/tx/cn20k/tx_96_111_vec_mseg.c new file mode 100644 index 0000000000..b6af4e34c0 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_96_111_vec_mseg.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if !defined(CNXK_DIS_TMPLT_FUNC) + +#define T(name, sz, flags) NIX_TX_XMIT_VEC_MSEG(cn20k_nix_xmit_pkts_vec_mseg_##name, sz, flags) + +NIX_TX_FASTPATH_MODES_96_111 +#undef T + +#endif diff --git a/drivers/net/cnxk/tx/cn20k/tx_all_offload.c b/drivers/net/cnxk/tx/cn20k/tx_all_offload.c new file mode 100644 index 0000000000..c7258b5df7 --- /dev/null +++ b/drivers/net/cnxk/tx/cn20k/tx_all_offload.c @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2024 Marvell. + */ + +#include "cn20k_tx.h" + +#ifdef _ROC_API_H_ +#error "roc_api.h is included" +#endif + +#if defined(CNXK_DIS_TMPLT_FUNC) + +uint16_t __rte_noinline __rte_hot +cn20k_nix_xmit_pkts_all_offload(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) +{ + uint64_t cmd[8 + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; + + return cn20k_nix_xmit_pkts_mseg( + tx_queue, NULL, tx_pkts, pkts, cmd, + NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_MBUF_NOFF_F | + NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_TSTAMP_F | NIX_TX_OFFLOAD_SECURITY_F | + NIX_TX_MULTI_SEG_F); +} + +uint16_t __rte_noinline __rte_hot +cn20k_nix_xmit_pkts_vec_all_offload(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) +{ + uint64_t cmd[8 + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; + + return cn20k_nix_xmit_pkts_vector( + tx_queue, NULL, tx_pkts, pkts, cmd, + NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_MBUF_NOFF_F | + NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_TSTAMP_F | NIX_TX_OFFLOAD_SECURITY_F | + NIX_TX_MULTI_SEG_F); +} + +#endif -- 2.34.1