From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 664DCA04A2; Sun, 23 Jan 2022 13:00:07 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 59BAC41199; Sun, 23 Jan 2022 13:00:07 +0100 (CET) Received: from mail-io1-f43.google.com (mail-io1-f43.google.com [209.85.166.43]) by mails.dpdk.org (Postfix) with ESMTP id 2ED5240040 for ; Sun, 23 Jan 2022 13:00:06 +0100 (CET) Received: by mail-io1-f43.google.com with SMTP id 9so612121iou.2 for ; Sun, 23 Jan 2022 04:00:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1WkRDKFYoKt5AGUxeAvISdWX8R5PxErJVeUTPzb/Oao=; b=fjZtHtbwNjbgOTSZ01+JirdSkxRbdnT2N8/d+bpGtDMQDCXgTP8/hy72Yo+EJuaOK/ ShYTdEFOFCQ7OG/hNqHxZGohhO1bjwvAz5g6W0hxcFOfPwEoSlVoMW2Y6z4DDtQsrNpV OJcCSLvGgj5VjViMfs4lmiDCjqcr3UTh6wo7rg5JJgAmHvKCfEVZA+LdCJ4UBdZMRPrD Y7etXMdMbO7fySMisjfIvBcaX1sI0Jwqk2rj849PPFNJAm7HwEg0AnqTYAIpWRKQPWLB kvw+XkzfzHJsPV+MGgYXRJtxQBN5q1u7hToBefnKpSKNIvNh13Sz1nkaQ9gJldo3+IUn zqPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1WkRDKFYoKt5AGUxeAvISdWX8R5PxErJVeUTPzb/Oao=; b=sgoTMG0HXIL3nnQCLv2HLvR+yPm3lUS8opP+jglBtyZ0ocTNxLr0uJKFzhWH0KWpkA EPBs1CtZrQpdSSo0VACbbf/BM9QubJ9a6gZXEHpgJTJxeh8sWXbqf/h3IbhZGpVEK3Ac vlBtm2uHdXBsvPi7ivSekpZq9vm5PNKZHtCslO2U1x8b9+iejsQzgu7XqS0AJsf5b3PA FYDfTinCIFnOBU7OjAD66RrOGNyUHivX5LFHLYz5EbGUG8J1ihURmbcCIOxsTNUhVqOF JlpJziMaC2AC/2Bn73ZmH/cmkn0BZY5OiAqqdPzoNxGrLH7yj/ILO3InMKt4FEaglaDA nQxw== X-Gm-Message-State: AOAM533aZiCa4jl2GgEoVs7W7xBUvyKtvblyqo+dFJOU1U9iQK3tNNv2 e9mC87BA8q3Rz1MUgdDX59PMuxdEkmALIjMJQDlVZpuklz4= X-Google-Smtp-Source: ABdhPJyPJb+dkXhLJ4Cegvf8jVwXQS/K3BbkUdF+21opkRr5TF45M9g9d+iYvGe/Csb/btK8RQEW/MN2Gce9+ZfUXSI= X-Received: by 2002:a5d:9306:: with SMTP id l6mr5801980ion.154.1642939204943; Sun, 23 Jan 2022 04:00:04 -0800 (PST) MIME-Version: 1.0 References: <20220120145136.7844-1-pbhagavatula@marvell.com> <20220120145136.7844-8-pbhagavatula@marvell.com> In-Reply-To: <20220120145136.7844-8-pbhagavatula@marvell.com> From: Jerin Jacob Date: Sun, 23 Jan 2022 17:29:38 +0530 Message-ID: Subject: Re: [PATCH v2 8/8] net/cnxk: add CN10K template Tx functions to build To: Pavan Nikhilesh , Ferruh Yigit Cc: Jerin Jacob , Shijith Thotton , Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , dpdk-dev Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Jan 20, 2022 at 8:23 PM wrote: > > From: Pavan Nikhilesh > > Add CN10K segregated Tx and event Tx template functions to build, > add macros to make future modifications simpler. > > Signed-off-by: Pavan Nikhilesh Series Acked-by: Jerin Jacob Tested and captured build time improvement at [3]. Series applied to dpdk-next-net-mrvl/for-next-net with following changes[2]. Thanks. Hi Ferruh. Noticed ICC failure with this patch[1]. Not sure is this a CI issue or not? I don't have set up with ICC. If it some ICC compiler issue then it is OK to remove ICC build support for cnxk drivers. [1] http://mails.dpdk.org/archives/test-report/2022-January/253920.html [2] - Changed CN9K/CN10K to cn9k/cn10k in commit messages - Removed extra line from meson.build. [main]dell[dpdk.org] $ git diff diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 375c75d1c7..cd8c13bd1c 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -181,7 +181,6 @@ sources += files( 'tx/cn10k/tx_112_127_vec_mseg.c', ) - deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] [3] Test result: only lib: ccache -C && rm -rf build && meson --cross-file config/arm/arm64_cn10k_linux_gcc build && time ninja -C build Without Fix: real 3m30.233s user 33m7.904s sys 1m53.748 With fix: real 1m24.363s user 52m9.708s sys 2m57.342s lib + examples: ccache -C && rm -rf build && meson --cross-file config/arm/arm64_cn10k_linux_gcc -Dexamples=all build && time ninja -C build without fix: real 3m31.282s user 35m45.109s sys 2m19.244s with fix: real 1m27.245s user 55m7.614s sys 3m28.513s > --- > drivers/event/cnxk/cn10k_eventdev.c | 19 +- > drivers/event/cnxk/cn10k_worker.h | 36 +- > drivers/event/cnxk/cn10k_worker_tx_enq.c | 23 - > drivers/event/cnxk/cn10k_worker_tx_enq_seg.c | 23 - > drivers/event/cnxk/meson.build | 21 +- > drivers/net/cnxk/cn10k_tx.c | 90 ---- > drivers/net/cnxk/cn10k_tx.h | 522 +++++++++---------- > drivers/net/cnxk/cn10k_tx_mseg.c | 26 - > drivers/net/cnxk/cn10k_tx_select.c | 63 +++ > drivers/net/cnxk/cn10k_tx_vec.c | 25 - > drivers/net/cnxk/cn10k_tx_vec_mseg.c | 24 - > drivers/net/cnxk/meson.build | 41 +- > 12 files changed, 413 insertions(+), 500 deletions(-) > delete mode 100644 drivers/event/cnxk/cn10k_worker_tx_enq.c > delete mode 100644 drivers/event/cnxk/cn10k_worker_tx_enq_seg.c > delete mode 100644 drivers/net/cnxk/cn10k_tx.c > delete mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c > create mode 100644 drivers/net/cnxk/cn10k_tx_select.c > delete mode 100644 drivers/net/cnxk/cn10k_tx_vec.c > delete mode 100644 drivers/net/cnxk/cn10k_tx_vec_mseg.c > > diff --git a/drivers/event/cnxk/cn10k_eventdev.c b/drivers/event/cnxk/cn10k_eventdev.c > index 02f3d8235d..b56426960a 100644 > --- a/drivers/event/cnxk/cn10k_eventdev.c > +++ b/drivers/event/cnxk/cn10k_eventdev.c > @@ -10,14 +10,7 @@ > deq_op = deq_ops[dev->rx_offloads & (NIX_RX_OFFLOAD_MAX - 1)] > > #define CN10K_SET_EVDEV_ENQ_OP(dev, enq_op, enq_ops) \ > - (enq_op = \ > - enq_ops[!!(dev->tx_offloads & NIX_TX_OFFLOAD_SECURITY_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSTAMP_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_TSO_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_MBUF_NOFF_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_VLAN_QINQ_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] \ > - [!!(dev->tx_offloads & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]) > + enq_op = enq_ops[dev->tx_offloads & (NIX_TX_OFFLOAD_MAX - 1)] > > static uint32_t > cn10k_sso_gw_mode_wdata(struct cnxk_sso_evdev *dev) > @@ -390,17 +383,15 @@ cn10k_sso_fp_fns_set(struct rte_eventdev *event_dev) > > /* Tx modes */ > const event_tx_adapter_enqueue_t > - sso_hws_tx_adptr_enq[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_sso_hws_tx_adptr_enq_##name, > + sso_hws_tx_adptr_enq[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_sso_hws_tx_adptr_enq_##name, > NIX_TX_FASTPATH_MODES > #undef T > }; > > const event_tx_adapter_enqueue_t > - sso_hws_tx_adptr_enq_seg[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_sso_hws_tx_adptr_enq_seg_##name, > + sso_hws_tx_adptr_enq_seg[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_sso_hws_tx_adptr_enq_seg_##name, > NIX_TX_FASTPATH_MODES > #undef T > }; > diff --git a/drivers/event/cnxk/cn10k_worker.h b/drivers/event/cnxk/cn10k_worker.h > index 160b90aa27..78d029baaa 100644 > --- a/drivers/event/cnxk/cn10k_worker.h > +++ b/drivers/event/cnxk/cn10k_worker.h > @@ -613,17 +613,43 @@ cn10k_sso_hws_event_tx(struct cn10k_sso_hws *ws, struct rte_event *ev, > return 1; > } > > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > +#define T(name, sz, flags) \ > uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_##name( \ > void *port, struct rte_event ev[], uint16_t nb_events); \ > uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_seg_##name( \ > - void *port, struct rte_event ev[], uint16_t nb_events); \ > - uint16_t __rte_hot cn10k_sso_hws_dual_tx_adptr_enq_##name( \ > - void *port, struct rte_event ev[], uint16_t nb_events); \ > - uint16_t __rte_hot cn10k_sso_hws_dual_tx_adptr_enq_seg_##name( \ > void *port, struct rte_event ev[], uint16_t nb_events); > > NIX_TX_FASTPATH_MODES > #undef T > > +#define SSO_TX(fn, sz, flags) \ > + uint16_t __rte_hot fn(void *port, struct rte_event ev[], \ > + uint16_t nb_events) \ > + { \ > + struct cn10k_sso_hws *ws = port; \ > + uint64_t cmd[sz]; \ > + \ > + RTE_SET_USED(nb_events); \ > + return cn10k_sso_hws_event_tx( \ > + ws, &ev[0], cmd, \ > + (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ > + ws->tx_adptr_data, \ > + flags); \ > + } > + > +#define SSO_TX_SEG(fn, sz, flags) \ > + uint16_t __rte_hot fn(void *port, struct rte_event ev[], \ > + uint16_t nb_events) \ > + { \ > + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ > + struct cn10k_sso_hws *ws = port; \ > + \ > + RTE_SET_USED(nb_events); \ > + return cn10k_sso_hws_event_tx( \ > + ws, &ev[0], cmd, \ > + (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ > + ws->tx_adptr_data, \ > + (flags) | NIX_TX_MULTI_SEG_F); \ > + } > + > #endif > diff --git a/drivers/event/cnxk/cn10k_worker_tx_enq.c b/drivers/event/cnxk/cn10k_worker_tx_enq.c > deleted file mode 100644 > index f14c7fc223..0000000000 > --- a/drivers/event/cnxk/cn10k_worker_tx_enq.c > +++ /dev/null > @@ -1,23 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_worker.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_##name( \ > - void *port, struct rte_event ev[], uint16_t nb_events) \ > - { \ > - struct cn10k_sso_hws *ws = port; \ > - uint64_t cmd[sz]; \ > - \ > - RTE_SET_USED(nb_events); \ > - return cn10k_sso_hws_event_tx( \ > - ws, &ev[0], cmd, \ > - (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ > - ws->tx_adptr_data, \ > - flags); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > diff --git a/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c b/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c > deleted file mode 100644 > index 2ea61e5d88..0000000000 > --- a/drivers/event/cnxk/cn10k_worker_tx_enq_seg.c > +++ /dev/null > @@ -1,23 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_worker.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_hot cn10k_sso_hws_tx_adptr_enq_seg_##name( \ > - void *port, struct rte_event ev[], uint16_t nb_events) \ > - { \ > - uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ > - struct cn10k_sso_hws *ws = port; \ > - \ > - RTE_SET_USED(nb_events); \ > - return cn10k_sso_hws_event_tx( \ > - ws, &ev[0], cmd, \ > - (const uint64_t(*)[RTE_MAX_QUEUES_PER_PORT]) & \ > - ws->tx_adptr_data, \ > - (flags) | NIX_TX_MULTI_SEG_F); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > diff --git a/drivers/event/cnxk/meson.build b/drivers/event/cnxk/meson.build > index ac452fe451..b27bae7b12 100644 > --- a/drivers/event/cnxk/meson.build > +++ b/drivers/event/cnxk/meson.build > @@ -13,8 +13,6 @@ sources = files( > 'cn9k_worker.c', > 'cn10k_eventdev.c', > 'cn10k_worker.c', > - 'cn10k_worker_tx_enq.c', > - 'cn10k_worker_tx_enq_seg.c', > 'cnxk_eventdev.c', > 'cnxk_eventdev_adptr.c', > 'cnxk_eventdev_selftest.c', > @@ -454,6 +452,25 @@ sources += files( > 'deq/cn10k/deq_112_127_ca_tmo_seg_burst.c', > ) > > +sources += files( > + 'tx/cn10k/tx_0_15.c', > + 'tx/cn10k/tx_16_31.c', > + 'tx/cn10k/tx_32_47.c', > + 'tx/cn10k/tx_48_63.c', > + 'tx/cn10k/tx_64_79.c', > + 'tx/cn10k/tx_80_95.c', > + 'tx/cn10k/tx_96_111.c', > + 'tx/cn10k/tx_112_127.c', > + 'tx/cn10k/tx_0_15_seg.c', > + 'tx/cn10k/tx_16_31_seg.c', > + 'tx/cn10k/tx_32_47_seg.c', > + 'tx/cn10k/tx_48_63_seg.c', > + 'tx/cn10k/tx_64_79_seg.c', > + 'tx/cn10k/tx_80_95_seg.c', > + 'tx/cn10k/tx_96_111_seg.c', > + 'tx/cn10k/tx_112_127_seg.c', > +) > + > extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] > foreach flag: extra_flags > if cc.has_argument(flag) > diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c > deleted file mode 100644 > index 5e6c5ee111..0000000000 > --- a/drivers/net/cnxk/cn10k_tx.c > +++ /dev/null > @@ -1,90 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_ethdev.h" > -#include "cn10k_tx.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ > - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > - { \ > - uint64_t cmd[sz]; \ > - \ > - /* For TSO inner checksum is a must */ \ > - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > - return 0; \ > - return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ > - 0, flags); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > - > -static inline void > -pick_tx_func(struct rte_eth_dev *eth_dev, > - const eth_tx_burst_t tx_burst[2][2][2][2][2][2][2]) > -{ > - struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > - > - /* [SEC] [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ > - eth_dev->tx_pkt_burst = tx_burst > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_SECURITY_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] > - [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; > -} > - > -void > -cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) > -{ > - struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > - > - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, > - > - NIX_TX_FASTPATH_MODES > -#undef T > - }; > - > - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, > - > - NIX_TX_FASTPATH_MODES > -#undef T > - }; > - > - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, > - > - NIX_TX_FASTPATH_MODES > -#undef T > - }; > - > - const eth_tx_burst_t nix_eth_tx_vec_burst_mseg[2][2][2][2][2][2][2] = { > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - [f6][f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_mseg_##name, > - > - NIX_TX_FASTPATH_MODES > -#undef T > - }; > - > - if (dev->scalar_ena) { > - pick_tx_func(eth_dev, nix_eth_tx_burst); > - if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) > - pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); > - } else { > - pick_tx_func(eth_dev, nix_eth_tx_vec_burst); > - if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) > - pick_tx_func(eth_dev, nix_eth_tx_vec_burst_mseg); > - } > - > - rte_mb(); > -} > diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h > index f3a282f429..2c9411f42c 100644 > --- a/drivers/net/cnxk/cn10k_tx.h > +++ b/drivers/net/cnxk/cn10k_tx.h > @@ -16,6 +16,7 @@ > #define NIX_TX_OFFLOAD_TSO_F BIT(4) > #define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) > #define NIX_TX_OFFLOAD_SECURITY_F BIT(6) > +#define NIX_TX_OFFLOAD_MAX (NIX_TX_OFFLOAD_SECURITY_F << 1) > > /* Flags to control xmit_prepare function. > * Defining it from backwards to denote its been > @@ -2675,279 +2676,272 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, > #define T_SEC_F NIX_TX_OFFLOAD_SECURITY_F > > /* [T_SEC_F] [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ > -#define NIX_TX_FASTPATH_MODES \ > -T(no_offload, 0, 0, 0, 0, 0, 0, 0, 4, \ > - NIX_TX_OFFLOAD_NONE) \ > -T(l3l4csum, 0, 0, 0, 0, 0, 0, 1, 4, \ > - L3L4CSUM_F) \ > -T(ol3ol4csum, 0, 0, 0, 0, 0, 1, 0, 4, \ > - OL3OL4CSUM_F) \ > -T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 0, 1, 1, 4, \ > - OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(vlan, 0, 0, 0, 0, 1, 0, 0, 6, \ > - VLAN_F) \ > -T(vlan_l3l4csum, 0, 0, 0, 0, 1, 0, 1, 6, \ > - VLAN_F | L3L4CSUM_F) \ > -T(vlan_ol3ol4csum, 0, 0, 0, 0, 1, 1, 0, 6, \ > - VLAN_F | OL3OL4CSUM_F) \ > -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 1, 6, \ > - VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(noff, 0, 0, 0, 1, 0, 0, 0, 4, \ > - NOFF_F) \ > -T(noff_l3l4csum, 0, 0, 0, 1, 0, 0, 1, 4, \ > - NOFF_F | L3L4CSUM_F) \ > -T(noff_ol3ol4csum, 0, 0, 0, 1, 0, 1, 0, 4, \ > - NOFF_F | OL3OL4CSUM_F) \ > -T(noff_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 0, 1, 1, 4, \ > - NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(noff_vlan, 0, 0, 0, 1, 1, 0, 0, 6, \ > - NOFF_F | VLAN_F) \ > -T(noff_vlan_l3l4csum, 0, 0, 0, 1, 1, 0, 1, 6, \ > - NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(noff_vlan_ol3ol4csum, 0, 0, 0, 1, 1, 1, 0, 6, \ > - NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 1, 6, \ > - NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(tso, 0, 0, 1, 0, 0, 0, 0, 6, \ > - TSO_F) \ > -T(tso_l3l4csum, 0, 0, 1, 0, 0, 0, 1, 6, \ > - TSO_F | L3L4CSUM_F) \ > -T(tso_ol3ol4csum, 0, 0, 1, 0, 0, 1, 0, 6, \ > - TSO_F | OL3OL4CSUM_F) \ > -T(tso_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 0, 1, 1, 6, \ > - TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(tso_vlan, 0, 0, 1, 0, 1, 0, 0, 6, \ > - TSO_F | VLAN_F) \ > -T(tso_vlan_l3l4csum, 0, 0, 1, 0, 1, 0, 1, 6, \ > - TSO_F | VLAN_F | L3L4CSUM_F) \ > -T(tso_vlan_ol3ol4csum, 0, 0, 1, 0, 1, 1, 0, 6, \ > - TSO_F | VLAN_F | OL3OL4CSUM_F) \ > -T(tso_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 1, 6, \ > - TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(tso_noff, 0, 0, 1, 1, 0, 0, 0, 6, \ > - TSO_F | NOFF_F) \ > -T(tso_noff_l3l4csum, 0, 0, 1, 1, 0, 0, 1, 6, \ > - TSO_F | NOFF_F | L3L4CSUM_F) \ > -T(tso_noff_ol3ol4csum, 0, 0, 1, 1, 0, 1, 0, 6, \ > - TSO_F | NOFF_F | OL3OL4CSUM_F) \ > -T(tso_noff_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 0, 1, 1, 6, \ > - TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(tso_noff_vlan, 0, 0, 1, 1, 1, 0, 0, 6, \ > - TSO_F | NOFF_F | VLAN_F) \ > -T(tso_noff_vlan_l3l4csum, 0, 0, 1, 1, 1, 0, 1, 6, \ > - TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(tso_noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 1, 0, 6, \ > - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 1, 6, \ > - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts, 0, 1, 0, 0, 0, 0, 0, 8, \ > - TSP_F) \ > -T(ts_l3l4csum, 0, 1, 0, 0, 0, 0, 1, 8, \ > - TSP_F | L3L4CSUM_F) \ > -T(ts_ol3ol4csum, 0, 1, 0, 0, 0, 1, 0, 8, \ > - TSP_F | OL3OL4CSUM_F) \ > -T(ts_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 0, 1, 1, 8, \ > - TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_vlan, 0, 1, 0, 0, 1, 0, 0, 8, \ > - TSP_F | VLAN_F) \ > -T(ts_vlan_l3l4csum, 0, 1, 0, 0, 1, 0, 1, 8, \ > - TSP_F | VLAN_F | L3L4CSUM_F) \ > -T(ts_vlan_ol3ol4csum, 0, 1, 0, 0, 1, 1, 0, 8, \ > - TSP_F | VLAN_F | OL3OL4CSUM_F) \ > -T(ts_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 1, 8, \ > - TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_noff, 0, 1, 0, 1, 0, 0, 0, 8, \ > - TSP_F | NOFF_F) \ > -T(ts_noff_l3l4csum, 0, 1, 0, 1, 0, 0, 1, 8, \ > - TSP_F | NOFF_F | L3L4CSUM_F) \ > -T(ts_noff_ol3ol4csum, 0, 1, 0, 1, 0, 1, 0, 8, \ > - TSP_F | NOFF_F | OL3OL4CSUM_F) \ > -T(ts_noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 0, 1, 1, 8, \ > - TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_noff_vlan, 0, 1, 0, 1, 1, 0, 0, 8, \ > - TSP_F | NOFF_F | VLAN_F) \ > -T(ts_noff_vlan_l3l4csum, 0, 1, 0, 1, 1, 0, 1, 8, \ > - TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(ts_noff_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 1, 0, 8, \ > - TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(ts_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 1, 8, \ > - TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_tso, 0, 1, 1, 0, 0, 0, 0, 8, \ > - TSP_F | TSO_F) \ > -T(ts_tso_l3l4csum, 0, 1, 1, 0, 0, 0, 1, 8, \ > - TSP_F | TSO_F | L3L4CSUM_F) \ > -T(ts_tso_ol3ol4csum, 0, 1, 1, 0, 0, 1, 0, 8, \ > - TSP_F | TSO_F | OL3OL4CSUM_F) \ > -T(ts_tso_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 0, 1, 1, 8, \ > - TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_tso_vlan, 0, 1, 1, 0, 1, 0, 0, 8, \ > - TSP_F | TSO_F | VLAN_F) \ > -T(ts_tso_vlan_l3l4csum, 0, 1, 1, 0, 1, 0, 1, 8, \ > - TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > -T(ts_tso_vlan_ol3ol4csum, 0, 1, 1, 0, 1, 1, 0, 8, \ > - TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > -T(ts_tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 1, 8, \ > - TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_tso_noff, 0, 1, 1, 1, 0, 0, 0, 8, \ > - TSP_F | TSO_F | NOFF_F) \ > -T(ts_tso_noff_l3l4csum, 0, 1, 1, 1, 0, 0, 1, 8, \ > - TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > -T(ts_tso_noff_ol3ol4csum, 0, 1, 1, 1, 0, 1, 0, 8, \ > - TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > -T(ts_tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 0, 1, 1, 8, \ > - TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(ts_tso_noff_vlan, 0, 1, 1, 1, 1, 0, 0, 8, \ > - TSP_F | TSO_F | NOFF_F | VLAN_F) \ > -T(ts_tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 1, 0, 1, 8, \ > - TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(ts_tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 1, 0, 8, \ > - TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 1, 8, \ > - TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ > -T(sec, 1, 0, 0, 0, 0, 0, 0, 4, \ > - T_SEC_F) \ > -T(sec_l3l4csum, 1, 0, 0, 0, 0, 0, 1, 4, \ > - T_SEC_F | L3L4CSUM_F) \ > -T(sec_ol3ol4csum, 1, 0, 0, 0, 0, 1, 0, 4, \ > - T_SEC_F | OL3OL4CSUM_F) \ > -T(sec_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 0, 1, 1, 4, \ > - T_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_vlan, 1, 0, 0, 0, 1, 0, 0, 6, \ > - T_SEC_F | VLAN_F) \ > -T(sec_vlan_l3l4csum, 1, 0, 0, 0, 1, 0, 1, 6, \ > - T_SEC_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_vlan_ol3ol4csum, 1, 0, 0, 0, 1, 1, 0, 6, \ > - T_SEC_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 1, 6, \ > - T_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_noff, 1, 0, 0, 1, 0, 0, 0, 4, \ > - T_SEC_F | NOFF_F) \ > -T(sec_noff_l3l4csum, 1, 0, 0, 1, 0, 0, 1, 4, \ > - T_SEC_F | NOFF_F | L3L4CSUM_F) \ > -T(sec_noff_ol3ol4csum, 1, 0, 0, 1, 0, 1, 0, 4, \ > - T_SEC_F | NOFF_F | OL3OL4CSUM_F) \ > -T(sec_noff_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 0, 1, 1, 4, \ > - T_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_noff_vlan, 1, 0, 0, 1, 1, 0, 0, 6, \ > - T_SEC_F | NOFF_F | VLAN_F) \ > -T(sec_noff_vlan_l3l4csum, 1, 0, 0, 1, 1, 0, 1, 6, \ > - T_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_noff_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 1, 0, 6, \ > - T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 1, 6, \ > - T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_tso, 1, 0, 1, 0, 0, 0, 0, 6, \ > - T_SEC_F | TSO_F) \ > -T(sec_tso_l3l4csum, 1, 0, 1, 0, 0, 0, 1, 6, \ > - T_SEC_F | TSO_F | L3L4CSUM_F) \ > -T(sec_tso_ol3ol4csum, 1, 0, 1, 0, 0, 1, 0, 6, \ > - T_SEC_F | TSO_F | OL3OL4CSUM_F) \ > -T(sec_tso_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 0, 1, 1, 6, \ > - T_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_tso_vlan, 1, 0, 1, 0, 1, 0, 0, 6, \ > - T_SEC_F | TSO_F | VLAN_F) \ > -T(sec_tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 0, 1, 6, \ > - T_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_tso_vlan_ol3ol4csum, 1, 0, 1, 0, 1, 1, 0, 6, \ > - T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 1, 6, \ > - T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_tso_noff, 1, 0, 1, 1, 0, 0, 0, 6, \ > - T_SEC_F | TSO_F | NOFF_F) \ > -T(sec_tso_noff_l3l4csum, 1, 0, 1, 1, 0, 0, 1, 6, \ > - T_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > -T(sec_tso_noff_ol3ol4csum, 1, 0, 1, 1, 0, 1, 0, 6, \ > - T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > -T(sec_tso_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 0, 1, 1, 6, \ > - T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_tso_noff_vlan, 1, 0, 1, 1, 1, 0, 0, 6, \ > - T_SEC_F | TSO_F | NOFF_F | VLAN_F) \ > -T(sec_tso_noff_vlan_l3l4csum, 1, 0, 1, 1, 1, 0, 1, 6, \ > - T_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_tso_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 1, 0, 6, \ > - T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 1, 6, \ > - T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ > -T(sec_ts, 1, 1, 0, 0, 0, 0, 0, 8, \ > - T_SEC_F | TSP_F) \ > -T(sec_ts_l3l4csum, 1, 1, 0, 0, 0, 0, 1, 8, \ > - T_SEC_F | TSP_F | L3L4CSUM_F) \ > -T(sec_ts_ol3ol4csum, 1, 1, 0, 0, 0, 1, 0, 8, \ > - T_SEC_F | TSP_F | OL3OL4CSUM_F) \ > -T(sec_ts_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 0, 1, 1, 8, \ > - T_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_ts_vlan, 1, 1, 0, 0, 1, 0, 0, 8, \ > - T_SEC_F | TSP_F | VLAN_F) \ > -T(sec_ts_vlan_l3l4csum, 1, 1, 0, 0, 1, 0, 1, 8, \ > - T_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_ts_vlan_ol3ol4csum, 1, 1, 0, 0, 1, 1, 0, 8, \ > - T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_ts_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 1, 8, \ > - T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_ts_noff, 1, 1, 0, 1, 0, 0, 0, 8, \ > - T_SEC_F | TSP_F | NOFF_F) \ > -T(sec_ts_noff_l3l4csum, 1, 1, 0, 1, 0, 0, 1, 8, \ > - T_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ > -T(sec_ts_noff_ol3ol4csum, 1, 1, 0, 1, 0, 1, 0, 8, \ > - T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ > -T(sec_ts_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 0, 1, 1, 8, \ > - T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_ts_noff_vlan, 1, 1, 0, 1, 1, 0, 0, 8, \ > - T_SEC_F | TSP_F | NOFF_F | VLAN_F) \ > -T(sec_ts_noff_vlan_l3l4csum, 1, 1, 0, 1, 1, 0, 1, 8, \ > - T_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_ts_noff_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 1, 0, 8, \ > - T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 1, 8, \ > - T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F)\ > -T(sec_ts_tso, 1, 1, 1, 0, 0, 0, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F) \ > -T(sec_ts_tso_l3l4csum, 1, 1, 1, 0, 0, 0, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | L3L4CSUM_F) \ > -T(sec_ts_tso_ol3ol4csum, 1, 1, 1, 0, 0, 1, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F) \ > -T(sec_ts_tso_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 0, 1, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_ts_tso_vlan, 1, 1, 1, 0, 1, 0, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | VLAN_F) \ > -T(sec_ts_tso_vlan_l3l4csum, 1, 1, 1, 0, 1, 0, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_ts_tso_vlan_ol3ol4csum, 1, 1, 1, 0, 1, 1, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > -T(sec_ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > -T(sec_ts_tso_noff, 1, 1, 1, 1, 0, 0, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F) \ > -T(sec_ts_tso_noff_l3l4csum, 1, 1, 1, 1, 0, 0, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > -T(sec_ts_tso_noff_ol3ol4csum, 1, 1, 1, 1, 0, 1, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > -T(sec_ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 0, 1, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F)\ > -T(sec_ts_tso_noff_vlan, 1, 1, 1, 1, 1, 0, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F) \ > -T(sec_ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 1, 0, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > -T(sec_ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 1, 0, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F)\ > -T(sec_ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 1, 8, \ > - T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ > - L3L4CSUM_F) > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > +#define NIX_TX_FASTPATH_MODES_0_15 \ > + T(no_offload, 4, NIX_TX_OFFLOAD_NONE) \ > + T(l3l4csum, 4, L3L4CSUM_F) \ > + T(ol3ol4csum, 4, OL3OL4CSUM_F) \ > + T(ol3ol4csum_l3l4csum, 4, OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(vlan, 6, VLAN_F) \ > + T(vlan_l3l4csum, 6, VLAN_F | L3L4CSUM_F) \ > + T(vlan_ol3ol4csum, 6, VLAN_F | OL3OL4CSUM_F) \ > + T(vlan_ol3ol4csum_l3l4csum, 6, VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(noff, 4, NOFF_F) \ > + T(noff_l3l4csum, 4, NOFF_F | L3L4CSUM_F) \ > + T(noff_ol3ol4csum, 4, NOFF_F | OL3OL4CSUM_F) \ > + T(noff_ol3ol4csum_l3l4csum, 4, NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(noff_vlan, 6, NOFF_F | VLAN_F) \ > + T(noff_vlan_l3l4csum, 6, NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(noff_vlan_ol3ol4csum, 6, NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(noff_vlan_ol3ol4csum_l3l4csum, 6, \ > + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_16_31 \ > + T(tso, 6, TSO_F) \ > + T(tso_l3l4csum, 6, TSO_F | L3L4CSUM_F) \ > + T(tso_ol3ol4csum, 6, TSO_F | OL3OL4CSUM_F) \ > + T(tso_ol3ol4csum_l3l4csum, 6, TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(tso_vlan, 6, TSO_F | VLAN_F) \ > + T(tso_vlan_l3l4csum, 6, TSO_F | VLAN_F | L3L4CSUM_F) \ > + T(tso_vlan_ol3ol4csum, 6, TSO_F | VLAN_F | OL3OL4CSUM_F) \ > + T(tso_vlan_ol3ol4csum_l3l4csum, 6, \ > + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(tso_noff, 6, TSO_F | NOFF_F) \ > + T(tso_noff_l3l4csum, 6, TSO_F | NOFF_F | L3L4CSUM_F) \ > + T(tso_noff_ol3ol4csum, 6, TSO_F | NOFF_F | OL3OL4CSUM_F) \ > + T(tso_noff_ol3ol4csum_l3l4csum, 6, \ > + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(tso_noff_vlan, 6, TSO_F | NOFF_F | VLAN_F) \ > + T(tso_noff_vlan_l3l4csum, 6, TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(tso_noff_vlan_ol3ol4csum, 6, TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(tso_noff_vlan_ol3ol4csum_l3l4csum, 6, \ > + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_32_47 \ > + T(ts, 8, TSP_F) \ > + T(ts_l3l4csum, 8, TSP_F | L3L4CSUM_F) \ > + T(ts_ol3ol4csum, 8, TSP_F | OL3OL4CSUM_F) \ > + T(ts_ol3ol4csum_l3l4csum, 8, TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_vlan, 8, TSP_F | VLAN_F) \ > + T(ts_vlan_l3l4csum, 8, TSP_F | VLAN_F | L3L4CSUM_F) \ > + T(ts_vlan_ol3ol4csum, 8, TSP_F | VLAN_F | OL3OL4CSUM_F) \ > + T(ts_vlan_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_noff, 8, TSP_F | NOFF_F) \ > + T(ts_noff_l3l4csum, 8, TSP_F | NOFF_F | L3L4CSUM_F) \ > + T(ts_noff_ol3ol4csum, 8, TSP_F | NOFF_F | OL3OL4CSUM_F) \ > + T(ts_noff_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_noff_vlan, 8, TSP_F | NOFF_F | VLAN_F) \ > + T(ts_noff_vlan_l3l4csum, 8, TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(ts_noff_vlan_ol3ol4csum, 8, TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(ts_noff_vlan_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_48_63 \ > + T(ts_tso, 8, TSP_F | TSO_F) \ > + T(ts_tso_l3l4csum, 8, TSP_F | TSO_F | L3L4CSUM_F) \ > + T(ts_tso_ol3ol4csum, 8, TSP_F | TSO_F | OL3OL4CSUM_F) \ > + T(ts_tso_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_tso_vlan, 8, TSP_F | TSO_F | VLAN_F) \ > + T(ts_tso_vlan_l3l4csum, 8, TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > + T(ts_tso_vlan_ol3ol4csum, 8, TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > + T(ts_tso_vlan_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_tso_noff, 8, TSP_F | TSO_F | NOFF_F) \ > + T(ts_tso_noff_l3l4csum, 8, TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > + T(ts_tso_noff_ol3ol4csum, 8, TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > + T(ts_tso_noff_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(ts_tso_noff_vlan, 8, TSP_F | TSO_F | NOFF_F | VLAN_F) \ > + T(ts_tso_noff_vlan_l3l4csum, 8, \ > + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(ts_tso_noff_vlan_ol3ol4csum, 8, \ > + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ > + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_64_79 \ > + T(sec, 4, T_SEC_F) \ > + T(sec_l3l4csum, 4, T_SEC_F | L3L4CSUM_F) \ > + T(sec_ol3ol4csum, 4, T_SEC_F | OL3OL4CSUM_F) \ > + T(sec_ol3ol4csum_l3l4csum, 4, T_SEC_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_vlan, 6, T_SEC_F | VLAN_F) \ > + T(sec_vlan_l3l4csum, 6, T_SEC_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_vlan_ol3ol4csum, 6, T_SEC_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_vlan_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_noff, 4, T_SEC_F | NOFF_F) \ > + T(sec_noff_l3l4csum, 4, T_SEC_F | NOFF_F | L3L4CSUM_F) \ > + T(sec_noff_ol3ol4csum, 4, T_SEC_F | NOFF_F | OL3OL4CSUM_F) \ > + T(sec_noff_ol3ol4csum_l3l4csum, 4, \ > + T_SEC_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_noff_vlan, 6, T_SEC_F | NOFF_F | VLAN_F) \ > + T(sec_noff_vlan_l3l4csum, 6, T_SEC_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_noff_vlan_ol3ol4csum, 6, \ > + T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_noff_vlan_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_80_95 \ > + T(sec_tso, 6, T_SEC_F | TSO_F) \ > + T(sec_tso_l3l4csum, 6, T_SEC_F | TSO_F | L3L4CSUM_F) \ > + T(sec_tso_ol3ol4csum, 6, T_SEC_F | TSO_F | OL3OL4CSUM_F) \ > + T(sec_tso_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_tso_vlan, 6, T_SEC_F | TSO_F | VLAN_F) \ > + T(sec_tso_vlan_l3l4csum, 6, T_SEC_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_tso_vlan_ol3ol4csum, 6, T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_tso_vlan_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_tso_noff, 6, T_SEC_F | TSO_F | NOFF_F) \ > + T(sec_tso_noff_l3l4csum, 6, T_SEC_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > + T(sec_tso_noff_ol3ol4csum, 6, T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > + T(sec_tso_noff_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_tso_noff_vlan, 6, T_SEC_F | TSO_F | NOFF_F | VLAN_F) \ > + T(sec_tso_noff_vlan_l3l4csum, 6, \ > + T_SEC_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_tso_noff_vlan_ol3ol4csum, 6, \ > + T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_tso_noff_vlan_ol3ol4csum_l3l4csum, 6, \ > + T_SEC_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_96_111 \ > + T(sec_ts, 8, T_SEC_F | TSP_F) \ > + T(sec_ts_l3l4csum, 8, T_SEC_F | TSP_F | L3L4CSUM_F) \ > + T(sec_ts_ol3ol4csum, 8, T_SEC_F | TSP_F | OL3OL4CSUM_F) \ > + T(sec_ts_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_vlan, 8, T_SEC_F | TSP_F | VLAN_F) \ > + T(sec_ts_vlan_l3l4csum, 8, T_SEC_F | TSP_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_ts_vlan_ol3ol4csum, 8, T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_ts_vlan_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_noff, 8, T_SEC_F | TSP_F | NOFF_F) \ > + T(sec_ts_noff_l3l4csum, 8, T_SEC_F | TSP_F | NOFF_F | L3L4CSUM_F) \ > + T(sec_ts_noff_ol3ol4csum, 8, T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F) \ > + T(sec_ts_noff_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_noff_vlan, 8, T_SEC_F | TSP_F | NOFF_F | VLAN_F) \ > + T(sec_ts_noff_vlan_l3l4csum, 8, \ > + T_SEC_F | TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_ts_noff_vlan_ol3ol4csum, 8, \ > + T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_ts_noff_vlan_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES_112_127 \ > + T(sec_ts_tso, 8, T_SEC_F | TSP_F | TSO_F) \ > + T(sec_ts_tso_l3l4csum, 8, T_SEC_F | TSP_F | TSO_F | L3L4CSUM_F) \ > + T(sec_ts_tso_ol3ol4csum, 8, T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F) \ > + T(sec_ts_tso_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_tso_vlan, 8, T_SEC_F | TSP_F | TSO_F | VLAN_F) \ > + T(sec_ts_tso_vlan_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_ts_tso_vlan_ol3ol4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_ts_tso_vlan_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_tso_noff, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F) \ > + T(sec_ts_tso_noff_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ > + T(sec_ts_tso_noff_ol3ol4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ > + T(sec_ts_tso_noff_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ > + T(sec_ts_tso_noff_vlan, 8, T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F) \ > + T(sec_ts_tso_noff_vlan_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ > + T(sec_ts_tso_noff_vlan_ol3ol4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ > + T(sec_ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 8, \ > + T_SEC_F | TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | \ > + L3L4CSUM_F) > + > +#define NIX_TX_FASTPATH_MODES \ > + NIX_TX_FASTPATH_MODES_0_15 \ > + NIX_TX_FASTPATH_MODES_16_31 \ > + NIX_TX_FASTPATH_MODES_32_47 \ > + NIX_TX_FASTPATH_MODES_48_63 \ > + NIX_TX_FASTPATH_MODES_64_79 \ > + NIX_TX_FASTPATH_MODES_80_95 \ > + NIX_TX_FASTPATH_MODES_96_111 \ > + NIX_TX_FASTPATH_MODES_112_127 > + > +#define T(name, sz, flags) \ > uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ > void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ > - \ > + \ > uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ > void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ > - \ > + \ > uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_##name( \ > void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ > - \ > + \ > uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_mseg_##name( \ > - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ > + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); > > NIX_TX_FASTPATH_MODES > #undef T > > +#define NIX_TX_XMIT(fn, sz, flags) \ > + uint16_t __rte_noinline __rte_hot fn( \ > + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > + { \ > + uint64_t cmd[sz]; \ > + \ > + /* For TSO inner checksum is a must */ \ > + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > + return 0; \ > + return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, 0, \ > + flags); \ > + } > + > +#define NIX_TX_XMIT_MSEG(fn, sz, flags) \ > + uint16_t __rte_noinline __rte_hot fn( \ > + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > + { \ > + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ > + \ > + /* For TSO inner checksum is a must */ \ > + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > + return 0; \ > + return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ > + 0, \ > + flags | NIX_TX_MULTI_SEG_F); \ > + } > + > +#define NIX_TX_XMIT_VEC(fn, sz, flags) \ > + uint16_t __rte_noinline __rte_hot fn( \ > + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > + { \ > + uint64_t cmd[sz]; \ > + \ > + /* For TSO inner checksum is a must */ \ > + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > + return 0; \ > + return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, \ > + cmd, 0, (flags)); \ > + } > + > +#define NIX_TX_XMIT_VEC_MSEG(fn, sz, flags) \ > + uint16_t __rte_noinline __rte_hot fn( \ > + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > + { \ > + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ > + \ > + /* For TSO inner checksum is a must */ \ > + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > + return 0; \ > + return cn10k_nix_xmit_pkts_vector( \ > + tx_queue, tx_pkts, pkts, cmd, 0, \ > + (flags) | NIX_TX_MULTI_SEG_F); \ > + } > + > #endif /* __CN10K_TX_H__ */ > diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c > deleted file mode 100644 > index 2b834095cf..0000000000 > --- a/drivers/net/cnxk/cn10k_tx_mseg.c > +++ /dev/null > @@ -1,26 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_ethdev.h" > -#include "cn10k_tx.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_noinline __rte_hot \ > - cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ > - struct rte_mbuf **tx_pkts, \ > - uint16_t pkts) \ > - { \ > - uint64_t cmd[(sz)]; \ > - \ > - /* For TSO inner checksum is a must */ \ > - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > - return 0; \ > - return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ > - 0, (flags) \ > - | NIX_TX_MULTI_SEG_F); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > diff --git a/drivers/net/cnxk/cn10k_tx_select.c b/drivers/net/cnxk/cn10k_tx_select.c > new file mode 100644 > index 0000000000..0b8e3f35b9 > --- /dev/null > +++ b/drivers/net/cnxk/cn10k_tx_select.c > @@ -0,0 +1,63 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > + > +#include "cn10k_ethdev.h" > +#include "cn10k_tx.h" > + > +static inline void > +pick_tx_func(struct rte_eth_dev *eth_dev, > + const eth_tx_burst_t tx_burst[NIX_TX_OFFLOAD_MAX]) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + > + /* [SEC] [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ > + eth_dev->tx_pkt_burst = > + tx_burst[dev->tx_offload_flags & (NIX_TX_OFFLOAD_MAX - 1)]; > +} > + > +void > +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + > + const eth_tx_burst_t nix_eth_tx_burst[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_##name, > + > + NIX_TX_FASTPATH_MODES > +#undef T > + }; > + > + const eth_tx_burst_t nix_eth_tx_burst_mseg[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_mseg_##name, > + > + NIX_TX_FASTPATH_MODES > +#undef T > + }; > + > + const eth_tx_burst_t nix_eth_tx_vec_burst[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_vec_##name, > + > + NIX_TX_FASTPATH_MODES > +#undef T > + }; > + > + const eth_tx_burst_t nix_eth_tx_vec_burst_mseg[NIX_TX_OFFLOAD_MAX] = { > +#define T(name, sz, flags) [flags] = cn10k_nix_xmit_pkts_vec_mseg_##name, > + > + NIX_TX_FASTPATH_MODES > +#undef T > + }; > + > + if (dev->scalar_ena) { > + pick_tx_func(eth_dev, nix_eth_tx_burst); > + if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) > + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); > + } else { > + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); > + if (dev->tx_offloads & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) > + pick_tx_func(eth_dev, nix_eth_tx_vec_burst_mseg); > + } > + > + rte_mb(); > +} > diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c > deleted file mode 100644 > index 2789b13d60..0000000000 > --- a/drivers/net/cnxk/cn10k_tx_vec.c > +++ /dev/null > @@ -1,25 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_ethdev.h" > -#include "cn10k_tx.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_noinline __rte_hot \ > - cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ > - struct rte_mbuf **tx_pkts, \ > - uint16_t pkts) \ > - { \ > - uint64_t cmd[sz]; \ > - \ > - /* For TSO inner checksum is a must */ \ > - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > - return 0; \ > - return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ > - 0, (flags)); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > diff --git a/drivers/net/cnxk/cn10k_tx_vec_mseg.c b/drivers/net/cnxk/cn10k_tx_vec_mseg.c > deleted file mode 100644 > index 98000df101..0000000000 > --- a/drivers/net/cnxk/cn10k_tx_vec_mseg.c > +++ /dev/null > @@ -1,24 +0,0 @@ > -/* SPDX-License-Identifier: BSD-3-Clause > - * Copyright(C) 2021 Marvell. > - */ > - > -#include "cn10k_ethdev.h" > -#include "cn10k_tx.h" > - > -#define T(name, f6, f5, f4, f3, f2, f1, f0, sz, flags) \ > - uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_mseg_##name( \ > - void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ > - { \ > - uint64_t cmd[sz]; \ > - \ > - /* For TSO inner checksum is a must */ \ > - if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ > - !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ > - return 0; \ > - return cn10k_nix_xmit_pkts_vector( \ > - tx_queue, tx_pkts, pkts, cmd, 0, \ > - (flags) | NIX_TX_MULTI_SEG_F); \ > - } > - > -NIX_TX_FASTPATH_MODES > -#undef T > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build > index 1fd388a1f0..375c75d1c7 100644 > --- a/drivers/net/cnxk/meson.build > +++ b/drivers/net/cnxk/meson.build > @@ -107,11 +107,8 @@ sources += files( > 'cn10k_ethdev.c', > 'cn10k_ethdev_sec.c', > 'cn10k_rte_flow.c', > - 'cn10k_tx.c', > - 'cn10k_tx_mseg.c', > - 'cn10k_tx_vec.c', > - 'cn10k_tx_vec_mseg.c', > 'cn10k_rx_select.c', > + 'cn10k_tx_select.c', > ) > > sources += files( > @@ -149,6 +146,42 @@ sources += files( > 'rx/cn10k/rx_112_127_vec_mseg.c', > ) > > +sources += files( > + 'tx/cn10k/tx_0_15.c', > + 'tx/cn10k/tx_16_31.c', > + 'tx/cn10k/tx_32_47.c', > + 'tx/cn10k/tx_48_63.c', > + 'tx/cn10k/tx_64_79.c', > + 'tx/cn10k/tx_80_95.c', > + 'tx/cn10k/tx_96_111.c', > + 'tx/cn10k/tx_112_127.c', > + 'tx/cn10k/tx_0_15_mseg.c', > + 'tx/cn10k/tx_16_31_mseg.c', > + 'tx/cn10k/tx_32_47_mseg.c', > + 'tx/cn10k/tx_48_63_mseg.c', > + 'tx/cn10k/tx_64_79_mseg.c', > + 'tx/cn10k/tx_80_95_mseg.c', > + 'tx/cn10k/tx_96_111_mseg.c', > + 'tx/cn10k/tx_112_127_mseg.c', > + 'tx/cn10k/tx_0_15_vec.c', > + 'tx/cn10k/tx_16_31_vec.c', > + 'tx/cn10k/tx_32_47_vec.c', > + 'tx/cn10k/tx_48_63_vec.c', > + 'tx/cn10k/tx_64_79_vec.c', > + 'tx/cn10k/tx_80_95_vec.c', > + 'tx/cn10k/tx_96_111_vec.c', > + 'tx/cn10k/tx_112_127_vec.c', > + 'tx/cn10k/tx_0_15_vec_mseg.c', > + 'tx/cn10k/tx_16_31_vec_mseg.c', > + 'tx/cn10k/tx_32_47_vec_mseg.c', > + 'tx/cn10k/tx_48_63_vec_mseg.c', > + 'tx/cn10k/tx_64_79_vec_mseg.c', > + 'tx/cn10k/tx_80_95_vec_mseg.c', > + 'tx/cn10k/tx_96_111_vec_mseg.c', > + 'tx/cn10k/tx_112_127_vec_mseg.c', > +) > + > + > deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] > deps += ['common_cnxk', 'mempool_cnxk'] > > -- > 2.17.1 >