* [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver
@ 2021-03-06 15:33 Nithin Dabilpuram
2021-03-06 15:33 ` [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe Nithin Dabilpuram
` (46 more replies)
0 siblings, 47 replies; 262+ messages in thread
From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw)
To: dev
Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh,
asekhar, Nithin Dabilpuram
This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk'
driver. In future, CN9K a.k.a octeontx2 will also be supported by same
driver when code is ready and 'net/octeontx2' will be deprecated.
Depends-on: series-15511 ("Add Marvell CNXK mempool driver")
Jerin Jacob (6):
net/cnxk: add Rx support for cn9k
net/cnxk: add Rx vector version for cn9k
net/cnxk: add Tx support for cn9k
net/cnxk: add Rx support for cn10k
net/cnxk: add Rx vector version for cn10k
net/cnxk: add Tx support for cn10k
Kiran Kumar K (2):
net/cnxk: add support to configure npc
net/cnxk: add initial version of rte flow support
Nithin Dabilpuram (17):
net/cnxk: add build infra and common probe
net/cnxk: add platform specific probe and remove
net/cnxk: add common devargs parsing function
net/cnxk: add common dev infos get support
net/cnxk: add device configuration operation
net/cnxk: add link status update support
net/cnxk: add Rx queue setup and release
net/cnxk: add Tx queue setup and release
net/cnxk: add packet type support
net/cnxk: add queue start and stop support
net/cnxk: add Rx multi-segmented version for cn9k
net/cnxk: add Tx multi-segment version for cn9k
net/cnxk: add Tx vector version for cn9k
net/cnxk: add Rx multi-segment version for cn10k
net/cnxk: add Tx multi-segment version for cn10k
net/cnxk: add Tx vector version for cn10k
net/cnxk: add device start and stop operations
Satha Rao (5):
net/cnxk: add port/queue stats
net/cnxk: add xstats apis
net/cnxk: add rxq/txq info get operations
net/cnxk: add ethdev firmware version get
net/cnxk: add get register operation
Satheesh Paul (1):
net/cnxk: add filter ctrl operation
Sunil Kumar Kori (13):
net/cnxk: add MAC address set ops
net/cnxk: add MTU set device operation
net/cnxk: add promiscuous mode enable and disable
net/cnxk: add DMAC filter support
net/cnxk: add all multicast enable/disable ethops
net/cnxk: add Rx/Tx burst mode get ops
net/cnxk: add flow ctrl set/get ops
net/cnxk: add link up/down operations
net/cnxk: add EEPROM module info get operations
net/cnxk: add Rx queue interrupt enable/disable ops
net/cnxk: add validation API for mempool ops
net/cnxk: add device close and reset operations
net/cnxk: add pending Tx mbuf cleanup operation
MAINTAINERS | 3 +
doc/guides/nics/cnxk.rst | 343 ++++++++
doc/guides/nics/features/cnxk.ini | 44 +
doc/guides/nics/features/cnxk_vec.ini | 42 +
doc/guides/nics/features/cnxk_vf.ini | 39 +
doc/guides/nics/index.rst | 1 +
doc/guides/platform/cnxk.rst | 3 +
drivers/common/cnxk/roc_npc.c | 2 +
drivers/net/cnxk/cn10k_ethdev.c | 374 +++++++++
drivers/net/cnxk/cn10k_ethdev.h | 39 +
drivers/net/cnxk/cn10k_rx.c | 388 +++++++++
drivers/net/cnxk/cn10k_rx.h | 212 +++++
drivers/net/cnxk/cn10k_tx.c | 1284 ++++++++++++++++++++++++++++
drivers/net/cnxk/cn10k_tx.h | 442 ++++++++++
drivers/net/cnxk/cn9k_ethdev.c | 404 +++++++++
drivers/net/cnxk/cn9k_ethdev.h | 37 +
drivers/net/cnxk/cn9k_rx.c | 388 +++++++++
drivers/net/cnxk/cn9k_rx.h | 215 +++++
drivers/net/cnxk/cn9k_tx.c | 1122 +++++++++++++++++++++++++
drivers/net/cnxk/cn9k_tx.h | 475 +++++++++++
drivers/net/cnxk/cnxk_ethdev.c | 1449 ++++++++++++++++++++++++++++++++
drivers/net/cnxk/cnxk_ethdev.h | 387 +++++++++
drivers/net/cnxk/cnxk_ethdev_devargs.c | 169 ++++
drivers/net/cnxk/cnxk_ethdev_ops.c | 729 ++++++++++++++++
drivers/net/cnxk/cnxk_link.c | 113 +++
drivers/net/cnxk/cnxk_lookup.c | 326 +++++++
drivers/net/cnxk/cnxk_rte_flow.c | 280 ++++++
drivers/net/cnxk/cnxk_rte_flow.h | 69 ++
drivers/net/cnxk/cnxk_stats.c | 217 +++++
drivers/net/cnxk/meson.build | 36 +
drivers/net/cnxk/version.map | 3 +
drivers/net/meson.build | 1 +
32 files changed, 9636 insertions(+)
create mode 100644 doc/guides/nics/cnxk.rst
create mode 100644 doc/guides/nics/features/cnxk.ini
create mode 100644 doc/guides/nics/features/cnxk_vec.ini
create mode 100644 doc/guides/nics/features/cnxk_vf.ini
create mode 100644 drivers/net/cnxk/cn10k_ethdev.c
create mode 100644 drivers/net/cnxk/cn10k_ethdev.h
create mode 100644 drivers/net/cnxk/cn10k_rx.c
create mode 100644 drivers/net/cnxk/cn10k_rx.h
create mode 100644 drivers/net/cnxk/cn10k_tx.c
create mode 100644 drivers/net/cnxk/cn10k_tx.h
create mode 100644 drivers/net/cnxk/cn9k_ethdev.c
create mode 100644 drivers/net/cnxk/cn9k_ethdev.h
create mode 100644 drivers/net/cnxk/cn9k_rx.c
create mode 100644 drivers/net/cnxk/cn9k_rx.h
create mode 100644 drivers/net/cnxk/cn9k_tx.c
create mode 100644 drivers/net/cnxk/cn9k_tx.h
create mode 100644 drivers/net/cnxk/cnxk_ethdev.c
create mode 100644 drivers/net/cnxk/cnxk_ethdev.h
create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c
create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c
create mode 100644 drivers/net/cnxk/cnxk_link.c
create mode 100644 drivers/net/cnxk/cnxk_lookup.c
create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c
create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h
create mode 100644 drivers/net/cnxk/cnxk_stats.c
create mode 100644 drivers/net/cnxk/meson.build
create mode 100644 drivers/net/cnxk/version.map
--
2.8.4
^ permalink raw reply [flat|nested] 262+ messages in thread* [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 02/44] net/cnxk: add platform specific probe and remove Nithin Dabilpuram ` (45 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add build infrastructure and common probe and remove for cnxk driver which is used by both CN10K and CN9K SoC. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- MAINTAINERS | 3 + doc/guides/nics/cnxk.rst | 29 +++++ doc/guides/nics/features/cnxk.ini | 9 ++ doc/guides/nics/features/cnxk_vec.ini | 9 ++ doc/guides/nics/features/cnxk_vf.ini | 9 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ drivers/net/cnxk/meson.build | 21 ++++ drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 12 files changed, 364 insertions(+) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 67c179f..efabc3c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -745,6 +745,9 @@ M: Sunil Kumar Kori <skori@marvell.com> M: Satha Rao <skoteshwar@marvell.com> T: git://dpdk.org/next/dpdk-next-net-mrvl F: drivers/common/cnxk/ +F: drivers/net/cnxk/ +F: doc/guides/nics/cnxk.rst +F: doc/guides/nics/features/cnxk*.ini F: doc/guides/platform/cnxk.rst Marvell mvpp2 diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst new file mode 100644 index 0000000..ca21842 --- /dev/null +++ b/doc/guides/nics/cnxk.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK Poll Mode driver +===================== + +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** +SoC family as well as for their virtual functions (VF) in SR-IOV context. + +More information can be found at `Marvell Official Website +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. + +Features +-------- + +Features of the CNXK Ethdev PMD are: + +Prerequisites +------------- + +See :doc:`../platform/cnxk` for setup information. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini new file mode 100644 index 0000000..2c23464 --- /dev/null +++ b/doc/guides/nics/features/cnxk.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini new file mode 100644 index 0000000..de78516 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vec' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini new file mode 100644 index 0000000..9c96351 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 799697c..c1a04d9 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -19,6 +19,7 @@ Network Interface Controller Drivers axgbe bnx2x bnxt + cnxk cxgbe dpaa dpaa2 diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index 9bbba65..a9c050e 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -141,6 +141,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in CNXK SoC. +#. **Ethdev Driver** + See :doc:`../nics/cnxk` for NIX Ethdev driver information. + #. **Mempool Driver** See :doc:`../mempool/cnxk` for NPA mempool driver information. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c new file mode 100644 index 0000000..6717410 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -0,0 +1,219 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_ethdev.h> + +/* CNXK platform independent eth dev ops */ +struct eth_dev_ops cnxk_eth_dev_ops; + +static int +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct rte_pci_device *pci_dev; + int rc, max_entries; + + eth_dev->dev_ops = &cnxk_eth_dev_ops; + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + rte_eth_copy_pci_info(eth_dev, pci_dev); + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + + /* Initialize base roc nix */ + nix->pci_dev = pci_dev; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to initialize roc nix rc=%d", rc); + goto error; + } + + dev->eth_dev = eth_dev; + + /* For vfs, returned max_entries will be 0. but to keep default mac + * address, one entry must be allocated. so setting up to 1. + */ + if (roc_nix_is_vf_or_sdp(nix)) + max_entries = 1; + else + max_entries = roc_nix_mac_max_entries_get(nix); + + if (max_entries <= 0) { + plt_err("Failed to get max entries for mac addr"); + rc = -ENOTSUP; + goto dev_fini; + } + + eth_dev->data->mac_addrs = + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + plt_err("Failed to allocate memory for mac addr"); + rc = -ENOMEM; + goto dev_fini; + } + + dev->max_mac_entries = max_entries; + + /* Get mac address */ + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to get mac addr, rc=%d", rc); + goto free_mac_addrs; + } + + /* Update the mac address */ + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); + + if (!roc_nix_is_vf_or_sdp(nix)) { + /* Sync same MAC address to CGX/RPM table */ + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to set mac addr, rc=%d", rc); + goto free_mac_addrs; + } + } + + /* Initialize roc npc */ + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, + eth_dev->data->port_id, roc_nix_get_pf(nix), + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, + dev->rx_offload_capa, dev->tx_offload_capa); + return 0; + +free_mac_addrs: + rte_free(eth_dev->data->mac_addrs); +dev_fini: + roc_nix_dev_fini(nix); +error: + plt_err("Failed to init nix eth_dev rc=%d", rc); + return rc; +} + +static int +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct roc_nix *nix = &dev->nix; + int rc, i; + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + roc_nix_npc_rx_ena_dis(nix, false); + + /* Free up SQs */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); + eth_dev->data->tx_queues[i] = NULL; + } + eth_dev->data->nb_tx_queues = 0; + + /* Free up RQ's and CQ's */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); + eth_dev->data->rx_queues[i] = NULL; + } + eth_dev->data->nb_rx_queues = 0; + + /* Free tm resources */ + roc_nix_tm_fini(nix); + + /* Unregister queue irqs */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister cq irqs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Free nix lf resources */ + rc = roc_nix_lf_free(nix); + if (rc) + plt_err("Failed to free nix lf, rc=%d", rc); + + rte_free(eth_dev->data->mac_addrs); + eth_dev->data->mac_addrs = NULL; + + /* Check if mbox close is needed */ + if (!mbox_close) + return 0; + + rc = roc_nix_dev_fini(nix); + /* Can be freed later by PMD if NPA LF is in use */ + if (rc == -EAGAIN) { + eth_dev->data->dev_private = NULL; + return 0; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + } + + return rc; +} + +int +cnxk_nix_remove(struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct roc_nix *nix; + int rc = -EINVAL; + + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (eth_dev) { + /* Cleanup eth dev */ + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + rte_eth_dev_release_port(eth_dev); + } + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (nix->pci_dev != pci_dev) + return 0; + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("%s: common resource in use by other devices", + pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + /* Free device pointer as rte_ethdev does not have it anymore */ + rte_free(nix); +exit: + return rc; +} + +int +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + int rc; + + RTE_SET_USED(pci_drv); + + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), + cnxk_eth_dev_init); + + /* On error on secondary, recheck if port exists in primary or + * in mid of detach state. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) + if (!rte_eth_dev_allocated(pci_dev->device.name)) + return 0; + return rc; +} diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h new file mode 100644 index 0000000..0460d1e --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_ETHDEV_H__ +#define __CNXK_ETHDEV_H__ + +#include <math.h> +#include <stdint.h> + +#include <ethdev_driver.h> +#include <ethdev_pci.h> + +#include "roc_api.h" + +#define CNXK_ETH_DEV_PMD_VERSION "1.0" + +struct cnxk_eth_dev { + /* ROC NIX */ + struct roc_nix nix; + + /* Max macfilter entries */ + uint8_t max_mac_entries; + + uint16_t flags; + + /* Pointer back to rte */ + struct rte_eth_dev *eth_dev; + + /* HW capabilities / Limitations */ + union { + uint64_t hwcap; + }; + + /* Rx and Tx offload capabilities */ + uint64_t rx_offload_capa; + uint64_t tx_offload_capa; + uint32_t speed_capa; + + /* Default mac address */ + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; +}; + +static inline struct cnxk_eth_dev * +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) +{ + return eth_dev->data->dev_private; +} + +/* Common ethdev ops */ +extern struct eth_dev_ops cnxk_eth_dev_ops; + +/* Ops */ +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); +int cnxk_nix_remove(struct rte_pci_device *pci_dev); + +#endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build new file mode 100644 index 0000000..77b2f18 --- /dev/null +++ b/drivers/net/cnxk/meson.build @@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit' + subdir_done() +endif + +sources = files('cnxk_ethdev.c') + +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] +deps += ['common_cnxk', 'mempool_cnxk'] + +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] +foreach flag: extra_flags + if cc.has_argument(flag) + cflags += flag + endif +endforeach diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map new file mode 100644 index 0000000..ee80c51 --- /dev/null +++ b/drivers/net/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index fb9ff05..9f69842 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -10,6 +10,7 @@ drivers = ['af_packet', 'axgbe', 'bonding', 'bnx2x', 'bnxt', + 'cnxk', 'cxgbe', 'dpaa', 'dpaa2', 'e1000', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 02/44] net/cnxk: add platform specific probe and remove 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 03/44] net/cnxk: add common devargs parsing function Nithin Dabilpuram ` (44 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add platform specific probe and remove callbacks for CN9K and CN10K which use common probe and remove functions. Register ethdev driver for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ drivers/net/cnxk/meson.build | 5 +++ 7 files changed, 230 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c new file mode 100644 index 0000000..54711ea --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn10k_ethdev.h" + +static int +cn10k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 64) { + plt_err("Driver not compiled for CN10K"); + return -EFAULT; + } + + rc = plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + } + return 0; +} + +static const struct rte_pci_id cn10k_pci_nix_map[] = { + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn10k_pci_nix = { + .id_table = cn10k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn10k_nix_probe, + .remove = cn10k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h new file mode 100644 index 0000000..1bf4a65 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_ETHDEV_H__ +#define __CN10K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c new file mode 100644 index 0000000..bd97d5f --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn9k_ethdev.h" + +static int +cn9k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 128) { + plt_err("Driver not compiled for CN9K"); + return -EFAULT; + } + + rc = plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + + dev = cnxk_eth_pmd_priv(eth_dev); + /* Update capabilities already set for TSO. + * TSO not supported for earlier chip revisions + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_GRE_TNL_TSO); + + /* 50G and 100G to be supported for board version C0 + * and above of CN9K. + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) { + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; + } + + dev->hwcap = 0; + + /* Update HW erratas */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->cq_min_4k = 1; + return 0; +} + +static const struct rte_pci_id cn9k_pci_nix_map[] = { + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn9k_pci_nix = { + .id_table = cn9k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn9k_nix_probe, + .remove = cn9k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h new file mode 100644 index 0000000..15d9397 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_ETHDEV_H__ +#define __CN9K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6717410..b836fc2 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -3,6 +3,40 @@ */ #include <cnxk_ethdev.h> +static inline uint64_t +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) +{ + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; + + return capa; +} + +static inline uint64_t +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) +{ + RTE_SET_USED(dev); + return CNXK_NIX_TX_OFFLOAD_CAPA; +} + +static inline uint32_t +nix_get_speed_capa(struct cnxk_eth_dev *dev) +{ + uint32_t speed_capa; + + /* Auto negotiation disabled */ + speed_capa = ETH_LINK_SPEED_FIXED; + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; + } + + return speed_capa; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops; @@ -77,6 +111,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } } + /* Union of all capabilities supported by CNXK. + * Platform specific capabilities will be + * updated later. + */ + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); + dev->speed_capa = nix_get_speed_capa(dev); + /* Initialize roc npc */ plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0460d1e..ba2bfcd 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -14,6 +14,22 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +#define CNXK_NIX_TX_OFFLOAD_CAPA \ + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ + DEV_TX_OFFLOAD_IPV4_CKSUM) + +#define CNXK_NIX_RX_OFFLOAD_CAPA \ + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_RX_OFFLOAD_RSS_HASH) + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -28,6 +44,9 @@ struct cnxk_eth_dev { /* HW capabilities / Limitations */ union { + struct { + uint64_t cq_min_4k : 1; + }; uint64_t hwcap; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 77b2f18..57dea5e 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,6 +10,11 @@ endif sources = files('cnxk_ethdev.c') +# CN9K +sources += files('cn9k_ethdev.c') +# CN10K +sources += files('cn10k_ethdev.c') + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 03/44] net/cnxk: add common devargs parsing function 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 02/44] net/cnxk: add platform specific probe and remove Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 04/44] net/cnxk: add common dev infos get support Nithin Dabilpuram ` (43 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add various devargs parsing command line arguments parsing functions supported by CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 7 ++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ca21842..611ffb4 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -27,3 +27,97 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. + +Runtime Config Options +---------------------- + +- ``Rx&Tx scalar mode enable`` (default ``0``) + + Ethdev supports both scalar and vector mode, it may be selected at runtime + using ``scalar_enable`` ``devargs`` parameter. + +- ``RSS reta size`` (default ``64``) + + RSS redirection table size may be configured during runtime using ``reta_size`` + ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,reta_size=256 + + With the above configuration, reta table of size 256 is populated. + +- ``Flow priority levels`` (default ``3``) + + RTE Flow priority levels can be configured during runtime using + ``flow_max_priority`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_max_priority=10 + + With the above configuration, priority level was set to 10 (0-9). Max + priority level supported is 32. + +- ``Reserve Flow entries`` (default ``8``) + + RTE flow entries can be pre allocated and the size of pre allocation can be + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_prealloc_size=4 + + With the above configuration, pre alloc size was set to 4. Max pre alloc + size supported is 32. + +- ``Max SQB buffer count`` (default ``512``) + + Send queue descriptor buffer count may be limited during runtime using + ``max_sqb_count`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,max_sqb_count=64 + + With the above configuration, each send queue's decscriptor buffer count is + limited to a maximum of 64 buffers. + +- ``Switch header enable`` (default ``none``) + + A port can be configured to a specific switch header type by using + ``switch_header`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,switch_header="higig2" + + With the above configuration, higig2 will be enabled on that port and the + traffic on this port should be higig2 traffic only. Supported switch header + types are "higig2", "dsa", "chlen90b" and "chlen24b". + +- ``RSS tag as XOR`` (default ``0``) + + The HW gives two options to configure the RSS adder i.e + + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` + + * ``rss_adder<7:0> = flow_tag<7:0>`` + + Latter one aligns with standard NIC behavior vs former one is a legacy + RSS adder scheme used in OCTEON TX2 products. + + By default, the driver runs in the latter mode. + Setting this flag to 1 to select the legacy mode. + + For example to select the legacy mode(RSS tag adder as XOR):: + + -a 0002:02:00.0,tag_as_xor=1 + + + +.. note:: + + Above devarg parameters are configurable per device, user needs to pass the + parameters to all the PCIe devices if application requires to configure on + all the ethdev ports. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b836fc2..3a2309e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -58,6 +58,13 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) rte_eth_copy_pci_info(eth_dev, pci_dev); eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + /* Parse devargs string */ + rc = cnxk_ethdev_parse_devargs(eth_dev->device->devargs, dev); + if (rc) { + plt_err("Failed to parse devargs rc=%d", rc); + goto error; + } + /* Initialize base roc nix */ nix->pci_dev = pci_dev; rc = roc_nix_dev_init(nix); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ba2bfcd..97e3a15 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -9,11 +9,15 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> +#include <rte_kvargs.h> #include "roc_api.h" #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Max supported SQB count */ +#define CNXK_NIX_TX_MAX_SQB 512 + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -38,6 +42,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + bool scalar_ena; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -73,4 +78,8 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +/* Devargs */ +int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, + struct cnxk_eth_dev *dev); + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c new file mode 100644 index 0000000..4af2803 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <inttypes.h> +#include <math.h> + +#include "cnxk_ethdev.h" + +static int +parse_flow_max_priority(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the max priority to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flow_prealloc_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the prealloc size to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_reta_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val <= ETH_RSS_RETA_SIZE_64) + val = ROC_NIX_RSS_RETA_SZ_64; + else if (val > ETH_RSS_RETA_SIZE_64 && val <= ETH_RSS_RETA_SIZE_128) + val = ROC_NIX_RSS_RETA_SZ_128; + else if (val > ETH_RSS_RETA_SIZE_128 && val <= ETH_RSS_RETA_SIZE_256) + val = ROC_NIX_RSS_RETA_SZ_256; + else + val = ROC_NIX_RSS_RETA_SZ_64; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flag(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + *(uint16_t *)extra_args = atoi(value); + + return 0; +} + +static int +parse_sqb_count(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_switch_header_type(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + if (strcmp(value, "higig2") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_HIGIG; + + if (strcmp(value, "dsa") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_EDSA; + + if (strcmp(value, "chlen90b") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_LEN_90B; + return 0; +} + +#define CNXK_RSS_RETA_SIZE "reta_size" +#define CNXK_SCL_ENABLE "scalar_enable" +#define CNXK_MAX_SQB_COUNT "max_sqb_count" +#define CNXK_FLOW_PREALLOC_SIZE "flow_prealloc_size" +#define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" +#define CNXK_SWITCH_HEADER_TYPE "switch_header" +#define CNXK_RSS_TAG_AS_XOR "tag_as_xor" + +int +cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) +{ + uint16_t reta_sz = ROC_NIX_RSS_RETA_SZ_64; + uint16_t sqb_count = CNXK_NIX_TX_MAX_SQB; + uint16_t flow_prealloc_size = 8; + uint16_t switch_header_type = 0; + uint16_t flow_max_priority = 3; + uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto null_devargs; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_RSS_RETA_SIZE, &parse_reta_size, + &reta_sz); + rte_kvargs_process(kvlist, CNXK_SCL_ENABLE, &parse_flag, + &scalar_enable); + rte_kvargs_process(kvlist, CNXK_MAX_SQB_COUNT, &parse_sqb_count, + &sqb_count); + rte_kvargs_process(kvlist, CNXK_FLOW_PREALLOC_SIZE, + &parse_flow_prealloc_size, &flow_prealloc_size); + rte_kvargs_process(kvlist, CNXK_FLOW_MAX_PRIORITY, + &parse_flow_max_priority, &flow_max_priority); + rte_kvargs_process(kvlist, CNXK_SWITCH_HEADER_TYPE, + &parse_switch_header_type, &switch_header_type); + rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, + &rss_tag_as_xor); + rte_kvargs_free(kvlist); + +null_devargs: + dev->scalar_ena = !!scalar_enable; + dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; + dev->nix.max_sqb_count = sqb_count; + dev->nix.reta_sz = reta_sz; + return 0; + +exit: + return -EINVAL; +} + +RTE_PMD_REGISTER_PARAM_STRING(net_cnxk, + CNXK_RSS_RETA_SIZE "=<64|128|256>" + CNXK_SCL_ENABLE "=1" + CNXK_MAX_SQB_COUNT "=<8-512>" + CNXK_FLOW_PREALLOC_SIZE "=<1-32>" + CNXK_FLOW_MAX_PRIORITY "=<1-32>" + CNXK_SWITCH_HEADER_TYPE "=<higig2|dsa|chlen90b>" + CNXK_RSS_TAG_AS_XOR "=1"); diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 57dea5e..00c4722 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -8,7 +8,8 @@ if not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_ethdev.c') +sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_devargs.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 04/44] net/cnxk: add common dev infos get support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (2 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 03/44] net/cnxk: add common devargs parsing function Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 05/44] net/cnxk: add device configuration operation Nithin Dabilpuram ` (42 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add support to retrieve dev infos get for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 3 ++ doc/guides/nics/features/cnxk.ini | 4 ++ doc/guides/nics/features/cnxk_vec.ini | 4 ++ doc/guides/nics/features/cnxk_vf.ini | 3 ++ drivers/net/cnxk/cnxk_ethdev.c | 4 +- drivers/net/cnxk/cnxk_ethdev.h | 33 ++++++++++++++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 71 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 8 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 611ffb4..dfe2e7a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,9 @@ Features Features of the CNXK Ethdev PMD are: +- SR-IOV VF +- Lock-free Tx queue + Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c23464..b426340 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index de78516..292ac1e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 9c96351..bc2eb8a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3a2309e..1567007 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -38,7 +38,9 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) } /* CNXK platform independent eth dev ops */ -struct eth_dev_ops cnxk_eth_dev_ops; +struct eth_dev_ops cnxk_eth_dev_ops = { + .dev_infos_get = cnxk_nix_info_get, +}; static int cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 97e3a15..8d9a7e0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,9 +15,40 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* VLAN tag inserted by NIX_TX_VTAG_ACTION. + * In Tx space is always reserved for this in FRS. + */ +#define CNXK_NIX_MAX_VTAG_INS 2 +#define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) + +/* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) + +#define CNXK_NIX_RX_MIN_DESC 16 +#define CNXK_NIX_RX_MIN_DESC_ALIGN 16 +#define CNXK_NIX_RX_NB_SEG_MAX 6 +#define CNXK_NIX_RX_DEFAULT_RING_SZ 4096 /* Max supported SQB count */ #define CNXK_NIX_TX_MAX_SQB 512 +/* If PTP is enabled additional SEND MEM DESC is required which + * takes 2 words, hence max 7 iova address are possible + */ +#if defined(RTE_LIBRTE_IEEE1588) +#define CNXK_NIX_TX_NB_SEG_MAX 7 +#else +#define CNXK_NIX_TX_NB_SEG_MAX 9 +#endif + +#define CNXK_NIX_RSS_L3_L4_SRC_DST \ + (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ + ETH_RSS_L4_DST_ONLY) + +#define CNXK_NIX_RSS_OFFLOAD \ + (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | \ + ETH_RSS_SCTP | ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \ + CNXK_NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK | ETH_RSS_C_VLAN) + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -77,6 +108,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_info *dev_info); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c new file mode 100644 index 0000000..4a45956 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <cnxk_ethdev.h> + +int +cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) +{ + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int max_rx_pktlen; + + max_rx_pktlen = (roc_nix_max_pkt_len(&dev->nix) + RTE_ETHER_CRC_LEN - + CNXK_NIX_MAX_VTAG_ACT_SIZE); + + devinfo->min_rx_bufsize = NIX_MIN_HW_FRS + RTE_ETHER_CRC_LEN; + devinfo->max_rx_pktlen = max_rx_pktlen; + devinfo->max_rx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_mac_addrs = dev->max_mac_entries; + devinfo->max_vfs = pci_dev->max_vfs; + devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; + + devinfo->rx_offload_capa = dev->rx_offload_capa; + devinfo->tx_offload_capa = dev->tx_offload_capa; + devinfo->rx_queue_offload_capa = 0; + devinfo->tx_queue_offload_capa = 0; + + devinfo->reta_size = dev->nix.reta_sz; + devinfo->hash_key_size = ROC_NIX_RSS_KEY_LEN; + devinfo->flow_type_rss_offloads = CNXK_NIX_RSS_OFFLOAD; + + devinfo->default_rxconf = (struct rte_eth_rxconf){ + .rx_drop_en = 0, + .offloads = 0, + }; + + devinfo->default_txconf = (struct rte_eth_txconf){ + .offloads = 0, + }; + + devinfo->default_rxportconf = (struct rte_eth_dev_portconf){ + .ring_size = CNXK_NIX_RX_DEFAULT_RING_SZ, + }; + + devinfo->rx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = CNXK_NIX_RX_MIN_DESC, + .nb_align = CNXK_NIX_RX_MIN_DESC_ALIGN, + .nb_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + }; + devinfo->rx_desc_lim.nb_max = + RTE_ALIGN_MUL_FLOOR(devinfo->rx_desc_lim.nb_max, + CNXK_NIX_RX_MIN_DESC_ALIGN); + + devinfo->tx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = 1, + .nb_align = 1, + .nb_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + }; + + devinfo->speed_capa = dev->speed_capa; + devinfo->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 00c4722..45ccbe3 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -9,6 +9,7 @@ if not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 05/44] net/cnxk: add device configuration operation 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (3 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 04/44] net/cnxk: add common dev infos get support Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 06/44] net/cnxk: add link status update support Nithin Dabilpuram ` (41 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add device configuration op for CN9K and CN10K. Most of the device configuration is common between two platforms except for some supported offloads. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 2 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cn10k_ethdev.c | 34 +++ drivers/net/cnxk/cn9k_ethdev.c | 45 +++ drivers/net/cnxk/cnxk_ethdev.c | 521 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 70 +++++ 8 files changed, 678 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index dfe2e7a..73eb62a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -18,6 +18,8 @@ Features of the CNXK Ethdev PMD are: - SR-IOV VF - Lock-free Tx queue +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b426340..96dba2a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 292ac1e..616991c 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index bc2eb8a..a0bd2f1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 54711ea..9cf0f9e 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,38 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; +} + +static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -26,6 +58,8 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index bd97d5f..4f50949 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_conf *conf = ð_dev->data->dev_conf; + struct rte_eth_txmode *txmode = &conf->txmode; + int rc; + + /* Platform specific checks */ + if ((roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) && + (txmode->offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) && + ((txmode->offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) || + (txmode->offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM))) { + plt_err("Outer IP and SCTP checksum unsupported"); + return -EINVAL; + } + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; +} + +static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -27,6 +70,8 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1567007..f141027 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,520 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint32_t +cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level) +{ + uint32_t flow_key_type[RSS_MAX_LEVELS][6] = { + {FLOW_KEY_TYPE_IPV4, FLOW_KEY_TYPE_IPV6, FLOW_KEY_TYPE_TCP, + FLOW_KEY_TYPE_UDP, FLOW_KEY_TYPE_SCTP, FLOW_KEY_TYPE_ETH_DMAC}, + {FLOW_KEY_TYPE_INNR_IPV4, FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_INNR_TCP, FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_INNR_SCTP, FLOW_KEY_TYPE_INNR_ETH_DMAC}, + {FLOW_KEY_TYPE_IPV4 | FLOW_KEY_TYPE_INNR_IPV4, + FLOW_KEY_TYPE_IPV6 | FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_TCP | FLOW_KEY_TYPE_INNR_TCP, + FLOW_KEY_TYPE_UDP | FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_SCTP | FLOW_KEY_TYPE_INNR_SCTP, + FLOW_KEY_TYPE_ETH_DMAC | FLOW_KEY_TYPE_INNR_ETH_DMAC} + }; + uint32_t flowkey_cfg = 0; + + dev->ethdev_rss_hf = ethdev_rss; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + + if (ethdev_rss & ETH_RSS_C_VLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VLAN; + + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC; + + if (ethdev_rss & ETH_RSS_L3_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST; + + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC; + + if (ethdev_rss & ETH_RSS_L4_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST; + + if (ethdev_rss & RSS_IPV4_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX]; + + if (ethdev_rss & RSS_IPV6_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV6_INDEX]; + + if (ethdev_rss & ETH_RSS_TCP) + flowkey_cfg |= flow_key_type[rss_level][RSS_TCP_INDEX]; + + if (ethdev_rss & ETH_RSS_UDP) + flowkey_cfg |= flow_key_type[rss_level][RSS_UDP_INDEX]; + + if (ethdev_rss & ETH_RSS_SCTP) + flowkey_cfg |= flow_key_type[rss_level][RSS_SCTP_INDEX]; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= flow_key_type[rss_level][RSS_DMAC_INDEX]; + + if (ethdev_rss & RSS_IPV6_EX_ENABLE) + flowkey_cfg |= FLOW_KEY_TYPE_IPV6_EXT; + + if (ethdev_rss & ETH_RSS_PORT) + flowkey_cfg |= FLOW_KEY_TYPE_PORT; + + if (ethdev_rss & ETH_RSS_NVGRE) + flowkey_cfg |= FLOW_KEY_TYPE_NVGRE; + + if (ethdev_rss & ETH_RSS_VXLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VXLAN; + + if (ethdev_rss & ETH_RSS_GENEVE) + flowkey_cfg |= FLOW_KEY_TYPE_GENEVE; + + if (ethdev_rss & ETH_RSS_GTPU) + flowkey_cfg |= FLOW_KEY_TYPE_GTPU; + + return flowkey_cfg; +} + +static void +nix_free_queue_mem(struct cnxk_eth_dev *dev) +{ + plt_free(dev->rqs); + plt_free(dev->cqs); + plt_free(dev->sqs); + dev->rqs = NULL; + dev->cqs = NULL; + dev->sqs = NULL; +} + +static int +nix_rss_default_setup(struct cnxk_eth_dev *dev) +{ + struct rte_eth_dev *eth_dev = dev->eth_dev; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + uint64_t rss_hf; + + rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_hash_level = ETH_RSS_LEVEL(rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + + flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level); + return roc_nix_rss_default_setup(&dev->nix, flowkey_cfg); +} + +static int +nix_store_queue_cfg_and_then_release(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = NULL; + struct cnxk_eth_qconf *rx_qconf = NULL; + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_txq_sp *txq_sp; + int i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + tx_qconf = malloc(nb_txq * sizeof(*tx_qconf)); + if (tx_qconf == NULL) { + plt_err("Failed to allocate memory for tx_qconf"); + goto fail; + } + + rx_qconf = malloc(nb_rxq * sizeof(*rx_qconf)); + if (rx_qconf == NULL) { + plt_err("Failed to allocate memory for rx_qconf"); + goto fail; + } + + txq = eth_dev->data->tx_queues; + for (i = 0; i < nb_txq; i++) { + if (txq[i] == NULL) { + tx_qconf[i].valid = false; + plt_info("txq[%d] is already released", i); + continue; + } + txq_sp = ((struct cnxk_eth_txq_sp *)txq[i]) - 1; + memcpy(&tx_qconf[i], &txq_sp->qconf, sizeof(*tx_qconf)); + tx_qconf[i].valid = true; + dev_ops->tx_queue_release(txq[i]); + eth_dev->data->tx_queues[i] = NULL; + } + + rxq = eth_dev->data->rx_queues; + for (i = 0; i < nb_rxq; i++) { + if (rxq[i] == NULL) { + rx_qconf[i].valid = false; + plt_info("rxq[%d] is already released", i); + continue; + } + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq[i]) - 1; + memcpy(&rx_qconf[i], &rxq_sp->qconf, sizeof(*rx_qconf)); + rx_qconf[i].valid = true; + dev_ops->rx_queue_release(rxq[i]); + eth_dev->data->rx_queues[i] = NULL; + } + + dev->tx_qconf = tx_qconf; + dev->rx_qconf = rx_qconf; + return 0; + +fail: + free(tx_qconf); + free(rx_qconf); + return -ENOMEM; +} + +static int +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; + int rc, i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + rc = -ENOMEM; + /* Setup tx & rx queues with previous configuration so + * that the queues can be functional in cases like ports + * are started without re configuring queues. + * + * Usual re config sequence is like below: + * port_configure() { + * if(reconfigure) { + * queue_release() + * queue_setup() + * } + * queue_configure() { + * queue_release() + * queue_setup() + * } + * } + * port_start() + * + * In some application's control path, queue_configure() would + * NOT be invoked for TXQs/RXQs in port_configure(). + * In such cases, queues can be functional after start as the + * queues are already setup in port_configure(). + */ + for (i = 0; i < nb_txq; i++) { + if (!tx_qconf[i].valid) + continue; + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, + &tx_qconf[i].conf.tx); + if (rc) { + plt_err("Failed to setup tx queue rc=%d", rc); + txq = eth_dev->data->tx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->tx_queue_release(txq[i]); + goto fail; + } + } + + free(tx_qconf); + tx_qconf = NULL; + + for (i = 0; i < nb_rxq; i++) { + if (!rx_qconf[i].valid) + continue; + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, + &rx_qconf[i].conf.rx, + rx_qconf[i].mp); + if (rc) { + plt_err("Failed to setup rx queue rc=%d", rc); + rxq = eth_dev->data->rx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->rx_queue_release(rxq[i]); + goto tx_queue_release; + } + } + + free(rx_qconf); + rx_qconf = NULL; + + return 0; + +tx_queue_release: + txq = eth_dev->data->tx_queues; + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_release(txq[i]); +fail: + if (tx_qconf) + free(tx_qconf); + if (rx_qconf) + free(rx_qconf); + + return rc; +} + +static uint16_t +nix_eth_nop_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + RTE_SET_USED(queue); + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + return 0; +} + +static void +nix_set_nop_rxtx_function(struct rte_eth_dev *eth_dev) +{ + /* These dummy functions are required for supporting + * some applications which reconfigure queues without + * stopping tx burst and rx burst threads(eg kni app) + * When the queues context is saved, txq/rxqs are released + * which caused app crash since rx/tx burst is still + * on different lcores + */ + eth_dev->tx_pkt_burst = nix_eth_nop_burst; + eth_dev->rx_pkt_burst = nix_eth_nop_burst; + rte_mb(); +} + +int +cnxk_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + struct rte_eth_txmode *txmode = &conf->txmode; + char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix *nix = &dev->nix; + struct rte_ether_addr *ea; + uint8_t nb_rxq, nb_txq; + uint64_t rx_cfg; + void *qs; + int rc; + + rc = -EINVAL; + + /* Sanity checks */ + if (rte_eal_has_hugepages() == 0) { + plt_err("Huge page is not configured"); + goto fail_configure; + } + + if (conf->dcb_capability_en == 1) { + plt_err("dcb enable is not supported"); + goto fail_configure; + } + + if (conf->fdir_conf.mode != RTE_FDIR_MODE_NONE) { + plt_err("Flow director is not supported"); + goto fail_configure; + } + + if (rxmode->mq_mode != ETH_MQ_RX_NONE && + rxmode->mq_mode != ETH_MQ_RX_RSS) { + plt_err("Unsupported mq rx mode %d", rxmode->mq_mode); + goto fail_configure; + } + + if (txmode->mq_mode != ETH_MQ_TX_NONE) { + plt_err("Unsupported mq tx mode %d", txmode->mq_mode); + goto fail_configure; + } + + /* Free the resources allocated from the previous configure */ + if (dev->configured == 1) { + /* Unregister queue irq's */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister CQ irqs if present */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Set no-op functions */ + nix_set_nop_rxtx_function(eth_dev); + /* Store queue config for later */ + rc = nix_store_queue_cfg_and_then_release(eth_dev); + if (rc) + goto fail_configure; + roc_nix_tm_fini(nix); + roc_nix_lf_free(nix); + } + + dev->rx_offloads = rxmode->offloads; + dev->tx_offloads = txmode->offloads; + + /* Prepare rx cfg */ + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) { + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_OL4; + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_IL4; + } + rx_cfg |= (ROC_NIX_LF_RX_CFG_DROP_RE | ROC_NIX_LF_RX_CFG_L2_LEN_ERR | + ROC_NIX_LF_RX_CFG_LEN_IL4 | ROC_NIX_LF_RX_CFG_LEN_IL3 | + ROC_NIX_LF_RX_CFG_LEN_OL4 | ROC_NIX_LF_RX_CFG_LEN_OL3); + + nb_rxq = RTE_MAX(data->nb_rx_queues, 1); + nb_txq = RTE_MAX(data->nb_tx_queues, 1); + + /* Alloc a nix lf */ + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("Failed to init nix_lf rc=%d", rc); + goto fail_configure; + } + + nb_rxq = data->nb_rx_queues; + nb_txq = data->nb_tx_queues; + rc = -ENOMEM; + if (nb_rxq) { + /* Allocate memory for roc rq's and cq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_rq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc rqs"); + goto free_nix_lf; + } + dev->rqs = qs; + + qs = plt_zmalloc(sizeof(struct roc_nix_cq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc cqs"); + goto free_nix_lf; + } + dev->cqs = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_sq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc sqs"); + goto free_nix_lf; + } + dev->sqs = qs; + } + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + if (nix->rx_ptp_ena) { + plt_err("Both PTP and switch header enabled"); + goto free_nix_lf; + } + + rc = roc_nix_lso_fmt_setup(nix); + if (rc) { + plt_err("failed to setup nix lso format fields, rc=%d", rc); + goto free_nix_lf; + } + + /* Configure RSS */ + rc = nix_rss_default_setup(dev); + if (rc) { + plt_err("Failed to configure rss rc=%d", rc); + goto free_nix_lf; + } + + /* Init the default TM scheduler hierarchy */ + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("Failed to init traffic manager, rc=%d", rc); + goto free_nix_lf; + } + + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("Failed to enable default tm hierarchy, rc=%d", rc); + goto tm_fini; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Register cq IRQs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) { + if (eth_dev->data->nb_rx_queues > dev->nix.cints) { + plt_err("Rx interrupt cannot be enabled, rxq > %d", + dev->nix.cints); + goto q_irq_fini; + } + /* Rx interrupt feature cannot work with vector mode because, + * vector mode does not process packets unless min 4 pkts are + * received, while cq interrupts are generated even for 1 pkt + * in the CQ. + */ + dev->scalar_ena = true; + + rc = roc_nix_register_cq_irqs(nix); + if (rc) { + plt_err("Failed to register CQ interrupts rc=%d", rc); + goto q_irq_fini; + } + } + + /* Configure loop back mode */ + rc = roc_nix_mac_loopback_enable(nix, + eth_dev->data->dev_conf.lpbk_mode); + if (rc) { + plt_err("Failed to configure cgx loop back mode rc=%d", rc); + goto cq_fini; + } + + /* + * Restore queue config when reconfigure followed by + * reconfigure and no queue configure invoked from application case. + */ + if (dev->configured == 1) { + rc = nix_restore_queue_cfg(eth_dev); + if (rc) + goto cq_fini; + } + + /* Update the mac address */ + ea = eth_dev->data->mac_addrs; + memcpy(ea, dev->mac_addr, RTE_ETHER_ADDR_LEN); + if (rte_is_zero_ether_addr(ea)) + rte_eth_random_addr((uint8_t *)ea); + + rte_ether_format_addr(ea_fmt, RTE_ETHER_ADDR_FMT_SIZE, ea); + + plt_nix_dbg("Configured port%d mac=%s nb_rxq=%d nb_txq=%d" + " rx_offloads=0x%" PRIx64 " tx_offloads=0x%" PRIx64 "", + eth_dev->data->port_id, ea_fmt, nb_rxq, nb_txq, + dev->rx_offloads, dev->tx_offloads); + + /* All good */ + dev->configured = 1; + dev->nb_rxq = data->nb_rx_queues; + dev->nb_txq = data->nb_tx_queues; + return 0; + +cq_fini: + roc_nix_unregister_cq_irqs(nix); +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_nix_lf: + nix_free_queue_mem(dev); + rc |= roc_nix_lf_free(nix); +fail_configure: + dev->configured = 0; + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, @@ -76,6 +590,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->eth_dev = eth_dev; + dev->configured = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. @@ -157,6 +672,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; + /* Clear the flag since we are closing down */ + dev->configured = 0; + roc_nix_npc_rx_ena_dis(nix, false); /* Free up SQs */ @@ -183,6 +701,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (eth_dev->data->dev_conf.intr_conf.rxq) roc_nix_unregister_cq_irqs(nix); + /* Free ROC RQ's, SQ's and CQ's memory */ + nix_free_queue_mem(dev); + /* Free nix lf resources */ rc = roc_nix_lf_free(nix); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8d9a7e0..55da1da 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -65,10 +65,50 @@ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ DEV_RX_OFFLOAD_RSS_HASH) +#define RSS_IPV4_ENABLE \ + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV4_TCP | ETH_RSS_NONFRAG_IPV4_SCTP) + +#define RSS_IPV6_ENABLE \ + (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_UDP | \ + ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_NONFRAG_IPV6_SCTP) + +#define RSS_IPV6_EX_ENABLE \ + (ETH_RSS_IPV6_EX | ETH_RSS_IPV6_TCP_EX | ETH_RSS_IPV6_UDP_EX) + +#define RSS_MAX_LEVELS 3 + +#define RSS_IPV4_INDEX 0 +#define RSS_IPV6_INDEX 1 +#define RSS_TCP_INDEX 2 +#define RSS_UDP_INDEX 3 +#define RSS_SCTP_INDEX 4 +#define RSS_DMAC_INDEX 5 + +struct cnxk_eth_qconf { + union { + struct rte_eth_txconf tx; + struct rte_eth_rxconf rx; + } conf; + struct rte_mempool *mp; + uint16_t nb_desc; + uint8_t valid; +}; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC RQs, SQs and CQs */ + struct roc_nix_rq *rqs; + struct roc_nix_sq *sqs; + struct roc_nix_cq *cqs; + + /* Configured queue count */ + uint16_t nb_rxq; + uint16_t nb_txq; + uint8_t configured; + /* Max macfilter entries */ uint8_t max_mac_entries; @@ -90,11 +130,36 @@ struct cnxk_eth_dev { uint64_t rx_offload_capa; uint64_t tx_offload_capa; uint32_t speed_capa; + /* Configured Rx and Tx offloads */ + uint64_t rx_offloads; + uint64_t tx_offloads; + /* Platform specific offload flags */ + uint16_t rx_offload_flags; + uint16_t tx_offload_flags; + + /* ETHDEV RSS HF bitmask */ + uint64_t ethdev_rss_hf; + + /* Saved qconf before lf realloc */ + struct cnxk_eth_qconf *tx_qconf; + struct cnxk_eth_qconf *rx_qconf; /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; +struct cnxk_eth_rxq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + +struct cnxk_eth_txq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + static inline struct cnxk_eth_dev * cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) { @@ -110,6 +175,11 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_configure(struct rte_eth_dev *eth_dev); + +/* RSS */ +uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 06/44] net/cnxk: add link status update support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (4 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 05/44] net/cnxk: add device configuration operation Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 07/44] net/cnxk: add Rx queue setup and release Nithin Dabilpuram ` (40 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add link status update callback to get current link status. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 7 +++ drivers/net/cnxk/cnxk_ethdev.h | 8 +++ drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_link.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 73eb62a..a982450 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -20,6 +20,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Link state information Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 96dba2a..affbbd9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 616991c..836cc9f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a0bd2f1..29bb24f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f141027..c07827c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -554,6 +554,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, + .link_update = cnxk_nix_link_update, }; static int @@ -589,6 +590,9 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) goto error; } + /* Register up msg callbacks */ + roc_nix_mac_link_cb_register(nix, cnxk_eth_dev_link_status_cb); + dev->eth_dev = eth_dev; dev->configured = 0; @@ -677,6 +681,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable link status events */ + roc_nix_mac_link_event_start_stop(nix, false); + /* Free up SQs */ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 55da1da..6dad8ac 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,6 +15,9 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Used for struct cnxk_eth_dev::flags */ +#define CNXK_LINK_CFG_IN_PROGRESS_F BIT_ULL(0) + /* VLAN tag inserted by NIX_TX_VTAG_ACTION. * In Tx space is always reserved for this in FRS. */ @@ -181,6 +184,11 @@ int cnxk_nix_configure(struct rte_eth_dev *eth_dev); uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +/* Link */ +void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, + struct roc_nix_link_info *link); +int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c new file mode 100644 index 0000000..0223d68 --- /dev/null +++ b/drivers/net/cnxk/cnxk_link.c @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +static inline int +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) +{ + uint16_t wait = 1000; + + do { + rte_rmb(); + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) + break; + wait--; + rte_delay_ms(1); + } while (wait); + + return wait ? 0 : -1; +} + +static void +nix_link_status_print(struct rte_eth_dev *eth_dev, struct rte_eth_link *link) +{ + if (link && link->link_status) + plt_info("Port %d: Link Up - speed %u Mbps - %s", + (int)(eth_dev->data->port_id), + (uint32_t)link->link_speed, + link->link_duplex == ETH_LINK_FULL_DUPLEX + ? "full-duplex" + : "half-duplex"); + else + plt_info("Port %d: Link Down", (int)(eth_dev->data->port_id)); +} + +void +cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_link eth_link; + struct rte_eth_dev *eth_dev; + + if (!link || !nix) + return; + + eth_dev = dev->eth_dev; + if (!eth_dev || !eth_dev->data->dev_conf.intr_conf.lsc) + return; + + if (nix_wait_for_link_cfg(dev)) { + plt_err("Timeout waiting for link_cfg to complete"); + return; + } + + eth_link.link_status = link->status; + eth_link.link_speed = link->speed; + eth_link.link_autoneg = ETH_LINK_AUTONEG; + eth_link.link_duplex = link->full_duplex; + + /* Print link info */ + nix_link_status_print(eth_dev, ð_link); + + /* Update link info */ + rte_eth_linkstatus_set(eth_dev, ð_link); + + /* Set the flag and execute application callbacks */ + rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); +} + +int +cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_link_info info; + struct rte_eth_link link; + int rc; + + RTE_SET_USED(wait_to_complete); + memset(&link, 0, sizeof(struct rte_eth_link)); + + if (roc_nix_is_sdp(&dev->nix)) + return 0; + + if (roc_nix_is_lbk(&dev->nix)) { + link.link_status = ETH_LINK_UP; + link.link_speed = ETH_SPEED_NUM_100G; + link.link_autoneg = ETH_LINK_FIXED; + link.link_duplex = ETH_LINK_FULL_DUPLEX; + } else { + rc = roc_nix_mac_link_info_get(&dev->nix, &info); + if (rc) + return rc; + link.link_status = info.status; + link.link_speed = info.speed; + link.link_autoneg = ETH_LINK_AUTONEG; + if (info.full_duplex) + link.link_duplex = info.full_duplex; + } + + return rte_eth_linkstatus_set(eth_dev, &link); +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 45ccbe3..c0ec91a 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,7 +10,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', - 'cnxk_ethdev_devargs.c') + 'cnxk_ethdev_devargs.c', + 'cnxk_link.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 07/44] net/cnxk: add Rx queue setup and release 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (5 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 06/44] net/cnxk: add link status update support Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 08/44] net/cnxk: add Tx " Nithin Dabilpuram ` (39 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 9cf0f9e..f7e2f7b 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 4f50949..79c30aa 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index c07827c..dc41f78 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq) - 1; + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -555,6 +726,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6dad8ac..e938c64 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> #include <rte_kvargs.h> +#include <rte_mbuf.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> #include "roc_api.h" @@ -179,6 +182,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 08/44] net/cnxk: add Tx queue setup and release 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (6 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 07/44] net/cnxk: add Rx queue setup and release Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 09/44] net/cnxk: add packet type support Nithin Dabilpuram ` (38 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx queue setup and release for CN9K and CN10K. Release is common while setup is platform dependent due to differences in fast path Tx queue structures. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 71 +++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 12 +++++ drivers/net/cnxk/cn10k_tx.h | 13 +++++ drivers/net/cnxk/cn9k_ethdev.c | 69 ++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 10 ++++ drivers/net/cnxk/cn9k_tx.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev.c | 98 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++ 11 files changed, 292 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx.h diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index a9d2b03..462d7c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6a8ca1f..09e0d3a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index f761638..4a93a35 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -10,6 +10,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index f7e2f7b..e194b13 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,76 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + union nix_send_hdr_w0_u send_hdr_w0; + union nix_send_sg_s sg_w0; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + send_hdr_w0.u = 0; + sg_w0.u = 0; + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr_w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + } else { + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr_w0.sizem1 = 1; + } + + send_hdr_w0.sq = qid; + sg_w0.subdc = NIX_SUBDC_SG; + sg_w0.segs = 1; + sg_w0.ld_type = NIX_SENDLDTYPE_LDD; + + txq->send_hdr_w0 = send_hdr_w0.u; + txq->sg_w0 = sg_w0.u; + + rte_wmb(); +} + +static int +cn10k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + /* Store lmt base in tx queue for easy access */ + txq->lmt_base = dev->nix.lmt_base; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -76,6 +146,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 08e11bb..2157b16 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,6 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_txq { + uint64_t send_hdr_w0; + uint64_t sg_w0; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + uintptr_t lmt_base; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t cmd[4]; +} __plt_cache_aligned; + struct cn10k_eth_rxq { uint64_t mbuf_initializer; uintptr_t desc; diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h new file mode 100644 index 0000000..39d4755 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_TX_H__ +#define __CN10K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 79c30aa..e97ce15 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,74 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr->w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + sg = (union nix_send_sg_s *)&txq->cmd[4]; + } else { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr->w0.sizem1 = 1; + sg = (union nix_send_sg_s *)&txq->cmd[2]; + } + + send_hdr->w0.sq = qid; + sg->subdc = NIX_SUBDC_SG; + sg->segs = 1; + sg->ld_type = NIX_SENDLDTYPE_LDD; + + rte_wmb(); +} + +static int +cn9k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + txq->lmt_addr = sq->lmt_addr; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -87,6 +155,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 6384609..9ebf68f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,6 +6,16 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_txq { + uint64_t cmd[8]; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + void *lmt_addr; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; +} __plt_cache_aligned; + struct cn9k_eth_rxq { uint64_t mbuf_initializer; uint64_t data_off; diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h new file mode 100644 index 0000000..bb6379b --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_TX_H__ +#define __CN9K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index dc41f78..5772345 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -66,6 +66,103 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) return *tmp; } +static inline uint8_t +nix_sq_max_sqe_sz(struct cnxk_eth_dev *dev) +{ + /* + * Maximum three segments can be supported with W8, Choose + * NIX_MAXSQESZ_W16 for multi segment offload. + */ + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + return NIX_MAXSQESZ_W16; + else + return NIX_MAXSQESZ_W8; +} + +int +cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_txq_sp *txq_sp; + struct roc_nix_sq *sq; + size_t txq_sz; + int rc; + + /* Free memory prior to re-allocation if needed. */ + if (eth_dev->data->tx_queues[qid] != NULL) { + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->tx_queue_release(eth_dev->data->tx_queues[qid]); + eth_dev->data->tx_queues[qid] = NULL; + } + + /* Setup ROC SQ */ + sq = &dev->sqs[qid]; + sq->qid = qid; + sq->nb_desc = nb_desc; + sq->max_sqe_sz = nix_sq_max_sqe_sz(dev); + + rc = roc_nix_sq_init(&dev->nix, sq); + if (rc) { + plt_err("Failed to init sq=%d, rc=%d", qid, rc); + return rc; + } + + rc = -ENOMEM; + txq_sz = sizeof(struct cnxk_eth_txq_sp) + fp_tx_q_sz; + txq_sp = plt_zmalloc(txq_sz, PLT_CACHE_LINE_SIZE); + if (!txq_sp) { + plt_err("Failed to alloc tx queue mem"); + rc |= roc_nix_sq_fini(sq); + return rc; + } + + txq_sp->dev = dev; + txq_sp->qid = qid; + txq_sp->qconf.conf.tx = *tx_conf; + txq_sp->qconf.nb_desc = nb_desc; + + plt_nix_dbg("sq=%d fc=%p offload=0x%" PRIx64 " lmt_addr=%p" + " nb_sqb_bufs=%d sqes_per_sqb_log2=%d", + qid, sq->fc, dev->tx_offloads, sq->lmt_addr, + sq->nb_sqb_bufs, sq->sqes_per_sqb_log2); + + /* Store start of fast path area */ + eth_dev->data->tx_queues[qid] = txq_sp + 1; + eth_dev->data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + +static void +cnxk_nix_tx_queue_release(void *txq) +{ + struct cnxk_eth_txq_sp *txq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_sq *sq; + uint16_t qid; + int rc; + + if (!txq) + return; + + txq_sp = ((struct cnxk_eth_txq_sp *)txq) - 1; + dev = txq_sp->dev; + qid = txq_sp->qid; + + plt_nix_dbg("Releasing txq %u", qid); + + /* Cleanup ROC SQ */ + sq = &dev->sqs[qid]; + rc = roc_nix_sq_fini(sq); + if (rc) + plt_err("Failed to cleanup sq, rc=%d", rc); + + /* Finally free */ + plt_free(txq_sp); +} + int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, @@ -726,6 +823,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e938c64..90c8ff6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -182,6 +182,9 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf); int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 09/44] net/cnxk: add packet type support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (7 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 08/44] net/cnxk: add Tx " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 10/44] net/cnxk: add queue start and stop support Nithin Dabilpuram ` (37 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add support for packet type lookup on Rx to translate HW specific types to RTE_PTYPE_* defines Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 21 +++ drivers/net/cnxk/cn10k_rx.h | 11 ++ drivers/net/cnxk/cn9k_ethdev.c | 21 +++ drivers/net/cnxk/cn9k_rx.h | 12 ++ drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 14 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 12 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cnxk_lookup.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index a982450..4f1b58c 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,7 @@ Features Features of the CNXK Ethdev PMD are: +- Packet type information - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 462d7c4..503582c 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 09e0d3a..9ad225a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4a93a35..8c93ba7 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -13,6 +13,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index e194b13..efd5b67 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rx.h" #include "cn10k_tx.h" +static int +cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, uint16_t qid) @@ -113,6 +130,9 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -148,6 +168,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h new file mode 100644 index 0000000..d3d1661 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_RX_H__ +#define __CN10K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index e97ce15..3f3de4f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rx.h" #include "cn9k_tx.h" +static int +cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, uint16_t qid) @@ -111,6 +128,9 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -157,6 +177,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h new file mode 100644 index 0000000..95a1e69 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef __CN9K_RX_H__ +#define __CN9K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 5772345..96acf90 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -825,6 +825,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; static int @@ -865,6 +866,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->eth_dev = eth_dev; dev->configured = 0; + dev->ptype_disable = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 90c8ff6..6b7261c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,15 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ \ + ((PTYPE_NON_TUNNEL_ARRAY_SZ + PTYPE_TUNNEL_ARRAY_SZ) * sizeof(uint16_t)) +/* Fastpath lookup */ +#define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -119,6 +128,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + uint8_t ptype_disable; bool scalar_ena; /* Pointer back to rte */ @@ -201,6 +211,10 @@ void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +/* Lookup configuration */ +const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); +void *cnxk_nix_fastpath_lookup_mem_get(void); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_lookup.c b/drivers/net/cnxk/cnxk_lookup.c new file mode 100644 index 0000000..a9e51e5 --- /dev/null +++ b/drivers/net/cnxk/cnxk_lookup.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <rte_common.h> +#include <rte_memzone.h> + +#include "cnxk_ethdev.h" + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) * sizeof(uint32_t)) + +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ + SA_TBL_SZ) +const uint32_t * +cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) +{ + RTE_SET_USED(eth_dev); + + static const uint32_t ptypes[] = { + RTE_PTYPE_L2_ETHER_QINQ, /* LB */ + RTE_PTYPE_L2_ETHER_VLAN, /* LB */ + RTE_PTYPE_L2_ETHER_TIMESYNC, /* LB */ + RTE_PTYPE_L2_ETHER_ARP, /* LC */ + RTE_PTYPE_L2_ETHER_NSH, /* LC */ + RTE_PTYPE_L2_ETHER_FCOE, /* LC */ + RTE_PTYPE_L2_ETHER_MPLS, /* LC */ + RTE_PTYPE_L3_IPV4, /* LC */ + RTE_PTYPE_L3_IPV4_EXT, /* LC */ + RTE_PTYPE_L3_IPV6, /* LC */ + RTE_PTYPE_L3_IPV6_EXT, /* LC */ + RTE_PTYPE_L4_TCP, /* LD */ + RTE_PTYPE_L4_UDP, /* LD */ + RTE_PTYPE_L4_SCTP, /* LD */ + RTE_PTYPE_L4_ICMP, /* LD */ + RTE_PTYPE_L4_IGMP, /* LD */ + RTE_PTYPE_TUNNEL_GRE, /* LD */ + RTE_PTYPE_TUNNEL_ESP, /* LD */ + RTE_PTYPE_TUNNEL_NVGRE, /* LD */ + RTE_PTYPE_TUNNEL_VXLAN, /* LE */ + RTE_PTYPE_TUNNEL_GENEVE, /* LE */ + RTE_PTYPE_TUNNEL_GTPC, /* LE */ + RTE_PTYPE_TUNNEL_GTPU, /* LE */ + RTE_PTYPE_TUNNEL_VXLAN_GPE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_GRE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_UDP, /* LE */ + RTE_PTYPE_INNER_L2_ETHER, /* LF */ + RTE_PTYPE_INNER_L3_IPV4, /* LG */ + RTE_PTYPE_INNER_L3_IPV6, /* LG */ + RTE_PTYPE_INNER_L4_TCP, /* LH */ + RTE_PTYPE_INNER_L4_UDP, /* LH */ + RTE_PTYPE_INNER_L4_SCTP, /* LH */ + RTE_PTYPE_INNER_L4_ICMP, /* LH */ + RTE_PTYPE_UNKNOWN, + }; + + return ptypes; +} + +/* + * +------------------ +------------------ + + * | | IL4 | IL3| IL2 | TU | L4 | L3 | L2 | + * +-------------------+-------------------+ + * + * +-------------------+------------------ + + * | | LH | LG | LF | LE | LD | LC | LB | + * +-------------------+-------------------+ + * + * ptype [LE - LD - LC - LB] = TU - L4 - L3 - T2 + * ptype_tunnel[LH - LG - LF] = IL4 - IL3 - IL2 - TU + * + */ +static void +nix_create_non_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lb, lc, ld, le; + uint16_t val; + uint32_t idx; + + for (idx = 0; idx < PTYPE_NON_TUNNEL_ARRAY_SZ; idx++) { + lb = idx & 0xF; + lc = (idx & 0xF0) >> 4; + ld = (idx & 0xF00) >> 8; + le = (idx & 0xF000) >> 12; + val = RTE_PTYPE_UNKNOWN; + + switch (lb) { + case NPC_LT_LB_STAG_QINQ: + val |= RTE_PTYPE_L2_ETHER_QINQ; + break; + case NPC_LT_LB_CTAG: + val |= RTE_PTYPE_L2_ETHER_VLAN; + break; + } + + switch (lc) { + case NPC_LT_LC_ARP: + val |= RTE_PTYPE_L2_ETHER_ARP; + break; + case NPC_LT_LC_NSH: + val |= RTE_PTYPE_L2_ETHER_NSH; + break; + case NPC_LT_LC_FCOE: + val |= RTE_PTYPE_L2_ETHER_FCOE; + break; + case NPC_LT_LC_MPLS: + val |= RTE_PTYPE_L2_ETHER_MPLS; + break; + case NPC_LT_LC_IP: + val |= RTE_PTYPE_L3_IPV4; + break; + case NPC_LT_LC_IP_OPT: + val |= RTE_PTYPE_L3_IPV4_EXT; + break; + case NPC_LT_LC_IP6: + val |= RTE_PTYPE_L3_IPV6; + break; + case NPC_LT_LC_IP6_EXT: + val |= RTE_PTYPE_L3_IPV6_EXT; + break; + case NPC_LT_LC_PTP: + val |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + } + + switch (ld) { + case NPC_LT_LD_TCP: + val |= RTE_PTYPE_L4_TCP; + break; + case NPC_LT_LD_UDP: + val |= RTE_PTYPE_L4_UDP; + break; + case NPC_LT_LD_SCTP: + val |= RTE_PTYPE_L4_SCTP; + break; + case NPC_LT_LD_ICMP: + case NPC_LT_LD_ICMP6: + val |= RTE_PTYPE_L4_ICMP; + break; + case NPC_LT_LD_IGMP: + val |= RTE_PTYPE_L4_IGMP; + break; + case NPC_LT_LD_GRE: + val |= RTE_PTYPE_TUNNEL_GRE; + break; + case NPC_LT_LD_NVGRE: + val |= RTE_PTYPE_TUNNEL_NVGRE; + break; + } + + switch (le) { + case NPC_LT_LE_VXLAN: + val |= RTE_PTYPE_TUNNEL_VXLAN; + break; + case NPC_LT_LE_ESP: + val |= RTE_PTYPE_TUNNEL_ESP; + break; + case NPC_LT_LE_VXLANGPE: + val |= RTE_PTYPE_TUNNEL_VXLAN_GPE; + break; + case NPC_LT_LE_GENEVE: + val |= RTE_PTYPE_TUNNEL_GENEVE; + break; + case NPC_LT_LE_GTPC: + val |= RTE_PTYPE_TUNNEL_GTPC; + break; + case NPC_LT_LE_GTPU: + val |= RTE_PTYPE_TUNNEL_GTPU; + break; + case NPC_LT_LE_TU_MPLS_IN_GRE: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + break; + case NPC_LT_LE_TU_MPLS_IN_UDP: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_UDP; + break; + } + ptype[idx] = val; + } +} + +#define TU_SHIFT(x) ((x) >> PTYPE_NON_TUNNEL_WIDTH) +static void +nix_create_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lf, lg, lh; + uint16_t val; + uint32_t idx; + + /* Skip non tunnel ptype array memory */ + ptype = ptype + PTYPE_NON_TUNNEL_ARRAY_SZ; + + for (idx = 0; idx < PTYPE_TUNNEL_ARRAY_SZ; idx++) { + lf = idx & 0xF; + lg = (idx & 0xF0) >> 4; + lh = (idx & 0xF00) >> 8; + val = RTE_PTYPE_UNKNOWN; + + switch (lf) { + case NPC_LT_LF_TU_ETHER: + val |= TU_SHIFT(RTE_PTYPE_INNER_L2_ETHER); + break; + } + switch (lg) { + case NPC_LT_LG_TU_IP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV4); + break; + case NPC_LT_LG_TU_IP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV6); + break; + } + switch (lh) { + case NPC_LT_LH_TU_TCP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_TCP); + break; + case NPC_LT_LH_TU_UDP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_UDP); + break; + case NPC_LT_LH_TU_SCTP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_SCTP); + break; + case NPC_LT_LH_TU_ICMP: + case NPC_LT_LH_TU_ICMP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_ICMP); + break; + } + + ptype[idx] = val; + } +} + +static void +nix_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)((uint8_t *)mem + PTYPE_ARRAY_SZ); + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errlev = idx & 0xf; + errcode = (idx & 0xff0) >> 4; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case NPC_ERRLEV_RE: + /* Mark all errors as BAD checksum errors + * including Outer L2 length mismatch error + */ + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LC: + if (errcode == NPC_EC_OIP4_CSUM || + errcode == NPC_EC_IP_FRAG_OFFSET_1) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_EIP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LG: + if (errcode == NPC_EC_IIP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case NPC_ERRLEV_NIX: + if (errcode == NIX_RX_PERRCODE_OL4_CHK || + errcode == NIX_RX_PERRCODE_OL4_LEN || + errcode == NIX_RX_PERRCODE_OL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL4_CHK || + errcode == NIX_RX_PERRCODE_IL4_LEN || + errcode == NIX_RX_PERRCODE_IL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL3_LEN || + errcode == NIX_RX_PERRCODE_OL3_LEN) { + val |= PKT_RX_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + } + ol_flags[idx] = val; + } +} + +void * +cnxk_nix_fastpath_lookup_mem_get(void) +{ + const char name[] = CNXK_NIX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, SOCKET_ID_ANY, + 0, ROC_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the ptype array lookup memory */ + nix_create_non_tunnel_ptype_array(mem); + nix_create_tunnel_ptype_array(mem); + /* Form the rx ol_flags based on errcode */ + nix_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c0ec91a..b6b6989 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -11,7 +11,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', - 'cnxk_link.c') + 'cnxk_link.c', + 'cnxk_lookup.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 10/44] net/cnxk: add queue start and stop support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (8 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 09/44] net/cnxk: add packet type support Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 11/44] net/cnxk: add Rx support for cn9k Nithin Dabilpuram ` (36 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx/Tx queue start and stop callbacks for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 16 ++++++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++++++ drivers/net/cnxk/cnxk_ethdev.c | 92 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 1 + 7 files changed, 128 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 503582c..712f8d5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 9ad225a..82f2af0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8c93ba7..61fed11 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index efd5b67..1a9fcbb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -137,6 +137,21 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn10k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn10k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -168,6 +183,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 3f3de4f..3561632 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -135,6 +135,21 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn9k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn9k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -177,6 +192,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 96acf90..f1ba04f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -819,12 +819,104 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } +static int +cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc = -EINVAL; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, true); + if (rc) { + plt_err("Failed to enable sq aura fc, txq=%u, rc=%d", qid, rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +int +cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, false); + if (rc) { + plt_err("Failed to disable sqb aura fc, txq=%u, rc=%d", qid, + rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, true); + if (rc) { + plt_err("Failed to enable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, false); + if (rc) { + plt_err("Failed to disable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .tx_queue_start = cnxk_nix_tx_queue_start, + .rx_queue_start = cnxk_nix_rx_queue_start, + .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6b7261c..7e79a8d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -199,6 +199,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 11/44] net/cnxk: add Rx support for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (9 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 10/44] net/cnxk: add queue start and stop support Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram ` (35 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Jerin Jacob <jerinj@marvell.com> Add Rx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 3 + drivers/net/cnxk/cn9k_rx.c | 124 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 152 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/meson.build | 3 +- 5 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_rx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 9ebf68f..84dcc2c 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -30,4 +30,7 @@ struct cn9k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c new file mode 100644 index 0000000..1c05cf3 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.c @@ -0,0 +1,124 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +static inline uint16_t +nix_rx_nb_pkts(struct cn9k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define R(name, f3, f2, f1, f0, flags) \ + static uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 95a1e69..949fd95 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -7,6 +7,158 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->cn9k.pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = + nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 7e79a8d..58c6710 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,9 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +/* Default mark value used when none is provided. */ +#define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b6b6989..c2bfd94 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -15,7 +15,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_lookup.c') # CN9K -sources += files('cn9k_ethdev.c') +sources += files('cn9k_ethdev.c', + 'cn9k_rx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (10 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 11/44] net/cnxk: add Rx support for cn9k Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 13/44] net/cnxk: add Rx vector " Nithin Dabilpuram ` (34 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 26 ++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 55 ++++++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ 3 files changed, 82 insertions(+), 2 deletions(-) diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 1c05cf3..5535735 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -88,6 +88,15 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_recv_pkts_mseg_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ } NIX_RX_FASTPATH_MODES @@ -110,6 +119,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -118,7 +129,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 949fd95..a6b245f 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -99,6 +99,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -133,8 +180,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } #define RSS_F NIX_RX_OFFLOAD_RSS_F diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58c6710..a94f5eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 13/44] net/cnxk: add Rx vector version for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (11 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 14/44] net/cnxk: add Tx support " Nithin Dabilpuram ` (33 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 240 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 239 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 5535735..391f1e2 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include <rte_vect.h> + #include "cn9k_ethdev.h" #include "cn9k_rx.h" @@ -83,6 +85,223 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += nix_recv_pkts(rx_queue, &rx_pkts[packets], pkts_left, + flags); + + return packets; +} + +#else + +static inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define R(name, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -97,6 +316,14 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ (flags) | NIX_RX_MULTI_SEG_F); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, (flags)); \ } NIX_RX_FASTPATH_MODES @@ -137,7 +364,18 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 14/44] net/cnxk: add Tx support for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (12 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 13/44] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 15/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (32 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_tx.c | 103 ++++++++++++ drivers/net/cnxk/cn9k_tx.h | 357 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 71 ++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 534 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 84dcc2c..cd0938f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -32,5 +32,6 @@ struct cn9k_eth_rxq { /* Rx and Tx routines */ void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c new file mode 100644 index 0000000..06e9618 --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.c @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +static __rte_always_inline uint16_t +nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint16_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index bb6379b..5f915e8 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,10 +4,367 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn9k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(++sg) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, + const uint32_t flags) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_prep_lmt(uint64_t *cmd, void *lmt_addr, const uint32_t flags) +{ + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeor(io_addr); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeorl(io_addr); +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + #endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a94f5eb..8facafc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -103,6 +103,10 @@ /* Fastpath lookup */ #define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" +#define CNXK_NIX_UDP_TUN_BITMASK \ + ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ + (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -226,4 +230,71 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Inlines */ +static __rte_always_inline uint64_t +cnxk_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +cnxk_nix_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c2bfd94..d152f08 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -16,7 +16,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', - 'cn9k_rx.c') + 'cn9k_rx.c', + 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 15/44] net/cnxk: add Tx multi-segment version for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (13 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 14/44] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 16/44] net/cnxk: add Tx vector " Nithin Dabilpuram ` (31 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst multi-segment version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 70 +++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx.h | 105 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++ 3 files changed, 179 insertions(+) diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 06e9618..a474eb5 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -55,6 +55,44 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint64_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint16_t segdw; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -71,6 +109,25 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -89,6 +146,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, @@ -97,7 +156,18 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 5f915e8..d653b3c 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -294,6 +294,111 @@ cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) return roc_lmt_submit_ldeorl(io_addr); } +static __rte_always_inline uint16_t +cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Commit changes to mbuf */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + rte_io_wmb(); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_prep_lmt(uint64_t *cmd, void *lmt_addr, uint16_t segdw) +{ + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, rte_iova_t io_addr, + uint16_t segdw) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_io_wmb(); + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8facafc..2f31cba 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -46,6 +46,10 @@ #define CNXK_NIX_TX_NB_SEG_MAX 9 #endif +#define CNXK_NIX_TX_MSEG_SG_DWORDS \ + ((RTE_ALIGN_MUL_CEIL(CNXK_NIX_TX_NB_SEG_MAX, 3) / 3) + \ + CNXK_NIX_TX_NB_SEG_MAX) + #define CNXK_NIX_RSS_L3_L4_SRC_DST \ (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ ETH_RSS_L4_DST_ONLY) -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 16/44] net/cnxk: add Tx vector version for cn9k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (14 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 15/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 17/44] net/cnxk: add Rx support for cn10k Nithin Dabilpuram ` (30 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst vector version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 951 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 950 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index a474eb5..300ccd2 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include <rte_vect.h> + #include "cn9k_ethdev.h" #include "cn9k_tx.h" @@ -93,6 +95,921 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn9k_eth_txq *txq = tx_queue; + uint64_t *lmt_addr = txq->lmt_addr; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + uint64_t lmt_status, i; + uint16_t pkts_left; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->cmd[2]); + sgdesc23_w0 = sgdesc01_w0; + + for (i = 0; i < pkts; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + do { + vst1q_u64(lmt_addr, cmd00); + vst1q_u64(lmt_addr + 2, cmd01); + vst1q_u64(lmt_addr + 4, cmd10); + vst1q_u64(lmt_addr + 6, cmd11); + vst1q_u64(lmt_addr + 8, cmd20); + vst1q_u64(lmt_addr + 10, cmd21); + vst1q_u64(lmt_addr + 12, cmd30); + vst1q_u64(lmt_addr + 14, cmd31); + lmt_status = roc_lmt_submit_ldeor(io_addr); + + } while (lmt_status == 0); + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + if (unlikely(pkts_left)) + pkts += nix_xmit_pkts(tx_queue, tx_pkts, pkts_left, cmd, flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -128,6 +1045,25 @@ NIX_TX_FASTPATH_MODES NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -164,7 +1100,20 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 17/44] net/cnxk: add Rx support for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (15 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 16/44] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 18/44] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram ` (29 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Rx burst support for CN10K SoC. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.h | 3 + drivers/net/cnxk/cn10k_rx.c | 123 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_rx.h | 151 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.c diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 2157b16..e4332d3 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -32,4 +32,7 @@ struct cn10k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c new file mode 100644 index 0000000..1ff1b04 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.c @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +static inline uint16_t +nix_rx_nb_pkts(struct cn10k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define R(name, f3, f2, f1, f0, flags) \ + static uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index d3d1661..f43f320 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -6,6 +6,157 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index d152f08..1ac6b85 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -19,7 +19,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_tx.c') # CN10K -sources += files('cn10k_ethdev.c') +sources += files('cn10k_ethdev.c', + 'cn10k_rx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 18/44] net/cnxk: add Rx multi-segment version for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (16 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 17/44] net/cnxk: add Rx support for cn10k Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 19/44] net/cnxk: add Rx vector " Nithin Dabilpuram ` (28 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- doc/guides/nics/cnxk.rst | 2 ++ doc/guides/nics/features/cnxk.ini | 2 ++ doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 2 ++ drivers/net/cnxk/cn10k_rx.c | 27 ++++++++++++++++++ drivers/net/cnxk/cn10k_rx.h | 54 +++++++++++++++++++++++++++++++++-- 6 files changed, 86 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4f1b58c..789ec29 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,11 +17,13 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Jumbo frames - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - Link state information +- Scatter-Gather IO support Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 712f8d5..23564b7 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 82f2af0..421048d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 61fed11..e901fa2 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 1ff1b04..b98e7a1 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -88,6 +88,15 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_mseg_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ } NIX_RX_FASTPATH_MODES @@ -110,6 +119,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, @@ -118,6 +129,22 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index f43f320..7887a81 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -98,6 +98,52 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + ((rx->desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -131,8 +177,12 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } #define RSS_F NIX_RX_OFFLOAD_RSS_F -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 19/44] net/cnxk: add Rx vector version for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (17 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 18/44] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 20/44] net/cnxk: add Tx support " Nithin Dabilpuram ` (27 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + drivers/net/cnxk/cn10k_rx.c | 240 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 240 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 789ec29..4187e9d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Link state information - Scatter-Gather IO support +- Vector Poll mode driver Prerequisites ------------- diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index b98e7a1..2bc952d 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include <rte_vect.h> + #include "cn10k_ethdev.h" #include "cn10k_rx.h" @@ -83,6 +85,223 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += nix_recv_pkts(rx_queue, &rx_pkts[packets], pkts_left, + flags); + + return packets; +} + +#else + +static inline uint16_t +nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define R(name, f3, f2, f1, f0, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ @@ -97,6 +316,14 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ (flags) | NIX_RX_MULTI_SEG_F); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, (flags)); \ } NIX_RX_FASTPATH_MODES @@ -137,7 +364,18 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 20/44] net/cnxk: add Tx support for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (18 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 19/44] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 21/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (26 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 7 + doc/guides/nics/features/cnxk_vec.ini | 6 + doc/guides/nics/features/cnxk_vf.ini | 7 + drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_tx.c | 174 +++++++++++++++++ drivers/net/cnxk/cn10k_tx.h | 358 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 556 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4187e9d..555730d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -22,6 +22,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Inner and Outer Checksum offload - Link state information - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 23564b7..02be26b 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,11 +12,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 421048d..8c63853 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,10 +12,16 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index e901fa2..a1bd49b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,11 +11,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index e4332d3..58c51ab 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -34,5 +34,6 @@ struct cn10k_eth_rxq { /* Rx and Tx routines */ void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c new file mode 100644 index 0000000..0fad4c0 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.c @@ -0,0 +1,174 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_data(const uint16_t flags) +{ + const uint64_t dw_m1 = cn10k_nix_tx_ext_subs(flags) + 1; + uint64_t data = 0; + + /* This will be moved to addr area */ + data |= dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + +static __rte_always_inline uint16_t +nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + uintptr_t pa, lmt_addr = txq->lmt_base; + uint16_t lmt_id, burst, left, i; + uint64_t data; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + /* Get cmd skeleton */ + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 39d4755..ce1d4a0 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,10 +4,368 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn10k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, + const uint16_t flags) +{ + /* Send hdr */ + cmd[0] = txq->send_hdr_w0; + cmd[1] = 0; + cmd += 2; + + /* Send ext if present */ + if (flags & NIX_TX_NEED_EXT_HDR) { + *(__uint128_t *)cmd = *(const __uint128_t *)txq->cmd; + cmd += 2; + } + + /* Send sg */ + cmd[0] = txq->sg_w0; + cmd[1] = 0; +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, + const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(sg + 1) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } + + /* With minimal offloads, 'cmd' being local could be optimized out to + * registers. In other cases, 'cmd' will be in stack. Intent is + * 'cmd' stores content from txq->cmd which is copied only once. + */ + *((struct nix_send_hdr_s *)lmt_addr) = *send_hdr; + lmt_addr += 16; + if (flags & NIX_TX_NEED_EXT_HDR) { + *((struct nix_send_ext_s *)lmt_addr) = *send_hdr_ext; + lmt_addr += 16; + } + /* In case of multi-seg, sg template is stored here */ + *((union nix_send_sg_s *)lmt_addr) = *sg; + *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1ac6b85..3aa9f70 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -20,7 +20,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c', - 'cn10k_rx.c') + 'cn10k_rx.c', + 'cn10k_tx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 21/44] net/cnxk: add Tx multi-segment version for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (19 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 20/44] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 22/44] net/cnxk: add Tx vector " Nithin Dabilpuram ` (25 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 124 ++++++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx.h | 71 +++++++++++++++++++++++++ 2 files changed, 195 insertions(+) diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 0fad4c0..d170f31 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -125,6 +125,98 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; + const rte_iova_t io_addr = txq->io_addr; + uint16_t segdw, lmt_id, burst, left, i; + uint64_t data0, data1; + __uint128_t data128; + uint16_t shft; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + shft = 16; + data128 = 0; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + /* Store sg list directly on lmt line */ + segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, + flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + data128 |= (((__uint128_t)(segdw - 1)) << shft); + shft += 3; + } + + data0 = (uint64_t)data128; + data1 = (uint64_t)(data128 >> 64); + /* Make data0 similar to data1 */ + data0 >>= 16; + /* Trigger LMTST */ + if (burst > 16) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= (15ULL << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + + pa1 = io_addr | (data1 & 0x7) << 4; + data1 &= ~0x7ULL; + data1 <<= 16; + data1 |= ((uint64_t)(burst - 17)) << 12; + data1 |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data1, pa1); + } else if (burst) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= ((burst - 1) << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ @@ -142,6 +234,25 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz)]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -160,6 +271,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, @@ -168,7 +281,18 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index ce1d4a0..22f7a2b 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -295,6 +295,77 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); } +static __rte_always_inline uint16_t +cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Mark mempool object as "put" since it is freed by NIX + */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 22/44] net/cnxk: add Tx vector version for cn10k 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (20 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 21/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 23/44] net/cnxk: add device start and stop operations Nithin Dabilpuram ` (24 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst vector version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 988 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 987 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index d170f31..c487c83 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -2,6 +2,8 @@ * Copyright(C) 2021 Marvell. */ +#include <rte_vect.h> + #include "cn10k_ethdev.h" #include "cn10k_tx.h" @@ -217,6 +219,958 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3, data, pa; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint16_t left, scalar, burst, i, lmt_id; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t lmt_addr = txq->lmt_base; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + senddesc01_w0 = vld1q_dup_u64(&txq->send_hdr_w0); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->sg_w0); + sgdesc23_w0 = sgdesc01_w0; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + /* Store the prepared send desc to LMT lines */ + vst1q_u64((void *)lmt_addr, cmd00); + vst1q_u64((void *)(lmt_addr + 16), cmd01); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd10); + vst1q_u64((void *)(lmt_addr + 16), cmd11); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd20); + vst1q_u64((void *)(lmt_addr + 16), cmd21); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd30); + vst1q_u64((void *)(lmt_addr + 16), cmd31); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Reset lmt base addr to start another burst */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + if (unlikely(scalar)) + pkts += nix_xmit_pkts(tx_queue, tx_pkts, scalar, cmd, flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif #define T(name, f4, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ @@ -253,6 +1207,25 @@ NIX_TX_FASTPATH_MODES NIX_TX_FASTPATH_MODES #undef T +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + static uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -289,7 +1262,20 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 23/44] net/cnxk: add device start and stop operations 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (21 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 22/44] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 24/44] net/cnxk: add MAC address set ops Nithin Dabilpuram ` (23 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add device start and stop operation callbacks for CN9K and CN10K. Device stop is common for both platforms while device start as some platform dependent portion where the platform specific offload flags are recomputed and the right Rx/Tx burst function is chosen. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 84 ++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 127 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 90 ++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 2 + drivers/net/cnxk/cnxk_link.c | 11 ++++ 6 files changed, 438 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 555730d..42aa7a5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -39,6 +39,58 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1 + EAL: Detected 4 lcore(s) + EAL: Detected 1 NUMA nodes + EAL: Multi-process socket /var/run/dpdk/rte/mp_socket + EAL: Selected IOVA mode 'VA' + EAL: No available hugepages reported in hugepages-16777216kB + EAL: No available hugepages reported in hugepages-2048kB + EAL: Probing VFIO support... + EAL: VFIO support initialized + EAL: using IOMMU type 1 (Type 1) + [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98 + EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0) + PMD: RoC Model: cn10k + EAL: No legacy callbacks, legacy socket not created + testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0 + testpmd: preferred mempool ops selected: cn10k_mempool_ops + Configuring Port 0 (socket 0) + PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex + + Port 0: link state change event + Port 0: 96:D4:99:72:A5:BF + Checking link statuses... + Done + No commandline core given, start packet forwarding + io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native + Logical Core 3 (socket 0) forwards packets on 1 streams: + RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 + + io packet forwarding packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=1 + port 0: RX queue number: 1 Tx queue number: 1 + Rx offloads=0x0 Tx offloads=0x10000 + RX queue: 0 + RX desc=4096 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + RX Offloads=0x0 + TX queue: 0 + TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX offloads=0x0 - TX RS bit threshold=0 + Press enter to exit + Runtime Config Options ---------------------- @@ -132,3 +184,35 @@ Runtime Config Options Above devarg parameters are configurable per device, user needs to pass the parameters to all the PCIe devices if application requires to configure on all the ethdev ports. + +Limitations +----------- + +``mempool_cnxk`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager. +``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler +as it is performance wise most effective way for packet allocation and Tx buffer +recycling on OCTEON TX2 SoC platform. + +CRC stripping +~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by +the host interface irrespective of the offload configuration. + +Debugging Options +----------------- + +.. _table_cnxk_ethdev_debug_options: + +.. table:: cnxk ethdev debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NIX | --log-level='pmd\.net.cnxk,8' | + +---+------------+-------------------------------------------------------+ + | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | + +---+------------+-------------------------------------------------------+ diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 1a9fcbb..f9e0274 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -5,6 +5,98 @@ #include "cn10k_rx.h" #include "cn10k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn10k_eth_set_rx_function(eth_dev); return 0; } @@ -162,6 +255,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -169,6 +266,28 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -184,6 +303,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } @@ -221,6 +341,10 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (!eth_dev) return -ENOENT; + + /* Setup callbacks for secondary process */ + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); } return 0; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 3561632..c0ec0c6 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -5,6 +5,98 @@ #include "cn9k_rx.h" #include "cn9k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn9k_eth_set_rx_function(eth_dev); return 0; } @@ -171,6 +264,10 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -178,6 +275,28 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -193,6 +312,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } @@ -232,6 +352,13 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (!eth_dev) return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* Setup callbacks for secondary process */ + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; + } + dev = cnxk_eth_pmd_priv(eth_dev); /* Update capabilities already set for TSO. * TSO not supported for earlier chip revisions diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f1ba04f..ba05711 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -908,12 +908,102 @@ cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) return rc; } +static int +cnxk_nix_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct rte_mbuf *rx_pkts[32]; + int count, i, j, rc; + void *rxq; + + /* Disable switch hdr pkind */ + roc_nix_switch_hdr_set(&dev->nix, 0); + + /* Stop link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) + roc_nix_mac_link_event_start_stop(&dev->nix, false); + + /* Disable Rx via NPC */ + roc_nix_npc_rx_ena_dis(&dev->nix, false); + + /* Stop rx queues and free up pkts pending */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = dev_ops->rx_queue_stop(eth_dev, i); + if (rc) + continue; + + rxq = eth_dev->data->rx_queues[i]; + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + while (count) { + for (j = 0; j < count; j++) + rte_pktmbuf_free(rx_pkts[j]); + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + } + } + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_stop(eth_dev, i); + + return 0; +} + +int +cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc, i; + + /* Start rx queues */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = cnxk_nix_rx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Enable Rx in NPC */ + rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); + if (rc) { + plt_err("Failed to enable NPC rx %d", rc); + return rc; + } + + cnxk_nix_toggle_flag_link_cfg(dev, true); + + /* Start link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) { + rc = roc_nix_mac_link_event_start_stop(&dev->nix, true); + if (rc) { + plt_err("Failed to start cgx link event %d", rc); + goto rx_disable; + } + } + + cnxk_nix_toggle_flag_link_cfg(dev, false); + + return 0; + +rx_disable: + roc_nix_npc_rx_ena_dis(&dev->nix, false); + cnxk_nix_toggle_flag_link_cfg(dev, false); + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_stop = cnxk_nix_dev_stop, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 2f31cba..984f4fe 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -214,6 +214,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); +int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -222,6 +223,7 @@ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); /* Link */ +void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c index 0223d68..8728eb1 100644 --- a/drivers/net/cnxk/cnxk_link.c +++ b/drivers/net/cnxk/cnxk_link.c @@ -4,6 +4,17 @@ #include "cnxk_ethdev.h" +void +cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set) +{ + if (set) + dev->flags |= CNXK_LINK_CFG_IN_PROGRESS_F; + else + dev->flags &= ~CNXK_LINK_CFG_IN_PROGRESS_F; + + rte_wmb(); +} + static inline int nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 24/44] net/cnxk: add MAC address set ops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (22 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 23/44] net/cnxk: add device start and stop operations Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 25/44] net/cnxk: add MTU set device operation Nithin Dabilpuram ` (22 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Default mac address set operation is implemented for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ba05711..ed01087 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -999,6 +999,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 984f4fe..717a8d8 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -203,6 +203,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4a45956..87cf4ee 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -69,3 +69,32 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; return 0; } + +int +cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Update mac address at NPC */ + rc = roc_nix_npc_mac_addr_set(nix, addr->addr_bytes); + if (rc) + goto exit; + + /* Update mac address at CGX for PFs only */ + if (!roc_nix_is_vf_or_sdp(nix)) { + rc = roc_nix_mac_addr_set(nix, addr->addr_bytes); + if (rc) { + /* Rollback to previous mac address */ + roc_nix_npc_mac_addr_set(nix, dev->mac_addr); + goto exit; + } + } + + /* Update mac address to cnxk ethernet device */ + rte_memcpy(dev->mac_addr, addr->addr_bytes, RTE_ETHER_ADDR_LEN); + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 25/44] net/cnxk: add MTU set device operation 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (23 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 24/44] net/cnxk: add MAC address set ops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 26/44] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram ` (21 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> This Patch implements mtu set dev op for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 51 +++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 77 ++++++++++++++++++++++++++++++++++- 7 files changed, 135 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 42aa7a5..6cb90a7 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Inner and Outer Checksum offload - Link state information +- MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 02be26b..6fef725 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8c63853..79cb1e2 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a1bd49b..5cc9f3f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ed01087..9040ce6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,50 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +static void +nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + uint32_t buffsz; + + dev = rxq->dev; + eth_dev = dev->eth_dev; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->qconf.mp); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + if (eth_dev->data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + dev->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + dev->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } +} + +static int +nix_recalc_mtu(struct rte_eth_dev *eth_dev) +{ + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_eth_rxq_sp *rxq; + uint16_t mtu; + int rc; + + rxq = ((struct cnxk_eth_rxq_sp *)data->rx_queues[0]) - 1; + /* Setup scatter mode if needed by jumbo */ + nix_enable_mseg_on_jumbo(rxq); + + /* Setup MTU based on max_rx_pkt_len */ + mtu = data->dev_conf.rxmode.max_rx_pkt_len - CNXK_NIX_L2_OVERHEAD + + CNXK_NIX_MAX_VTAG_ACT_SIZE; + + rc = cnxk_nix_mtu_set(eth_dev, mtu); + if (rc) + plt_err("Failed to set default MTU size, rc=%d", rc); + + return rc; +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -955,6 +999,12 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; + if (eth_dev->data->nb_rx_queues != 0) { + rc = nix_recalc_mtu(eth_dev); + if (rc) + return rc; + } + /* Start rx queues */ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { rc = cnxk_nix_rx_queue_start(eth_dev, i); @@ -999,6 +1049,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mtu_set = cnxk_nix_mtu_set, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 717a8d8..3838573 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -28,7 +28,9 @@ #define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) /* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ -#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + \ + RTE_ETHER_CRC_LEN + \ + CNXK_NIX_MAX_VTAG_ACT_SIZE) #define CNXK_NIX_RX_MIN_DESC 16 #define CNXK_NIX_RX_MIN_DESC_ALIGN 16 @@ -203,6 +205,7 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 87cf4ee..21b55c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -20,7 +20,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; - devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->max_mtu = devinfo->max_rx_pktlen - + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN); devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; devinfo->rx_offload_capa = dev->rx_offload_capa; @@ -98,3 +99,77 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) exit: return rc; } + +int +cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) +{ + uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + uint32_t buffsz; + + /* Check if MTU is within the allowed range */ + if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { + plt_err("MTU is lesser than minimum"); + goto exit; + } + + if ((frame_size - RTE_ETHER_CRC_LEN) > + ((uint32_t)roc_nix_max_pkt_len(nix))) { + plt_err("MTU is greater than maximum"); + goto exit; + } + + buffsz = data->min_rx_buf_size - RTE_PKTMBUF_HEADROOM; + old_frame_size = data->mtu + CNXK_NIX_L2_OVERHEAD; + + /* Refuse MTU that requires the support of scattered packets + * when this feature has not been enabled before. + */ + if (data->dev_started && frame_size > buffsz && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER)) { + plt_err("Scatter offload is not enabled for mtu"); + goto exit; + } + + /* Check <seg size> * <max_seg> >= max_frame */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) && + frame_size > (buffsz * CNXK_NIX_RX_NB_SEG_MAX)) { + plt_err("Greater than maximum supported packet length"); + goto exit; + } + + frame_size -= RTE_ETHER_CRC_LEN; + + /* Update mtu on Tx */ + rc = roc_nix_mac_mtu_set(nix, frame_size); + if (rc) { + plt_err("Failed to set MTU, rc=%d", rc); + goto exit; + } + + /* Sync same frame size on Rx */ + rc = roc_nix_mac_max_rx_len_set(nix, frame_size); + if (rc) { + /* Rollback to older mtu */ + roc_nix_mac_mtu_set(nix, + old_frame_size - RTE_ETHER_CRC_LEN); + plt_err("Failed to max Rx frame length, rc=%d", rc); + goto exit; + } + + frame_size += RTE_ETHER_CRC_LEN; + + if (frame_size > RTE_ETHER_MAX_LEN) + dev->rx_offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; + else + dev->rx_offloads &= ~DEV_RX_OFFLOAD_JUMBO_FRAME; + + /* Update max_rx_pkt_len */ + data->dev_conf.rxmode.max_rx_pkt_len = frame_size; + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 26/44] net/cnxk: add promiscuous mode enable and disable 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (24 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 25/44] net/cnxk: add MTU set device operation Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 27/44] net/cnxk: add DMAC filter support Nithin Dabilpuram ` (20 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Add device operations to enable and disable promisc mode for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 56 +++++++++++++++++++++++++++++++++++ 6 files changed, 63 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6cb90a7..364e511 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,6 +17,7 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Promiscuous mode - Jumbo frames - SR-IOV VF - Lock-free Tx queue diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 6fef725..9b2e163 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y TSO = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 79cb1e2..31471e0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -16,6 +16,7 @@ Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 9040ce6..8d16dec 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1060,6 +1060,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, + .promiscuous_enable = cnxk_nix_promisc_enable, + .promiscuous_disable = cnxk_nix_promisc_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 3838573..73aef34 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -208,6 +208,8 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); +int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 21b55c4..6feb3a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -173,3 +173,59 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) exit: return rc; } + +int +cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, false); + return rc; + } + + return 0; +} + +int +cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, false); + if (rc < 0) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, false); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, true); + return rc; + } + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 27/44] net/cnxk: add DMAC filter support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (25 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 26/44] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 28/44] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram ` (19 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> DMAC filter support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 44 ++++++++++++++++++++++++++++++++--- 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 364e511..ce33f17 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -23,6 +23,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- MAC filtering - Inner and Outer Checksum offload - Link state information - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9b2e163..20d4d12 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 31471e0..e1de8ab 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8d16dec..171418a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1050,6 +1050,8 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, + .mac_addr_add = cnxk_nix_mac_addr_add, + .mac_addr_remove = cnxk_nix_mac_addr_del, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 73aef34..38ac654 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -139,6 +139,7 @@ struct cnxk_eth_dev { /* Max macfilter entries */ uint8_t max_mac_entries; + bool dmac_filter_enable; uint16_t flags; uint8_t ptype_disable; @@ -206,6 +207,10 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr, uint32_t index, + uint32_t pool); +void cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6feb3a9..fc60576 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -101,6 +101,43 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) } int +cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, + uint32_t index, uint32_t pool) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + PLT_SET_USED(index); + PLT_SET_USED(pool); + + rc = roc_nix_mac_addr_add(nix, addr->addr_bytes); + if (rc < 0) { + plt_err("Failed to add mac address, rc=%d", rc); + return rc; + } + + /* Enable promiscuous mode at NIX level */ + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} + +void +cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_mac_addr_del(nix, index); + if (rc) + plt_err("Failed to delete mac address, rc=%d", rc); +} + +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) { uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; @@ -212,8 +249,8 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (roc_nix_is_vf_or_sdp(nix)) return rc; - rc = roc_nix_npc_promisc_ena_dis(nix, false); - if (rc < 0) { + rc = roc_nix_npc_promisc_ena_dis(nix, dev->dmac_filter_enable); + if (rc) { plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, roc_error_msg_get(rc)); return rc; @@ -223,9 +260,10 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (rc) { plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, roc_error_msg_get(rc)); - roc_nix_npc_promisc_ena_dis(nix, true); + roc_nix_npc_promisc_ena_dis(nix, !dev->dmac_filter_enable); return rc; } + dev->dmac_filter_enable = false; return 0; } -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 28/44] net/cnxk: add all multicast enable/disable ethops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (26 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 27/44] net/cnxk: add DMAC filter support Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 29/44] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram ` (18 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> L2 multicast packets can be allowed or blocked. Patch implements corresponding ethops. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 17 +++++++++++++++++ 5 files changed, 23 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 20d4d12..b41af2d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e1de8ab..7fe8018 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 171418a..77a8c09 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1064,6 +1064,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, .promiscuous_enable = cnxk_nix_promisc_enable, .promiscuous_disable = cnxk_nix_promisc_disable, + .allmulticast_enable = cnxk_nix_allmulticast_enable, + .allmulticast_disable = cnxk_nix_allmulticast_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 38ac654..09031e9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -215,6 +215,8 @@ int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index fc60576..61ecbab 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -267,3 +267,20 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) dev->dmac_filter_enable = false; return 0; } + +int +cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, true, false); +} + +int +cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, false, + eth_dev->data->promiscuous); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 29/44] net/cnxk: add Rx/Tx burst mode get ops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (27 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 28/44] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 30/44] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram ` (17 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements ethdev operations to get Rx and Tx burst mode. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 127 ++++++++++++++++++++++++++++++++++ 6 files changed, 136 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b41af2d..298f167 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 7fe8018..a673cc1 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5cc9f3f..335d082 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 77a8c09..28fcf8c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1066,6 +1066,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .promiscuous_disable = cnxk_nix_promisc_disable, .allmulticast_enable = cnxk_nix_allmulticast_enable, .allmulticast_disable = cnxk_nix_allmulticast_disable, + .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, + .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 09031e9..481ede9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -219,6 +219,10 @@ int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); +int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 61ecbab..7ae961a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -72,6 +72,133 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) } int +cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } rx_offload_map[] = { + {DEV_RX_OFFLOAD_VLAN_STRIP, " VLAN Strip,"}, + {DEV_RX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_LRO, " TCP LRO,"}, + {DEV_RX_OFFLOAD_QINQ_STRIP, " QinQ VLAN Strip,"}, + {DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_MACSEC_STRIP, " MACsec Strip,"}, + {DEV_RX_OFFLOAD_HEADER_SPLIT, " Header Split,"}, + {DEV_RX_OFFLOAD_VLAN_FILTER, " VLAN Filter,"}, + {DEV_RX_OFFLOAD_VLAN_EXTEND, " VLAN Extend,"}, + {DEV_RX_OFFLOAD_JUMBO_FRAME, " Jumbo Frame,"}, + {DEV_RX_OFFLOAD_SCATTER, " Scattered,"}, + {DEV_RX_OFFLOAD_TIMESTAMP, " Timestamp,"}, + {DEV_RX_OFFLOAD_SECURITY, " Security,"}, + {DEV_RX_OFFLOAD_KEEP_CRC, " Keep CRC,"}, + {DEV_RX_OFFLOAD_SCTP_CKSUM, " SCTP,"}, + {DEV_RX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_RX_OFFLOAD_RSS_HASH, " RSS,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Rx Offloads:", + "Scalar, Rx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Rx offload info */ + for (i = 0; i < RTE_DIM(rx_offload_map); i++) { + if (dev->rx_offloads & rx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + rx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int +cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } tx_offload_map[] = { + {DEV_TX_OFFLOAD_VLAN_INSERT, " VLAN Insert,"}, + {DEV_TX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_TX_OFFLOAD_SCTP_CKSUM, " SCTP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_TSO, " TCP TSO,"}, + {DEV_TX_OFFLOAD_UDP_TSO, " UDP TSO,"}, + {DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_QINQ_INSERT, " QinQ VLAN Insert,"}, + {DEV_TX_OFFLOAD_VXLAN_TNL_TSO, " VXLAN Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GRE_TNL_TSO, " GRE Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IPIP_TNL_TSO, " IP-in-IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GENEVE_TNL_TSO, " Geneve Tunnel TSO,"}, + {DEV_TX_OFFLOAD_MACSEC_INSERT, " MACsec Insert,"}, + {DEV_TX_OFFLOAD_MT_LOCKFREE, " Multi Thread Lockless Tx,"}, + {DEV_TX_OFFLOAD_MULTI_SEGS, " Scattered,"}, + {DEV_TX_OFFLOAD_MBUF_FAST_FREE, " H/W MBUF Free,"}, + {DEV_TX_OFFLOAD_SECURITY, " Security,"}, + {DEV_TX_OFFLOAD_UDP_TNL_TSO, " UDP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IP_TNL_TSO, " IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP, " Timestamp,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Tx Offloads:", + "Scalar, Tx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Tx offload info */ + for (i = 0; i < RTE_DIM(tx_offload_map); i++) { + if (dev->tx_offloads & tx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + tx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 30/44] net/cnxk: add flow ctrl set/get ops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (28 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 29/44] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 31/44] net/cnxk: add link up/down operations Nithin Dabilpuram ` (16 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements set and get operations for flow control. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 74 +++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 95 +++++++++++++++++++++++++++++++++++ 6 files changed, 185 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ce33f17..96b2c5d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -26,6 +26,7 @@ Features of the CNXK Ethdev PMD are: - MAC filtering - Inner and Outer Checksum offload - Link state information +- Link flow control - MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 298f167..afd0f01 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index a673cc1..4bd11ce 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 28fcf8c..0ffc45b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -81,6 +81,55 @@ nix_recalc_mtu(struct rte_eth_dev *eth_dev) return rc; } +static int +nix_init_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_conf = {0}; + int rc; + + /* Both Rx & Tx flow ctrl get enabled(RTE_FC_FULL) in HW + * by AF driver, update those info in PMD structure. + */ + rc = cnxk_nix_flow_ctrl_get(eth_dev, &fc_conf); + if (rc) + goto exit; + + fc->mode = fc_conf.mode; + fc->rx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_RX_PAUSE); + fc->tx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_TX_PAUSE); + +exit: + return rc; +} + +static int +nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_cfg = {0}; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + return 0; + + fc_cfg.mode = fc->mode; + + /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ + if (roc_model_is_cn96_Ax() && + (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { + fc_cfg.mode = + (fc_cfg.mode == RTE_FC_FULL || + fc_cfg.mode == RTE_FC_TX_PAUSE) ? + RTE_FC_TX_PAUSE : RTE_FC_NONE; + } + + return cnxk_nix_flow_ctrl_set(eth_dev, &fc_cfg); +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -640,6 +689,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct rte_eth_rxmode *rxmode = &conf->rxmode; struct rte_eth_txmode *txmode = &conf->txmode; char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix_fc_cfg fc_cfg = {0}; struct roc_nix *nix = &dev->nix; struct rte_ether_addr *ea; uint8_t nb_rxq, nb_txq; @@ -820,6 +870,21 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Init flow control configuration */ + fc_cfg.cq_cfg_valid = false; + fc_cfg.rxchan_cfg.enable = true; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } + + /* Update flow control configuration to PMD */ + rc = nix_init_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } /* * Restore queue config when reconfigure followed by * reconfigure and no queue configure invoked from application case. @@ -1019,6 +1084,13 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } + /* Update Flow control configuration */ + rc = nix_update_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to enable flow control. error code(%d)", rc); + return rc; + } + /* Enable Rx in NPC */ rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); if (rc) { @@ -1068,6 +1140,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .allmulticast_disable = cnxk_nix_allmulticast_disable, .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, + .flow_ctrl_get = cnxk_nix_flow_ctrl_get, + .flow_ctrl_set = cnxk_nix_flow_ctrl_set, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 481ede9..77139d0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -113,6 +113,12 @@ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) +struct cnxk_fc_cfg { + enum rte_eth_fc_mode mode; + uint8_t rx_pause; + uint8_t tx_pause; +}; + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -174,6 +180,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Flow control configuration */ + struct cnxk_fc_cfg fc_cfg; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -223,6 +232,10 @@ int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); +int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7ae961a..eac50a2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -199,6 +199,101 @@ cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, } int +cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum rte_eth_fc_mode mode_map[] = { + RTE_FC_NONE, RTE_FC_RX_PAUSE, + RTE_FC_TX_PAUSE, RTE_FC_FULL + }; + struct roc_nix *nix = &dev->nix; + int mode; + + mode = roc_nix_fc_mode_get(nix); + if (mode < 0) + return mode; + + memset(fc_conf, 0, sizeof(struct rte_eth_fc_conf)); + fc_conf->mode = mode_map[mode]; + return 0; +} + +int +cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum roc_nix_fc_mode mode_map[] = { + ROC_NIX_FC_NONE, ROC_NIX_FC_RX, + ROC_NIX_FC_TX, ROC_NIX_FC_FULL + }; + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct roc_nix *nix = &dev->nix; + struct roc_nix_fc_cfg fc_cfg; + struct cnxk_eth_rxq_sp *rxq; + uint8_t rx_pause, tx_pause; + struct roc_nix_cq *cq; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) { + plt_err("Flow control configuration is not allowed on VFs"); + return -ENOTSUP; + } + + if (fc_conf->high_water || fc_conf->low_water || fc_conf->pause_time || + fc_conf->mac_ctrl_frame_fwd || fc_conf->autoneg) { + plt_info("Only MODE configuration is supported"); + return -EINVAL; + } + + if (fc_conf->mode == fc->mode) + return 0; + + rx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_RX_PAUSE); + tx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_TX_PAUSE); + + /* Check if TX pause frame is already enabled or not */ + if (fc->tx_pause ^ tx_pause) { + if (roc_model_is_cn96_Ax() && data->dev_started) { + /* On Ax, CQ should be in disabled state + * while setting flow control configuration. + */ + plt_info("Stop the port=%d for setting flow control", + data->port_id); + return 0; + } + + for (i = 0; i < data->nb_rx_queues; i++) { + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + rxq = ((struct cnxk_eth_rxq_sp *) + data->rx_queues[i]) - 1; + cq = &dev->cqs[rxq->qid]; + fc_cfg.cq_cfg_valid = true; + fc_cfg.cq_cfg.enable = tx_pause; + fc_cfg.cq_cfg.rq = rxq->qid; + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) + return rc; + } + } + + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); + if (rc) + return rc; + + fc->rx_pause = rx_pause; + fc->tx_pause = tx_pause; + fc->mode = fc_conf->mode; + + return rc; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 31/44] net/cnxk: add link up/down operations 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (29 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 30/44] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 32/44] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram ` (15 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements link up/down ethdev operations for cn9k and cn10k platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 4 +++- drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 47 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 0ffc45b..cb7404f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -928,7 +928,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } -static int +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -1142,6 +1142,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, .flow_ctrl_get = cnxk_nix_flow_ctrl_get, .flow_ctrl_set = cnxk_nix_flow_ctrl_set, + .dev_set_link_up = cnxk_nix_set_link_up, + .dev_set_link_down = cnxk_nix_set_link_down, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 77139d0..6500433 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -236,6 +236,9 @@ int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); +int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); + int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, @@ -244,6 +247,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index eac50a2..37ba211 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -506,3 +506,50 @@ cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) return roc_nix_npc_mcast_config(&dev->nix, false, eth_dev->data->promiscuous); } + +int +cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + rc = roc_nix_mac_link_state_set(nix, true); + if (rc) + goto exit; + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + goto exit; + } + +exit: + return rc; +} + +int +cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_stop(eth_dev, i); + if (rc) + goto exit; + } + + rc = roc_nix_mac_link_state_set(nix, false); +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 32/44] net/cnxk: add EEPROM module info get operations 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (30 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 31/44] net/cnxk: add link up/down operations Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 33/44] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram ` (14 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements eeprom module info get ethops for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 39 +++++++++++++++++++++++++++++++++++ 6 files changed, 48 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index afd0f01..b1e8641 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -31,6 +31,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4bd11ce..0f99634 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -29,6 +29,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 335d082..cecced9 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -26,6 +26,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cb7404f..97d8e6d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1144,6 +1144,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .flow_ctrl_set = cnxk_nix_flow_ctrl_set, .dev_set_link_up = cnxk_nix_set_link_up, .dev_set_link_down = cnxk_nix_set_link_down, + .get_module_info = cnxk_nix_get_module_info, + .get_module_eeprom = cnxk_nix_get_module_eeprom, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6500433..c4a562b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -238,6 +238,10 @@ int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); +int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo); +int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 37ba211..a1a963a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -553,3 +553,42 @@ cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) exit: return rc; } + +int +cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + modinfo->type = eeprom_info.sff_id; + modinfo->eeprom_len = ROC_NIX_EEPROM_SIZE; + return 0; +} + +int +cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + + if (!info->data || !info->length || + (info->offset + info->length > ROC_NIX_EEPROM_SIZE)) + return rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 33/44] net/cnxk: add Rx queue interrupt enable/disable ops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (31 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 32/44] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 34/44] net/cnxk: add validation API for mempool ops Nithin Dabilpuram ` (13 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Application may choose to enable/disable interrupts on Rx queues so that application can select its processing if no packets are available on queues for a longer period. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 7 files changed, 29 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 96b2c5d..6a001d9 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -30,6 +30,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Support Rx interrupt Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b1e8641..e5669f5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 0f99634..dff0c9b 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index cecced9..b950d2f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 97d8e6d..bfcc456 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1146,6 +1146,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_set_link_down = cnxk_nix_set_link_down, .get_module_info = cnxk_nix_get_module_info, .get_module_eeprom = cnxk_nix_get_module_eeprom, + .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, + .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c4a562b..76e1049 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -242,6 +242,10 @@ int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, struct rte_eth_dev_module_info *modinfo); int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, struct rte_dev_eeprom_info *info); +int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); +int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index a1a963a..34d4a42 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -592,3 +592,22 @@ cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); return 0; } + +int +cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_enable(&dev->nix, rx_queue_id); + return 0; +} + +int +cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 34/44] net/cnxk: add validation API for mempool ops 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (32 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 33/44] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 35/44] net/cnxk: add port/queue stats Nithin Dabilpuram ` (12 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> cn9k and cn10k supports platform specific mempool ops. This patch implements API to validate whether given mempool ops is supported or not. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 11 +++++++++++ 3 files changed, 13 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index bfcc456..8a76486 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1148,6 +1148,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_module_eeprom = cnxk_nix_get_module_eeprom, .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, + .pool_ops_supported = cnxk_nix_pool_ops_supported, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 76e1049..0b501f6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -246,6 +246,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); +int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 34d4a42..5b8bc53 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -611,3 +611,14 @@ cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); return 0; } + +int +cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) +{ + RTE_SET_USED(eth_dev); + + if (!strcmp(pool, rte_mbuf_platform_mempool_ops())) + return 0; + + return -ENOTSUP; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 35/44] net/cnxk: add port/queue stats 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (33 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 34/44] net/cnxk: add validation API for mempool ops Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 36/44] net/cnxk: add xstats apis Nithin Dabilpuram ` (11 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> This patch implements regular port statistics and queue mapping set api to get queue statistics Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 8 ++++ drivers/net/cnxk/cnxk_stats.c | 85 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_stats.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6a001d9..c2a6fbb 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -25,6 +25,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - MAC filtering - Inner and Outer Checksum offload +- Port hardware statistics - Link state information - Link flow control - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e5669f5..40952a9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index dff0c9b..32035bb 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -30,6 +30,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b950d2f..8060a68 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -27,6 +27,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8a76486..a798b14 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1149,6 +1149,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, .pool_ops_supported = cnxk_nix_pool_ops_supported, + .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, + .stats_get = cnxk_nix_stats_get, + .stats_reset = cnxk_nix_stats_reset, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0b501f6..5075e7c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -188,6 +188,10 @@ struct cnxk_eth_dev { /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + + /* Per queue statistics counters */ + uint32_t txq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; + uint32_t rxq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; struct cnxk_eth_rxq_sp { @@ -271,6 +275,10 @@ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx); +int cnxk_nix_stats_reset(struct rte_eth_dev *dev); +int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c new file mode 100644 index 0000000..24bff0b --- /dev/null +++ b/drivers/net/cnxk/cnxk_stats.c @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +int +cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct roc_nix_stats nix_stats; + int rc = 0, i; + + rc = roc_nix_stats_get(nix, &nix_stats); + if (rc) + goto exit; + + stats->opackets = nix_stats.tx_ucast; + stats->opackets += nix_stats.tx_mcast; + stats->opackets += nix_stats.tx_bcast; + stats->oerrors = nix_stats.tx_drop; + stats->obytes = nix_stats.tx_octs; + + stats->ipackets = nix_stats.rx_ucast; + stats->ipackets += nix_stats.rx_mcast; + stats->ipackets += nix_stats.rx_bcast; + stats->imissed = nix_stats.rx_drop; + stats->ibytes = nix_stats.rx_octs; + stats->ierrors = nix_stats.rx_err; + + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) { + struct roc_nix_stats_queue qstats; + uint16_t qidx; + + if (dev->txq_stat_map[i] & (1U << 31)) { + qidx = dev->txq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 0, &qstats); + if (rc) + goto exit; + stats->q_opackets[i] = qstats.tx_pkts; + stats->q_obytes[i] = qstats.tx_octs; + stats->q_errors[i] = qstats.tx_drop_pkts; + } + + if (dev->rxq_stat_map[i] & (1U << 31)) { + qidx = dev->rxq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 1, &qstats); + if (rc) + goto exit; + stats->q_ipackets[i] = qstats.rx_pkts; + stats->q_ibytes[i] = qstats.rx_octs; + stats->q_errors[i] += qstats.rx_drop_pkts; + } + } +exit: + return rc; +} + +int +cnxk_nix_stats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_stats_reset(&dev->nix); +} + +int +cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (is_rx) { + if (queue_id >= dev->nb_rxq) + return -EINVAL; + dev->rxq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } else { + if (queue_id >= dev->nb_txq) + return -EINVAL; + dev->txq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } + + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 3aa9f70..0e141d7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -12,7 +12,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', - 'cnxk_lookup.c') + 'cnxk_lookup.c', + 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 36/44] net/cnxk: add xstats apis 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (34 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 35/44] net/cnxk: add port/queue stats Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 37/44] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram ` (10 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Initial implementation of xstats operations. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 5 ++ drivers/net/cnxk/cnxk_ethdev.h | 11 +++ drivers/net/cnxk/cnxk_stats.c | 132 ++++++++++++++++++++++++++++++++++ 6 files changed, 151 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 40952a9..192c15a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -34,6 +34,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 32035bb..e990480 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -32,6 +32,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8060a68..3a4417c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -29,6 +29,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a798b14..a145aaa 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1152,6 +1152,11 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, .stats_get = cnxk_nix_stats_get, .stats_reset = cnxk_nix_stats_reset, + .xstats_get = cnxk_nix_xstats_get, + .xstats_get_names = cnxk_nix_xstats_get_names, + .xstats_reset = cnxk_nix_xstats_reset, + .xstats_get_by_id = cnxk_nix_xstats_get_by_id, + .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5075e7c..dd05c24 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -279,6 +279,17 @@ int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx); int cnxk_nix_stats_reset(struct rte_eth_dev *dev); int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); +int cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat *xstats, unsigned int n); +int cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit); +int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit); +int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n); +int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c index 24bff0b..ce9f9f4 100644 --- a/drivers/net/cnxk/cnxk_stats.c +++ b/drivers/net/cnxk/cnxk_stats.c @@ -83,3 +83,135 @@ cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, return 0; } + +int +cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat roc_xstats[n]; + int size, i; + + size = roc_nix_xstats_get(&dev->nix, roc_xstats, n); + + /* If requested array do not have space then return with count */ + if (size < 0 || size > (int)n) + return size; + + for (i = 0; i < size; i++) { + xstats[i].id = roc_xstats[i].id; + xstats[i].value = roc_xstats[i].value; + } + + return size; +} + +int +cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat_name roc_xstats_name[limit]; + struct roc_nix *nix = &dev->nix; + int size, i; + + if ((int)limit < roc_nix_num_xstats_get(nix) && xstats_names == NULL) + return roc_nix_num_xstats_get(nix); + + size = roc_nix_xstats_names_get(nix, roc_xstats_name, limit); + + for (i = 0; i < size; i++) + strlcpy(xstats_names[i].name, roc_xstats_name[i].name, + sizeof(xstats_names[i].name)); + + return size; +} + +int +cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat_name xnames[xstat_cnt]; + + if (limit < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (limit > xstat_cnt) + return -EINVAL; + + if (xstats_names == NULL) + return -ENOMEM; + + roc_nix_xstats_names_get(&dev->nix, xnames, limit); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + + strlcpy(xstats_names[i].name, xnames[ids[i]].name, + sizeof(xstats_names[i].name)); + } + + return limit; +} + +int +cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat xstats[xstat_cnt]; + + if (n < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (n > xstat_cnt) + return -EINVAL; + + if (values == NULL) + return -ENOMEM; + + roc_nix_xstats_get(&dev->nix, xstats, n); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + values[i] = xstats[ids[i]].value; + } + + return n; +} + +int +cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0, i; + + rc = roc_nix_stats_reset(nix); + if (rc) + goto exit; + + /* Reset Rx Queues */ + for (i = 0; i < dev->nb_rxq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 1); + if (rc) + goto exit; + } + + /* Reset Tx Queues */ + for (i = 0; i < dev->nb_txq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 0); + if (rc) + goto exit; + } + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 37/44] net/cnxk: add rxq/txq info get operations 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (35 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 36/44] net/cnxk: add xstats apis Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 38/44] net/cnxk: add device close and reset operations Nithin Dabilpuram ` (9 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Initial apis to get default queue information. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a145aaa..24c51b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1157,6 +1157,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .rxq_info_get = cnxk_nix_rxq_info_get, + .txq_info_get = cnxk_nix_txq_info_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index dd05c24..eeb6a53 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -290,6 +290,10 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo); +void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 5b8bc53..0bcba0c 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -622,3 +622,33 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } + +void +cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo) +{ + struct cnxk_eth_rxq_sp *rxq_sp = + ((struct cnxk_eth_rxq_sp *)eth_dev->data->rx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->mp = rxq_sp->qconf.mp; + qinfo->scattered_rx = eth_dev->data->scattered_rx; + qinfo->nb_desc = rxq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &rxq_sp->qconf.conf.rx, sizeof(qinfo->conf)); +} + +void +cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo) +{ + struct cnxk_eth_txq_sp *txq_sp = + ((struct cnxk_eth_txq_sp *)eth_dev->data->tx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->nb_desc = txq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 38/44] net/cnxk: add device close and reset operations 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (36 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 37/44] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 39/44] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram ` (8 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements device close and reset operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 24c51b4..86dabad 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1119,6 +1119,9 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } +static int cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev); +static int cnxk_nix_dev_close(struct rte_eth_dev *eth_dev); + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, @@ -1130,6 +1133,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, .dev_stop = cnxk_nix_dev_stop, + .dev_close = cnxk_nix_dev_close, + .dev_reset = cnxk_nix_dev_reset, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, @@ -1270,7 +1275,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } static int -cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; @@ -1324,14 +1329,11 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* Check if mbox close is needed */ - if (!mbox_close) - return 0; - rc = roc_nix_dev_fini(nix); /* Can be freed later by PMD if NPA LF is in use */ if (rc == -EAGAIN) { - eth_dev->data->dev_private = NULL; + if (!reset) + eth_dev->data->dev_private = NULL; return 0; } else if (rc) { plt_err("Failed in nix dev fini, rc=%d", rc); @@ -1340,6 +1342,25 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) return rc; } +static int +cnxk_nix_dev_close(struct rte_eth_dev *eth_dev) +{ + cnxk_eth_dev_uninit(eth_dev, false); + return 0; +} + +static int +cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev) +{ + int rc; + + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + return cnxk_eth_dev_init(eth_dev); +} + int cnxk_nix_remove(struct rte_pci_device *pci_dev) { @@ -1350,7 +1371,7 @@ cnxk_nix_remove(struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (eth_dev) { /* Cleanup eth dev */ - rc = cnxk_eth_dev_uninit(eth_dev, true); + rc = cnxk_eth_dev_uninit(eth_dev, false); if (rc) return rc; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 39/44] net/cnxk: add pending Tx mbuf cleanup operation 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (37 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 38/44] net/cnxk: add device close and reset operations Nithin Dabilpuram @ 2021-03-06 15:33 ` Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 40/44] net/cnxk: add support to configure npc Nithin Dabilpuram ` (7 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:33 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are accumalated as a pending mbuf. Hence operation is NOP for cnxk platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 3 files changed, 12 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 86dabad..5a2f90b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1164,6 +1164,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, + .tx_done_cleanup = cnxk_nix_tx_done_cleanup, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index eeb6a53..1ca52bc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -251,6 +251,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); +int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 0bcba0c..ff8afac 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -652,3 +652,13 @@ cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); } + +/* It is a NOP for cnxk as HW frees the buffer on xmit */ +int +cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) +{ + RTE_SET_USED(txq); + RTE_SET_USED(free_cnt); + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 40/44] net/cnxk: add support to configure npc 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (38 preceding siblings ...) 2021-03-06 15:33 ` [dpdk-dev] [PATCH 39/44] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram @ 2021-03-06 15:34 ` Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 41/44] net/cnxk: add initial version of rte flow support Nithin Dabilpuram ` (6 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:34 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Kiran Kumar K <kirankumark@marvell.com> Adding support to configure NPC on device initialization. This involves reading the MKEX and initializing the necessary data. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 25 ++++++++++++++++++++++--- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 3 +++ 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 5a2f90b..afe97f1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -8,7 +8,8 @@ nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) { uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; - if (roc_nix_is_vf_or_sdp(&dev->nix)) + if (roc_nix_is_vf_or_sdp(&dev->nix) || + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; return capa; @@ -120,6 +121,7 @@ nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ if (roc_model_is_cn96_Ax() && + dev->npc.switch_header_type != ROC_PRIV_FLAGS_HIGIG && (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { fc_cfg.mode = (fc_cfg.mode == RTE_FC_FULL || @@ -419,8 +421,10 @@ cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, dev->ethdev_rss_hf = ethdev_rss; - if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + if (ethdev_rss & ETH_RSS_L2_PAYLOAD && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_LEN_90B) { flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + } if (ethdev_rss & ETH_RSS_C_VLAN) flowkey_cfg |= FLOW_KEY_TYPE_VLAN; @@ -803,11 +807,18 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) roc_nix_err_intr_ena_dis(nix, true); roc_nix_ras_intr_ena_dis(nix, true); - if (nix->rx_ptp_ena) { + if (nix->rx_ptp_ena && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { plt_err("Both PTP and switch header enabled"); goto free_nix_lf; } + rc = roc_nix_switch_hdr_set(nix, dev->npc.switch_header_type); + if (rc) { + plt_err("Failed to enable switch type nix_lf rc=%d", rc); + goto free_nix_lf; + } + rc = roc_nix_lso_fmt_setup(nix); if (rc) { plt_err("failed to setup nix lso format fields, rc=%d", rc); @@ -1259,6 +1270,11 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->speed_capa = nix_get_speed_capa(dev); /* Initialize roc npc */ + dev->npc.roc_nix = nix; + rc = roc_npc_init(&dev->npc); + if (rc) + goto free_mac_addrs; + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, eth_dev->data->port_id, roc_nix_get_pf(nix), @@ -1292,6 +1308,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable and free rte_flow entries */ + roc_npc_fini(&dev->npc); + /* Disable link status events */ roc_nix_mac_link_event_start_stop(nix, false); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1ca52bc..e3b0bc1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -133,6 +133,9 @@ struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC NPC */ + struct roc_npc npc; + /* ROC RQs, SQs and CQs */ struct roc_nix_rq *rqs; struct roc_nix_sq *sqs; diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 4af2803..7fd06eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -150,6 +150,9 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->npc.flow_prealloc_size = flow_prealloc_size; + dev->npc.flow_max_priority = flow_max_priority; + dev->npc.switch_header_type = switch_header_type; return 0; exit: -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 41/44] net/cnxk: add initial version of rte flow support 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (39 preceding siblings ...) 2021-03-06 15:34 ` [dpdk-dev] [PATCH 40/44] net/cnxk: add support to configure npc Nithin Dabilpuram @ 2021-03-06 15:34 ` Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 42/44] net/cnxk: add filter ctrl operation Nithin Dabilpuram ` (5 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:34 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Kiran Kumar K <kirankumark@marvell.com> Adding initial version of rte_flow support for cnxk family device. Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_flush, flow_query, flow_isolate. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- doc/guides/nics/cnxk.rst | 118 ++++++++++++++ doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_rte_flow.c | 280 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 69 +++++++++ drivers/net/cnxk/meson.build | 1 + 7 files changed, 471 insertions(+) create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index c2a6fbb..87401f0 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - MAC filtering +- Generic flow API - Inner and Outer Checksum offload - Port hardware statistics - Link state information @@ -222,3 +223,120 @@ Debugging Options +---+------------+-------------------------------------------------------+ | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | +---+------------+-------------------------------------------------------+ + +RTE Flow Support +---------------- + +The OCTEON CN9K/CN10K SoC family NIC has support for the following patterns and +actions. + +Patterns: + +.. _table_cnxk_supported_flow_item_types: + +.. table:: Item types + + +----+--------------------------------+ + | # | Pattern Type | + +====+================================+ + | 1 | RTE_FLOW_ITEM_TYPE_ETH | + +----+--------------------------------+ + | 2 | RTE_FLOW_ITEM_TYPE_VLAN | + +----+--------------------------------+ + | 3 | RTE_FLOW_ITEM_TYPE_E_TAG | + +----+--------------------------------+ + | 4 | RTE_FLOW_ITEM_TYPE_IPV4 | + +----+--------------------------------+ + | 5 | RTE_FLOW_ITEM_TYPE_IPV6 | + +----+--------------------------------+ + | 6 | RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4| + +----+--------------------------------+ + | 7 | RTE_FLOW_ITEM_TYPE_MPLS | + +----+--------------------------------+ + | 8 | RTE_FLOW_ITEM_TYPE_ICMP | + +----+--------------------------------+ + | 9 | RTE_FLOW_ITEM_TYPE_UDP | + +----+--------------------------------+ + | 10 | RTE_FLOW_ITEM_TYPE_TCP | + +----+--------------------------------+ + | 11 | RTE_FLOW_ITEM_TYPE_SCTP | + +----+--------------------------------+ + | 12 | RTE_FLOW_ITEM_TYPE_ESP | + +----+--------------------------------+ + | 13 | RTE_FLOW_ITEM_TYPE_GRE | + +----+--------------------------------+ + | 14 | RTE_FLOW_ITEM_TYPE_NVGRE | + +----+--------------------------------+ + | 15 | RTE_FLOW_ITEM_TYPE_VXLAN | + +----+--------------------------------+ + | 16 | RTE_FLOW_ITEM_TYPE_GTPC | + +----+--------------------------------+ + | 17 | RTE_FLOW_ITEM_TYPE_GTPU | + +----+--------------------------------+ + | 18 | RTE_FLOW_ITEM_TYPE_GENEVE | + +----+--------------------------------+ + | 19 | RTE_FLOW_ITEM_TYPE_VXLAN_GPE | + +----+--------------------------------+ + | 20 | RTE_FLOW_ITEM_TYPE_IPV6_EXT | + +----+--------------------------------+ + | 21 | RTE_FLOW_ITEM_TYPE_VOID | + +----+--------------------------------+ + | 22 | RTE_FLOW_ITEM_TYPE_ANY | + +----+--------------------------------+ + | 23 | RTE_FLOW_ITEM_TYPE_GRE_KEY | + +----+--------------------------------+ + | 24 | RTE_FLOW_ITEM_TYPE_HIGIG2 | + +----+--------------------------------+ + +.. note:: + + ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing + bits in the GRE header are equal to 0. + +Actions: + +.. _table_cnxk_supported_ingress_action_types: + +.. table:: Ingress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_VOID | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_MARK | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_FLAG | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 6 | RTE_FLOW_ACTION_TYPE_QUEUE | + +----+-----------------------------------------+ + | 7 | RTE_FLOW_ACTION_TYPE_RSS | + +----+-----------------------------------------+ + | 8 | RTE_FLOW_ACTION_TYPE_PF | + +----+-----------------------------------------+ + | 9 | RTE_FLOW_ACTION_TYPE_VF | + +----+-----------------------------------------+ + | 10 | RTE_FLOW_ACTION_TYPE_OF_POP_VLAN | + +----+-----------------------------------------+ + +.. _table_cnxk_supported_egress_action_types: + +.. table:: Egress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP | + +----+-----------------------------------------+ diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 192c15a..7b6d832 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -25,6 +25,7 @@ Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Flow control = Y +Flow API = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e990480..ef37088 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -24,6 +24,7 @@ Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Flow control = Y +Flow API = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 3a4417c..69419d1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -20,6 +20,7 @@ MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y +Flow API = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c new file mode 100644 index 0000000..344e01f --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_rte_flow.h> + +static int +cnxk_map_actions(struct rte_eth_dev *dev, + const struct rte_flow_action actions[], + struct roc_npc_action in_actions[]) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + const struct rte_flow_action_count *act_count; + const struct rte_flow_action_queue *act_q; + int rq; + int i = 0; + + RTE_SET_USED(hw); + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VOID; + break; + + case RTE_FLOW_ACTION_TYPE_MARK: + in_actions[i].type = ROC_NPC_ACTION_TYPE_MARK; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_FLAG: + in_actions[i].type = ROC_NPC_ACTION_TYPE_FLAG; + break; + + case RTE_FLOW_ACTION_TYPE_COUNT: + act_count = (const struct rte_flow_action_count *) + actions->conf; + + if (act_count->shared == 1) { + plt_npc_dbg("Shared counter is not supported"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_COUNT; + break; + + case RTE_FLOW_ACTION_TYPE_DROP: + in_actions[i].type = ROC_NPC_ACTION_TYPE_DROP; + break; + + case RTE_FLOW_ACTION_TYPE_PF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_PF; + break; + + case RTE_FLOW_ACTION_TYPE_VF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VF; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_QUEUE: + act_q = (const struct rte_flow_action_queue *) + actions->conf; + rq = act_q->index; + if (rq >= dev->data->nb_rx_queues) { + plt_npc_dbg("Invalid queue index"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_QUEUE; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_RSS: + in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + break; + + case RTE_FLOW_ACTION_TYPE_SECURITY: + in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; + break; + default: + plt_npc_dbg("Action is not supported = %d", + actions->type); + goto err_exit; + } + i++; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_END; + return 0; + +err_exit: + return -EINVAL; +} + +static int +cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct roc_npc_attr *in_attr, + struct roc_npc_item_info in_pattern[], + struct roc_npc_action in_actions[]) +{ + int i = 0; + + in_attr->priority = attr->priority; + in_attr->ingress = attr->ingress; + in_attr->egress = attr->egress; + + while (pattern->type != RTE_FLOW_ITEM_TYPE_END) { + in_pattern[i].spec = pattern->spec; + in_pattern[i].last = pattern->last; + in_pattern[i].mask = pattern->mask; + in_pattern[i].type = term[pattern->type].item_type; + in_pattern[i].size = term[pattern->type].item_size; + pattern++; + i++; + } + in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; + + return cnxk_map_actions(dev, actions, in_actions); +} + +static int +cnxk_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow flow; + int rc; + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return rc; + } + + return roc_npc_flow_parse(npc, &in_attr, in_pattern, in_actions, &flow); +} + +static struct rte_flow * +cnxk_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow *flow; + int errcode; + int rc; + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return NULL; + } + + flow = roc_npc_flow_create(npc, &in_attr, in_pattern, in_actions, + &errcode); + if (errcode != 0) { + rte_flow_error_set(error, errcode, errcode, NULL, + roc_error_msg_get(errcode)); + return NULL; + } + + return (struct rte_flow *)flow; +} + +static int +cnxk_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_flow_destroy(npc, in_flow); + if (rc) + rte_flow_error_set(error, rc, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow Destroy failed"); + return rc; +} + +static int +cnxk_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_mcam_free_all_resources(npc); + if (rc) { + rte_flow_error_set(error, EIO, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Failed to flush filter"); + return -rte_errno; + } + + return 0; +} + +static int +cnxk_flow_query(struct rte_eth_dev *dev, struct rte_flow *flow, + const struct rte_flow_action *action, void *data, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct rte_flow_query_count *query = data; + const char *errmsg = NULL; + int errcode = ENOTSUP; + int rc; + + if (action->type != RTE_FLOW_ACTION_TYPE_COUNT) { + errmsg = "Only COUNT is supported in query"; + goto err_exit; + } + + if (in_flow->ctr_id == NPC_COUNTER_NONE) { + errmsg = "Counter is not available"; + goto err_exit; + } + + rc = roc_npc_mcam_read_counter(npc, in_flow->ctr_id, &query->hits); + if (rc != 0) { + errcode = EIO; + errmsg = "Error reading flow counter"; + goto err_exit; + } + query->hits_set = 1; + query->bytes_set = 0; + + if (query->reset) + rc = roc_npc_mcam_clear_counter(npc, in_flow->ctr_id); + if (rc != 0) { + errcode = EIO; + errmsg = "Error clearing flow counter"; + goto err_exit; + } + + return 0; + +err_exit: + rte_flow_error_set(error, errcode, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, errmsg); + return -rte_errno; +} + +static int +cnxk_flow_isolate(struct rte_eth_dev *dev __rte_unused, int enable __rte_unused, + struct rte_flow_error *error) +{ + /* If we support, we need to un-install the default mcam + * entry for this port. + */ + + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow isolation not supported"); + + return -rte_errno; +} + +const struct rte_flow_ops cnxk_flow_ops = { + .validate = cnxk_flow_validate, + .create = cnxk_flow_create, + .destroy = cnxk_flow_destroy, + .flush = cnxk_flow_flush, + .query = cnxk_flow_query, + .isolate = cnxk_flow_isolate, +}; diff --git a/drivers/net/cnxk/cnxk_rte_flow.h b/drivers/net/cnxk/cnxk_rte_flow.h new file mode 100644 index 0000000..3740226 --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_RTE_FLOW_H__ +#define __CNXK_RTE_FLOW_H__ + +#include <rte_flow_driver.h> +#include <rte_malloc.h> + +#include "cnxk_ethdev.h" +#include "roc_api.h" +#include "roc_npc_priv.h" + +struct cnxk_rte_flow_term_info { + uint16_t item_type; + uint16_t item_size; +}; + +struct cnxk_rte_flow_term_info term[] = { + [RTE_FLOW_ITEM_TYPE_ETH] = {ROC_NPC_ITEM_TYPE_ETH, + sizeof(struct rte_flow_item_eth)}, + [RTE_FLOW_ITEM_TYPE_VLAN] = {ROC_NPC_ITEM_TYPE_VLAN, + sizeof(struct rte_flow_item_vlan)}, + [RTE_FLOW_ITEM_TYPE_E_TAG] = {ROC_NPC_ITEM_TYPE_E_TAG, + sizeof(struct rte_flow_item_e_tag)}, + [RTE_FLOW_ITEM_TYPE_IPV4] = {ROC_NPC_ITEM_TYPE_IPV4, + sizeof(struct rte_flow_item_ipv4)}, + [RTE_FLOW_ITEM_TYPE_IPV6] = {ROC_NPC_ITEM_TYPE_IPV6, + sizeof(struct rte_flow_item_ipv6)}, + [RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4] = {ROC_NPC_ITEM_TYPE_ARP_ETH_IPV4, + sizeof(struct rte_flow_item_arp_eth_ipv4)}, + [RTE_FLOW_ITEM_TYPE_MPLS] = {ROC_NPC_ITEM_TYPE_MPLS, + sizeof(struct rte_flow_item_mpls)}, + [RTE_FLOW_ITEM_TYPE_ICMP] = {ROC_NPC_ITEM_TYPE_ICMP, + sizeof(struct rte_flow_item_icmp)}, + [RTE_FLOW_ITEM_TYPE_UDP] = {ROC_NPC_ITEM_TYPE_UDP, + sizeof(struct rte_flow_item_udp)}, + [RTE_FLOW_ITEM_TYPE_TCP] = {ROC_NPC_ITEM_TYPE_TCP, + sizeof(struct rte_flow_item_tcp)}, + [RTE_FLOW_ITEM_TYPE_SCTP] = {ROC_NPC_ITEM_TYPE_SCTP, + sizeof(struct rte_flow_item_sctp)}, + [RTE_FLOW_ITEM_TYPE_ESP] = {ROC_NPC_ITEM_TYPE_ESP, + sizeof(struct rte_flow_item_esp)}, + [RTE_FLOW_ITEM_TYPE_GRE] = {ROC_NPC_ITEM_TYPE_GRE, + sizeof(struct rte_flow_item_gre)}, + [RTE_FLOW_ITEM_TYPE_NVGRE] = {ROC_NPC_ITEM_TYPE_NVGRE, + sizeof(struct rte_flow_item_nvgre)}, + [RTE_FLOW_ITEM_TYPE_VXLAN] = {ROC_NPC_ITEM_TYPE_VXLAN, + sizeof(struct rte_flow_item_vxlan)}, + [RTE_FLOW_ITEM_TYPE_GTPC] = {ROC_NPC_ITEM_TYPE_GTPC, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GTPU] = {ROC_NPC_ITEM_TYPE_GTPU, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GENEVE] = {ROC_NPC_ITEM_TYPE_GENEVE, + sizeof(struct rte_flow_item_geneve)}, + [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = {ROC_NPC_ITEM_TYPE_VXLAN_GPE, + sizeof(struct rte_flow_item_vxlan_gpe)}, + [RTE_FLOW_ITEM_TYPE_IPV6_EXT] = {ROC_NPC_ITEM_TYPE_IPV6_EXT, + sizeof(struct rte_flow_item_ipv6_ext)}, + [RTE_FLOW_ITEM_TYPE_VOID] = {ROC_NPC_ITEM_TYPE_VOID, 0}, + [RTE_FLOW_ITEM_TYPE_ANY] = {ROC_NPC_ITEM_TYPE_ANY, 0}, + [RTE_FLOW_ITEM_TYPE_GRE_KEY] = {ROC_NPC_ITEM_TYPE_GRE_KEY, + sizeof(uint32_t)}, + [RTE_FLOW_ITEM_TYPE_HIGIG2] = { + ROC_NPC_ITEM_TYPE_HIGIG2, + sizeof(struct rte_flow_item_higig2_hdr)} +}; + +#endif /* __CNXK_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 0e141d7..49aa016 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_rte_flow.c', 'cnxk_stats.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 42/44] net/cnxk: add filter ctrl operation 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (40 preceding siblings ...) 2021-03-06 15:34 ` [dpdk-dev] [PATCH 41/44] net/cnxk: add initial version of rte flow support Nithin Dabilpuram @ 2021-03-06 15:34 ` Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 43/44] net/cnxk: add ethdev firmware version get Nithin Dabilpuram ` (4 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:34 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> This patch adds filter_ctrl operation to enable rte_flow_ops. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 3 +++ drivers/net/cnxk/cnxk_ethdev.h | 6 +++++- drivers/net/cnxk/cnxk_ethdev_ops.c | 21 +++++++++++++++++++++ 4 files changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 0efe080..b862e23 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -645,6 +645,8 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, struct npc_flow_list *list; int rc; + npc->channel = roc_npc->channel; + flow = plt_zmalloc(sizeof(*flow), 0); if (flow == NULL) { *errcode = NPC_ERR_NO_MEM; diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index afe97f1..347428e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -773,6 +773,8 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto fail_configure; } + dev->npc.channel = roc_nix_get_base_chan(nix); + nb_rxq = data->nb_rx_queues; nb_txq = data->nb_tx_queues; rc = -ENOMEM; @@ -1176,6 +1178,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, + .filter_ctrl = cnxk_nix_filter_ctrl, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e3b0bc1..7cf7cf7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -218,6 +218,8 @@ cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; +extern const struct rte_flow_ops cnxk_flow_ops; + /* Ops */ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); @@ -255,7 +257,9 @@ int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); - +int cnxk_nix_filter_ctrl(struct rte_eth_dev *eth_dev, + enum rte_filter_type filter_type, + enum rte_filter_op filter_op, void *arg); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index ff8afac..00f1fe7 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -294,6 +294,27 @@ cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, } int +cnxk_nix_filter_ctrl(struct rte_eth_dev *eth_dev, + enum rte_filter_type filter_type, + enum rte_filter_op filter_op, void *arg) +{ + RTE_SET_USED(eth_dev); + + if (filter_type != RTE_ETH_FILTER_GENERIC) { + plt_err("Unsupported filter type %d", filter_type); + return -ENOTSUP; + } + + if (filter_op == RTE_ETH_FILTER_GET) { + *(const void **)arg = &cnxk_flow_ops; + return 0; + } + + plt_err("Invalid filter_op %d", filter_op); + return -EINVAL; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 43/44] net/cnxk: add ethdev firmware version get 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (41 preceding siblings ...) 2021-03-06 15:34 ` [dpdk-dev] [PATCH 42/44] net/cnxk: add filter ctrl operation Nithin Dabilpuram @ 2021-03-06 15:34 ` Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 44/44] net/cnxk: add get register operation Nithin Dabilpuram ` (3 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:34 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Add callback to get ethdev firmware version. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 6 files changed, 25 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 7b6d832..2c83bfb 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -36,6 +36,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index ef37088..c8ad253 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -34,6 +34,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 69419d1..4dbdfcb 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -31,6 +31,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 347428e..f006718 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1175,6 +1175,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .fw_version_get = cnxk_nix_fw_version_get, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 7cf7cf7..4b25593 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -298,6 +298,8 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +int cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size); void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo); void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 00f1fe7..bf89ede 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -644,6 +644,25 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } +int +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const char *str = roc_npc_profile_name_get(&dev->npc); + uint32_t size = strlen(str) + 1; + + if (fw_size > size) + fw_size = size; + + strlcpy(fw_version, str, fw_size); + + if (fw_size < size) + return size; + + return 0; +} + void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo) -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH 44/44] net/cnxk: add get register operation 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (42 preceding siblings ...) 2021-03-06 15:34 ` [dpdk-dev] [PATCH 43/44] net/cnxk: add ethdev firmware version get Nithin Dabilpuram @ 2021-03-06 15:34 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (2 subsequent siblings) 46 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-03-06 15:34 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> With this patch implemented api to dump platform registers for debug purposes. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 25 +++++++++++++++++++++++++ 7 files changed, 34 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 87401f0..98bcb51 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -32,6 +32,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Debug utilities - Context dump and error interrupt support - Support Rx interrupt Prerequisites diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c83bfb..d1c6f9d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -38,6 +38,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index c8ad253..5f2478d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -36,6 +36,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4dbdfcb..3cbc369 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -33,6 +33,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f006718..99fb091 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1180,6 +1180,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .filter_ctrl = cnxk_nix_filter_ctrl, + .get_reg = cnxk_nix_dev_get_reg, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4b25593..74573f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -313,6 +313,10 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Debug */ +int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, + struct rte_dev_reg_info *regs); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index bf89ede..41c6d37 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -702,3 +702,28 @@ cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) return 0; } + +int +cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t *data = regs->data; + int rc = -ENOTSUP; + + if (data == NULL) { + rc = roc_nix_lf_get_reg_count(nix); + if (rc > 0) { + regs->length = rc; + regs->width = 8; + rc = 0; + } + return rc; + } + + if (!regs->length || + regs->length == (uint32_t)roc_nix_lf_get_reg_count(nix)) + return roc_nix_lf_reg_dump(nix, data); + + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (43 preceding siblings ...) 2021-03-06 15:34 ` [dpdk-dev] [PATCH 44/44] net/cnxk: add get register operation Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram ` (62 more replies) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram 46 siblings, 63 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' driver. In future, CN9K a.k.a octeontx2 will also be supported by same driver when code is ready and 'net/octeontx2' will be deprecated. Harman Kalra (1): common/cnxk: allocate lmt region in userspace Jerin Jacob (7): common/cnxk: fix batch alloc completion poll logic net/cnxk: add Rx support for cn9k net/cnxk: add Rx vector version for cn9k net/cnxk: add Tx support for cn9k net/cnxk: add Rx support for cn10k net/cnxk: add Rx vector version for cn10k net/cnxk: add Tx support for cn10k Kiran Kumar K (2): net/cnxk: add support to configure npc net/cnxk: add initial version of rte flow support Nithin Dabilpuram (17): net/cnxk: add build infra and common probe net/cnxk: add platform specific probe and remove net/cnxk: add common devargs parsing function net/cnxk: add common dev infos get support net/cnxk: add device configuration operation net/cnxk: add link status update support net/cnxk: add Rx queue setup and release net/cnxk: add Tx queue setup and release net/cnxk: add packet type support net/cnxk: add queue start and stop support net/cnxk: add Rx multi-segmented version for cn9k net/cnxk: add Tx multi-segment version for cn9k net/cnxk: add Tx vector version for cn9k net/cnxk: add Rx multi-segment version for cn10k net/cnxk: add Tx multi-segment version for cn10k net/cnxk: add Tx vector version for cn10k net/cnxk: add device start and stop operations Satha Rao (8): common/cnxk: add support to lock NIX RQ contexts common/cnxk: add provision to enable RED on RQ net/cnxk: add port/queue stats net/cnxk: add xstats apis net/cnxk: add rxq/txq info get operations net/cnxk: add ethdev firmware version get net/cnxk: add get register operation net/cnxk: added reta and rss_hash operations Satheesh Paul (6): common/cnxk: add support to dump flow entries common/cnxk: support for mark and flag flow actions common/cnxk: fix flow create on CN98xx net/cnxk: add flow ops get operation net/cnxk: support for rss in rte_flow net/cnxk: support for rte flow dev dump API Srujana Challa (1): common/cnxk: update Rx inline IPsec mbox message format Sunil Kumar Kori (20): net/cnxk: add MAC address set ops net/cnxk: add MTU set device operation net/cnxk: add promiscuous mode enable and disable net/cnxk: add DMAC filter support net/cnxk: add all multicast enable/disable ethops net/cnxk: add Rx/Tx burst mode get ops net/cnxk: add flow ctrl set/get ops net/cnxk: add link up/down operations net/cnxk: add EEPROM module info get operations net/cnxk: add Rx queue interrupt enable/disable ops net/cnxk: add validation API for mempool ops net/cnxk: add device close and reset operations net/cnxk: add pending Tx mbuf cleanup operation net/cnxk: register callback to get PTP status net/cnxk: add base PTP timesync support net/cnxk: add timesync enable/disable operations net/cnxk: add Rx/Tx timestamp read operations net/cnxk: add time read/write/adjust operations net/cnxk: add read clock operation net/cnxk: add multicast filter support -- v2: - Fixed issue with flow validate and flow create for 98xx - Fixed issue batch alloc logic - Fix lmtline allocation to be cached - Sync Inline IPSec Rx mbox with kernel - Add support for mark and flag flow actions - Add reta key and hash update ops - Added PTP and multicast filter support MAINTAINERS | 3 + doc/guides/nics/cnxk.rst | 343 ++++++ doc/guides/nics/features/cnxk.ini | 90 ++ doc/guides/nics/features/cnxk_vec.ini | 44 + doc/guides/nics/features/cnxk_vf.ini | 40 + doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 +- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 6 + drivers/common/cnxk/roc_model.h | 6 + drivers/common/cnxk/roc_nix.h | 39 +- drivers/common/cnxk/roc_nix_queue.c | 52 + drivers/common/cnxk/roc_nix_rss.c | 51 +- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +- drivers/common/cnxk/roc_npa.c | 10 +- drivers/common/cnxk/roc_npa.h | 35 +- drivers/common/cnxk/roc_npc.c | 41 +- drivers/common/cnxk/roc_npc.h | 15 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 +++++++++++ drivers/common/cnxk/roc_npc_priv.h | 2 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/roc_platform.h | 13 + drivers/common/cnxk/version.map | 5 + drivers/net/cnxk/cn10k_ethdev.c | 534 ++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 40 + drivers/net/cnxk/cn10k_rx.c | 78 ++ drivers/net/cnxk/cn10k_rx.h | 546 ++++++++++ drivers/net/cnxk/cn10k_rx_mseg.c | 17 + drivers/net/cnxk/cn10k_rx_vec.c | 22 + drivers/net/cnxk/cn10k_tx.c | 82 ++ drivers/net/cnxk/cn10k_tx.h | 1737 +++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 + drivers/net/cnxk/cn10k_tx_vec.c | 26 + drivers/net/cnxk/cn9k_ethdev.c | 557 ++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 38 + drivers/net/cnxk/cn9k_rx.c | 78 ++ drivers/net/cnxk/cn9k_rx.h | 548 ++++++++++ drivers/net/cnxk/cn9k_rx_mseg.c | 17 + drivers/net/cnxk/cn9k_rx_vec.c | 20 + drivers/net/cnxk/cn9k_tx.c | 81 ++ drivers/net/cnxk/cn9k_tx.h | 1605 ++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 + drivers/net/cnxk/cn9k_tx_vec.c | 26 + drivers/net/cnxk/cnxk_ethdev.c | 1511 +++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 478 +++++++++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 173 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 904 ++++++++++++++++ drivers/net/cnxk/cnxk_link.c | 113 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++ drivers/net/cnxk/cnxk_ptp.c | 287 +++++ drivers/net/cnxk/cnxk_rte_flow.c | 366 +++++++ drivers/net/cnxk/cnxk_rte_flow.h | 69 ++ drivers/net/cnxk/cnxk_stats.c | 217 ++++ drivers/net/cnxk/meson.build | 45 + drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 60 files changed, 12116 insertions(+), 83 deletions(-) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn10k_rx.c create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c create mode 100644 drivers/net/cnxk/cn10k_tx.c create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_rx.c create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c create mode 100644 drivers/net/cnxk/cn9k_tx.c create mode 100644 drivers/net/cnxk/cn9k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c create mode 100644 drivers/net/cnxk/cnxk_link.c create mode 100644 drivers/net/cnxk/cnxk_lookup.c create mode 100644 drivers/net/cnxk/cnxk_ptp.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h create mode 100644 drivers/net/cnxk/cnxk_stats.c create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 18:25 ` Stephen Hemminger 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format Nithin Dabilpuram ` (61 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> This patch will consider device argument to lock rss table in NIX. This patch also adds few misc fixes such as disabling NIX Tx vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ for Tx completions and TM related stats API. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 31 ++++++++++-- drivers/common/cnxk/roc_nix_queue.c | 2 + drivers/common/cnxk/roc_nix_rss.c | 51 ++++++++++++++++++-- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +++++++++++++++++++++++++++++++++- drivers/common/cnxk/roc_platform.h | 2 + drivers/common/cnxk/version.map | 1 + 6 files changed, 163 insertions(+), 10 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index b39f461..6d9ac10 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -85,10 +85,11 @@ struct roc_nix_eeprom_info { #define ROC_NIX_LF_RX_CFG_LEN_OL3 BIT_ULL(41) /* Group 0 will be used for RSS, 1 -7 will be used for npc_flow RSS action*/ -#define ROC_NIX_RSS_GROUP_DEFAULT 0 -#define ROC_NIX_RSS_GRPS 8 -#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 -#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_GROUP_DEFAULT 0 +#define ROC_NIX_RSS_GRPS 8 +#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 +#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_MCAM_IDX_DEFAULT (-1) #define ROC_NIX_DEFAULT_HW_FRS 1514 @@ -184,6 +185,7 @@ struct roc_nix_sq { enum roc_nix_sq_max_sqe_sz max_sqe_sz; uint32_t nb_desc; uint16_t qid; + bool sso_ena; /* End of Input parameters */ uint16_t sqes_per_sqb_log2; struct roc_nix *roc_nix; @@ -241,6 +243,8 @@ struct roc_nix { uint16_t max_sqb_count; enum roc_nix_rss_reta_sz reta_sz; bool enable_loop; + bool hw_vlan_ins; + uint8_t lock_rx_ctx; /* End of input parameters */ /* LMT line base for "Per Core Tx LMT line" mode*/ uintptr_t lmt_base; @@ -371,6 +375,22 @@ struct roc_nix_tm_shaper_profile { void (*free_fn)(void *profile); }; +enum roc_nix_tm_node_stats_type { + ROC_NIX_TM_NODE_PKTS_DROPPED, + ROC_NIX_TM_NODE_BYTES_DROPPED, + ROC_NIX_TM_NODE_GREEN_PKTS, + ROC_NIX_TM_NODE_GREEN_BYTES, + ROC_NIX_TM_NODE_YELLOW_PKTS, + ROC_NIX_TM_NODE_YELLOW_BYTES, + ROC_NIX_TM_NODE_RED_PKTS, + ROC_NIX_TM_NODE_RED_BYTES, + ROC_NIX_TM_NODE_STATS_MAX, +}; + +struct roc_nix_tm_node_stats { + uint64_t stats[ROC_NIX_TM_NODE_STATS_MAX]; +}; + int __roc_api roc_nix_tm_node_add(struct roc_nix *roc_nix, struct roc_nix_tm_node *roc_node); int __roc_api roc_nix_tm_node_delete(struct roc_nix *roc_nix, uint32_t node_id, @@ -408,6 +428,9 @@ roc_nix_tm_shaper_profile_get(struct roc_nix *roc_nix, uint32_t profile_id); struct roc_nix_tm_shaper_profile *__roc_api roc_nix_tm_shaper_profile_next( struct roc_nix *roc_nix, struct roc_nix_tm_shaper_profile *__prev); +int __roc_api roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, + uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *stats); /* * TM ratelimit tree API. */ diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index fbf7efa..1c62aa2 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -582,6 +582,7 @@ sq_cn9k_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); @@ -679,6 +680,7 @@ sq_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); diff --git a/drivers/common/cnxk/roc_nix_rss.c b/drivers/common/cnxk/roc_nix_rss.c index 2d7b84a..7de69aa 100644 --- a/drivers/common/cnxk/roc_nix_rss.c +++ b/drivers/common/cnxk/roc_nix_rss.c @@ -52,7 +52,7 @@ roc_nix_rss_key_get(struct roc_nix *roc_nix, uint8_t key[ROC_NIX_RSS_KEY_LEN]) static int nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_aq_enq_req *req; @@ -77,6 +77,27 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -88,7 +109,7 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, static int nix_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_cn10k_aq_enq_req *req; @@ -113,6 +134,27 @@ nix_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -133,9 +175,10 @@ roc_nix_rss_reta_set(struct roc_nix *roc_nix, uint8_t group, return NIX_ERR_PARAM; if (roc_model_is_cn9k()) - rc = nix_cn9k_rss_reta_set(nix, group, reta); + rc = nix_cn9k_rss_reta_set(nix, group, reta, + roc_nix->lock_rx_ctx); else - rc = nix_rss_reta_set(nix, group, reta); + rc = nix_rss_reta_set(nix, group, reta, roc_nix->lock_rx_ctx); if (rc) return rc; diff --git a/drivers/common/cnxk/roc_nix_tm_utils.c b/drivers/common/cnxk/roc_nix_tm_utils.c index 1d7dd68..6b9543e 100644 --- a/drivers/common/cnxk/roc_nix_tm_utils.c +++ b/drivers/common/cnxk/roc_nix_tm_utils.c @@ -409,6 +409,7 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, volatile uint64_t *reg, volatile uint64_t *regval, volatile uint64_t *regval_mask) { + struct roc_nix *roc_nix = nix_priv_to_roc_nix(nix); uint8_t k = 0, hw_lvl, parent_lvl; uint64_t parent = 0, child = 0; enum roc_nix_tm_tree tree; @@ -454,8 +455,11 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, reg[k] = NIX_AF_SMQX_CFG(schq); regval[k] = (BIT_ULL(50) | NIX_MIN_HW_FRS | ((nix->mtu & 0xFFFF) << 8)); - regval_mask[k] = - ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | GENMASK_ULL(23, 8)); + /* Maximum Vtag insertion size as a multiple of four bytes */ + if (roc_nix->hw_vlan_ins) + regval[k] |= (0x2ULL << 36); + regval_mask[k] = ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | + GENMASK_ULL(23, 8) | GENMASK_ULL(38, 36)); k++; /* Parent and schedule conf */ @@ -1000,3 +1004,81 @@ nix_tm_shaper_profile_free(struct nix_tm_shaper_profile *profile) (profile->free_fn)(profile); } + +int +roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *n_stats) +{ + struct nix *nix = roc_nix_to_nix_priv(roc_nix); + struct mbox *mbox = (&nix->dev)->mbox; + struct nix_txschq_config *req, *rsp; + struct nix_tm_node *node; + uint32_t schq; + int rc, i; + + node = nix_tm_node_search(nix, node_id, ROC_NIX_TM_USER); + if (!node) + return NIX_ERR_TM_INVALID_NODE; + + if (node->hw_lvl != NIX_TXSCH_LVL_TL1) + return NIX_ERR_OP_NOTSUP; + + schq = node->hw_id; + /* Skip fetch if not requested */ + if (!n_stats) + goto clear_stats; + + memset(n_stats, 0, sizeof(struct roc_nix_tm_node_stats)); + /* Check if node has HW resource */ + if (!(node->flags & NIX_TM_NODE_HWRES)) + return 0; + + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->read = 1; + req->lvl = NIX_TXSCH_LVL_TL1; + + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + rc = mbox_process_msg(mbox, (void **)&rsp); + if (rc) + return rc; + + /* Return stats */ + n_stats->stats[ROC_NIX_TM_NODE_PKTS_DROPPED] = rsp->regval[0]; + n_stats->stats[ROC_NIX_TM_NODE_BYTES_DROPPED] = rsp->regval[1]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_PKTS] = rsp->regval[2]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_BYTES] = rsp->regval[3]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_PKTS] = rsp->regval[4]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_BYTES] = rsp->regval[5]; + n_stats->stats[ROC_NIX_TM_NODE_RED_PKTS] = rsp->regval[6]; + n_stats->stats[ROC_NIX_TM_NODE_RED_BYTES] = rsp->regval[7]; + +clear_stats: + if (!clear) + return 0; + + /* Clear all the stats */ + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->lvl = NIX_TXSCH_LVL_TL1; + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + return mbox_process_msg(mbox, (void **)&rsp); +} diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 7864fa4..911ae15 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -127,6 +127,8 @@ #define plt_memzone_reserve_cache_align(name, sz) \ rte_memzone_reserve_aligned(name, sz, 0, 0, RTE_CACHE_LINE_SIZE) #define plt_memzone_free rte_memzone_free +#define plt_memzone_reserve_aligned(name, len, flags, align) \ + rte_memzone_reserve_aligned((name), (len), 0, (flags), (align)) #define plt_tsc_hz rte_get_tsc_hz #define plt_delay_ms rte_delay_ms diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 8e67c83..c39d76f 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -123,6 +123,7 @@ INTERNAL { roc_nix_tm_node_parent_update; roc_nix_tm_node_pkt_mode_update; roc_nix_tm_node_shaper_update; + roc_nix_tm_node_stats_get; roc_nix_tm_node_suspend_resume; roc_nix_tm_prealloc_res; roc_nix_tm_rlimit_sq; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram @ 2021-06-07 18:25 ` Stephen Hemminger 2021-06-08 3:47 ` Jerin Jacob 0 siblings, 1 reply; 262+ messages in thread From: Stephen Hemminger @ 2021-06-07 18:25 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dev, jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar On Mon, 7 Jun 2021 23:28:42 +0530 Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > From: Satha Rao <skoteshwar@marvell.com> > > This patch will consider device argument to lock rss table > in NIX. > > This patch also adds few misc fixes such as disabling NIX Tx > vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ > for Tx completions and TM related stats API. > > Signed-off-by: Satha Rao <skoteshwar@marvell.com> In general, breaking patches into smaller parts is helpful. But in this case breaking a driver into 62 smaller pieces is getting ridiculous. It is too tedious to review the small chunks. ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts 2021-06-07 18:25 ` Stephen Hemminger @ 2021-06-08 3:47 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-08 3:47 UTC (permalink / raw) To: Stephen Hemminger Cc: Nithin Dabilpuram, dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:55 PM Stephen Hemminger <stephen@networkplumber.org> wrote: > > On Mon, 7 Jun 2021 23:28:42 +0530 > Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > > From: Satha Rao <skoteshwar@marvell.com> > > > > This patch will consider device argument to lock rss table > > in NIX. > > > > This patch also adds few misc fixes such as disabling NIX Tx > > vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ > > for Tx completions and TM related stats API. > > > > Signed-off-by: Satha Rao <skoteshwar@marvell.com> > > In general, breaking patches into smaller parts is helpful. > But in this case breaking a driver into 62 smaller pieces is getting > ridiculous. It is too tedious to review the small chunks. I think, it is other way around. More logically splited patches are easy to review. > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-08 12:26 ` Andrew Rybchenko 2021-06-14 3:30 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 03/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram ` (60 subsequent siblings) 62 siblings, 2 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Srujana Challa From: Srujana Challa <schalla@marvell.com> Updates Rx inline IPSEC mailbox message format to make it sync with latest CPT PF driver. Signed-off-by: Srujana Challa <schalla@marvell.com> --- drivers/common/cnxk/roc_mbox.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h index f6b11b6..fe4df21 100644 --- a/drivers/common/cnxk/roc_mbox.h +++ b/drivers/common/cnxk/roc_mbox.h @@ -1328,6 +1328,9 @@ struct cpt_rxc_time_cfg_req { struct cpt_rx_inline_lf_cfg_msg { struct mbox_msghdr hdr; uint16_t __io sso_pf_func; + uint16_t __io param1; + uint16_t __io param2; + uint16_t __io reserved; }; enum cpt_eng_type { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format Nithin Dabilpuram @ 2021-06-08 12:26 ` Andrew Rybchenko 2021-06-09 11:02 ` Nithin Dabilpuram 2021-06-14 3:30 ` Jerin Jacob 1 sibling, 1 reply; 262+ messages in thread From: Andrew Rybchenko @ 2021-06-08 12:26 UTC (permalink / raw) To: Nithin Dabilpuram, dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Srujana Challa On 6/7/21 8:58 PM, Nithin Dabilpuram wrote: > From: Srujana Challa <schalla@marvell.com> > > Updates Rx inline IPSEC mailbox message format to make it > sync with latest CPT PF driver. > > Signed-off-by: Srujana Challa <schalla@marvell.com> > --- > drivers/common/cnxk/roc_mbox.h | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h > index f6b11b6..fe4df21 100644 > --- a/drivers/common/cnxk/roc_mbox.h > +++ b/drivers/common/cnxk/roc_mbox.h > @@ -1328,6 +1328,9 @@ struct cpt_rxc_time_cfg_req { > struct cpt_rx_inline_lf_cfg_msg { > struct mbox_msghdr hdr; > uint16_t __io sso_pf_func; > + uint16_t __io param1; > + uint16_t __io param2; > + uint16_t __io reserved; > }; > > enum cpt_eng_type { > Isn't is a dead code? May be it should be added when it is actually used? ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format 2021-06-08 12:26 ` Andrew Rybchenko @ 2021-06-09 11:02 ` Nithin Dabilpuram 0 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-09 11:02 UTC (permalink / raw) To: Andrew Rybchenko Cc: dev, jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Srujana Challa On Tue, Jun 08, 2021 at 03:26:08PM +0300, Andrew Rybchenko wrote: > On 6/7/21 8:58 PM, Nithin Dabilpuram wrote: > > From: Srujana Challa <schalla@marvell.com> > > > > Updates Rx inline IPSEC mailbox message format to make it > > sync with latest CPT PF driver. > > > > Signed-off-by: Srujana Challa <schalla@marvell.com> > > --- > > drivers/common/cnxk/roc_mbox.h | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h > > index f6b11b6..fe4df21 100644 > > --- a/drivers/common/cnxk/roc_mbox.h > > +++ b/drivers/common/cnxk/roc_mbox.h > > @@ -1328,6 +1328,9 @@ struct cpt_rxc_time_cfg_req { > > struct cpt_rx_inline_lf_cfg_msg { > > struct mbox_msghdr hdr; > > uint16_t __io sso_pf_func; > > + uint16_t __io param1; > > + uint16_t __io param2; > > + uint16_t __io reserved; > > }; > > > > enum cpt_eng_type { > > > > Isn't is a dead code? > May be it should be added when it is actually used? This patch is just a sync of this data structure b/w Kernel and Userspace used for mbox communication. It is already used to generate mbox message request response API's via and drivers/common/cnxk/roc_mbox_priv.h:158 for all mbox messages. Since it is common code and just a datastructure update to be in sync with kernel, isn't it fine ? ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format Nithin Dabilpuram 2021-06-08 12:26 ` Andrew Rybchenko @ 2021-06-14 3:30 ` Jerin Jacob 1 sibling, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-14 3:30 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Srujana Challa On Mon, Jun 7, 2021 at 11:33 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Srujana Challa <schalla@marvell.com> > > Updates Rx inline IPSEC mailbox message format to make it > sync with latest CPT PF driver. > > Signed-off-by: Srujana Challa <schalla@marvell.com> > --- > drivers/common/cnxk/roc_mbox.h | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h > index f6b11b6..fe4df21 100644 > --- a/drivers/common/cnxk/roc_mbox.h > +++ b/drivers/common/cnxk/roc_mbox.h > @@ -1328,6 +1328,9 @@ struct cpt_rxc_time_cfg_req { > struct cpt_rx_inline_lf_cfg_msg { > struct mbox_msghdr hdr; > uint16_t __io sso_pf_func; > + uint16_t __io param1; > + uint16_t __io param2; > + uint16_t __io reserved; Items like 3 lines changes, Please squash to some other patches in the series. > }; > > enum cpt_eng_type { > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 03/62] common/cnxk: fix batch alloc completion poll logic 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 04/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram ` (59 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Jerin Jacob <jerinj@marvell.com> The instruction generation was not correct due to fact that volatile suppose to use with ccode variable as well. Change the logic to use gcc atomic builtin to simplify and avoid explicit volatile from the code. Fixes: 81af26789316 ("common/cnxk: support NPA batch alloc/free") Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com> --- drivers/common/cnxk/roc_npa.c | 2 +- drivers/common/cnxk/roc_npa.h | 30 +++++++++++++++--------------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index f1e03b7..5ba6e81 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -236,7 +236,7 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Block size should be cache line aligned and in range of 128B-128KB */ if (block_size % ROC_ALIGN || block_size < 128 || - block_size > 128 * 1024) + block_size > ROC_NPA_MAX_BLOCK_SZ) return NPA_ERR_INVALID_BLOCK_SZ; pos = 0; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 89f5c6f..59d6223 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -8,6 +8,7 @@ #define ROC_AURA_ID_MASK (BIT_ULL(16) - 1) #define ROC_AURA_OP_LIMIT_MASK (BIT_ULL(36) - 1) +#define ROC_NPA_MAX_BLOCK_SZ (128 * 1024) #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 @@ -219,6 +220,17 @@ roc_npa_aura_batch_alloc_issue(uint64_t aura_handle, uint64_t *buf, return 0; } +static inline void +roc_npa_batch_alloc_wait(uint64_t *cache_line) +{ + /* Batch alloc status code is updated in bits [5:6] of the first word + * of the 128 byte cache line. + */ + while (((__atomic_load_n(cache_line, __ATOMIC_RELAXED) >> 5) & 0x3) == + ALLOC_CCODE_INVAL) + ; +} + static inline unsigned int roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) { @@ -231,17 +243,10 @@ roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int ccode; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); - + roc_npa_batch_alloc_wait(&aligned_buf[i]); count += status->count; } @@ -261,16 +266,11 @@ roc_npa_aura_batch_alloc_extract(uint64_t *buf, uint64_t *aligned_buf, /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int line_count, ccode; + int line_count; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); + roc_npa_batch_alloc_wait(&aligned_buf[i]); line_count = status->count; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 04/62] common/cnxk: add support to dump flow entries 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (2 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 03/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 05/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram ` (58 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> Add NPC support API to dump created flow entries. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_npc.c | 20 ++ drivers/common/cnxk/roc_npc.h | 12 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npc_priv.h | 2 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/version.map | 2 + 8 files changed, 652 insertions(+), 2 deletions(-) create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c diff --git a/drivers/common/cnxk/hw/npc.h b/drivers/common/cnxk/hw/npc.h index e0f06bf..68c5037 100644 --- a/drivers/common/cnxk/hw/npc.h +++ b/drivers/common/cnxk/hw/npc.h @@ -193,6 +193,7 @@ enum npc_kpu_lb_ltype { NPC_LT_LB_EXDSA, NPC_LT_LB_EXDSA_VLAN, NPC_LT_LB_FDSA, + NPC_LT_LB_VLAN_EXDSA, NPC_LT_LB_CUSTOM0 = 0xE, NPC_LT_LB_CUSTOM1 = 0xF, }; @@ -208,6 +209,7 @@ enum npc_kpu_lc_ltype { NPC_LT_LC_MPLS, NPC_LT_LC_NSH, NPC_LT_LC_FCOE, + NPC_LT_LC_NGIO, NPC_LT_LC_CUSTOM0 = 0xE, NPC_LT_LC_CUSTOM1 = 0xF, }; diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index 178bce7..e7ab79f 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -37,6 +37,7 @@ sources = files( 'roc_npa_irq.c', 'roc_npc.c', 'roc_npc_mcam.c', + 'roc_npc_mcam_dump.c', 'roc_npc_parse.c', 'roc_npc_utils.c', 'roc_platform.c', diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index abaef77..81c7fd9 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -870,3 +870,23 @@ roc_npc_flow_destroy(struct roc_npc *roc_npc, struct roc_npc_flow *flow) plt_free(flow); return 0; } + +void +roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + struct roc_npc_flow *flow_iter; + struct npc_flow_list *list; + uint32_t max_prio, i; + + max_prio = npc->flow_max_priority; + + for (i = 0; i < max_prio; i++) { + list = &npc->flow_list[i]; + + /* List in ascending order of mcam entries */ + TAILQ_FOREACH(flow_iter, list, next) { + roc_npc_flow_mcam_dump(file, roc_npc, flow_iter); + } + } +} diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 223c4ba..115bcd5 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -90,6 +90,11 @@ struct roc_npc_attr { uint32_t reserved : 30; /**< Reserved, must be zero. */ }; +struct roc_npc_flow_dump_data { + uint8_t lid; + uint16_t ltype; +}; + struct roc_npc_flow { uint8_t nix_intf; uint8_t enable; @@ -102,6 +107,9 @@ struct roc_npc_flow { uint64_t mcam_mask[ROC_NPC_MAX_MCAM_WIDTH_DWORDS]; uint64_t npc_action; uint64_t vtag_action; +#define ROC_NPC_MAX_FLOW_PATTERNS 32 + struct roc_npc_flow_dump_data dump_data[ROC_NPC_MAX_FLOW_PATTERNS]; + uint16_t num_patterns; TAILQ_ENTRY(roc_npc_flow) next; }; @@ -185,5 +193,7 @@ int __roc_api roc_npc_mcam_clear_counter(struct roc_npc *roc_npc, uint32_t ctr_id); int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); - +void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); +void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *mcam); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/roc_npc_mcam_dump.c b/drivers/common/cnxk/roc_npc_mcam_dump.c new file mode 100644 index 0000000..19b4901 --- /dev/null +++ b/drivers/common/cnxk/roc_npc_mcam_dump.c @@ -0,0 +1,611 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#define NPC_MAX_FIELD_NAME_SIZE 80 +#define NPC_RX_ACTIONOP_MASK GENMASK(3, 0) +#define NPC_RX_ACTION_PFFUNC_MASK GENMASK(19, 4) +#define NPC_RX_ACTION_INDEX_MASK GENMASK(39, 20) +#define NPC_RX_ACTION_MATCH_MASK GENMASK(55, 40) +#define NPC_RX_ACTION_FLOWKEY_MASK GENMASK(60, 56) + +#define NPC_TX_ACTION_INDEX_MASK GENMASK(31, 12) +#define NPC_TX_ACTION_MATCH_MASK GENMASK(47, 32) + +#define NIX_RX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_RX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_RX_VTAGACT_VTAG0_TYPE_MASK GENMASK(14, 12) +#define NIX_RX_VTAGACT_VTAG0_VALID_MASK BIT_ULL(15) + +#define NIX_RX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_RX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_RX_VTAGACT_VTAG1_TYPE_MASK GENMASK(46, 44) +#define NIX_RX_VTAGACT_VTAG1_VALID_MASK BIT_ULL(47) + +#define NIX_TX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_TX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_TX_VTAGACT_VTAG0_OP_MASK GENMASK(13, 12) +#define NIX_TX_VTAGACT_VTAG0_DEF_MASK GENMASK(25, 16) + +#define NIX_TX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_TX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_TX_VTAGACT_VTAG1_OP_MASK GENMASK(45, 44) +#define NIX_TX_VTAGACT_VTAG1_DEF_MASK GENMASK(57, 48) + +struct npc_rx_parse_nibble_s { + uint16_t chan : 3; + uint16_t errlev : 1; + uint16_t errcode : 2; + uint16_t l2l3bm : 1; + uint16_t laflags : 2; + uint16_t latype : 1; + uint16_t lbflags : 2; + uint16_t lbtype : 1; + uint16_t lcflags : 2; + uint16_t lctype : 1; + uint16_t ldflags : 2; + uint16_t ldtype : 1; + uint16_t leflags : 2; + uint16_t letype : 1; + uint16_t lfflags : 2; + uint16_t lftype : 1; + uint16_t lgflags : 2; + uint16_t lgtype : 1; + uint16_t lhflags : 2; + uint16_t lhtype : 1; +} __plt_packed; + +static const char *const intf_str[] = { + "NIX-RX", + "NIX-TX", +}; + +static const char *const ltype_str[NPC_MAX_LID][NPC_MAX_LT] = { + [NPC_LID_LA][0] = "NONE", + [NPC_LID_LA][NPC_LT_LA_ETHER] = "LA_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_ETHER] = "LA_IH_NIX_ETHER", + [NPC_LID_LA][NPC_LT_LA_HIGIG2_ETHER] = "LA_HIGIG2_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_HIGIG2_ETHER] = "LA_IH_NIX_HIGIG2_ETHER", + [NPC_LID_LB][0] = "NONE", + [NPC_LID_LB][NPC_LT_LB_CTAG] = "LB_CTAG", + [NPC_LID_LB][NPC_LT_LB_STAG_QINQ] = "LB_STAG_QINQ", + [NPC_LID_LB][NPC_LT_LB_ETAG] = "LB_ETAG", + [NPC_LID_LB][NPC_LT_LB_EXDSA] = "LB_EXDSA", + [NPC_LID_LB][NPC_LT_LB_VLAN_EXDSA] = "LB_VLAN_EXDSA", + [NPC_LID_LC][0] = "NONE", + [NPC_LID_LC][NPC_LT_LC_IP] = "LC_IP", + [NPC_LID_LC][NPC_LT_LC_IP6] = "LC_IP6", + [NPC_LID_LC][NPC_LT_LC_ARP] = "LC_ARP", + [NPC_LID_LC][NPC_LT_LC_IP6_EXT] = "LC_IP6_EXT", + [NPC_LID_LC][NPC_LT_LC_NGIO] = "LC_NGIO", + [NPC_LID_LD][0] = "NONE", + [NPC_LID_LD][NPC_LT_LD_ICMP] = "LD_ICMP", + [NPC_LID_LD][NPC_LT_LD_ICMP6] = "LD_ICMP6", + [NPC_LID_LD][NPC_LT_LD_UDP] = "LD_UDP", + [NPC_LID_LD][NPC_LT_LD_TCP] = "LD_TCP", + [NPC_LID_LD][NPC_LT_LD_SCTP] = "LD_SCTP", + [NPC_LID_LD][NPC_LT_LD_GRE] = "LD_GRE", + [NPC_LID_LD][NPC_LT_LD_NVGRE] = "LD_NVGRE", + [NPC_LID_LE][0] = "NONE", + [NPC_LID_LE][NPC_LT_LE_VXLAN] = "LE_VXLAN", + [NPC_LID_LE][NPC_LT_LE_ESP] = "LE_ESP", + [NPC_LID_LE][NPC_LT_LE_GTPC] = "LE_GTPC", + [NPC_LID_LE][NPC_LT_LE_GTPU] = "LE_GTPU", + [NPC_LID_LE][NPC_LT_LE_GENEVE] = "LE_GENEVE", + [NPC_LID_LE][NPC_LT_LE_VXLANGPE] = "LE_VXLANGPE", + [NPC_LID_LF][0] = "NONE", + [NPC_LID_LF][NPC_LT_LF_TU_ETHER] = "LF_TU_ETHER", + [NPC_LID_LG][0] = "NONE", + [NPC_LID_LG][NPC_LT_LG_TU_IP] = "LG_TU_IP", + [NPC_LID_LG][NPC_LT_LG_TU_IP6] = "LG_TU_IP6", + [NPC_LID_LH][0] = "NONE", + [NPC_LID_LH][NPC_LT_LH_TU_UDP] = "LH_TU_UDP", + [NPC_LID_LH][NPC_LT_LH_TU_TCP] = "LH_TU_TCP", + [NPC_LID_LH][NPC_LT_LH_TU_SCTP] = "LH_TU_SCTP", + [NPC_LID_LH][NPC_LT_LH_TU_ESP] = "LH_TU_ESP", +}; + +static uint16_t +npc_get_nibbles(struct roc_npc_flow *flow, uint16_t size, uint32_t bit_offset) +{ + uint32_t byte_index, noffset; + uint16_t data, mask; + uint8_t *bytes; + + bytes = (uint8_t *)flow->mcam_data; + mask = (1ULL << (size * 4)) - 1; + byte_index = bit_offset / 8; + noffset = bit_offset % 8; + data = *(unaligned_uint16_t *)&bytes[byte_index]; + data >>= noffset; + data &= mask; + + return data; +} + +static void +npc_flow_print_parse_nibbles(FILE *file, struct roc_npc_flow *flow, + uint64_t parse_nibbles) +{ + struct npc_rx_parse_nibble_s *rx_parse; + uint32_t data, offset = 0; + + rx_parse = (struct npc_rx_parse_nibble_s *)&parse_nibbles; + + if (rx_parse->chan) { + data = npc_get_nibbles(flow, 3, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_CHAN:%#03X\n", data); + offset += 12; + } + + if (rx_parse->errlev) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRLEV:%#X\n", data); + offset += 4; + } + + if (rx_parse->errcode) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRCODE:%#02X\n", data); + offset += 8; + } + + if (rx_parse->l2l3bm) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_L2L3_BCAST:%#X\n", data); + offset += 4; + } + + if (rx_parse->latype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_LTYPE:%s\n", + ltype_str[NPC_LID_LA][data]); + offset += 4; + } + + if (rx_parse->laflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lbtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_LTYPE:%s\n", + ltype_str[NPC_LID_LB][data]); + offset += 4; + } + + if (rx_parse->lbflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lctype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_LTYPE:%s\n", + ltype_str[NPC_LID_LC][data]); + offset += 4; + } + + if (rx_parse->lcflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->ldtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_LTYPE:%s\n", + ltype_str[NPC_LID_LD][data]); + offset += 4; + } + + if (rx_parse->ldflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->letype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_LTYPE:%s\n", + ltype_str[NPC_LID_LE][data]); + offset += 4; + } + + if (rx_parse->leflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lftype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_LTYPE:%s\n", + ltype_str[NPC_LID_LF][data]); + offset += 4; + } + + if (rx_parse->lfflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lgtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_LTYPE:%s\n", + ltype_str[NPC_LID_LG][data]); + offset += 4; + } + + if (rx_parse->lgflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lhtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_LTYPE:%s\n", + ltype_str[NPC_LID_LH][data]); + offset += 4; + } + + if (rx_parse->lhflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_FLAGS:%#02X\n", data); + } +} + +static void +npc_flow_print_xtractinfo(FILE *file, struct npc_xtract_info *lfinfo, + struct roc_npc_flow *flow, int lid, int lt) +{ + uint8_t *datastart, *maskstart; + int i; + + datastart = (uint8_t *)&flow->mcam_data + lfinfo->key_off; + maskstart = (uint8_t *)&flow->mcam_mask + lfinfo->key_off; + + fprintf(file, "\t%s, hdr offset:%#X, len:%#X, key offset:%#X, ", + ltype_str[lid][lt], lfinfo->hdr_off, lfinfo->len, + lfinfo->key_off); + + fprintf(file, "Data:0X"); + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", datastart[i]); + + fprintf(file, ", Mask:0X"); + + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", maskstart[i]); + + fprintf(file, "\n"); +} + +static void +npc_flow_print_item(FILE *file, struct npc *npc, struct npc_xtract_info *xinfo, + struct roc_npc_flow *flow, int intf, int lid, int lt, + int ld) +{ + struct npc_xtract_info *lflags_info; + int i, lf_cfg = 0; + + npc_flow_print_xtractinfo(file, xinfo, flow, lid, lt); + + if (xinfo->flags_enable) { + lf_cfg = npc->prx_lfcfg[ld].i; + + if (lf_cfg != lid) + return; + + for (i = 0; i < NPC_MAX_LFL; i++) { + lflags_info = npc->prx_fxcfg[intf][ld][i].xtract; + + npc_flow_print_xtractinfo(file, lflags_info, flow, lid, + lt); + } + } +} + +static void +npc_flow_dump_patterns(FILE *file, struct npc *npc, struct roc_npc_flow *flow) +{ + struct npc_lid_lt_xtract_info *lt_xinfo; + struct npc_xtract_info *xinfo; + uint32_t intf, lid, ld, i; + uint64_t parse_nibbles; + uint16_t ltype; + + intf = flow->nix_intf; + parse_nibbles = npc->keyx_supp_nmask[intf]; + npc_flow_print_parse_nibbles(file, flow, parse_nibbles); + + for (i = 0; i < flow->num_patterns; i++) { + lid = flow->dump_data[i].lid; + ltype = flow->dump_data[i].ltype; + lt_xinfo = &npc->prx_dxcfg[intf][lid][ltype]; + + for (ld = 0; ld < NPC_MAX_LD; ld++) { + xinfo = <_xinfo->xtract[ld]; + if (!xinfo->enable) + continue; + npc_flow_print_item(file, npc, xinfo, flow, intf, lid, + ltype, ld); + } + } +} + +static void +npc_flow_dump_tx_action(FILE *file, uint64_t npc_action) +{ + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + uint32_t tx_op, index, match_id; + + tx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (tx_op) { + case NIX_TX_ACTIONOP_DROP: + fprintf(file, "NIX_TX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_TX_ACTIONOP_UCAST_DEFAULT: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_DEFAULT); + break; + case NIX_TX_ACTIONOP_UCAST_CHAN: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_CHAN); + plt_strlcpy(index_name, + "Transmit Channel:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_MCAST: + fprintf(file, "NIX_TX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_MCAST); + plt_strlcpy(index_name, + "Multicast Table Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_DROP_VIOL: + fprintf(file, "NIX_TX_ACTIONOP_DROP_VIOL (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_DROP_VIOL); + break; + default: + plt_err("Unknown NIX_TX_ACTIONOP found"); + return; + } + + index = ((npc_action & NPC_TX_ACTION_INDEX_MASK) >> 12) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_TX_ACTION_MATCH_MASK) >> 32) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); +} + +static void +npc_flow_dump_rx_action(FILE *file, uint64_t npc_action) +{ + uint32_t rx_op, pf_func, index, match_id, flowkey_alg; + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + + rx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (rx_op) { + case NIX_RX_ACTIONOP_DROP: + fprintf(file, "NIX_RX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_RX_ACTIONOP_UCAST: + fprintf(file, "NIX_RX_ACTIONOP_UCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST); + plt_strlcpy(index_name, "RQ Index", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_UCAST_IPSEC: + fprintf(file, "NIX_RX_ACTIONOP_UCAST_IPSEC (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST_IPSEC); + plt_strlcpy(index_name, "RQ Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_MCAST: + fprintf(file, "NIX_RX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MCAST); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_RSS: + fprintf(file, "NIX_RX_ACTIONOP_RSS (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_RSS); + plt_strlcpy(index_name, "RSS Group Index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_PF_FUNC_DROP: + fprintf(file, "NIX_RX_ACTIONOP_PF_FUNC_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_PF_FUNC_DROP); + break; + case NIX_RX_ACTIONOP_MIRROR: + fprintf(file, "NIX_RX_ACTIONOP_MIRROR (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MIRROR); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + default: + plt_err("Unknown NIX_RX_ACTIONOP found"); + return; + } + + pf_func = ((npc_action & NPC_RX_ACTION_PFFUNC_MASK) >> 4) & + GENMASK(15, 0); + + fprintf(file, "\tPF_FUNC: %#04X\n", pf_func); + + index = ((npc_action & NPC_RX_ACTION_INDEX_MASK) >> 20) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_RX_ACTION_MATCH_MASK) >> 40) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); + + flowkey_alg = ((npc_action & NPC_RX_ACTION_FLOWKEY_MASK) >> 56) & + GENMASK(4, 0); + + fprintf(file, "\tFlow Key Alg:%#X\n", flowkey_alg); +} + +static void +npc_flow_dump_parsed_action(FILE *file, uint64_t npc_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX Action:%#016lX\n", npc_action); + npc_flow_dump_rx_action(file, npc_action); + } else { + fprintf(file, "NPC TX Action:%#016lX\n", npc_action); + npc_flow_dump_tx_action(file, npc_action); + } +} + +static void +npc_flow_dump_rx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t type, lid, relptr; + + if (vtag_action & NIX_RX_VTAGACT_VTAG0_VALID_MASK) { + relptr = vtag_action & NIX_RX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG0_TYPE_MASK) >> 12) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG0:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } + + if (vtag_action & NIX_RX_VTAGACT_VTAG1_VALID_MASK) { + relptr = ((vtag_action & NIX_RX_VTAGACT_VTAG1_RELPTR_MASK) >> + 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG1_TYPE_MASK) >> 44) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } +} + +static void +npc_get_vtag_opname(uint32_t op, char *opname, int len) +{ + switch (op) { + case 0x0: + plt_strlcpy(opname, "NOP", len - 1); + break; + case 0x1: + plt_strlcpy(opname, "INSERT", len - 1); + break; + case 0x2: + plt_strlcpy(opname, "REPLACE", len - 1); + break; + default: + plt_err("Unknown vtag op found"); + break; + } +} + +static void +npc_flow_dump_tx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t relptr, lid, op, vtag_def; + char opname[10]; + + relptr = vtag_action & NIX_TX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG0_OP_MASK) >> 12) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG0_DEF_MASK) >> 16) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG0 relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); + + relptr = ((vtag_action & NIX_TX_VTAGACT_VTAG1_RELPTR_MASK) >> 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG1_OP_MASK) >> 44) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG1_DEF_MASK) >> 48) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); +} + +static void +npc_flow_dump_vtag_action(FILE *file, uint64_t vtag_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_rx_vtag_action(file, vtag_action); + } else { + fprintf(file, "NPC TX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_tx_vtag_action(file, vtag_action); + } +} + +void +roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *flow) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + bool is_rx = 0; + int i; + + fprintf(file, "MCAM Index:%d\n", flow->mcam_id); + fprintf(file, "Interface :%s (%d)\n", intf_str[flow->nix_intf], + flow->nix_intf); + fprintf(file, "Priority :%d\n", flow->priority); + + if (flow->nix_intf == NIX_INTF_RX) + is_rx = 1; + + npc_flow_dump_parsed_action(file, flow->npc_action, is_rx); + npc_flow_dump_vtag_action(file, flow->vtag_action, is_rx); + fprintf(file, "Patterns:\n"); + npc_flow_dump_patterns(file, npc, flow); + + fprintf(file, "MCAM Raw Data :\n"); + + for (i = 0; i < ROC_NPC_MAX_MCAM_WIDTH_DWORDS; i++) { + fprintf(file, "\tDW%d :%016lX\n", i, flow->mcam_data[i]); + fprintf(file, "\tDW%d_Mask:%016lX\n", i, flow->mcam_mask[i]); + } + + fprintf(file, "\n"); +} diff --git a/drivers/common/cnxk/roc_npc_priv.h b/drivers/common/cnxk/roc_npc_priv.h index 8bc5bac..961583b 100644 --- a/drivers/common/cnxk/roc_npc_priv.h +++ b/drivers/common/cnxk/roc_npc_priv.h @@ -47,7 +47,7 @@ #define NPC_RVUPF_MAX_9XXX 0x10 /* HRM: RVU_PRIV_CONST */ #define NPC_RVUPF_MAX_10XX 0x20 /* HRM: RVU_PRIV_CONST */ #define NPC_NIXLF_MAX 0x80 /* HRM: NIX_AF_CONST2 */ -#define NPC_MCAME_PER_PF 2 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ +#define NPC_MCAME_PER_PF 3 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ #define NPC_MCAME_PER_LF 1 /* DRV: RSVD_MCAM_ENTRIES_PER_NIXLF */ #define NPC_MCAME_RESVD_9XXX \ (NPC_NIXLF_MAX * NPC_MCAME_PER_LF + \ diff --git a/drivers/common/cnxk/roc_npc_utils.c b/drivers/common/cnxk/roc_npc_utils.c index 1fb8973..5c97588 100644 --- a/drivers/common/cnxk/roc_npc_utils.c +++ b/drivers/common/cnxk/roc_npc_utils.c @@ -206,6 +206,7 @@ npc_update_parse_state(struct npc_parse_state *pst, uint8_t flags) { struct npc_lid_lt_xtract_info *xinfo; + struct roc_npc_flow_dump_data *dump; struct npc_xtract_info *lfinfo; int intf, lf_cfg; int i, j, rc = 0; @@ -248,6 +249,9 @@ npc_update_parse_state(struct npc_parse_state *pst, } done: + dump = &pst->flow->dump_data[pst->flow->num_patterns++]; + dump->lid = lid; + dump->ltype = lt; pst->pattern++; return 0; } diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index c39d76f..a11ba4d 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -160,6 +160,8 @@ INTERNAL { roc_npc_fini; roc_npc_flow_create; roc_npc_flow_destroy; + roc_npc_flow_dump; + roc_npc_flow_mcam_dump; roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 05/62] common/cnxk: support for mark and flag flow actions 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (3 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 04/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram ` (57 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> Add roc API to get mark action. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 17 +++++++++++++++++ drivers/common/cnxk/roc_npc.h | 3 +++ drivers/common/cnxk/version.map | 2 ++ 3 files changed, 22 insertions(+) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 81c7fd9..e6a5036 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -757,6 +757,23 @@ npc_rss_action_program(struct roc_npc *roc_npc, return 0; } +int +roc_npc_mark_actions_get(struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + return npc->mark_actions; +} + +int +roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + npc->mark_actions -= count; + return npc->mark_actions; +} + struct roc_npc_flow * roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, const struct roc_npc_item_info pattern[], diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 115bcd5..cf6f732 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -196,4 +196,7 @@ int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, struct roc_npc_flow *mcam); +int __roc_api roc_npc_mark_actions_get(struct roc_npc *roc_npc); +int __roc_api roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, + uint32_t count); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index a11ba4d..554459b 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -165,6 +165,8 @@ INTERNAL { roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; + roc_npc_mark_actions_get; + roc_npc_mark_actions_sub_return; roc_npc_mcam_alloc_entries; roc_npc_mcam_alloc_entry; roc_npc_mcam_clear_counter; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (4 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 05/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-14 3:32 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 07/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram ` (56 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Harman Kalra From: Harman Kalra <hkalra@marvell.com> As per the new LMTST design, userspace shall allocate lmt region, setup the DMA translation and share the IOVA with kernel via MBOX. Kernel will convert this IOVA to physical memory and update the LMT table entry with the same. With this new design also shared mode (i.e. all pci funcs sharing the LMT region allocated by primary/base pci func) is intact. Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 ++++++++++++++++++-------------------- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 3 ++ drivers/common/cnxk/roc_platform.h | 11 +++++ 5 files changed, 63 insertions(+), 52 deletions(-) diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h index 67f5d13..32e383c 100644 --- a/drivers/common/cnxk/roc_api.h +++ b/drivers/common/cnxk/roc_api.h @@ -24,6 +24,8 @@ /* Platform definition */ #include "roc_platform.h" +#define ROC_LMT_LINE_SZ 128 +#define ROC_NUM_LMT_LINES 2048 #define ROC_LMT_LINES_PER_CORE_LOG2 5 #define ROC_LMT_LINE_SIZE_LOG2 7 #define ROC_LMT_BASE_PER_CORE_LOG2 \ diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index a39acc9..adff779 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -915,43 +915,30 @@ dev_vf_mbase_put(struct plt_pci_device *pci_dev, uintptr_t vf_mbase) mbox_mem_unmap((void *)vf_mbase, MBOX_SIZE * pci_dev->max_vfs); } -static uint16_t -dev_pf_total_vfs(struct plt_pci_device *pci_dev) -{ - uint16_t total_vfs = 0; - int sriov_pos, rc; - - sriov_pos = - plt_pci_find_ext_capability(pci_dev, ROC_PCI_EXT_CAP_ID_SRIOV); - if (sriov_pos <= 0) { - plt_warn("Unable to find SRIOV cap, rc=%d", sriov_pos); - return 0; - } - - rc = plt_pci_read_config(pci_dev, &total_vfs, 2, - sriov_pos + ROC_PCI_SRIOV_TOTAL_VF); - if (rc < 0) { - plt_warn("Unable to read SRIOV cap, rc=%d", rc); - return 0; - } - - return total_vfs; -} - static int -dev_setup_shared_lmt_region(struct mbox *mbox) +dev_setup_shared_lmt_region(struct mbox *mbox, bool valid_iova, uint64_t iova) { struct lmtst_tbl_setup_req *req; req = mbox_alloc_msg_lmtst_tbl_setup(mbox); - req->pcifunc = idev_lmt_pffunc_get(); + /* This pcifunc is defined with primary pcifunc whose LMT address + * will be shared. If call contains valid IOVA, following pcifunc + * field is of no use. + */ + req->pcifunc = valid_iova ? 0 : idev_lmt_pffunc_get(); + req->use_local_lmt_region = valid_iova; + req->lmt_iova = iova; return mbox_process(mbox); } +/* Total no of lines * size of each lmtline */ +#define LMT_REGION_SIZE (ROC_NUM_LMT_LINES * ROC_LMT_LINE_SZ) static int -dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) +dev_lmt_setup(struct dev *dev) { + char name[PLT_MEMZONE_NAMESIZE]; + const struct plt_memzone *mz; struct idev_cfg *idev; int rc; @@ -965,8 +952,11 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) /* Set common lmt region from second pf_func onwards. */ if (!dev->disable_shared_lmt && idev_lmt_pffunc_get() && dev->pf_func != idev_lmt_pffunc_get()) { - rc = dev_setup_shared_lmt_region(dev->mbox); + rc = dev_setup_shared_lmt_region(dev->mbox, false, 0); if (!rc) { + /* On success, updating lmt base of secondary pf_funcs + * with primary pf_func's lmt base. + */ dev->lmt_base = roc_idev_lmt_base_addr_get(); return rc; } @@ -975,34 +965,30 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) dev->pf_func, rc); } - if (dev_is_vf(dev)) { - /* VF BAR4 should always be sufficient enough to - * hold LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (RVU_LMT_LINE_MAX * RVU_LMT_SZ)) { - plt_err("Not enough bar4 space for lmt lines"); - return -EFAULT; - } + /* Allocating memory for LMT region */ + sprintf(name, "LMT_MAP%x", dev->pf_func); - dev->lmt_base = dev->bar4; - } else { - uint64_t bar4_mbox_sz = MBOX_SIZE; - - /* PF BAR4 should always be sufficient enough to - * hold PF-AF MBOX + PF-VF MBOX + LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (bar4_mbox_sz + (RVU_LMT_LINE_MAX * RVU_LMT_SZ))) { - plt_err("Not enough bar4 space for lmt lines and mbox"); - return -EFAULT; - } + /* Setting alignment to ensure correct masking for resetting to lmt base + * of a core after all lmt lines under that core are used. + * Alignment value LMT_REGION_SIZE to handle the case where all lines + * are used by 1 core. + */ + mz = plt_lmt_region_reserve_aligned(name, LMT_REGION_SIZE, + LMT_REGION_SIZE); + if (!mz) { + plt_err("Memory alloc failed: %s", strerror(errno)); + goto fail; + } - /* LMT base is just after total VF MBOX area */ - bar4_mbox_sz += (MBOX_SIZE * dev_pf_total_vfs(pci_dev)); - dev->lmt_base = dev->bar4 + bar4_mbox_sz; + /* Share the IOVA address with Kernel */ + rc = dev_setup_shared_lmt_region(dev->mbox, true, mz->iova); + if (rc) { + errno = rc; + goto free; } + dev->lmt_base = mz->iova; + dev->lmt_mz = mz; /* Base LMT address should be chosen from only those pci funcs which * participate in LMT shared mode. */ @@ -1016,6 +1002,10 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) } return 0; +free: + plt_memzone_free(mz); +fail: + return -errno; } int @@ -1130,7 +1120,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) goto iounmap; /* Setup LMT line base */ - rc = dev_lmt_setup(pci_dev, dev); + rc = dev_lmt_setup(dev); if (rc) goto iounmap; @@ -1161,6 +1151,10 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) /* Clear references to this pci dev */ npa_lf_fini(); + /* Releasing memory allocated for lmt region */ + if (dev->lmt_mz) + plt_memzone_free(dev->lmt_mz); + mbox_unregister_irq(pci_dev, dev); if (!dev_is_vf(dev)) diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h index 910cfb6..7ee604e 100644 --- a/drivers/common/cnxk/roc_dev_priv.h +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -84,6 +84,7 @@ struct dev { struct dev_ops *ops; void *roc_nix; bool disable_shared_lmt; /* false(default): shared lmt mode enabled */ + const struct plt_memzone *lmt_mz; } __plt_cache_aligned; struct npa { diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h index fe4df21..ea11382 100644 --- a/drivers/common/cnxk/roc_mbox.h +++ b/drivers/common/cnxk/roc_mbox.h @@ -403,6 +403,9 @@ struct lmtst_tbl_setup_req { uint64_t __io dis_line_pref : 1; uint64_t __io ssow_pf_func : 13; uint16_t __io pcifunc; + uint8_t __io use_local_lmt_region; + uint64_t __io lmt_iova; + uint64_t __io rsvd[2]; /* Future use */ }; /* CGX mbox message formats */ diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 911ae15..be58625 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -194,4 +194,15 @@ int roc_plt_init(void); typedef int (*roc_plt_init_cb_t)(void); int __roc_api roc_plt_init_cb_register(roc_plt_init_cb_t cb); +static inline const void * +plt_lmt_region_reserve_aligned(const char *name, size_t len, uint32_t align) +{ + /* To ensure returned memory is physically contiguous, bounding + * the start and end address in 2M range. + */ + return rte_memzone_reserve_bounded(name, len, SOCKET_ID_ANY, + RTE_MEMZONE_IOVA_CONTIG, + align, RTE_PGSIZE_2M); +} + #endif /* _ROC_PLATFORM_H_ */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram @ 2021-06-14 3:32 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-14 3:32 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra On Mon, Jun 7, 2021 at 11:33 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Harman Kalra <hkalra@marvell.com> > > As per the new LMTST design, userspace shall allocate lmt region, > setup the DMA translation and share the IOVA with kernel via MBOX. > Kernel will convert this IOVA to physical memory and update the > LMT table entry with the same. > With this new design also shared mode (i.e. all pci funcs sharing > the LMT region allocated by primary/base pci func) is intact. > > Signed-off-by: Harman Kalra <hkalra@marvell.com> Reviewed-by: Jerin Jacob <jerinj@marvell.com> ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 07/62] common/cnxk: add provision to enable RED on RQ 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (5 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram ` (55 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Send RED pass/drop levels based on rq configurations to kernel. Fixed the aura and pool shift value calculation. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 8 ++++++ drivers/common/cnxk/roc_nix_queue.c | 50 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npa.c | 8 ++++-- drivers/common/cnxk/roc_npa.h | 5 ++++ 4 files changed, 69 insertions(+), 2 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index 6d9ac10..bb69027 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -161,6 +161,14 @@ struct roc_nix_rq { uint32_t vwqe_max_sz_exp; uint64_t vwqe_wait_tmo; uint64_t vwqe_aura_handle; + /* Average LPB aura level drop threshold for RED */ + uint8_t red_drop; + /* Average LPB aura level pass threshold for RED */ + uint8_t red_pass; + /* Average SPB aura level drop threshold for RED */ + uint8_t spb_red_drop; + /* Average SPB aura level pass threshold for RED */ + uint8_t spb_red_pass; /* End of Input parameters */ struct roc_nix *roc_nix; }; diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index 1c62aa2..0604e7a 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -119,6 +119,15 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_aura_pass = rq->spb_red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + + aq->rq.spb_aura_drop = rq->spb_red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -155,6 +164,14 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + } } return 0; @@ -244,6 +261,23 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_pool_pass = rq->red_pass; + aq->rq.spb_aura_pass = rq->red_pass; + aq->rq.lpb_pool_pass = rq->red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + aq->rq.wqe_pool_pass = rq->red_pass; + aq->rq.xqe_pass = rq->red_pass; + + aq->rq.spb_pool_drop = rq->red_drop; + aq->rq.spb_aura_drop = rq->red_drop; + aq->rq.lpb_pool_drop = rq->red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + aq->rq.wqe_pool_drop = rq->red_drop; + aq->rq.xqe_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -296,6 +330,22 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_pool_pass = ~aq->rq_mask.spb_pool_pass; + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_pool_pass = ~aq->rq_mask.lpb_pool_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + aq->rq_mask.wqe_pool_pass = ~aq->rq_mask.wqe_pool_pass; + aq->rq_mask.xqe_pass = ~aq->rq_mask.xqe_pass; + + aq->rq_mask.spb_pool_drop = ~aq->rq_mask.spb_pool_drop; + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_pool_drop = ~aq->rq_mask.lpb_pool_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + aq->rq_mask.wqe_pool_drop = ~aq->rq_mask.wqe_pool_drop; + aq->rq_mask.xqe_drop = ~aq->rq_mask.xqe_drop; + } } return 0; diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index 5ba6e81..d064d12 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -278,13 +278,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Update aura fields */ aura->pool_addr = pool_id; /* AF will translate to associated poolctx */ aura->ena = 1; - aura->shift = __builtin_clz(block_count) - 8; + aura->shift = plt_log2_u32(block_count); + aura->shift = aura->shift < 8 ? 0 : aura->shift - 8; aura->limit = block_count; aura->pool_caching = 1; aura->err_int_ena = BIT(NPA_AURA_ERR_INT_AURA_ADD_OVER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_ADD_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_FREE_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_POOL_DIS); + aura->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ aura->err_qint_idx = aura_id % lf->qints; @@ -293,13 +295,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, pool->ena = 1; pool->buf_size = block_size / ROC_ALIGN; pool->stack_max_pages = stack_size; - pool->shift = __builtin_clz(block_count) - 8; + pool->shift = plt_log2_u32(block_count); + pool->shift = pool->shift < 8 ? 0 : pool->shift - 8; pool->ptr_start = 0; pool->ptr_end = ~0; pool->stack_caching = 1; pool->err_int_ena = BIT(NPA_POOL_ERR_INT_OVFLS); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_RANGE); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_PERR); + pool->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ pool->err_qint_idx = pool_id % lf->qints; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 59d6223..3fc6192 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -12,6 +12,11 @@ #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 +/* This value controls how much of the present average resource level is used to + * calculate the new resource level. + */ +#define ROC_NPA_AVG_CONT 0xE0 + /* 16 CASP instructions can be outstanding in CN9k, but we use only 15 * outstanding CASPs as we run out of registers. */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 08/62] common/cnxk: fix flow create on CN98xx 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (6 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 07/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram ` (54 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> CN96xx and CN98xx have 4096 and 16384 MCAM entries respectively. Aligning the code with the same numbers. Fixes: a07f7ced436d ("common/cnxk: add NPC init and fini") Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_model.h | 6 ++++++ drivers/common/cnxk/roc_npc.c | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_model.h b/drivers/common/cnxk/roc_model.h index fb774ac..d6ef459 100644 --- a/drivers/common/cnxk/roc_model.h +++ b/drivers/common/cnxk/roc_model.h @@ -88,6 +88,12 @@ roc_model_is_cn10k(void) } static inline uint64_t +roc_model_is_cn98xx(void) +{ + return (roc_model->flag & ROC_MODEL_CN98xx_A0); +} + +static inline uint64_t roc_model_is_cn96_A0(void) { return roc_model->flag & ROC_MODEL_CN96xx_A0; diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index e6a5036..a69be4f 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -101,7 +101,7 @@ npc_mcam_tot_entries(void) /* FIXME: change to reading in AF from NPC_AF_CONST1/2 * MCAM_BANK_DEPTH(_EXT) * MCAM_BANKS */ - if (roc_model_is_cn10k()) + if (roc_model_is_cn10k() || roc_model_is_cn98xx()) return 16 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 4096 */ else return 4 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 1024 */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (7 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-09 1:38 ` Huisong Li 2021-06-14 3:52 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram ` (53 subsequent siblings) 62 siblings, 2 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add build infrastructure and common probe and remove for cnxk driver which is used by both CN10K and CN9K SoC. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- MAINTAINERS | 3 + doc/guides/nics/cnxk.rst | 29 +++++ doc/guides/nics/features/cnxk.ini | 9 ++ doc/guides/nics/features/cnxk_vec.ini | 9 ++ doc/guides/nics/features/cnxk_vf.ini | 9 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ drivers/net/cnxk/meson.build | 21 ++++ drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 12 files changed, 364 insertions(+) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16..2be220e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -746,6 +746,9 @@ M: Sunil Kumar Kori <skori@marvell.com> M: Satha Rao <skoteshwar@marvell.com> T: git://dpdk.org/next/dpdk-next-net-mrvl F: drivers/common/cnxk/ +F: drivers/net/cnxk/ +F: doc/guides/nics/cnxk.rst +F: doc/guides/nics/features/cnxk*.ini F: doc/guides/platform/cnxk.rst Marvell mvpp2 diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst new file mode 100644 index 0000000..ca21842 --- /dev/null +++ b/doc/guides/nics/cnxk.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK Poll Mode driver +===================== + +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** +SoC family as well as for their virtual functions (VF) in SR-IOV context. + +More information can be found at `Marvell Official Website +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. + +Features +-------- + +Features of the CNXK Ethdev PMD are: + +Prerequisites +------------- + +See :doc:`../platform/cnxk` for setup information. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini new file mode 100644 index 0000000..2c23464 --- /dev/null +++ b/doc/guides/nics/features/cnxk.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini new file mode 100644 index 0000000..de78516 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vec' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini new file mode 100644 index 0000000..9c96351 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 799697c..c1a04d9 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -19,6 +19,7 @@ Network Interface Controller Drivers axgbe bnx2x bnxt + cnxk cxgbe dpaa dpaa2 diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index cebb3d0..b506c11 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -142,6 +142,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in cnxk SoC. +#. **Ethdev Driver** + See :doc:`../nics/cnxk` for NIX Ethdev driver information. + #. **Mempool Driver** See :doc:`../mempool/cnxk` for NPA mempool driver information. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c new file mode 100644 index 0000000..6717410 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -0,0 +1,219 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_ethdev.h> + +/* CNXK platform independent eth dev ops */ +struct eth_dev_ops cnxk_eth_dev_ops; + +static int +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct rte_pci_device *pci_dev; + int rc, max_entries; + + eth_dev->dev_ops = &cnxk_eth_dev_ops; + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + rte_eth_copy_pci_info(eth_dev, pci_dev); + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + + /* Initialize base roc nix */ + nix->pci_dev = pci_dev; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to initialize roc nix rc=%d", rc); + goto error; + } + + dev->eth_dev = eth_dev; + + /* For vfs, returned max_entries will be 0. but to keep default mac + * address, one entry must be allocated. so setting up to 1. + */ + if (roc_nix_is_vf_or_sdp(nix)) + max_entries = 1; + else + max_entries = roc_nix_mac_max_entries_get(nix); + + if (max_entries <= 0) { + plt_err("Failed to get max entries for mac addr"); + rc = -ENOTSUP; + goto dev_fini; + } + + eth_dev->data->mac_addrs = + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + plt_err("Failed to allocate memory for mac addr"); + rc = -ENOMEM; + goto dev_fini; + } + + dev->max_mac_entries = max_entries; + + /* Get mac address */ + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to get mac addr, rc=%d", rc); + goto free_mac_addrs; + } + + /* Update the mac address */ + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); + + if (!roc_nix_is_vf_or_sdp(nix)) { + /* Sync same MAC address to CGX/RPM table */ + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to set mac addr, rc=%d", rc); + goto free_mac_addrs; + } + } + + /* Initialize roc npc */ + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, + eth_dev->data->port_id, roc_nix_get_pf(nix), + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, + dev->rx_offload_capa, dev->tx_offload_capa); + return 0; + +free_mac_addrs: + rte_free(eth_dev->data->mac_addrs); +dev_fini: + roc_nix_dev_fini(nix); +error: + plt_err("Failed to init nix eth_dev rc=%d", rc); + return rc; +} + +static int +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct roc_nix *nix = &dev->nix; + int rc, i; + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + roc_nix_npc_rx_ena_dis(nix, false); + + /* Free up SQs */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); + eth_dev->data->tx_queues[i] = NULL; + } + eth_dev->data->nb_tx_queues = 0; + + /* Free up RQ's and CQ's */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); + eth_dev->data->rx_queues[i] = NULL; + } + eth_dev->data->nb_rx_queues = 0; + + /* Free tm resources */ + roc_nix_tm_fini(nix); + + /* Unregister queue irqs */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister cq irqs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Free nix lf resources */ + rc = roc_nix_lf_free(nix); + if (rc) + plt_err("Failed to free nix lf, rc=%d", rc); + + rte_free(eth_dev->data->mac_addrs); + eth_dev->data->mac_addrs = NULL; + + /* Check if mbox close is needed */ + if (!mbox_close) + return 0; + + rc = roc_nix_dev_fini(nix); + /* Can be freed later by PMD if NPA LF is in use */ + if (rc == -EAGAIN) { + eth_dev->data->dev_private = NULL; + return 0; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + } + + return rc; +} + +int +cnxk_nix_remove(struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct roc_nix *nix; + int rc = -EINVAL; + + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (eth_dev) { + /* Cleanup eth dev */ + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + rte_eth_dev_release_port(eth_dev); + } + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (nix->pci_dev != pci_dev) + return 0; + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("%s: common resource in use by other devices", + pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + /* Free device pointer as rte_ethdev does not have it anymore */ + rte_free(nix); +exit: + return rc; +} + +int +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + int rc; + + RTE_SET_USED(pci_drv); + + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), + cnxk_eth_dev_init); + + /* On error on secondary, recheck if port exists in primary or + * in mid of detach state. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) + if (!rte_eth_dev_allocated(pci_dev->device.name)) + return 0; + return rc; +} diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h new file mode 100644 index 0000000..0460d1e --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_ETHDEV_H__ +#define __CNXK_ETHDEV_H__ + +#include <math.h> +#include <stdint.h> + +#include <ethdev_driver.h> +#include <ethdev_pci.h> + +#include "roc_api.h" + +#define CNXK_ETH_DEV_PMD_VERSION "1.0" + +struct cnxk_eth_dev { + /* ROC NIX */ + struct roc_nix nix; + + /* Max macfilter entries */ + uint8_t max_mac_entries; + + uint16_t flags; + + /* Pointer back to rte */ + struct rte_eth_dev *eth_dev; + + /* HW capabilities / Limitations */ + union { + uint64_t hwcap; + }; + + /* Rx and Tx offload capabilities */ + uint64_t rx_offload_capa; + uint64_t tx_offload_capa; + uint32_t speed_capa; + + /* Default mac address */ + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; +}; + +static inline struct cnxk_eth_dev * +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) +{ + return eth_dev->data->dev_private; +} + +/* Common ethdev ops */ +extern struct eth_dev_ops cnxk_eth_dev_ops; + +/* Ops */ +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); +int cnxk_nix_remove(struct rte_pci_device *pci_dev); + +#endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build new file mode 100644 index 0000000..77b2f18 --- /dev/null +++ b/drivers/net/cnxk/meson.build @@ -0,0 +1,21 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit' + subdir_done() +endif + +sources = files('cnxk_ethdev.c') + +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] +deps += ['common_cnxk', 'mempool_cnxk'] + +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] +foreach flag: extra_flags + if cc.has_argument(flag) + cflags += flag + endif +endforeach diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map new file mode 100644 index 0000000..ee80c51 --- /dev/null +++ b/drivers/net/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index c8b5ce2..5b066fd 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -12,6 +12,7 @@ drivers = [ 'bnx2x', 'bnxt', 'bonding', + 'cnxk', 'cxgbe', 'dpaa', 'dpaa2', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-06-09 1:38 ` Huisong Li 2021-06-09 10:45 ` Nithin Dabilpuram 2021-06-14 3:52 ` Jerin Jacob 1 sibling, 1 reply; 262+ messages in thread From: Huisong Li @ 2021-06-09 1:38 UTC (permalink / raw) To: Nithin Dabilpuram, dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar 在 2021/6/8 1:58, Nithin Dabilpuram 写道: > Add build infrastructure and common probe and remove for cnxk driver > which is used by both CN10K and CN9K SoC. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > --- > MAINTAINERS | 3 + > doc/guides/nics/cnxk.rst | 29 +++++ > doc/guides/nics/features/cnxk.ini | 9 ++ > doc/guides/nics/features/cnxk_vec.ini | 9 ++ > doc/guides/nics/features/cnxk_vf.ini | 9 ++ > doc/guides/nics/index.rst | 1 + > doc/guides/platform/cnxk.rst | 3 + > drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ > drivers/net/cnxk/meson.build | 21 ++++ > drivers/net/cnxk/version.map | 3 + > drivers/net/meson.build | 1 + > 12 files changed, 364 insertions(+) > create mode 100644 doc/guides/nics/cnxk.rst > create mode 100644 doc/guides/nics/features/cnxk.ini > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > create mode 100644 drivers/net/cnxk/meson.build > create mode 100644 drivers/net/cnxk/version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5877a16..2be220e 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -746,6 +746,9 @@ M: Sunil Kumar Kori <skori@marvell.com> > M: Satha Rao <skoteshwar@marvell.com> > T: git://dpdk.org/next/dpdk-next-net-mrvl > F: drivers/common/cnxk/ > +F: drivers/net/cnxk/ > +F: doc/guides/nics/cnxk.rst > +F: doc/guides/nics/features/cnxk*.ini > F: doc/guides/platform/cnxk.rst > > Marvell mvpp2 > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst > new file mode 100644 > index 0000000..ca21842 > --- /dev/null > +++ b/doc/guides/nics/cnxk.rst > @@ -0,0 +1,29 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(C) 2021 Marvell. > + > +CNXK Poll Mode driver > +===================== > + > +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver > +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** > +SoC family as well as for their virtual functions (VF) in SR-IOV context. > + > +More information can be found at `Marvell Official Website > +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. > + > +Features > +-------- > + > +Features of the CNXK Ethdev PMD are: > + > +Prerequisites > +------------- > + > +See :doc:`../platform/cnxk` for setup information. > + > + > +Driver compilation and testing > +------------------------------ > + > +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` > +for details. > diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini > new file mode 100644 > index 0000000..2c23464 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini > new file mode 100644 > index 0000000..de78516 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk_vec.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk_vec' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini > new file mode 100644 > index 0000000..9c96351 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk_vf.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk_vf' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 799697c..c1a04d9 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -19,6 +19,7 @@ Network Interface Controller Drivers > axgbe > bnx2x > bnxt > + cnxk > cxgbe > dpaa > dpaa2 > diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst > index cebb3d0..b506c11 100644 > --- a/doc/guides/platform/cnxk.rst > +++ b/doc/guides/platform/cnxk.rst > @@ -142,6 +142,9 @@ HW Offload Drivers > > This section lists dataplane H/W block(s) available in cnxk SoC. > > +#. **Ethdev Driver** > + See :doc:`../nics/cnxk` for NIX Ethdev driver information. > + > #. **Mempool Driver** > See :doc:`../mempool/cnxk` for NPA mempool driver information. > > diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c > new file mode 100644 > index 0000000..6717410 > --- /dev/null > +++ b/drivers/net/cnxk/cnxk_ethdev.c > @@ -0,0 +1,219 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#include <cnxk_ethdev.h> > + > +/* CNXK platform independent eth dev ops */ > +struct eth_dev_ops cnxk_eth_dev_ops; > + > +static int > +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + struct roc_nix *nix = &dev->nix; > + struct rte_pci_device *pci_dev; > + int rc, max_entries; > + > + eth_dev->dev_ops = &cnxk_eth_dev_ops; > + > + /* For secondary processes, the primary has done all the work */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); > + rte_eth_copy_pci_info(eth_dev, pci_dev); > + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; Hi, It is recommended not to use this flag when add new driver. Queue stats will be moved to xstats and be filled by PMDs. Please check the following patch: commit f30e69b41f949cd4a9afb6ff39de196e661708e2 Author: Ferruh Yigit <ferruh.yigit@intel.com> Date: Wed Oct 14 03:26:47 2020 +0100 ethdev: add device flag to bypass auto-filled queue xstats Queue stats are stored in 'struct rte_eth_stats' as array and array size is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag. As a result of technical board discussion, decided to remove the queue statistics from 'struct rte_eth_stats' in the long term. Instead PMDs should represent the queue statistics via xstats, this gives more flexibility on the number of the queues supported. Currently queue stats in the xstats are filled by ethdev layer, using some basic stats, when queue stats removed from basic stats the responsibility to fill the relevant xstats will be pushed to the PMDs. During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' device flag is created. Initially all PMDs using xstats set this flag. The PMDs implemented queue stats in the xstats should clear the flag. When all PMDs switch to the xstats for the queue stats, queue stats related fields from 'struct rte_eth_stats' will be removed, as well as 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag. Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be removed. Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Acked-by: Haiyue Wang <haiyue.wang@intel.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Acked-by: Thomas Monjalon <thomas@monjalon.net> > + > + /* Initialize base roc nix */ > + nix->pci_dev = pci_dev; > + rc = roc_nix_dev_init(nix); > + if (rc) { > + plt_err("Failed to initialize roc nix rc=%d", rc); > + goto error; > + } > + > + dev->eth_dev = eth_dev; > + > + /* For vfs, returned max_entries will be 0. but to keep default mac > + * address, one entry must be allocated. so setting up to 1. > + */ > + if (roc_nix_is_vf_or_sdp(nix)) > + max_entries = 1; > + else > + max_entries = roc_nix_mac_max_entries_get(nix); > + > + if (max_entries <= 0) { > + plt_err("Failed to get max entries for mac addr"); > + rc = -ENOTSUP; > + goto dev_fini; > + } > + > + eth_dev->data->mac_addrs = > + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); > + if (eth_dev->data->mac_addrs == NULL) { > + plt_err("Failed to allocate memory for mac addr"); > + rc = -ENOMEM; > + goto dev_fini; > + } > + > + dev->max_mac_entries = max_entries; > + > + /* Get mac address */ > + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); > + if (rc) { > + plt_err("Failed to get mac addr, rc=%d", rc); > + goto free_mac_addrs; > + } > + > + /* Update the mac address */ > + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); > + > + if (!roc_nix_is_vf_or_sdp(nix)) { > + /* Sync same MAC address to CGX/RPM table */ > + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); > + if (rc) { > + plt_err("Failed to set mac addr, rc=%d", rc); > + goto free_mac_addrs; > + } > + } > + > + /* Initialize roc npc */ > + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 > + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, > + eth_dev->data->port_id, roc_nix_get_pf(nix), > + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, > + dev->rx_offload_capa, dev->tx_offload_capa); > + return 0; > + > +free_mac_addrs: > + rte_free(eth_dev->data->mac_addrs); > +dev_fini: > + roc_nix_dev_fini(nix); > +error: > + plt_err("Failed to init nix eth_dev rc=%d", rc); > + return rc; > +} > + > +static int > +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; > + struct roc_nix *nix = &dev->nix; > + int rc, i; > + > + /* Nothing to be done for secondary processes */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + roc_nix_npc_rx_ena_dis(nix, false); > + > + /* Free up SQs */ > + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { > + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); > + eth_dev->data->tx_queues[i] = NULL; > + } > + eth_dev->data->nb_tx_queues = 0; > + > + /* Free up RQ's and CQ's */ > + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { > + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); > + eth_dev->data->rx_queues[i] = NULL; > + } > + eth_dev->data->nb_rx_queues = 0; > + > + /* Free tm resources */ > + roc_nix_tm_fini(nix); > + > + /* Unregister queue irqs */ > + roc_nix_unregister_queue_irqs(nix); > + > + /* Unregister cq irqs */ > + if (eth_dev->data->dev_conf.intr_conf.rxq) > + roc_nix_unregister_cq_irqs(nix); > + > + /* Free nix lf resources */ > + rc = roc_nix_lf_free(nix); > + if (rc) > + plt_err("Failed to free nix lf, rc=%d", rc); > + > + rte_free(eth_dev->data->mac_addrs); > + eth_dev->data->mac_addrs = NULL; > + > + /* Check if mbox close is needed */ > + if (!mbox_close) > + return 0; > + > + rc = roc_nix_dev_fini(nix); > + /* Can be freed later by PMD if NPA LF is in use */ > + if (rc == -EAGAIN) { > + eth_dev->data->dev_private = NULL; > + return 0; > + } else if (rc) { > + plt_err("Failed in nix dev fini, rc=%d", rc); > + } > + > + return rc; > +} > + > +int > +cnxk_nix_remove(struct rte_pci_device *pci_dev) > +{ > + struct rte_eth_dev *eth_dev; > + struct roc_nix *nix; > + int rc = -EINVAL; > + > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); > + if (eth_dev) { > + /* Cleanup eth dev */ > + rc = cnxk_eth_dev_uninit(eth_dev, true); > + if (rc) > + return rc; > + > + rte_eth_dev_release_port(eth_dev); > + } > + > + /* Nothing to be done for secondary processes */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + /* Check if this device is hosting common resource */ > + nix = roc_idev_npa_nix_get(); > + if (nix->pci_dev != pci_dev) > + return 0; > + > + /* Try nix fini now */ > + rc = roc_nix_dev_fini(nix); > + if (rc == -EAGAIN) { > + plt_info("%s: common resource in use by other devices", > + pci_dev->name); > + goto exit; > + } else if (rc) { > + plt_err("Failed in nix dev fini, rc=%d", rc); > + goto exit; > + } > + > + /* Free device pointer as rte_ethdev does not have it anymore */ > + rte_free(nix); > +exit: > + return rc; > +} > + > +int > +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > +{ > + int rc; > + > + RTE_SET_USED(pci_drv); > + > + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), > + cnxk_eth_dev_init); > + > + /* On error on secondary, recheck if port exists in primary or > + * in mid of detach state. > + */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) > + if (!rte_eth_dev_allocated(pci_dev->device.name)) > + return 0; > + return rc; > +} > diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h > new file mode 100644 > index 0000000..0460d1e > --- /dev/null > +++ b/drivers/net/cnxk/cnxk_ethdev.h > @@ -0,0 +1,57 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#ifndef __CNXK_ETHDEV_H__ > +#define __CNXK_ETHDEV_H__ > + > +#include <math.h> > +#include <stdint.h> > + > +#include <ethdev_driver.h> > +#include <ethdev_pci.h> > + > +#include "roc_api.h" > + > +#define CNXK_ETH_DEV_PMD_VERSION "1.0" > + > +struct cnxk_eth_dev { > + /* ROC NIX */ > + struct roc_nix nix; > + > + /* Max macfilter entries */ > + uint8_t max_mac_entries; > + > + uint16_t flags; > + > + /* Pointer back to rte */ > + struct rte_eth_dev *eth_dev; > + > + /* HW capabilities / Limitations */ > + union { > + uint64_t hwcap; > + }; > + > + /* Rx and Tx offload capabilities */ > + uint64_t rx_offload_capa; > + uint64_t tx_offload_capa; > + uint32_t speed_capa; > + > + /* Default mac address */ > + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; > +}; > + > +static inline struct cnxk_eth_dev * > +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) > +{ > + return eth_dev->data->dev_private; > +} > + > +/* Common ethdev ops */ > +extern struct eth_dev_ops cnxk_eth_dev_ops; > + > +/* Ops */ > +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, > + struct rte_pci_device *pci_dev); > +int cnxk_nix_remove(struct rte_pci_device *pci_dev); > + > +#endif /* __CNXK_ETHDEV_H__ */ > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build > new file mode 100644 > index 0000000..77b2f18 > --- /dev/null > +++ b/drivers/net/cnxk/meson.build > @@ -0,0 +1,21 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(C) 2021 Marvell. > +# > + > +if not dpdk_conf.get('RTE_ARCH_64') > + build = false > + reason = 'only supported on 64-bit' > + subdir_done() > +endif > + > +sources = files('cnxk_ethdev.c') > + > +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] > +deps += ['common_cnxk', 'mempool_cnxk'] > + > +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] > +foreach flag: extra_flags > + if cc.has_argument(flag) > + cflags += flag > + endif > +endforeach > diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map > new file mode 100644 > index 0000000..ee80c51 > --- /dev/null > +++ b/drivers/net/cnxk/version.map > @@ -0,0 +1,3 @@ > +INTERNAL { > + local: *; > +}; > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index c8b5ce2..5b066fd 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -12,6 +12,7 @@ drivers = [ > 'bnx2x', > 'bnxt', > 'bonding', > + 'cnxk', > 'cxgbe', > 'dpaa', > 'dpaa2', ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe 2021-06-09 1:38 ` Huisong Li @ 2021-06-09 10:45 ` Nithin Dabilpuram 0 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-09 10:45 UTC (permalink / raw) To: Huisong Li Cc: dev, jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar On Wed, Jun 09, 2021 at 09:38:04AM +0800, Huisong Li wrote: > > 在 2021/6/8 1:58, Nithin Dabilpuram 写道: > > Add build infrastructure and common probe and remove for cnxk driver > > which is used by both CN10K and CN9K SoC. > > > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > > --- > > MAINTAINERS | 3 + > > doc/guides/nics/cnxk.rst | 29 +++++ > > doc/guides/nics/features/cnxk.ini | 9 ++ > > doc/guides/nics/features/cnxk_vec.ini | 9 ++ > > doc/guides/nics/features/cnxk_vf.ini | 9 ++ > > doc/guides/nics/index.rst | 1 + > > doc/guides/platform/cnxk.rst | 3 + > > drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ > > drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ > > drivers/net/cnxk/meson.build | 21 ++++ > > drivers/net/cnxk/version.map | 3 + > > drivers/net/meson.build | 1 + > > 12 files changed, 364 insertions(+) > > create mode 100644 doc/guides/nics/cnxk.rst > > create mode 100644 doc/guides/nics/features/cnxk.ini > > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > > create mode 100644 drivers/net/cnxk/meson.build > > create mode 100644 drivers/net/cnxk/version.map > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 5877a16..2be220e 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -746,6 +746,9 @@ M: Sunil Kumar Kori <skori@marvell.com> > > M: Satha Rao <skoteshwar@marvell.com> > > T: git://dpdk.org/next/dpdk-next-net-mrvl > > F: drivers/common/cnxk/ > > +F: drivers/net/cnxk/ > > +F: doc/guides/nics/cnxk.rst > > +F: doc/guides/nics/features/cnxk*.ini > > F: doc/guides/platform/cnxk.rst > > Marvell mvpp2 > > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst > > new file mode 100644 > > index 0000000..ca21842 > > --- /dev/null > > +++ b/doc/guides/nics/cnxk.rst > > @@ -0,0 +1,29 @@ > > +.. SPDX-License-Identifier: BSD-3-Clause > > + Copyright(C) 2021 Marvell. > > + > > +CNXK Poll Mode driver > > +===================== > > + > > +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver > > +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** > > +SoC family as well as for their virtual functions (VF) in SR-IOV context. > > + > > +More information can be found at `Marvell Official Website > > +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. > > + > > +Features > > +-------- > > + > > +Features of the CNXK Ethdev PMD are: > > + > > +Prerequisites > > +------------- > > + > > +See :doc:`../platform/cnxk` for setup information. > > + > > + > > +Driver compilation and testing > > +------------------------------ > > + > > +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` > > +for details. > > diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini > > new file mode 100644 > > index 0000000..2c23464 > > --- /dev/null > > +++ b/doc/guides/nics/features/cnxk.ini > > @@ -0,0 +1,9 @@ > > +; > > +; Supported features of the 'cnxk' network poll mode driver. > > +; > > +; Refer to default.ini for the full list of available PMD features. > > +; > > +[Features] > > +Linux = Y > > +ARMv8 = Y > > +Usage doc = Y > > diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini > > new file mode 100644 > > index 0000000..de78516 > > --- /dev/null > > +++ b/doc/guides/nics/features/cnxk_vec.ini > > @@ -0,0 +1,9 @@ > > +; > > +; Supported features of the 'cnxk_vec' network poll mode driver. > > +; > > +; Refer to default.ini for the full list of available PMD features. > > +; > > +[Features] > > +Linux = Y > > +ARMv8 = Y > > +Usage doc = Y > > diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini > > new file mode 100644 > > index 0000000..9c96351 > > --- /dev/null > > +++ b/doc/guides/nics/features/cnxk_vf.ini > > @@ -0,0 +1,9 @@ > > +; > > +; Supported features of the 'cnxk_vf' network poll mode driver. > > +; > > +; Refer to default.ini for the full list of available PMD features. > > +; > > +[Features] > > +Linux = Y > > +ARMv8 = Y > > +Usage doc = Y > > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > > index 799697c..c1a04d9 100644 > > --- a/doc/guides/nics/index.rst > > +++ b/doc/guides/nics/index.rst > > @@ -19,6 +19,7 @@ Network Interface Controller Drivers > > axgbe > > bnx2x > > bnxt > > + cnxk > > cxgbe > > dpaa > > dpaa2 > > diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst > > index cebb3d0..b506c11 100644 > > --- a/doc/guides/platform/cnxk.rst > > +++ b/doc/guides/platform/cnxk.rst > > @@ -142,6 +142,9 @@ HW Offload Drivers > > This section lists dataplane H/W block(s) available in cnxk SoC. > > +#. **Ethdev Driver** > > + See :doc:`../nics/cnxk` for NIX Ethdev driver information. > > + > > #. **Mempool Driver** > > See :doc:`../mempool/cnxk` for NPA mempool driver information. > > diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c > > new file mode 100644 > > index 0000000..6717410 > > --- /dev/null > > +++ b/drivers/net/cnxk/cnxk_ethdev.c > > @@ -0,0 +1,219 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2021 Marvell. > > + */ > > +#include <cnxk_ethdev.h> > > + > > +/* CNXK platform independent eth dev ops */ > > +struct eth_dev_ops cnxk_eth_dev_ops; > > + > > +static int > > +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) > > +{ > > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > > + struct roc_nix *nix = &dev->nix; > > + struct rte_pci_device *pci_dev; > > + int rc, max_entries; > > + > > + eth_dev->dev_ops = &cnxk_eth_dev_ops; > > + > > + /* For secondary processes, the primary has done all the work */ > > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > > + return 0; > > + > > + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); > > + rte_eth_copy_pci_info(eth_dev, pci_dev); > > + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; > Hi, > > It is recommended not to use this flag when add new driver. Queue stats will > be moved to xstats and be filled by PMDs. > > Please check the following patch: > > commit f30e69b41f949cd4a9afb6ff39de196e661708e2 > Author: Ferruh Yigit <ferruh.yigit@intel.com> > Date: Wed Oct 14 03:26:47 2020 +0100 > > ethdev: add device flag to bypass auto-filled queue xstats > > Queue stats are stored in 'struct rte_eth_stats' as array and array size > is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag. > > As a result of technical board discussion, decided to remove the queue > statistics from 'struct rte_eth_stats' in the long term. > > Instead PMDs should represent the queue statistics via xstats, this > gives more flexibility on the number of the queues supported. > > Currently queue stats in the xstats are filled by ethdev layer, using > some basic stats, when queue stats removed from basic stats the > responsibility to fill the relevant xstats will be pushed to the PMDs. > > During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' > device flag is created. Initially all PMDs using xstats set this flag. > The PMDs implemented queue stats in the xstats should clear the flag. > > When all PMDs switch to the xstats for the queue stats, queue stats > related fields from 'struct rte_eth_stats' will be removed, as well as > 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag. > Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be > removed. > > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> > Acked-by: Haiyue Wang <haiyue.wang@intel.com> > Acked-by: Xiao Wang <xiao.w.wang@intel.com> > Acked-by: Thomas Monjalon <thomas@monjalon.net> Ack, will fix it in V3. > > > + > > + /* Initialize base roc nix */ > > + nix->pci_dev = pci_dev; > > + rc = roc_nix_dev_init(nix); > > + if (rc) { > > + plt_err("Failed to initialize roc nix rc=%d", rc); > > + goto error; > > + } > > + > > + dev->eth_dev = eth_dev; > > + > > + /* For vfs, returned max_entries will be 0. but to keep default mac > > + * address, one entry must be allocated. so setting up to 1. > > + */ > > + if (roc_nix_is_vf_or_sdp(nix)) > > + max_entries = 1; > > + else > > + max_entries = roc_nix_mac_max_entries_get(nix); > > + > > + if (max_entries <= 0) { > > + plt_err("Failed to get max entries for mac addr"); > > + rc = -ENOTSUP; > > + goto dev_fini; > > + } > > + > > + eth_dev->data->mac_addrs = > > + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); > > + if (eth_dev->data->mac_addrs == NULL) { > > + plt_err("Failed to allocate memory for mac addr"); > > + rc = -ENOMEM; > > + goto dev_fini; > > + } > > + > > + dev->max_mac_entries = max_entries; > > + > > + /* Get mac address */ > > + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); > > + if (rc) { > > + plt_err("Failed to get mac addr, rc=%d", rc); > > + goto free_mac_addrs; > > + } > > + > > + /* Update the mac address */ > > + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); > > + > > + if (!roc_nix_is_vf_or_sdp(nix)) { > > + /* Sync same MAC address to CGX/RPM table */ > > + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); > > + if (rc) { > > + plt_err("Failed to set mac addr, rc=%d", rc); > > + goto free_mac_addrs; > > + } > > + } > > + > > + /* Initialize roc npc */ > > + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 > > + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, > > + eth_dev->data->port_id, roc_nix_get_pf(nix), > > + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, > > + dev->rx_offload_capa, dev->tx_offload_capa); > > + return 0; > > + > > +free_mac_addrs: > > + rte_free(eth_dev->data->mac_addrs); > > +dev_fini: > > + roc_nix_dev_fini(nix); > > +error: > > + plt_err("Failed to init nix eth_dev rc=%d", rc); > > + return rc; > > +} > > + > > +static int > > +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) > > +{ > > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > > + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; > > + struct roc_nix *nix = &dev->nix; > > + int rc, i; > > + > > + /* Nothing to be done for secondary processes */ > > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > > + return 0; > > + > > + roc_nix_npc_rx_ena_dis(nix, false); > > + > > + /* Free up SQs */ > > + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { > > + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); > > + eth_dev->data->tx_queues[i] = NULL; > > + } > > + eth_dev->data->nb_tx_queues = 0; > > + > > + /* Free up RQ's and CQ's */ > > + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { > > + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); > > + eth_dev->data->rx_queues[i] = NULL; > > + } > > + eth_dev->data->nb_rx_queues = 0; > > + > > + /* Free tm resources */ > > + roc_nix_tm_fini(nix); > > + > > + /* Unregister queue irqs */ > > + roc_nix_unregister_queue_irqs(nix); > > + > > + /* Unregister cq irqs */ > > + if (eth_dev->data->dev_conf.intr_conf.rxq) > > + roc_nix_unregister_cq_irqs(nix); > > + > > + /* Free nix lf resources */ > > + rc = roc_nix_lf_free(nix); > > + if (rc) > > + plt_err("Failed to free nix lf, rc=%d", rc); > > + > > + rte_free(eth_dev->data->mac_addrs); > > + eth_dev->data->mac_addrs = NULL; > > + > > + /* Check if mbox close is needed */ > > + if (!mbox_close) > > + return 0; > > + > > + rc = roc_nix_dev_fini(nix); > > + /* Can be freed later by PMD if NPA LF is in use */ > > + if (rc == -EAGAIN) { > > + eth_dev->data->dev_private = NULL; > > + return 0; > > + } else if (rc) { > > + plt_err("Failed in nix dev fini, rc=%d", rc); > > + } > > + > > + return rc; > > +} > > + > > +int > > +cnxk_nix_remove(struct rte_pci_device *pci_dev) > > +{ > > + struct rte_eth_dev *eth_dev; > > + struct roc_nix *nix; > > + int rc = -EINVAL; > > + > > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); > > + if (eth_dev) { > > + /* Cleanup eth dev */ > > + rc = cnxk_eth_dev_uninit(eth_dev, true); > > + if (rc) > > + return rc; > > + > > + rte_eth_dev_release_port(eth_dev); > > + } > > + > > + /* Nothing to be done for secondary processes */ > > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > > + return 0; > > + > > + /* Check if this device is hosting common resource */ > > + nix = roc_idev_npa_nix_get(); > > + if (nix->pci_dev != pci_dev) > > + return 0; > > + > > + /* Try nix fini now */ > > + rc = roc_nix_dev_fini(nix); > > + if (rc == -EAGAIN) { > > + plt_info("%s: common resource in use by other devices", > > + pci_dev->name); > > + goto exit; > > + } else if (rc) { > > + plt_err("Failed in nix dev fini, rc=%d", rc); > > + goto exit; > > + } > > + > > + /* Free device pointer as rte_ethdev does not have it anymore */ > > + rte_free(nix); > > +exit: > > + return rc; > > +} > > + > > +int > > +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > > +{ > > + int rc; > > + > > + RTE_SET_USED(pci_drv); > > + > > + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), > > + cnxk_eth_dev_init); > > + > > + /* On error on secondary, recheck if port exists in primary or > > + * in mid of detach state. > > + */ > > + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) > > + if (!rte_eth_dev_allocated(pci_dev->device.name)) > > + return 0; > > + return rc; > > +} > > diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h > > new file mode 100644 > > index 0000000..0460d1e > > --- /dev/null > > +++ b/drivers/net/cnxk/cnxk_ethdev.h > > @@ -0,0 +1,57 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2021 Marvell. > > + */ > > +#ifndef __CNXK_ETHDEV_H__ > > +#define __CNXK_ETHDEV_H__ > > + > > +#include <math.h> > > +#include <stdint.h> > > + > > +#include <ethdev_driver.h> > > +#include <ethdev_pci.h> > > + > > +#include "roc_api.h" > > + > > +#define CNXK_ETH_DEV_PMD_VERSION "1.0" > > + > > +struct cnxk_eth_dev { > > + /* ROC NIX */ > > + struct roc_nix nix; > > + > > + /* Max macfilter entries */ > > + uint8_t max_mac_entries; > > + > > + uint16_t flags; > > + > > + /* Pointer back to rte */ > > + struct rte_eth_dev *eth_dev; > > + > > + /* HW capabilities / Limitations */ > > + union { > > + uint64_t hwcap; > > + }; > > + > > + /* Rx and Tx offload capabilities */ > > + uint64_t rx_offload_capa; > > + uint64_t tx_offload_capa; > > + uint32_t speed_capa; > > + > > + /* Default mac address */ > > + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; > > +}; > > + > > +static inline struct cnxk_eth_dev * > > +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) > > +{ > > + return eth_dev->data->dev_private; > > +} > > + > > +/* Common ethdev ops */ > > +extern struct eth_dev_ops cnxk_eth_dev_ops; > > + > > +/* Ops */ > > +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, > > + struct rte_pci_device *pci_dev); > > +int cnxk_nix_remove(struct rte_pci_device *pci_dev); > > + > > +#endif /* __CNXK_ETHDEV_H__ */ > > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build > > new file mode 100644 > > index 0000000..77b2f18 > > --- /dev/null > > +++ b/drivers/net/cnxk/meson.build > > @@ -0,0 +1,21 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(C) 2021 Marvell. > > +# > > + > > +if not dpdk_conf.get('RTE_ARCH_64') > > + build = false > > + reason = 'only supported on 64-bit' > > + subdir_done() > > +endif > > + > > +sources = files('cnxk_ethdev.c') > > + > > +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] > > +deps += ['common_cnxk', 'mempool_cnxk'] > > + > > +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] > > +foreach flag: extra_flags > > + if cc.has_argument(flag) > > + cflags += flag > > + endif > > +endforeach > > diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map > > new file mode 100644 > > index 0000000..ee80c51 > > --- /dev/null > > +++ b/drivers/net/cnxk/version.map > > @@ -0,0 +1,3 @@ > > +INTERNAL { > > + local: *; > > +}; > > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > > index c8b5ce2..5b066fd 100644 > > --- a/drivers/net/meson.build > > +++ b/drivers/net/meson.build > > @@ -12,6 +12,7 @@ drivers = [ > > 'bnx2x', > > 'bnxt', > > 'bonding', > > + 'cnxk', > > 'cxgbe', > > 'dpaa', > > 'dpaa2', ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-06-09 1:38 ` Huisong Li @ 2021-06-14 3:52 ` Jerin Jacob 1 sibling, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-14 3:52 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:34 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > Add build infrastructure and common probe and remove for cnxk driver > which is used by both CN10K and CN9K SoC. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > --- > MAINTAINERS | 3 + > doc/guides/nics/cnxk.rst | 29 +++++ > doc/guides/nics/features/cnxk.ini | 9 ++ > doc/guides/nics/features/cnxk_vec.ini | 9 ++ > doc/guides/nics/features/cnxk_vf.ini | 9 ++ > doc/guides/nics/index.rst | 1 + > doc/guides/platform/cnxk.rst | 3 + > drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ > drivers/net/cnxk/meson.build | 21 ++++ > drivers/net/cnxk/version.map | 3 + > drivers/net/meson.build | 1 + > 12 files changed, 364 insertions(+) > create mode 100644 doc/guides/nics/cnxk.rst > create mode 100644 doc/guides/nics/features/cnxk.ini > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > create mode 100644 drivers/net/cnxk/meson.build > create mode 100644 drivers/net/cnxk/version.map 21.08 release note update also can be done in this patch. > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5877a16..2be220e 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -746,6 +746,9 @@ M: Sunil Kumar Kori <skori@marvell.com> > M: Satha Rao <skoteshwar@marvell.com> > T: git://dpdk.org/next/dpdk-next-net-mrvl > F: drivers/common/cnxk/ > +F: drivers/net/cnxk/ > +F: doc/guides/nics/cnxk.rst > +F: doc/guides/nics/features/cnxk*.ini > F: doc/guides/platform/cnxk.rst Sort this in alphabetical order. > + > +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] Move this to patch where this actually required and have one line comment on need. > +foreach flag: extra_flags > + if cc.has_argument(flag) > + cflags += flag > + endif > +endforeach ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (8 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-15 12:26 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram ` (52 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add platform specific probe and remove callbacks for CN9K and CN10K which use common probe and remove functions. Register ethdev driver for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ drivers/net/cnxk/meson.build | 5 +++ 7 files changed, 230 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c new file mode 100644 index 0000000..ff8ce31 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn10k_ethdev.h" + +static int +cn10k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 64) { + plt_err("Driver not compiled for CN10K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + } + return 0; +} + +static const struct rte_pci_id cn10k_pci_nix_map[] = { + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn10k_pci_nix = { + .id_table = cn10k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn10k_nix_probe, + .remove = cn10k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h new file mode 100644 index 0000000..1bf4a65 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_ETHDEV_H__ +#define __CN10K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c new file mode 100644 index 0000000..701dc12 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn9k_ethdev.h" + +static int +cn9k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 128) { + plt_err("Driver not compiled for CN9K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + + dev = cnxk_eth_pmd_priv(eth_dev); + /* Update capabilities already set for TSO. + * TSO not supported for earlier chip revisions + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_GRE_TNL_TSO); + + /* 50G and 100G to be supported for board version C0 + * and above of CN9K. + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) { + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; + } + + dev->hwcap = 0; + + /* Update HW erratas */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->cq_min_4k = 1; + return 0; +} + +static const struct rte_pci_id cn9k_pci_nix_map[] = { + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn9k_pci_nix = { + .id_table = cn9k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn9k_nix_probe, + .remove = cn9k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h new file mode 100644 index 0000000..15d9397 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_ETHDEV_H__ +#define __CN9K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6717410..b836fc2 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -3,6 +3,40 @@ */ #include <cnxk_ethdev.h> +static inline uint64_t +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) +{ + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; + + return capa; +} + +static inline uint64_t +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) +{ + RTE_SET_USED(dev); + return CNXK_NIX_TX_OFFLOAD_CAPA; +} + +static inline uint32_t +nix_get_speed_capa(struct cnxk_eth_dev *dev) +{ + uint32_t speed_capa; + + /* Auto negotiation disabled */ + speed_capa = ETH_LINK_SPEED_FIXED; + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; + } + + return speed_capa; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops; @@ -77,6 +111,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } } + /* Union of all capabilities supported by CNXK. + * Platform specific capabilities will be + * updated later. + */ + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); + dev->speed_capa = nix_get_speed_capa(dev); + /* Initialize roc npc */ plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0460d1e..ba2bfcd 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -14,6 +14,22 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +#define CNXK_NIX_TX_OFFLOAD_CAPA \ + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ + DEV_TX_OFFLOAD_IPV4_CKSUM) + +#define CNXK_NIX_RX_OFFLOAD_CAPA \ + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_RX_OFFLOAD_RSS_HASH) + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -28,6 +44,9 @@ struct cnxk_eth_dev { /* HW capabilities / Limitations */ union { + struct { + uint64_t cq_min_4k : 1; + }; uint64_t hwcap; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 77b2f18..57dea5e 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,6 +10,11 @@ endif sources = files('cnxk_ethdev.c') +# CN9K +sources += files('cn9k_ethdev.c') +# CN10K +sources += files('cn10k_ethdev.c') + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram @ 2021-06-15 12:26 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:26 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:34 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > Add platform specific probe and remove callbacks for CN9K > and CN10K which use common probe and remove functions. > Register ethdev driver for CN9K and CN10K. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Reviewed-by: Jerin Jacob <jerinj@marvell.com> > --- > drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ > drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ > drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ > drivers/net/cnxk/meson.build | 5 +++ > 7 files changed, 230 insertions(+) > create mode 100644 drivers/net/cnxk/cn10k_ethdev.c > create mode 100644 drivers/net/cnxk/cn10k_ethdev.h > create mode 100644 drivers/net/cnxk/cn9k_ethdev.c > create mode 100644 drivers/net/cnxk/cn9k_ethdev.h > > diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c > new file mode 100644 > index 0000000..ff8ce31 > --- /dev/null > +++ b/drivers/net/cnxk/cn10k_ethdev.c > @@ -0,0 +1,64 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#include "cn10k_ethdev.h" > + > +static int > +cn10k_nix_remove(struct rte_pci_device *pci_dev) > +{ > + return cnxk_nix_remove(pci_dev); > +} > + > +static int > +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > +{ > + struct rte_eth_dev *eth_dev; > + int rc; > + > + if (RTE_CACHE_LINE_SIZE != 64) { > + plt_err("Driver not compiled for CN10K"); > + return -EFAULT; > + } > + > + rc = roc_plt_init(); > + if (rc) { > + plt_err("Failed to initialize platform model, rc=%d", rc); > + return rc; > + } > + > + /* Common probe */ > + rc = cnxk_nix_probe(pci_drv, pci_dev); > + if (rc) > + return rc; > + > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); > + if (!eth_dev) > + return -ENOENT; > + } > + return 0; > +} > + > +static const struct rte_pci_id cn10k_pci_nix_map[] = { > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), > + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), > + { > + .vendor_id = 0, > + }, > +}; > + > +static struct rte_pci_driver cn10k_pci_nix = { > + .id_table = cn10k_pci_nix_map, > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | > + RTE_PCI_DRV_INTR_LSC, > + .probe = cn10k_nix_probe, > + .remove = cn10k_nix_remove, > +}; > + > +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); > +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); > +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); > diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h > new file mode 100644 > index 0000000..1bf4a65 > --- /dev/null > +++ b/drivers/net/cnxk/cn10k_ethdev.h > @@ -0,0 +1,9 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#ifndef __CN10K_ETHDEV_H__ > +#define __CN10K_ETHDEV_H__ > + > +#include <cnxk_ethdev.h> > + > +#endif /* __CN10K_ETHDEV_H__ */ > diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c > new file mode 100644 > index 0000000..701dc12 > --- /dev/null > +++ b/drivers/net/cnxk/cn9k_ethdev.c > @@ -0,0 +1,82 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#include "cn9k_ethdev.h" > + > +static int > +cn9k_nix_remove(struct rte_pci_device *pci_dev) > +{ > + return cnxk_nix_remove(pci_dev); > +} > + > +static int > +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > +{ > + struct rte_eth_dev *eth_dev; > + struct cnxk_eth_dev *dev; > + int rc; > + > + if (RTE_CACHE_LINE_SIZE != 128) { > + plt_err("Driver not compiled for CN9K"); > + return -EFAULT; > + } > + > + rc = roc_plt_init(); > + if (rc) { > + plt_err("Failed to initialize platform model, rc=%d", rc); > + return rc; > + } > + > + /* Common probe */ > + rc = cnxk_nix_probe(pci_drv, pci_dev); > + if (rc) > + return rc; > + > + /* Find eth dev allocated */ > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); > + if (!eth_dev) > + return -ENOENT; > + > + dev = cnxk_eth_pmd_priv(eth_dev); > + /* Update capabilities already set for TSO. > + * TSO not supported for earlier chip revisions > + */ > + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) > + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | > + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | > + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | > + DEV_TX_OFFLOAD_GRE_TNL_TSO); > + > + /* 50G and 100G to be supported for board version C0 > + * and above of CN9K. > + */ > + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) { > + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; > + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; > + } > + > + dev->hwcap = 0; > + > + /* Update HW erratas */ > + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) > + dev->cq_min_4k = 1; > + return 0; > +} > + > +static const struct rte_pci_id cn9k_pci_nix_map[] = { > + { > + .vendor_id = 0, > + }, > +}; > + > +static struct rte_pci_driver cn9k_pci_nix = { > + .id_table = cn9k_pci_nix_map, > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | > + RTE_PCI_DRV_INTR_LSC, > + .probe = cn9k_nix_probe, > + .remove = cn9k_nix_remove, > +}; > + > +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); > +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); > +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); > diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h > new file mode 100644 > index 0000000..15d9397 > --- /dev/null > +++ b/drivers/net/cnxk/cn9k_ethdev.h > @@ -0,0 +1,9 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#ifndef __CN9K_ETHDEV_H__ > +#define __CN9K_ETHDEV_H__ > + > +#include <cnxk_ethdev.h> > + > +#endif /* __CN9K_ETHDEV_H__ */ > diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c > index 6717410..b836fc2 100644 > --- a/drivers/net/cnxk/cnxk_ethdev.c > +++ b/drivers/net/cnxk/cnxk_ethdev.c > @@ -3,6 +3,40 @@ > */ > #include <cnxk_ethdev.h> > > +static inline uint64_t > +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) > +{ > + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; > + > + if (roc_nix_is_vf_or_sdp(&dev->nix)) > + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; > + > + return capa; > +} > + > +static inline uint64_t > +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) > +{ > + RTE_SET_USED(dev); > + return CNXK_NIX_TX_OFFLOAD_CAPA; > +} > + > +static inline uint32_t > +nix_get_speed_capa(struct cnxk_eth_dev *dev) > +{ > + uint32_t speed_capa; > + > + /* Auto negotiation disabled */ > + speed_capa = ETH_LINK_SPEED_FIXED; > + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { > + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | > + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | > + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; > + } > + > + return speed_capa; > +} > + > /* CNXK platform independent eth dev ops */ > struct eth_dev_ops cnxk_eth_dev_ops; > > @@ -77,6 +111,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) > } > } > > + /* Union of all capabilities supported by CNXK. > + * Platform specific capabilities will be > + * updated later. > + */ > + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); > + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); > + dev->speed_capa = nix_get_speed_capa(dev); > + > /* Initialize roc npc */ > plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 > " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, > diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h > index 0460d1e..ba2bfcd 100644 > --- a/drivers/net/cnxk/cnxk_ethdev.h > +++ b/drivers/net/cnxk/cnxk_ethdev.h > @@ -14,6 +14,22 @@ > > #define CNXK_ETH_DEV_PMD_VERSION "1.0" > > +#define CNXK_NIX_TX_OFFLOAD_CAPA \ > + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ > + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ > + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ > + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ > + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ > + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ > + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ > + DEV_TX_OFFLOAD_IPV4_CKSUM) > + > +#define CNXK_NIX_RX_OFFLOAD_CAPA \ > + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ > + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ > + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ > + DEV_RX_OFFLOAD_RSS_HASH) > + > struct cnxk_eth_dev { > /* ROC NIX */ > struct roc_nix nix; > @@ -28,6 +44,9 @@ struct cnxk_eth_dev { > > /* HW capabilities / Limitations */ > union { > + struct { > + uint64_t cq_min_4k : 1; > + }; > uint64_t hwcap; > }; > > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build > index 77b2f18..57dea5e 100644 > --- a/drivers/net/cnxk/meson.build > +++ b/drivers/net/cnxk/meson.build > @@ -10,6 +10,11 @@ endif > > sources = files('cnxk_ethdev.c') > > +# CN9K > +sources += files('cn9k_ethdev.c') > +# CN10K > +sources += files('cn10k_ethdev.c') > + > deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] > deps += ['common_cnxk', 'mempool_cnxk'] > > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (9 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-14 4:22 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram ` (51 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add various devargs parsing command line arguments parsing functions supported by CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 7 ++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ca21842..611ffb4 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -27,3 +27,97 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. + +Runtime Config Options +---------------------- + +- ``Rx&Tx scalar mode enable`` (default ``0``) + + Ethdev supports both scalar and vector mode, it may be selected at runtime + using ``scalar_enable`` ``devargs`` parameter. + +- ``RSS reta size`` (default ``64``) + + RSS redirection table size may be configured during runtime using ``reta_size`` + ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,reta_size=256 + + With the above configuration, reta table of size 256 is populated. + +- ``Flow priority levels`` (default ``3``) + + RTE Flow priority levels can be configured during runtime using + ``flow_max_priority`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_max_priority=10 + + With the above configuration, priority level was set to 10 (0-9). Max + priority level supported is 32. + +- ``Reserve Flow entries`` (default ``8``) + + RTE flow entries can be pre allocated and the size of pre allocation can be + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_prealloc_size=4 + + With the above configuration, pre alloc size was set to 4. Max pre alloc + size supported is 32. + +- ``Max SQB buffer count`` (default ``512``) + + Send queue descriptor buffer count may be limited during runtime using + ``max_sqb_count`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,max_sqb_count=64 + + With the above configuration, each send queue's decscriptor buffer count is + limited to a maximum of 64 buffers. + +- ``Switch header enable`` (default ``none``) + + A port can be configured to a specific switch header type by using + ``switch_header`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,switch_header="higig2" + + With the above configuration, higig2 will be enabled on that port and the + traffic on this port should be higig2 traffic only. Supported switch header + types are "higig2", "dsa", "chlen90b" and "chlen24b". + +- ``RSS tag as XOR`` (default ``0``) + + The HW gives two options to configure the RSS adder i.e + + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` + + * ``rss_adder<7:0> = flow_tag<7:0>`` + + Latter one aligns with standard NIC behavior vs former one is a legacy + RSS adder scheme used in OCTEON TX2 products. + + By default, the driver runs in the latter mode. + Setting this flag to 1 to select the legacy mode. + + For example to select the legacy mode(RSS tag adder as XOR):: + + -a 0002:02:00.0,tag_as_xor=1 + + + +.. note:: + + Above devarg parameters are configurable per device, user needs to pass the + parameters to all the PCIe devices if application requires to configure on + all the ethdev ports. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b836fc2..3a2309e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -58,6 +58,13 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) rte_eth_copy_pci_info(eth_dev, pci_dev); eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; + /* Parse devargs string */ + rc = cnxk_ethdev_parse_devargs(eth_dev->device->devargs, dev); + if (rc) { + plt_err("Failed to parse devargs rc=%d", rc); + goto error; + } + /* Initialize base roc nix */ nix->pci_dev = pci_dev; rc = roc_nix_dev_init(nix); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ba2bfcd..97e3a15 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -9,11 +9,15 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> +#include <rte_kvargs.h> #include "roc_api.h" #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Max supported SQB count */ +#define CNXK_NIX_TX_MAX_SQB 512 + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -38,6 +42,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + bool scalar_ena; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -73,4 +78,8 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +/* Devargs */ +int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, + struct cnxk_eth_dev *dev); + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c new file mode 100644 index 0000000..4af2803 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <inttypes.h> +#include <math.h> + +#include "cnxk_ethdev.h" + +static int +parse_flow_max_priority(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the max priority to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flow_prealloc_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the prealloc size to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_reta_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val <= ETH_RSS_RETA_SIZE_64) + val = ROC_NIX_RSS_RETA_SZ_64; + else if (val > ETH_RSS_RETA_SIZE_64 && val <= ETH_RSS_RETA_SIZE_128) + val = ROC_NIX_RSS_RETA_SZ_128; + else if (val > ETH_RSS_RETA_SIZE_128 && val <= ETH_RSS_RETA_SIZE_256) + val = ROC_NIX_RSS_RETA_SZ_256; + else + val = ROC_NIX_RSS_RETA_SZ_64; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flag(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + *(uint16_t *)extra_args = atoi(value); + + return 0; +} + +static int +parse_sqb_count(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_switch_header_type(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + if (strcmp(value, "higig2") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_HIGIG; + + if (strcmp(value, "dsa") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_EDSA; + + if (strcmp(value, "chlen90b") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_LEN_90B; + return 0; +} + +#define CNXK_RSS_RETA_SIZE "reta_size" +#define CNXK_SCL_ENABLE "scalar_enable" +#define CNXK_MAX_SQB_COUNT "max_sqb_count" +#define CNXK_FLOW_PREALLOC_SIZE "flow_prealloc_size" +#define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" +#define CNXK_SWITCH_HEADER_TYPE "switch_header" +#define CNXK_RSS_TAG_AS_XOR "tag_as_xor" + +int +cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) +{ + uint16_t reta_sz = ROC_NIX_RSS_RETA_SZ_64; + uint16_t sqb_count = CNXK_NIX_TX_MAX_SQB; + uint16_t flow_prealloc_size = 8; + uint16_t switch_header_type = 0; + uint16_t flow_max_priority = 3; + uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto null_devargs; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_RSS_RETA_SIZE, &parse_reta_size, + &reta_sz); + rte_kvargs_process(kvlist, CNXK_SCL_ENABLE, &parse_flag, + &scalar_enable); + rte_kvargs_process(kvlist, CNXK_MAX_SQB_COUNT, &parse_sqb_count, + &sqb_count); + rte_kvargs_process(kvlist, CNXK_FLOW_PREALLOC_SIZE, + &parse_flow_prealloc_size, &flow_prealloc_size); + rte_kvargs_process(kvlist, CNXK_FLOW_MAX_PRIORITY, + &parse_flow_max_priority, &flow_max_priority); + rte_kvargs_process(kvlist, CNXK_SWITCH_HEADER_TYPE, + &parse_switch_header_type, &switch_header_type); + rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, + &rss_tag_as_xor); + rte_kvargs_free(kvlist); + +null_devargs: + dev->scalar_ena = !!scalar_enable; + dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; + dev->nix.max_sqb_count = sqb_count; + dev->nix.reta_sz = reta_sz; + return 0; + +exit: + return -EINVAL; +} + +RTE_PMD_REGISTER_PARAM_STRING(net_cnxk, + CNXK_RSS_RETA_SIZE "=<64|128|256>" + CNXK_SCL_ENABLE "=1" + CNXK_MAX_SQB_COUNT "=<8-512>" + CNXK_FLOW_PREALLOC_SIZE "=<1-32>" + CNXK_FLOW_MAX_PRIORITY "=<1-32>" + CNXK_SWITCH_HEADER_TYPE "=<higig2|dsa|chlen90b>" + CNXK_RSS_TAG_AS_XOR "=1"); diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 57dea5e..00c4722 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -8,7 +8,8 @@ if not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_ethdev.c') +sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_devargs.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram @ 2021-06-14 4:22 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-14 4:22 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:34 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > Add various devargs parsing command line arguments > parsing functions supported by CN9K and CN10K. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > --- > doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.c | 7 ++ > drivers/net/cnxk/cnxk_ethdev.h | 9 ++ > drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ > drivers/net/cnxk/meson.build | 3 +- > 5 files changed, 278 insertions(+), 1 deletion(-) > create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c > > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst > index ca21842..611ffb4 100644 > --- a/doc/guides/nics/cnxk.rst > +++ b/doc/guides/nics/cnxk.rst > @@ -27,3 +27,97 @@ Driver compilation and testing > > Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` > for details. > + > +Runtime Config Options > +---------------------- > + > +- ``Rx&Tx scalar mode enable`` (default ``0``) > + > + Ethdev supports both scalar and vector mode, it may be selected at runtime Ethdev->PMD > + using ``scalar_enable`` ``devargs`` parameter. > + > +- ``RSS reta size`` (default ``64``) > + > + RSS redirection table size may be configured during runtime using ``reta_size`` > + ``devargs`` parameter. > + > + For example:: > + > + -a 0002:02:00.0,reta_size=256 > + > + With the above configuration, reta table of size 256 is populated. > + > +- ``Flow priority levels`` (default ``3``) > + > + RTE Flow priority levels can be configured during runtime using > + ``flow_max_priority`` ``devargs`` parameter. > + > + For example:: > + > + -a 0002:02:00.0,flow_max_priority=10 > + > + With the above configuration, priority level was set to 10 (0-9). Max > + priority level supported is 32. > + > +- ``Reserve Flow entries`` (default ``8``) > + > + RTE flow entries can be pre allocated and the size of pre allocation can be > + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. > + > + For example:: > + > + -a 0002:02:00.0,flow_prealloc_size=4 > + > + With the above configuration, pre alloc size was set to 4. Max pre alloc > + size supported is 32. > + > +- ``Max SQB buffer count`` (default ``512``) > + > + Send queue descriptor buffer count may be limited during runtime using > + ``max_sqb_count`` ``devargs`` parameter. > + > + For example:: > + > + -a 0002:02:00.0,max_sqb_count=64 > + > + With the above configuration, each send queue's decscriptor buffer count is Typo > + limited to a maximum of 64 buffers. > + > +- ``Switch header enable`` (default ``none``) > + > + A port can be configured to a specific switch header type by using > + ``switch_header`` ``devargs`` parameter. > + > + For example:: > + > + -a 0002:02:00.0,switch_header="higig2" > + > + With the above configuration, higig2 will be enabled on that port and the > + traffic on this port should be higig2 traffic only. Supported switch header > + types are "higig2", "dsa", "chlen90b" and "chlen24b". > + > +- ``RSS tag as XOR`` (default ``0``) > + > + The HW gives two options to configure the RSS adder i.e > + > + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` > + > + * ``rss_adder<7:0> = flow_tag<7:0>`` > + ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 12/62] net/cnxk: add common dev infos get support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (10 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram ` (50 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add support to retrieve dev infos get for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 3 ++ doc/guides/nics/features/cnxk.ini | 4 ++ doc/guides/nics/features/cnxk_vec.ini | 4 ++ doc/guides/nics/features/cnxk_vf.ini | 3 ++ drivers/net/cnxk/cnxk_ethdev.c | 4 +- drivers/net/cnxk/cnxk_ethdev.h | 33 ++++++++++++++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 71 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 8 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 611ffb4..dfe2e7a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,9 @@ Features Features of the CNXK Ethdev PMD are: +- SR-IOV VF +- Lock-free Tx queue + Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c23464..b426340 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index de78516..292ac1e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 9c96351..bc2eb8a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3a2309e..1567007 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -38,7 +38,9 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) } /* CNXK platform independent eth dev ops */ -struct eth_dev_ops cnxk_eth_dev_ops; +struct eth_dev_ops cnxk_eth_dev_ops = { + .dev_infos_get = cnxk_nix_info_get, +}; static int cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 97e3a15..8d9a7e0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,9 +15,40 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* VLAN tag inserted by NIX_TX_VTAG_ACTION. + * In Tx space is always reserved for this in FRS. + */ +#define CNXK_NIX_MAX_VTAG_INS 2 +#define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) + +/* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) + +#define CNXK_NIX_RX_MIN_DESC 16 +#define CNXK_NIX_RX_MIN_DESC_ALIGN 16 +#define CNXK_NIX_RX_NB_SEG_MAX 6 +#define CNXK_NIX_RX_DEFAULT_RING_SZ 4096 /* Max supported SQB count */ #define CNXK_NIX_TX_MAX_SQB 512 +/* If PTP is enabled additional SEND MEM DESC is required which + * takes 2 words, hence max 7 iova address are possible + */ +#if defined(RTE_LIBRTE_IEEE1588) +#define CNXK_NIX_TX_NB_SEG_MAX 7 +#else +#define CNXK_NIX_TX_NB_SEG_MAX 9 +#endif + +#define CNXK_NIX_RSS_L3_L4_SRC_DST \ + (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ + ETH_RSS_L4_DST_ONLY) + +#define CNXK_NIX_RSS_OFFLOAD \ + (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | \ + ETH_RSS_SCTP | ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \ + CNXK_NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK | ETH_RSS_C_VLAN) + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -77,6 +108,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_info *dev_info); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c new file mode 100644 index 0000000..4a45956 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <cnxk_ethdev.h> + +int +cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) +{ + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int max_rx_pktlen; + + max_rx_pktlen = (roc_nix_max_pkt_len(&dev->nix) + RTE_ETHER_CRC_LEN - + CNXK_NIX_MAX_VTAG_ACT_SIZE); + + devinfo->min_rx_bufsize = NIX_MIN_HW_FRS + RTE_ETHER_CRC_LEN; + devinfo->max_rx_pktlen = max_rx_pktlen; + devinfo->max_rx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_mac_addrs = dev->max_mac_entries; + devinfo->max_vfs = pci_dev->max_vfs; + devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; + + devinfo->rx_offload_capa = dev->rx_offload_capa; + devinfo->tx_offload_capa = dev->tx_offload_capa; + devinfo->rx_queue_offload_capa = 0; + devinfo->tx_queue_offload_capa = 0; + + devinfo->reta_size = dev->nix.reta_sz; + devinfo->hash_key_size = ROC_NIX_RSS_KEY_LEN; + devinfo->flow_type_rss_offloads = CNXK_NIX_RSS_OFFLOAD; + + devinfo->default_rxconf = (struct rte_eth_rxconf){ + .rx_drop_en = 0, + .offloads = 0, + }; + + devinfo->default_txconf = (struct rte_eth_txconf){ + .offloads = 0, + }; + + devinfo->default_rxportconf = (struct rte_eth_dev_portconf){ + .ring_size = CNXK_NIX_RX_DEFAULT_RING_SZ, + }; + + devinfo->rx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = CNXK_NIX_RX_MIN_DESC, + .nb_align = CNXK_NIX_RX_MIN_DESC_ALIGN, + .nb_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + }; + devinfo->rx_desc_lim.nb_max = + RTE_ALIGN_MUL_FLOOR(devinfo->rx_desc_lim.nb_max, + CNXK_NIX_RX_MIN_DESC_ALIGN); + + devinfo->tx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = 1, + .nb_align = 1, + .nb_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + }; + + devinfo->speed_capa = dev->speed_capa; + devinfo->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 00c4722..45ccbe3 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -9,6 +9,7 @@ if not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (11 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-15 12:29 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support Nithin Dabilpuram ` (49 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add device configuration op for CN9K and CN10K. Most of the device configuration is common between two platforms except for some supported offloads. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 2 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cn10k_ethdev.c | 34 +++ drivers/net/cnxk/cn9k_ethdev.c | 45 +++ drivers/net/cnxk/cnxk_ethdev.c | 539 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 70 +++++ 8 files changed, 696 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index dfe2e7a..73eb62a 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -18,6 +18,8 @@ Features of the CNXK Ethdev PMD are: - SR-IOV VF - Lock-free Tx queue +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b426340..96dba2a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 292ac1e..616991c 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index bc2eb8a..a0bd2f1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index ff8ce31..d971bbd 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,38 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; +} + +static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -26,6 +58,8 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 701dc12..7f3e910 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_conf *conf = ð_dev->data->dev_conf; + struct rte_eth_txmode *txmode = &conf->txmode; + int rc; + + /* Platform specific checks */ + if ((roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) && + (txmode->offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) && + ((txmode->offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) || + (txmode->offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM))) { + plt_err("Outer IP and SCTP checksum unsupported"); + return -EINVAL; + } + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; +} + +static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -27,6 +70,8 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1567007..7824f3b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,538 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint32_t +cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level) +{ + uint32_t flow_key_type[RSS_MAX_LEVELS][6] = { + {FLOW_KEY_TYPE_IPV4, FLOW_KEY_TYPE_IPV6, FLOW_KEY_TYPE_TCP, + FLOW_KEY_TYPE_UDP, FLOW_KEY_TYPE_SCTP, FLOW_KEY_TYPE_ETH_DMAC}, + {FLOW_KEY_TYPE_INNR_IPV4, FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_INNR_TCP, FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_INNR_SCTP, FLOW_KEY_TYPE_INNR_ETH_DMAC}, + {FLOW_KEY_TYPE_IPV4 | FLOW_KEY_TYPE_INNR_IPV4, + FLOW_KEY_TYPE_IPV6 | FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_TCP | FLOW_KEY_TYPE_INNR_TCP, + FLOW_KEY_TYPE_UDP | FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_SCTP | FLOW_KEY_TYPE_INNR_SCTP, + FLOW_KEY_TYPE_ETH_DMAC | FLOW_KEY_TYPE_INNR_ETH_DMAC} + }; + uint32_t flowkey_cfg = 0; + + dev->ethdev_rss_hf = ethdev_rss; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + + if (ethdev_rss & ETH_RSS_C_VLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VLAN; + + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC; + + if (ethdev_rss & ETH_RSS_L3_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST; + + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC; + + if (ethdev_rss & ETH_RSS_L4_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST; + + if (ethdev_rss & RSS_IPV4_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX]; + + if (ethdev_rss & RSS_IPV6_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV6_INDEX]; + + if (ethdev_rss & ETH_RSS_TCP) + flowkey_cfg |= flow_key_type[rss_level][RSS_TCP_INDEX]; + + if (ethdev_rss & ETH_RSS_UDP) + flowkey_cfg |= flow_key_type[rss_level][RSS_UDP_INDEX]; + + if (ethdev_rss & ETH_RSS_SCTP) + flowkey_cfg |= flow_key_type[rss_level][RSS_SCTP_INDEX]; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= flow_key_type[rss_level][RSS_DMAC_INDEX]; + + if (ethdev_rss & RSS_IPV6_EX_ENABLE) + flowkey_cfg |= FLOW_KEY_TYPE_IPV6_EXT; + + if (ethdev_rss & ETH_RSS_PORT) + flowkey_cfg |= FLOW_KEY_TYPE_PORT; + + if (ethdev_rss & ETH_RSS_NVGRE) + flowkey_cfg |= FLOW_KEY_TYPE_NVGRE; + + if (ethdev_rss & ETH_RSS_VXLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VXLAN; + + if (ethdev_rss & ETH_RSS_GENEVE) + flowkey_cfg |= FLOW_KEY_TYPE_GENEVE; + + if (ethdev_rss & ETH_RSS_GTPU) + flowkey_cfg |= FLOW_KEY_TYPE_GTPU; + + return flowkey_cfg; +} + +static void +nix_free_queue_mem(struct cnxk_eth_dev *dev) +{ + plt_free(dev->rqs); + plt_free(dev->cqs); + plt_free(dev->sqs); + dev->rqs = NULL; + dev->cqs = NULL; + dev->sqs = NULL; +} + +static int +nix_rss_default_setup(struct cnxk_eth_dev *dev) +{ + struct rte_eth_dev *eth_dev = dev->eth_dev; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + uint64_t rss_hf; + + rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_hash_level = ETH_RSS_LEVEL(rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + + flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level); + return roc_nix_rss_default_setup(&dev->nix, flowkey_cfg); +} + +static int +nix_store_queue_cfg_and_then_release(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = NULL; + struct cnxk_eth_qconf *rx_qconf = NULL; + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_txq_sp *txq_sp; + int i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + tx_qconf = malloc(nb_txq * sizeof(*tx_qconf)); + if (tx_qconf == NULL) { + plt_err("Failed to allocate memory for tx_qconf"); + goto fail; + } + + rx_qconf = malloc(nb_rxq * sizeof(*rx_qconf)); + if (rx_qconf == NULL) { + plt_err("Failed to allocate memory for rx_qconf"); + goto fail; + } + + txq = eth_dev->data->tx_queues; + for (i = 0; i < nb_txq; i++) { + if (txq[i] == NULL) { + tx_qconf[i].valid = false; + plt_info("txq[%d] is already released", i); + continue; + } + txq_sp = ((struct cnxk_eth_txq_sp *)txq[i]) - 1; + memcpy(&tx_qconf[i], &txq_sp->qconf, sizeof(*tx_qconf)); + tx_qconf[i].valid = true; + dev_ops->tx_queue_release(txq[i]); + eth_dev->data->tx_queues[i] = NULL; + } + + rxq = eth_dev->data->rx_queues; + for (i = 0; i < nb_rxq; i++) { + if (rxq[i] == NULL) { + rx_qconf[i].valid = false; + plt_info("rxq[%d] is already released", i); + continue; + } + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq[i]) - 1; + memcpy(&rx_qconf[i], &rxq_sp->qconf, sizeof(*rx_qconf)); + rx_qconf[i].valid = true; + dev_ops->rx_queue_release(rxq[i]); + eth_dev->data->rx_queues[i] = NULL; + } + + dev->tx_qconf = tx_qconf; + dev->rx_qconf = rx_qconf; + return 0; + +fail: + free(tx_qconf); + free(rx_qconf); + return -ENOMEM; +} + +static int +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; + int rc, i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + rc = -ENOMEM; + /* Setup tx & rx queues with previous configuration so + * that the queues can be functional in cases like ports + * are started without re configuring queues. + * + * Usual re config sequence is like below: + * port_configure() { + * if(reconfigure) { + * queue_release() + * queue_setup() + * } + * queue_configure() { + * queue_release() + * queue_setup() + * } + * } + * port_start() + * + * In some application's control path, queue_configure() would + * NOT be invoked for TXQs/RXQs in port_configure(). + * In such cases, queues can be functional after start as the + * queues are already setup in port_configure(). + */ + for (i = 0; i < nb_txq; i++) { + if (!tx_qconf[i].valid) + continue; + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, + &tx_qconf[i].conf.tx); + if (rc) { + plt_err("Failed to setup tx queue rc=%d", rc); + txq = eth_dev->data->tx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->tx_queue_release(txq[i]); + goto fail; + } + } + + free(tx_qconf); + tx_qconf = NULL; + + for (i = 0; i < nb_rxq; i++) { + if (!rx_qconf[i].valid) + continue; + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, + &rx_qconf[i].conf.rx, + rx_qconf[i].mp); + if (rc) { + plt_err("Failed to setup rx queue rc=%d", rc); + rxq = eth_dev->data->rx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->rx_queue_release(rxq[i]); + goto tx_queue_release; + } + } + + free(rx_qconf); + rx_qconf = NULL; + + return 0; + +tx_queue_release: + txq = eth_dev->data->tx_queues; + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_release(txq[i]); +fail: + if (tx_qconf) + free(tx_qconf); + if (rx_qconf) + free(rx_qconf); + + return rc; +} + +static uint16_t +nix_eth_nop_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + RTE_SET_USED(queue); + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + return 0; +} + +static void +nix_set_nop_rxtx_function(struct rte_eth_dev *eth_dev) +{ + /* These dummy functions are required for supporting + * some applications which reconfigure queues without + * stopping tx burst and rx burst threads(eg kni app) + * When the queues context is saved, txq/rxqs are released + * which caused app crash since rx/tx burst is still + * on different lcores + */ + eth_dev->tx_pkt_burst = nix_eth_nop_burst; + eth_dev->rx_pkt_burst = nix_eth_nop_burst; + rte_mb(); +} + +static int +nix_lso_fmt_setup(struct cnxk_eth_dev *dev) +{ + struct roc_nix *nix = &dev->nix; + + /* Nothing much to do if offload is not enabled */ + if (!(dev->tx_offloads & + (DEV_TX_OFFLOAD_TCP_TSO | DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO))) + return 0; + + /* Setup LSO formats in AF. Its a no-op if other ethdev has + * already set it up + */ + return roc_nix_lso_fmt_setup(nix); +} + +int +cnxk_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + struct rte_eth_txmode *txmode = &conf->txmode; + char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix *nix = &dev->nix; + struct rte_ether_addr *ea; + uint8_t nb_rxq, nb_txq; + uint64_t rx_cfg; + void *qs; + int rc; + + rc = -EINVAL; + + /* Sanity checks */ + if (rte_eal_has_hugepages() == 0) { + plt_err("Huge page is not configured"); + goto fail_configure; + } + + if (conf->dcb_capability_en == 1) { + plt_err("dcb enable is not supported"); + goto fail_configure; + } + + if (conf->fdir_conf.mode != RTE_FDIR_MODE_NONE) { + plt_err("Flow director is not supported"); + goto fail_configure; + } + + if (rxmode->mq_mode != ETH_MQ_RX_NONE && + rxmode->mq_mode != ETH_MQ_RX_RSS) { + plt_err("Unsupported mq rx mode %d", rxmode->mq_mode); + goto fail_configure; + } + + if (txmode->mq_mode != ETH_MQ_TX_NONE) { + plt_err("Unsupported mq tx mode %d", txmode->mq_mode); + goto fail_configure; + } + + /* Free the resources allocated from the previous configure */ + if (dev->configured == 1) { + /* Unregister queue irq's */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister CQ irqs if present */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Set no-op functions */ + nix_set_nop_rxtx_function(eth_dev); + /* Store queue config for later */ + rc = nix_store_queue_cfg_and_then_release(eth_dev); + if (rc) + goto fail_configure; + roc_nix_tm_fini(nix); + roc_nix_lf_free(nix); + } + + dev->rx_offloads = rxmode->offloads; + dev->tx_offloads = txmode->offloads; + + /* Prepare rx cfg */ + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) { + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_OL4; + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_IL4; + } + rx_cfg |= (ROC_NIX_LF_RX_CFG_DROP_RE | ROC_NIX_LF_RX_CFG_L2_LEN_ERR | + ROC_NIX_LF_RX_CFG_LEN_IL4 | ROC_NIX_LF_RX_CFG_LEN_IL3 | + ROC_NIX_LF_RX_CFG_LEN_OL4 | ROC_NIX_LF_RX_CFG_LEN_OL3); + + nb_rxq = RTE_MAX(data->nb_rx_queues, 1); + nb_txq = RTE_MAX(data->nb_tx_queues, 1); + + /* Alloc a nix lf */ + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("Failed to init nix_lf rc=%d", rc); + goto fail_configure; + } + + nb_rxq = data->nb_rx_queues; + nb_txq = data->nb_tx_queues; + rc = -ENOMEM; + if (nb_rxq) { + /* Allocate memory for roc rq's and cq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_rq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc rqs"); + goto free_nix_lf; + } + dev->rqs = qs; + + qs = plt_zmalloc(sizeof(struct roc_nix_cq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc cqs"); + goto free_nix_lf; + } + dev->cqs = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_sq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc sqs"); + goto free_nix_lf; + } + dev->sqs = qs; + } + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + if (nix->rx_ptp_ena) { + plt_err("Both PTP and switch header enabled"); + goto free_nix_lf; + } + + /* Setup LSO if needed */ + rc = nix_lso_fmt_setup(dev); + if (rc) { + plt_err("Failed to setup nix lso format fields, rc=%d", rc); + goto free_nix_lf; + } + + /* Configure RSS */ + rc = nix_rss_default_setup(dev); + if (rc) { + plt_err("Failed to configure rss rc=%d", rc); + goto free_nix_lf; + } + + /* Init the default TM scheduler hierarchy */ + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("Failed to init traffic manager, rc=%d", rc); + goto free_nix_lf; + } + + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("Failed to enable default tm hierarchy, rc=%d", rc); + goto tm_fini; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Register cq IRQs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) { + if (eth_dev->data->nb_rx_queues > dev->nix.cints) { + plt_err("Rx interrupt cannot be enabled, rxq > %d", + dev->nix.cints); + goto q_irq_fini; + } + /* Rx interrupt feature cannot work with vector mode because, + * vector mode does not process packets unless min 4 pkts are + * received, while cq interrupts are generated even for 1 pkt + * in the CQ. + */ + dev->scalar_ena = true; + + rc = roc_nix_register_cq_irqs(nix); + if (rc) { + plt_err("Failed to register CQ interrupts rc=%d", rc); + goto q_irq_fini; + } + } + + /* Configure loop back mode */ + rc = roc_nix_mac_loopback_enable(nix, + eth_dev->data->dev_conf.lpbk_mode); + if (rc) { + plt_err("Failed to configure cgx loop back mode rc=%d", rc); + goto cq_fini; + } + + /* + * Restore queue config when reconfigure followed by + * reconfigure and no queue configure invoked from application case. + */ + if (dev->configured == 1) { + rc = nix_restore_queue_cfg(eth_dev); + if (rc) + goto cq_fini; + } + + /* Update the mac address */ + ea = eth_dev->data->mac_addrs; + memcpy(ea, dev->mac_addr, RTE_ETHER_ADDR_LEN); + if (rte_is_zero_ether_addr(ea)) + rte_eth_random_addr((uint8_t *)ea); + + rte_ether_format_addr(ea_fmt, RTE_ETHER_ADDR_FMT_SIZE, ea); + + plt_nix_dbg("Configured port%d mac=%s nb_rxq=%d nb_txq=%d" + " rx_offloads=0x%" PRIx64 " tx_offloads=0x%" PRIx64 "", + eth_dev->data->port_id, ea_fmt, nb_rxq, nb_txq, + dev->rx_offloads, dev->tx_offloads); + + /* All good */ + dev->configured = 1; + dev->nb_rxq = data->nb_rx_queues; + dev->nb_txq = data->nb_tx_queues; + return 0; + +cq_fini: + roc_nix_unregister_cq_irqs(nix); +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_nix_lf: + nix_free_queue_mem(dev); + rc |= roc_nix_lf_free(nix); +fail_configure: + dev->configured = 0; + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, @@ -76,6 +608,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->eth_dev = eth_dev; + dev->configured = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. @@ -157,6 +690,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; + /* Clear the flag since we are closing down */ + dev->configured = 0; + roc_nix_npc_rx_ena_dis(nix, false); /* Free up SQs */ @@ -183,6 +719,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (eth_dev->data->dev_conf.intr_conf.rxq) roc_nix_unregister_cq_irqs(nix); + /* Free ROC RQ's, SQ's and CQ's memory */ + nix_free_queue_mem(dev); + /* Free nix lf resources */ rc = roc_nix_lf_free(nix); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8d9a7e0..55da1da 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -65,10 +65,50 @@ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ DEV_RX_OFFLOAD_RSS_HASH) +#define RSS_IPV4_ENABLE \ + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV4_TCP | ETH_RSS_NONFRAG_IPV4_SCTP) + +#define RSS_IPV6_ENABLE \ + (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_UDP | \ + ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_NONFRAG_IPV6_SCTP) + +#define RSS_IPV6_EX_ENABLE \ + (ETH_RSS_IPV6_EX | ETH_RSS_IPV6_TCP_EX | ETH_RSS_IPV6_UDP_EX) + +#define RSS_MAX_LEVELS 3 + +#define RSS_IPV4_INDEX 0 +#define RSS_IPV6_INDEX 1 +#define RSS_TCP_INDEX 2 +#define RSS_UDP_INDEX 3 +#define RSS_SCTP_INDEX 4 +#define RSS_DMAC_INDEX 5 + +struct cnxk_eth_qconf { + union { + struct rte_eth_txconf tx; + struct rte_eth_rxconf rx; + } conf; + struct rte_mempool *mp; + uint16_t nb_desc; + uint8_t valid; +}; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC RQs, SQs and CQs */ + struct roc_nix_rq *rqs; + struct roc_nix_sq *sqs; + struct roc_nix_cq *cqs; + + /* Configured queue count */ + uint16_t nb_rxq; + uint16_t nb_txq; + uint8_t configured; + /* Max macfilter entries */ uint8_t max_mac_entries; @@ -90,11 +130,36 @@ struct cnxk_eth_dev { uint64_t rx_offload_capa; uint64_t tx_offload_capa; uint32_t speed_capa; + /* Configured Rx and Tx offloads */ + uint64_t rx_offloads; + uint64_t tx_offloads; + /* Platform specific offload flags */ + uint16_t rx_offload_flags; + uint16_t tx_offload_flags; + + /* ETHDEV RSS HF bitmask */ + uint64_t ethdev_rss_hf; + + /* Saved qconf before lf realloc */ + struct cnxk_eth_qconf *tx_qconf; + struct cnxk_eth_qconf *rx_qconf; /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; +struct cnxk_eth_rxq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + +struct cnxk_eth_txq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + static inline struct cnxk_eth_dev * cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) { @@ -110,6 +175,11 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_configure(struct rte_eth_dev *eth_dev); + +/* RSS */ +uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram @ 2021-06-15 12:29 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:29 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:34 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > Add device configuration op for CN9K and CN10K. Most of the > device configuration is common between two platforms except for > some supported offloads. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > +static int > +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; > + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; > + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; > + int rc, i, nb_rxq, nb_txq; > + void **txq, **rxq; > + > + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); > + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); > + > + rc = -ENOMEM; > + /* Setup tx & rx queues with previous configuration so > + * that the queues can be functional in cases like ports > + * are started without re configuring queues. > + * > + * Usual re config sequence is like below: > + * port_configure() { > + * if(reconfigure) { > + * queue_release() > + * queue_setup() > + * } > + * queue_configure() { > + * queue_release() > + * queue_setup() > + * } > + * } > + * port_start() This logic no more required as the KNI application fixed the bug. > + * > + * In some application's control path, queue_configure() would > + * NOT be invoked for TXQs/RXQs in port_configure(). > + * In such cases, queues can be functional after start as the > + * queues are already setup in port_configure(). > + */ > + for (i = 0; i < nb_txq; i++) { > + if (!tx_qconf[i].valid) > + continue; > + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, > + &tx_qconf[i].conf.tx); > + if (rc) { > + plt_err("Failed to setup tx queue rc=%d", rc); > + txq = eth_dev->data->tx_queues; > + for (i -= 1; i >= 0; i--) > + dev_ops->tx_queue_release(txq[i]); > + goto fail; > + } > + } > + > + free(tx_qconf); > + tx_qconf = NULL; > + > + for (i = 0; i < nb_rxq; i++) { > + if (!rx_qconf[i].valid) > + continue; > + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, > + &rx_qconf[i].conf.rx, > + rx_qconf[i].mp); > + if (rc) { > + plt_err("Failed to setup rx queue rc=%d", rc); > + rxq = eth_dev->data->rx_queues; > + for (i -= 1; i >= 0; i--) > + dev_ops->rx_queue_release(rxq[i]); > + goto tx_queue_release; > + } > + } > + > + free(rx_qconf); > + rx_qconf = NULL; > + > + return 0; > + > +tx_queue_release: > + txq = eth_dev->data->tx_queues; > + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) > + dev_ops->tx_queue_release(txq[i]); > +fail: > + if (tx_qconf) > + free(tx_qconf); > + if (rx_qconf) > + free(rx_qconf); > + > + return rc; > +} > + > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (12 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-15 12:31 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram ` (48 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add link status update callback to get current link status. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 7 +++ drivers/net/cnxk/cnxk_ethdev.h | 8 +++ drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_link.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 73eb62a..a982450 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -20,6 +20,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Link state information Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 96dba2a..affbbd9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 616991c..836cc9f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a0bd2f1..29bb24f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 7824f3b..1d7e0a7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -572,6 +572,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, + .link_update = cnxk_nix_link_update, }; static int @@ -607,6 +608,9 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) goto error; } + /* Register up msg callbacks */ + roc_nix_mac_link_cb_register(nix, cnxk_eth_dev_link_status_cb); + dev->eth_dev = eth_dev; dev->configured = 0; @@ -695,6 +699,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable link status events */ + roc_nix_mac_link_event_start_stop(nix, false); + /* Free up SQs */ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 55da1da..6dad8ac 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,6 +15,9 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Used for struct cnxk_eth_dev::flags */ +#define CNXK_LINK_CFG_IN_PROGRESS_F BIT_ULL(0) + /* VLAN tag inserted by NIX_TX_VTAG_ACTION. * In Tx space is always reserved for this in FRS. */ @@ -181,6 +184,11 @@ int cnxk_nix_configure(struct rte_eth_dev *eth_dev); uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +/* Link */ +void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, + struct roc_nix_link_info *link); +int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c new file mode 100644 index 0000000..0223d68 --- /dev/null +++ b/drivers/net/cnxk/cnxk_link.c @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +static inline int +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) +{ + uint16_t wait = 1000; + + do { + rte_rmb(); + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) + break; + wait--; + rte_delay_ms(1); + } while (wait); + + return wait ? 0 : -1; +} + +static void +nix_link_status_print(struct rte_eth_dev *eth_dev, struct rte_eth_link *link) +{ + if (link && link->link_status) + plt_info("Port %d: Link Up - speed %u Mbps - %s", + (int)(eth_dev->data->port_id), + (uint32_t)link->link_speed, + link->link_duplex == ETH_LINK_FULL_DUPLEX + ? "full-duplex" + : "half-duplex"); + else + plt_info("Port %d: Link Down", (int)(eth_dev->data->port_id)); +} + +void +cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_link eth_link; + struct rte_eth_dev *eth_dev; + + if (!link || !nix) + return; + + eth_dev = dev->eth_dev; + if (!eth_dev || !eth_dev->data->dev_conf.intr_conf.lsc) + return; + + if (nix_wait_for_link_cfg(dev)) { + plt_err("Timeout waiting for link_cfg to complete"); + return; + } + + eth_link.link_status = link->status; + eth_link.link_speed = link->speed; + eth_link.link_autoneg = ETH_LINK_AUTONEG; + eth_link.link_duplex = link->full_duplex; + + /* Print link info */ + nix_link_status_print(eth_dev, ð_link); + + /* Update link info */ + rte_eth_linkstatus_set(eth_dev, ð_link); + + /* Set the flag and execute application callbacks */ + rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); +} + +int +cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_link_info info; + struct rte_eth_link link; + int rc; + + RTE_SET_USED(wait_to_complete); + memset(&link, 0, sizeof(struct rte_eth_link)); + + if (roc_nix_is_sdp(&dev->nix)) + return 0; + + if (roc_nix_is_lbk(&dev->nix)) { + link.link_status = ETH_LINK_UP; + link.link_speed = ETH_SPEED_NUM_100G; + link.link_autoneg = ETH_LINK_FIXED; + link.link_duplex = ETH_LINK_FULL_DUPLEX; + } else { + rc = roc_nix_mac_link_info_get(&dev->nix, &info); + if (rc) + return rc; + link.link_status = info.status; + link.link_speed = info.speed; + link.link_autoneg = ETH_LINK_AUTONEG; + if (info.full_duplex) + link.link_duplex = info.full_duplex; + } + + return rte_eth_linkstatus_set(eth_dev, &link); +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 45ccbe3..c0ec91a 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,7 +10,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', - 'cnxk_ethdev_devargs.c') + 'cnxk_ethdev_devargs.c', + 'cnxk_link.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support Nithin Dabilpuram @ 2021-06-15 12:31 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:31 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:35 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > Add link status update callback to get current > link status. > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > --- > doc/guides/nics/cnxk.rst | 1 + > doc/guides/nics/features/cnxk.ini | 2 + > doc/guides/nics/features/cnxk_vec.ini | 2 + > doc/guides/nics/features/cnxk_vf.ini | 2 + > drivers/net/cnxk/cnxk_ethdev.c | 7 +++ > drivers/net/cnxk/cnxk_ethdev.h | 8 +++ > drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/meson.build | 3 +- > 8 files changed, 126 insertions(+), 1 deletion(-) > create mode 100644 drivers/net/cnxk/cnxk_link.c > > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > + > +#include "cnxk_ethdev.h" > + > +static inline int > +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) > +{ > + uint16_t wait = 1000; > + > + do { > + rte_rmb(); Please use gcc atomics instead. With above change: Reviewed-by: Jerin Jacob <jerinj@marvell.com> > + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) > + break; > + wait--; > + rte_delay_ms(1); > + } while (wait); > + > + return wait ? 0 : -1; > +} > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 15/62] net/cnxk: add Rx queue setup and release 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (13 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 16/62] net/cnxk: add Tx " Nithin Dabilpuram ` (47 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d971bbd..b87c4e5 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 7f3e910..ed9a813 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1d7e0a7..da4af3e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq) - 1; + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -573,6 +744,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6dad8ac..e938c64 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> #include <rte_kvargs.h> +#include <rte_mbuf.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> #include "roc_api.h" @@ -179,6 +182,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 16/62] net/cnxk: add Tx queue setup and release 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (14 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 17/62] net/cnxk: add packet type support Nithin Dabilpuram ` (46 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx queue setup and release for CN9K and CN10K. Release is common while setup is platform dependent due to differences in fast path Tx queue structures. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 71 +++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 12 +++++ drivers/net/cnxk/cn10k_tx.h | 13 +++++ drivers/net/cnxk/cn9k_ethdev.c | 69 ++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 10 ++++ drivers/net/cnxk/cn9k_tx.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev.c | 98 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++ 11 files changed, 292 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx.h diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index a9d2b03..462d7c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6a8ca1f..09e0d3a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index f761638..4a93a35 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -10,6 +10,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index b87c4e5..39b8254 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,76 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + union nix_send_hdr_w0_u send_hdr_w0; + union nix_send_sg_s sg_w0; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + send_hdr_w0.u = 0; + sg_w0.u = 0; + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr_w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + } else { + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr_w0.sizem1 = 1; + } + + send_hdr_w0.sq = qid; + sg_w0.subdc = NIX_SUBDC_SG; + sg_w0.segs = 1; + sg_w0.ld_type = NIX_SENDLDTYPE_LDD; + + txq->send_hdr_w0 = send_hdr_w0.u; + txq->sg_w0 = sg_w0.u; + + rte_wmb(); +} + +static int +cn10k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + /* Store lmt base in tx queue for easy access */ + txq->lmt_base = dev->nix.lmt_base; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -76,6 +146,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 08e11bb..2157b16 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,6 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_txq { + uint64_t send_hdr_w0; + uint64_t sg_w0; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + uintptr_t lmt_base; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t cmd[4]; +} __plt_cache_aligned; + struct cn10k_eth_rxq { uint64_t mbuf_initializer; uintptr_t desc; diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h new file mode 100644 index 0000000..39d4755 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_TX_H__ +#define __CN10K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index ed9a813..72a880f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,74 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr->w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + sg = (union nix_send_sg_s *)&txq->cmd[4]; + } else { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr->w0.sizem1 = 1; + sg = (union nix_send_sg_s *)&txq->cmd[2]; + } + + send_hdr->w0.sq = qid; + sg->subdc = NIX_SUBDC_SG; + sg->segs = 1; + sg->ld_type = NIX_SENDLDTYPE_LDD; + + rte_wmb(); +} + +static int +cn9k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + txq->lmt_addr = sq->lmt_addr; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + return 0; +} static int cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -87,6 +155,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 6384609..9ebf68f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,6 +6,16 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_txq { + uint64_t cmd[8]; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + void *lmt_addr; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; +} __plt_cache_aligned; + struct cn9k_eth_rxq { uint64_t mbuf_initializer; uint64_t data_off; diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h new file mode 100644 index 0000000..bb6379b --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_TX_H__ +#define __CN9K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index da4af3e..41a22b1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -66,6 +66,103 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) return *tmp; } +static inline uint8_t +nix_sq_max_sqe_sz(struct cnxk_eth_dev *dev) +{ + /* + * Maximum three segments can be supported with W8, Choose + * NIX_MAXSQESZ_W16 for multi segment offload. + */ + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + return NIX_MAXSQESZ_W16; + else + return NIX_MAXSQESZ_W8; +} + +int +cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_txq_sp *txq_sp; + struct roc_nix_sq *sq; + size_t txq_sz; + int rc; + + /* Free memory prior to re-allocation if needed. */ + if (eth_dev->data->tx_queues[qid] != NULL) { + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->tx_queue_release(eth_dev->data->tx_queues[qid]); + eth_dev->data->tx_queues[qid] = NULL; + } + + /* Setup ROC SQ */ + sq = &dev->sqs[qid]; + sq->qid = qid; + sq->nb_desc = nb_desc; + sq->max_sqe_sz = nix_sq_max_sqe_sz(dev); + + rc = roc_nix_sq_init(&dev->nix, sq); + if (rc) { + plt_err("Failed to init sq=%d, rc=%d", qid, rc); + return rc; + } + + rc = -ENOMEM; + txq_sz = sizeof(struct cnxk_eth_txq_sp) + fp_tx_q_sz; + txq_sp = plt_zmalloc(txq_sz, PLT_CACHE_LINE_SIZE); + if (!txq_sp) { + plt_err("Failed to alloc tx queue mem"); + rc |= roc_nix_sq_fini(sq); + return rc; + } + + txq_sp->dev = dev; + txq_sp->qid = qid; + txq_sp->qconf.conf.tx = *tx_conf; + txq_sp->qconf.nb_desc = nb_desc; + + plt_nix_dbg("sq=%d fc=%p offload=0x%" PRIx64 " lmt_addr=%p" + " nb_sqb_bufs=%d sqes_per_sqb_log2=%d", + qid, sq->fc, dev->tx_offloads, sq->lmt_addr, + sq->nb_sqb_bufs, sq->sqes_per_sqb_log2); + + /* Store start of fast path area */ + eth_dev->data->tx_queues[qid] = txq_sp + 1; + eth_dev->data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + +static void +cnxk_nix_tx_queue_release(void *txq) +{ + struct cnxk_eth_txq_sp *txq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_sq *sq; + uint16_t qid; + int rc; + + if (!txq) + return; + + txq_sp = ((struct cnxk_eth_txq_sp *)txq) - 1; + dev = txq_sp->dev; + qid = txq_sp->qid; + + plt_nix_dbg("Releasing txq %u", qid); + + /* Cleanup ROC SQ */ + sq = &dev->sqs[qid]; + rc = roc_nix_sq_fini(sq); + if (rc) + plt_err("Failed to cleanup sq, rc=%d", rc); + + /* Finally free */ + plt_free(txq_sp); +} + int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, @@ -744,6 +841,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e938c64..90c8ff6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -182,6 +182,9 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf); int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 17/62] net/cnxk: add packet type support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (15 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 16/62] net/cnxk: add Tx " Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram ` (45 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add support for packet type lookup on Rx to translate HW specific types to RTE_PTYPE_* defines Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 21 +++ drivers/net/cnxk/cn10k_rx.h | 11 ++ drivers/net/cnxk/cn9k_ethdev.c | 21 +++ drivers/net/cnxk/cn9k_rx.h | 12 ++ drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 14 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 12 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cnxk_lookup.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index a982450..4f1b58c 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,7 @@ Features Features of the CNXK Ethdev PMD are: +- Packet type information - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 462d7c4..503582c 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 09e0d3a..9ad225a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4a93a35..8c93ba7 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -13,6 +13,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 39b8254..56b92ae 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rx.h" #include "cn10k_tx.h" +static int +cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, uint16_t qid) @@ -113,6 +130,9 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -148,6 +168,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h new file mode 100644 index 0000000..d3d1661 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_RX_H__ +#define __CN10K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 72a880f..b1cbdc7 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rx.h" #include "cn9k_tx.h" +static int +cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, uint16_t qid) @@ -111,6 +128,9 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -157,6 +177,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h new file mode 100644 index 0000000..95a1e69 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef __CN9K_RX_H__ +#define __CN9K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 41a22b1..a0a601f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -843,6 +843,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; static int @@ -883,6 +884,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->eth_dev = eth_dev; dev->configured = 0; + dev->ptype_disable = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 90c8ff6..6b7261c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,15 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ \ + ((PTYPE_NON_TUNNEL_ARRAY_SZ + PTYPE_TUNNEL_ARRAY_SZ) * sizeof(uint16_t)) +/* Fastpath lookup */ +#define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -119,6 +128,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + uint8_t ptype_disable; bool scalar_ena; /* Pointer back to rte */ @@ -201,6 +211,10 @@ void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +/* Lookup configuration */ +const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); +void *cnxk_nix_fastpath_lookup_mem_get(void); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_lookup.c b/drivers/net/cnxk/cnxk_lookup.c new file mode 100644 index 0000000..0152ad9 --- /dev/null +++ b/drivers/net/cnxk/cnxk_lookup.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <rte_common.h> +#include <rte_memzone.h> + +#include "cnxk_ethdev.h" + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) * sizeof(uint32_t)) + +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ + SA_TBL_SZ) +const uint32_t * +cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) +{ + RTE_SET_USED(eth_dev); + + static const uint32_t ptypes[] = { + RTE_PTYPE_L2_ETHER_QINQ, /* LB */ + RTE_PTYPE_L2_ETHER_VLAN, /* LB */ + RTE_PTYPE_L2_ETHER_TIMESYNC, /* LB */ + RTE_PTYPE_L2_ETHER_ARP, /* LC */ + RTE_PTYPE_L2_ETHER_NSH, /* LC */ + RTE_PTYPE_L2_ETHER_FCOE, /* LC */ + RTE_PTYPE_L2_ETHER_MPLS, /* LC */ + RTE_PTYPE_L3_IPV4, /* LC */ + RTE_PTYPE_L3_IPV4_EXT, /* LC */ + RTE_PTYPE_L3_IPV6, /* LC */ + RTE_PTYPE_L3_IPV6_EXT, /* LC */ + RTE_PTYPE_L4_TCP, /* LD */ + RTE_PTYPE_L4_UDP, /* LD */ + RTE_PTYPE_L4_SCTP, /* LD */ + RTE_PTYPE_L4_ICMP, /* LD */ + RTE_PTYPE_L4_IGMP, /* LD */ + RTE_PTYPE_TUNNEL_GRE, /* LD */ + RTE_PTYPE_TUNNEL_ESP, /* LD */ + RTE_PTYPE_TUNNEL_NVGRE, /* LD */ + RTE_PTYPE_TUNNEL_VXLAN, /* LE */ + RTE_PTYPE_TUNNEL_GENEVE, /* LE */ + RTE_PTYPE_TUNNEL_GTPC, /* LE */ + RTE_PTYPE_TUNNEL_GTPU, /* LE */ + RTE_PTYPE_TUNNEL_VXLAN_GPE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_GRE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_UDP, /* LE */ + RTE_PTYPE_INNER_L2_ETHER, /* LF */ + RTE_PTYPE_INNER_L3_IPV4, /* LG */ + RTE_PTYPE_INNER_L3_IPV6, /* LG */ + RTE_PTYPE_INNER_L4_TCP, /* LH */ + RTE_PTYPE_INNER_L4_UDP, /* LH */ + RTE_PTYPE_INNER_L4_SCTP, /* LH */ + RTE_PTYPE_INNER_L4_ICMP, /* LH */ + RTE_PTYPE_UNKNOWN, + }; + + return ptypes; +} + +/* + * +------------------ +------------------ + + * | | IL4 | IL3| IL2 | TU | L4 | L3 | L2 | + * +-------------------+-------------------+ + * + * +-------------------+------------------ + + * | | LH | LG | LF | LE | LD | LC | LB | + * +-------------------+-------------------+ + * + * ptype [LE - LD - LC - LB] = TU - L4 - L3 - T2 + * ptype_tunnel[LH - LG - LF] = IL4 - IL3 - IL2 - TU + * + */ +static void +nix_create_non_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lb, lc, ld, le; + uint16_t val; + uint32_t idx; + + for (idx = 0; idx < PTYPE_NON_TUNNEL_ARRAY_SZ; idx++) { + lb = idx & 0xF; + lc = (idx & 0xF0) >> 4; + ld = (idx & 0xF00) >> 8; + le = (idx & 0xF000) >> 12; + val = RTE_PTYPE_UNKNOWN; + + switch (lb) { + case NPC_LT_LB_STAG_QINQ: + val |= RTE_PTYPE_L2_ETHER_QINQ; + break; + case NPC_LT_LB_CTAG: + val |= RTE_PTYPE_L2_ETHER_VLAN; + break; + } + + switch (lc) { + case NPC_LT_LC_ARP: + val |= RTE_PTYPE_L2_ETHER_ARP; + break; + case NPC_LT_LC_NSH: + val |= RTE_PTYPE_L2_ETHER_NSH; + break; + case NPC_LT_LC_FCOE: + val |= RTE_PTYPE_L2_ETHER_FCOE; + break; + case NPC_LT_LC_MPLS: + val |= RTE_PTYPE_L2_ETHER_MPLS; + break; + case NPC_LT_LC_IP: + val |= RTE_PTYPE_L3_IPV4; + break; + case NPC_LT_LC_IP_OPT: + val |= RTE_PTYPE_L3_IPV4_EXT; + break; + case NPC_LT_LC_IP6: + val |= RTE_PTYPE_L3_IPV6; + break; + case NPC_LT_LC_IP6_EXT: + val |= RTE_PTYPE_L3_IPV6_EXT; + break; + case NPC_LT_LC_PTP: + val |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + } + + switch (ld) { + case NPC_LT_LD_TCP: + val |= RTE_PTYPE_L4_TCP; + break; + case NPC_LT_LD_UDP: + val |= RTE_PTYPE_L4_UDP; + break; + case NPC_LT_LD_SCTP: + val |= RTE_PTYPE_L4_SCTP; + break; + case NPC_LT_LD_ICMP: + case NPC_LT_LD_ICMP6: + val |= RTE_PTYPE_L4_ICMP; + break; + case NPC_LT_LD_IGMP: + val |= RTE_PTYPE_L4_IGMP; + break; + case NPC_LT_LD_GRE: + val |= RTE_PTYPE_TUNNEL_GRE; + break; + case NPC_LT_LD_NVGRE: + val |= RTE_PTYPE_TUNNEL_NVGRE; + break; + } + + switch (le) { + case NPC_LT_LE_VXLAN: + val |= RTE_PTYPE_TUNNEL_VXLAN; + break; + case NPC_LT_LE_ESP: + val |= RTE_PTYPE_TUNNEL_ESP; + break; + case NPC_LT_LE_VXLANGPE: + val |= RTE_PTYPE_TUNNEL_VXLAN_GPE; + break; + case NPC_LT_LE_GENEVE: + val |= RTE_PTYPE_TUNNEL_GENEVE; + break; + case NPC_LT_LE_GTPC: + val |= RTE_PTYPE_TUNNEL_GTPC; + break; + case NPC_LT_LE_GTPU: + val |= RTE_PTYPE_TUNNEL_GTPU; + break; + case NPC_LT_LE_TU_MPLS_IN_GRE: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + break; + case NPC_LT_LE_TU_MPLS_IN_UDP: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_UDP; + break; + } + ptype[idx] = val; + } +} + +#define TU_SHIFT(x) ((x) >> PTYPE_NON_TUNNEL_WIDTH) +static void +nix_create_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lf, lg, lh; + uint16_t val; + uint32_t idx; + + /* Skip non tunnel ptype array memory */ + ptype = ptype + PTYPE_NON_TUNNEL_ARRAY_SZ; + + for (idx = 0; idx < PTYPE_TUNNEL_ARRAY_SZ; idx++) { + lf = idx & 0xF; + lg = (idx & 0xF0) >> 4; + lh = (idx & 0xF00) >> 8; + val = RTE_PTYPE_UNKNOWN; + + switch (lf) { + case NPC_LT_LF_TU_ETHER: + val |= TU_SHIFT(RTE_PTYPE_INNER_L2_ETHER); + break; + } + switch (lg) { + case NPC_LT_LG_TU_IP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV4); + break; + case NPC_LT_LG_TU_IP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV6); + break; + } + switch (lh) { + case NPC_LT_LH_TU_TCP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_TCP); + break; + case NPC_LT_LH_TU_UDP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_UDP); + break; + case NPC_LT_LH_TU_SCTP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_SCTP); + break; + case NPC_LT_LH_TU_ICMP: + case NPC_LT_LH_TU_ICMP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_ICMP); + break; + } + + ptype[idx] = val; + } +} + +static void +nix_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)((uint8_t *)mem + PTYPE_ARRAY_SZ); + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errlev = idx & 0xf; + errcode = (idx & 0xff0) >> 4; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case NPC_ERRLEV_RE: + /* Mark all errors as BAD checksum errors + * including Outer L2 length mismatch error + */ + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LC: + if (errcode == NPC_EC_OIP4_CSUM || + errcode == NPC_EC_IP_FRAG_OFFSET_1) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_OUTER_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LG: + if (errcode == NPC_EC_IIP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case NPC_ERRLEV_NIX: + if (errcode == NIX_RX_PERRCODE_OL4_CHK || + errcode == NIX_RX_PERRCODE_OL4_LEN || + errcode == NIX_RX_PERRCODE_OL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL4_CHK || + errcode == NIX_RX_PERRCODE_IL4_LEN || + errcode == NIX_RX_PERRCODE_IL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL3_LEN || + errcode == NIX_RX_PERRCODE_OL3_LEN) { + val |= PKT_RX_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + } + ol_flags[idx] = val; + } +} + +void * +cnxk_nix_fastpath_lookup_mem_get(void) +{ + const char name[] = CNXK_NIX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, SOCKET_ID_ANY, + 0, ROC_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the ptype array lookup memory */ + nix_create_non_tunnel_ptype_array(mem); + nix_create_tunnel_ptype_array(mem); + /* Form the rx ol_flags based on errcode */ + nix_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c0ec91a..b6b6989 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -11,7 +11,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', - 'cnxk_link.c') + 'cnxk_link.c', + 'cnxk_lookup.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 18/62] net/cnxk: add queue start and stop support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (16 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 17/62] net/cnxk: add packet type support Nithin Dabilpuram @ 2021-06-07 17:58 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram ` (44 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:58 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx/Tx queue start and stop callbacks for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 16 ++++++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++++++ drivers/net/cnxk/cnxk_ethdev.c | 92 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 1 + 7 files changed, 128 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 503582c..712f8d5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 9ad225a..82f2af0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8c93ba7..61fed11 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 56b92ae..9df30ae 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -137,6 +137,21 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn10k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn10k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -168,6 +183,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index b1cbdc7..22f5527 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -135,6 +135,21 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn9k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn9k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -177,6 +192,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a0a601f..2a0ce7f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -837,12 +837,104 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } +static int +cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc = -EINVAL; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, true); + if (rc) { + plt_err("Failed to enable sq aura fc, txq=%u, rc=%d", qid, rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +int +cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, false); + if (rc) { + plt_err("Failed to disable sqb aura fc, txq=%u, rc=%d", qid, + rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, true); + if (rc) { + plt_err("Failed to enable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, false); + if (rc) { + plt_err("Failed to disable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .tx_queue_start = cnxk_nix_tx_queue_start, + .rx_queue_start = cnxk_nix_rx_queue_start, + .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6b7261c..7e79a8d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -199,6 +199,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 19/62] net/cnxk: add Rx support for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (17 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram ` (43 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Jerin Jacob <jerinj@marvell.com> Add Rx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 3 + drivers/net/cnxk/cn9k_rx.c | 46 ++++++++ drivers/net/cnxk/cn9k_rx.h | 237 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/meson.build | 3 +- 5 files changed, 291 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_rx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 9ebf68f..84dcc2c 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -30,4 +30,7 @@ struct cn9k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c new file mode 100644 index 0000000..a4297f9 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 95a1e69..92f3c7c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -7,6 +7,243 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->cn9k.pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = + nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn9k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 7e79a8d..58c6710 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,9 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +/* Default mark value used when none is provided. */ +#define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b6b6989..c2bfd94 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -15,7 +15,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_lookup.c') # CN9K -sources += files('cn9k_ethdev.c') +sources += files('cn9k_ethdev.c', + 'cn9k_rx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 20/62] net/cnxk: add Rx multi-segmented version for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (18 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (42 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 17 ++++++++++++ drivers/net/cnxk/cn9k_rx.h | 60 ++++++++++++++++++++++++++++++++++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 17 ++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/meson.build | 3 ++- 5 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index a4297f9..87a62c9 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -40,7 +42,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 92f3c7c..49f80ce 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -104,6 +104,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -138,8 +185,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,8 +290,11 @@ R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c new file mode 100644 index 0000000..6ad8c1d --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58c6710..a94f5eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c2bfd94..9737234 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -16,7 +16,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', - 'cn9k_rx.c') + 'cn9k_rx.c', + 'cn9k_rx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 21/62] net/cnxk: add Rx vector version for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (19 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 22/62] net/cnxk: add Tx support " Nithin Dabilpuram ` (41 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 13 ++- drivers/net/cnxk/cn9k_rx.h | 221 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx_vec.c | 17 ++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 252 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 87a62c9..01eb21f 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -50,7 +50,18 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 49f80ce..bc04f5c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -6,6 +6,7 @@ #define __CN9K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -266,6 +267,223 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn9k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -295,6 +513,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c new file mode 100644 index 0000000..997177f --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 9737234..c02d9bd 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -17,7 +17,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', 'cn9k_rx.c', - 'cn9k_rx_mseg.c') + 'cn9k_rx_mseg.c', + 'cn9k_rx_vec.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 22/62] net/cnxk: add Tx support for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (20 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (40 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_tx.c | 53 ++++++ drivers/net/cnxk/cn9k_tx.h | 414 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 71 +++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 541 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 84dcc2c..cd0938f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -32,5 +32,6 @@ struct cn9k_eth_rxq { /* Rx and Tx routines */ void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c new file mode 100644 index 0000000..a0b022a --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.c @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags);\ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index bb6379b..6d5fc59 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,10 +4,424 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn9k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(++sg) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, + const uint32_t flags) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_prep_lmt(uint64_t *cmd, void *lmt_addr, const uint32_t flags) +{ + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeor(io_addr); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeorl(io_addr); +} + +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint16_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a94f5eb..8facafc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -103,6 +103,10 @@ /* Fastpath lookup */ #define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" +#define CNXK_NIX_UDP_TUN_BITMASK \ + ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ + (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -226,4 +230,71 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Inlines */ +static __rte_always_inline uint64_t +cnxk_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +cnxk_nix_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c02d9bd..2c1ce72 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -18,7 +18,8 @@ sources = files('cnxk_ethdev.c', sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', - 'cn9k_rx_vec.c') + 'cn9k_rx_vec.c', + 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 23/62] net/cnxk: add Tx multi-segment version for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (21 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 22/62] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (39 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst multi-segment version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 14 ++++ drivers/net/cnxk/cn9k_tx.h | 147 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 +++++++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 192 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index a0b022a..8f1d5f5 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,6 +21,7 @@ NIX_TX_FASTPATH_MODES #undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -39,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, @@ -47,7 +50,18 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 6d5fc59..239db66 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -309,6 +309,111 @@ cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) } static __rte_always_inline uint16_t +cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Commit changes to mbuf */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + rte_io_wmb(); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_prep_lmt(uint64_t *cmd, void *lmt_addr, uint16_t segdw) +{ + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, rte_iova_t io_addr, + uint16_t segdw) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_io_wmb(); + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline uint16_t cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -344,6 +449,45 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + uint64_t i; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint16_t segdw; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -419,6 +563,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c new file mode 100644 index 0000000..65c5f36 --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8facafc..2f31cba 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -46,6 +46,10 @@ #define CNXK_NIX_TX_NB_SEG_MAX 9 #endif +#define CNXK_NIX_TX_MSEG_SG_DWORDS \ + ((RTE_ALIGN_MUL_CEIL(CNXK_NIX_TX_NB_SEG_MAX, 3) / 3) + \ + CNXK_NIX_TX_NB_SEG_MAX) + #define CNXK_NIX_RSS_L3_L4_SRC_DST \ (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ ETH_RSS_L4_DST_ONLY) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 2c1ce72..c727a09 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -19,7 +19,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', - 'cn9k_tx.c') + 'cn9k_tx.c', + 'cn9k_tx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 24/62] net/cnxk: add Tx vector version for cn9k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (22 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram ` (38 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst vector version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 16 +- drivers/net/cnxk/cn9k_tx.h | 920 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 961 insertions(+), 3 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 8f1d5f5..2ff9720 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,7 +21,6 @@ NIX_TX_FASTPATH_MODES #undef T - static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -58,7 +57,20 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 239db66..c633008 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -487,6 +489,921 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn9k_eth_txq *txq = tx_queue; + uint64_t *lmt_addr = txq->lmt_addr; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + uint64_t lmt_status, i; + uint16_t pkts_left; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->cmd[2]); + sgdesc23_w0 = sgdesc01_w0; + + for (i = 0; i < pkts; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + do { + vst1q_u64(lmt_addr, cmd00); + vst1q_u64(lmt_addr + 2, cmd01); + vst1q_u64(lmt_addr + 4, cmd10); + vst1q_u64(lmt_addr + 6, cmd11); + vst1q_u64(lmt_addr + 8, cmd20); + vst1q_u64(lmt_addr + 10, cmd21); + vst1q_u64(lmt_addr + 12, cmd30); + vst1q_u64(lmt_addr + 14, cmd31); + lmt_status = roc_lmt_submit_ldeor(io_addr); + + } while (lmt_status == 0); + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + if (unlikely(pkts_left)) + pkts += cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts_left, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F @@ -566,6 +1483,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c new file mode 100644 index 0000000..21ffc2c --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index c727a09..ce771d1 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -20,7 +20,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', 'cn9k_tx.c', - 'cn9k_tx_mseg.c') + 'cn9k_tx_mseg.c', + 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 25/62] net/cnxk: add Rx support for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (23 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram ` (37 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Rx burst support for CN10K SoC. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.h | 3 + drivers/net/cnxk/cn10k_rx.c | 45 ++++++++ drivers/net/cnxk/cn10k_rx.h | 236 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.c diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 2157b16..e4332d3 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -32,4 +32,7 @@ struct cn10k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c new file mode 100644 index 0000000..8b422d0 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.c @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index d3d1661..01c9d29 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -6,6 +6,242 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn10k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index ce771d1..48ead8c 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -23,7 +23,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_mseg.c', 'cn9k_tx_vec.c') # CN10K -sources += files('cn10k_ethdev.c') +sources += files('cn10k_ethdev.c', + 'cn10k_rx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 26/62] net/cnxk: add Rx multi-segment version for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (24 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (36 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Rx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- doc/guides/nics/cnxk.rst | 2 ++ doc/guides/nics/features/cnxk.ini | 2 ++ doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 2 ++ drivers/net/cnxk/cn10k_rx.c | 20 +++++++++++- drivers/net/cnxk/cn10k_rx.h | 57 +++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx_mseg.c | 17 +++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 100 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4f1b58c..789ec29 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,11 +17,13 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Jumbo frames - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - Link state information +- Scatter-Gather IO support Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 712f8d5..23564b7 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 82f2af0..421048d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 61fed11..e901fa2 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 8b422d0..ce2cfee 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -10,7 +10,7 @@ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ - } + } \ NIX_RX_FASTPATH_MODES #undef R @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, @@ -40,6 +42,22 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 01c9d29..c667c9a 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -103,6 +103,52 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + ((rx->desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -136,8 +182,12 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,6 +289,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c new file mode 100644 index 0000000..9d283f7 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 48ead8c..b1e9245 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -24,7 +24,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c', - 'cn10k_rx.c') + 'cn10k_rx.c', + 'cn10k_rx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 27/62] net/cnxk: add Rx vector version for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (25 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 28/62] net/cnxk: add Tx support " Nithin Dabilpuram ` (35 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + drivers/net/cnxk/cn10k_rx.c | 13 ++- drivers/net/cnxk/cn10k_rx.h | 222 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_rx_vec.c | 19 ++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 256 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 789ec29..4187e9d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Link state information - Scatter-Gather IO support +- Vector Poll mode driver Prerequisites ------------- diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index ce2cfee..0598111 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -50,7 +50,18 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index c667c9a..7bb9dd8 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -5,6 +5,7 @@ #define __CN10K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -263,6 +264,224 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn10k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -292,6 +511,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c new file mode 100644 index 0000000..0fa079c --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b1e9245..9386005 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -25,7 +25,8 @@ sources += files('cn9k_ethdev.c', # CN10K sources += files('cn10k_ethdev.c', 'cn10k_rx.c', - 'cn10k_rx_mseg.c') + 'cn10k_rx_mseg.c', + 'cn10k_rx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 28/62] net/cnxk: add Tx support for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (26 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (34 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram, Harman Kalra From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 7 + doc/guides/nics/features/cnxk_vec.ini | 6 + doc/guides/nics/features/cnxk_vf.ini | 7 + drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_tx.c | 54 ++++ drivers/net/cnxk/cn10k_tx.h | 485 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 563 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 4187e9d..555730d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -22,6 +22,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Inner and Outer Checksum offload - Link state information - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 23564b7..02be26b 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,11 +12,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 421048d..8c63853 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,10 +12,16 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index e901fa2..a1bd49b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,11 +11,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index e4332d3..58c51ab 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -34,5 +34,6 @@ struct cn10k_eth_rxq { /* Rx and Tx routines */ void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c new file mode 100644 index 0000000..13c605f --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ + flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 39d4755..a6b75af 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,10 +4,495 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn10k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_data(const uint16_t flags) +{ + const uint64_t dw_m1 = cn10k_nix_tx_ext_subs(flags) + 1; + uint64_t data = 0; + + /* This will be moved to addr area */ + data |= dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + +static __rte_always_inline void +cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, + const uint16_t flags) +{ + /* Send hdr */ + cmd[0] = txq->send_hdr_w0; + cmd[1] = 0; + cmd += 2; + + /* Send ext if present */ + if (flags & NIX_TX_NEED_EXT_HDR) { + *(__uint128_t *)cmd = *(const __uint128_t *)txq->cmd; + cmd += 2; + } + + /* Send sg */ + cmd[0] = txq->sg_w0; + cmd[1] = 0; +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, + const uint16_t flags) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format += is_udp_tun ? 2 : 6; + + send_hdr_ext->w0.lso_format += + !!(ol_flags & PKT_TX_OUTER_IPV6) << 1; + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(sg + 1) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } + + /* With minimal offloads, 'cmd' being local could be optimized out to + * registers. In other cases, 'cmd' will be in stack. Intent is + * 'cmd' stores content from txq->cmd which is copied only once. + */ + *((struct nix_send_hdr_s *)lmt_addr) = *send_hdr; + lmt_addr += 16; + if (flags & NIX_TX_NEED_EXT_HDR) { + *((struct nix_send_ext_s *)lmt_addr) = *send_hdr_ext; + lmt_addr += 16; + } + /* In case of multi-seg, sg template is stored here */ + *((union nix_send_sg_s *)lmt_addr) = *sg; + *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); +} + +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + uintptr_t pa, lmt_addr = txq->lmt_base; + uint16_t lmt_id, burst, left, i; + uint64_t data; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + /* Get cmd skeleton */ + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 9386005..faec738 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -26,7 +26,8 @@ sources += files('cn9k_ethdev.c', sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', - 'cn10k_rx_vec.c') + 'cn10k_rx_vec.c', + 'cn10k_tx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 29/62] net/cnxk: add Tx multi-segment version for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (27 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 28/62] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (33 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 18 ++++- drivers/net/cnxk/cn10k_tx.h | 166 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 ++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 210 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 13c605f..9803002 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -40,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, @@ -48,7 +50,21 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index a6b75af..4a201fd 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -338,6 +338,77 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, } static __rte_always_inline uint16_t +cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Mark mempool object as "put" since it is freed by NIX + */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline uint16_t cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -415,6 +486,98 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; + const rte_iova_t io_addr = txq->io_addr; + uint16_t segdw, lmt_id, burst, left, i; + uint64_t data0, data1; + __uint128_t data128; + uint16_t shft; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + shft = 16; + data128 = 0; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + /* Store sg list directly on lmt line */ + segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, + flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + data128 |= (((__uint128_t)(segdw - 1)) << shft); + shft += 3; + } + + data0 = (uint64_t)data128; + data1 = (uint64_t)(data128 >> 64); + /* Make data0 similar to data1 */ + data0 >>= 16; + /* Trigger LMTST */ + if (burst > 16) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= (15ULL << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + + pa1 = io_addr | (data1 & 0x7) << 4; + data1 &= ~0x7ULL; + data1 <<= 16; + data1 |= ((uint64_t)(burst - 17)) << 12; + data1 |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data1, pa1); + } else if (burst) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= ((burst - 1) << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -490,6 +653,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c new file mode 100644 index 0000000..6ae6907 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz)]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index faec738..f04e2d7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -27,7 +27,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', - 'cn10k_tx.c') + 'cn10k_tx.c', + 'cn10k_tx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 30/62] net/cnxk: add Tx vector version for cn10k 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (28 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram ` (32 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add Tx burst vector version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 10 + drivers/net/cnxk/cn10k_tx.h | 959 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 996 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 9803002..e6eb101 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -58,10 +58,20 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + if (dev->scalar_ena || (dev->tx_offload_flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 4a201fd..5d649e0 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -578,6 +580,960 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3, data, pa; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint16_t left, scalar, burst, i, lmt_id; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t lmt_addr = txq->lmt_base; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64x2_t cmd00, cmd01; + uint64x2_t cmd10, cmd11; + uint64x2_t cmd20, cmd21; + uint64x2_t cmd30, cmd31; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + senddesc01_w0 = vld1q_dup_u64(&txq->send_hdr_w0); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->sg_w0); + sgdesc23_w0 = sgdesc01_w0; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + struct rte_mbuf *mbuf; + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + if (cnxk_nix_prefree_seg(mbuf)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies(mbuf->pool, + (void **)&mbuf, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + } else { + struct rte_mbuf *mbuf; + /* Mark mempool object as "put" since + * it is freed by NIX + */ + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf0 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf1 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf2 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + + mbuf = (struct rte_mbuf *)((uintptr_t)mbuf3 - + offsetof(struct rte_mbuf, + buf_iova)); + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, + 0); + RTE_SET_USED(mbuf); + } + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create first half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Create second half of 4W cmd for 4 mbufs (sgdesc) */ + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + + /* Create first half of 4W cmd for 4 mbufs (sendhdr) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + } else { + /* Just use ld1q to retrieve aura + * when we don't need tx_offload + */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, + pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, + pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, + pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, + pool_id)); + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd00 = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd01 = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd10 = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd11 = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd20 = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd21 = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd30 = vzip2q_u64(senddesc23_w0, senddesc23_w1); + cmd31 = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + } + + /* Store the prepared send desc to LMT lines */ + vst1q_u64((void *)lmt_addr, cmd00); + vst1q_u64((void *)(lmt_addr + 16), cmd01); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd10); + vst1q_u64((void *)(lmt_addr + 16), cmd11); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd20); + vst1q_u64((void *)(lmt_addr + 16), cmd21); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + vst1q_u64((void *)lmt_addr, cmd30); + vst1q_u64((void *)(lmt_addr + 16), cmd31); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Reset lmt base addr to start another burst */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + if (unlikely(scalar)) + pkts += cn10k_nix_xmit_pkts(tx_queue, tx_pkts, scalar, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -656,6 +1612,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c new file mode 100644 index 0000000..42baeb5 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index f04e2d7..748a49a 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -28,7 +28,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', 'cn10k_tx.c', - 'cn10k_tx_mseg.c') + 'cn10k_tx_mseg.c', + 'cn10k_tx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 31/62] net/cnxk: add device start and stop operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (29 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram ` (31 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, Nithin Dabilpuram Add device start and stop operation callbacks for CN9K and CN10K. Device stop is common for both platforms while device start as some platform dependent portion where the platform specific offload flags are recomputed and the right Rx/Tx burst function is chosen. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 84 ++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 127 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 90 ++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 2 + drivers/net/cnxk/cnxk_link.c | 11 ++++ 6 files changed, 438 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 555730d..42aa7a5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -39,6 +39,58 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1 + EAL: Detected 4 lcore(s) + EAL: Detected 1 NUMA nodes + EAL: Multi-process socket /var/run/dpdk/rte/mp_socket + EAL: Selected IOVA mode 'VA' + EAL: No available hugepages reported in hugepages-16777216kB + EAL: No available hugepages reported in hugepages-2048kB + EAL: Probing VFIO support... + EAL: VFIO support initialized + EAL: using IOMMU type 1 (Type 1) + [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98 + EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0) + PMD: RoC Model: cn10k + EAL: No legacy callbacks, legacy socket not created + testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0 + testpmd: preferred mempool ops selected: cn10k_mempool_ops + Configuring Port 0 (socket 0) + PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex + + Port 0: link state change event + Port 0: 96:D4:99:72:A5:BF + Checking link statuses... + Done + No commandline core given, start packet forwarding + io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native + Logical Core 3 (socket 0) forwards packets on 1 streams: + RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 + + io packet forwarding packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=1 + port 0: RX queue number: 1 Tx queue number: 1 + Rx offloads=0x0 Tx offloads=0x10000 + RX queue: 0 + RX desc=4096 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + RX Offloads=0x0 + TX queue: 0 + TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX offloads=0x0 - TX RS bit threshold=0 + Press enter to exit + Runtime Config Options ---------------------- @@ -132,3 +184,35 @@ Runtime Config Options Above devarg parameters are configurable per device, user needs to pass the parameters to all the PCIe devices if application requires to configure on all the ethdev ports. + +Limitations +----------- + +``mempool_cnxk`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager. +``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler +as it is performance wise most effective way for packet allocation and Tx buffer +recycling on OCTEON TX2 SoC platform. + +CRC stripping +~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by +the host interface irrespective of the offload configuration. + +Debugging Options +----------------- + +.. _table_cnxk_ethdev_debug_options: + +.. table:: cnxk ethdev debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NIX | --log-level='pmd\.net.cnxk,8' | + +---+------------+-------------------------------------------------------+ + | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | + +---+------------+-------------------------------------------------------+ diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 9df30ae..fa7b835 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -5,6 +5,98 @@ #include "cn10k_rx.h" #include "cn10k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn10k_eth_set_rx_function(eth_dev); return 0; } @@ -162,6 +255,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -169,6 +266,28 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -184,6 +303,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } @@ -221,6 +341,10 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (!eth_dev) return -ENOENT; + + /* Setup callbacks for secondary process */ + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); } return 0; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 22f5527..e47fa9a 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -5,6 +5,98 @@ #include "cn9k_rx.h" #include "cn9k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn9k_eth_set_rx_function(eth_dev); return 0; } @@ -171,6 +264,10 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -178,6 +275,28 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -193,6 +312,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } @@ -232,6 +352,13 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (!eth_dev) return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* Setup callbacks for secondary process */ + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; + } + dev = cnxk_eth_pmd_priv(eth_dev); /* Update capabilities already set for TSO. * TSO not supported for earlier chip revisions diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 2a0ce7f..156d962 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -926,12 +926,102 @@ cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) return rc; } +static int +cnxk_nix_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct rte_mbuf *rx_pkts[32]; + int count, i, j, rc; + void *rxq; + + /* Disable switch hdr pkind */ + roc_nix_switch_hdr_set(&dev->nix, 0); + + /* Stop link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) + roc_nix_mac_link_event_start_stop(&dev->nix, false); + + /* Disable Rx via NPC */ + roc_nix_npc_rx_ena_dis(&dev->nix, false); + + /* Stop rx queues and free up pkts pending */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = dev_ops->rx_queue_stop(eth_dev, i); + if (rc) + continue; + + rxq = eth_dev->data->rx_queues[i]; + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + while (count) { + for (j = 0; j < count; j++) + rte_pktmbuf_free(rx_pkts[j]); + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + } + } + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_stop(eth_dev, i); + + return 0; +} + +int +cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc, i; + + /* Start rx queues */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = cnxk_nix_rx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Enable Rx in NPC */ + rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); + if (rc) { + plt_err("Failed to enable NPC rx %d", rc); + return rc; + } + + cnxk_nix_toggle_flag_link_cfg(dev, true); + + /* Start link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) { + rc = roc_nix_mac_link_event_start_stop(&dev->nix, true); + if (rc) { + plt_err("Failed to start cgx link event %d", rc); + goto rx_disable; + } + } + + cnxk_nix_toggle_flag_link_cfg(dev, false); + + return 0; + +rx_disable: + roc_nix_npc_rx_ena_dis(&dev->nix, false); + cnxk_nix_toggle_flag_link_cfg(dev, false); + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_stop = cnxk_nix_dev_stop, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 2f31cba..984f4fe 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -214,6 +214,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); +int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -222,6 +223,7 @@ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); /* Link */ +void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c index 0223d68..8728eb1 100644 --- a/drivers/net/cnxk/cnxk_link.c +++ b/drivers/net/cnxk/cnxk_link.c @@ -4,6 +4,17 @@ #include "cnxk_ethdev.h" +void +cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set) +{ + if (set) + dev->flags |= CNXK_LINK_CFG_IN_PROGRESS_F; + else + dev->flags &= ~CNXK_LINK_CFG_IN_PROGRESS_F; + + rte_wmb(); +} + static inline int nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 32/62] net/cnxk: add MAC address set ops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (30 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram ` (30 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Default mac address set operation is implemented for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 156d962..4e02be2 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1017,6 +1017,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 984f4fe..717a8d8 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -203,6 +203,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4a45956..87cf4ee 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -69,3 +69,32 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; return 0; } + +int +cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Update mac address at NPC */ + rc = roc_nix_npc_mac_addr_set(nix, addr->addr_bytes); + if (rc) + goto exit; + + /* Update mac address at CGX for PFs only */ + if (!roc_nix_is_vf_or_sdp(nix)) { + rc = roc_nix_mac_addr_set(nix, addr->addr_bytes); + if (rc) { + /* Rollback to previous mac address */ + roc_nix_npc_mac_addr_set(nix, dev->mac_addr); + goto exit; + } + } + + /* Update mac address to cnxk ethernet device */ + rte_memcpy(dev->mac_addr, addr->addr_bytes, RTE_ETHER_ADDR_LEN); + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 33/62] net/cnxk: add MTU set device operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (31 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram ` (29 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> This Patch implements mtu set dev op for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 51 +++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 77 ++++++++++++++++++++++++++++++++++- 7 files changed, 135 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 42aa7a5..6cb90a7 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Inner and Outer Checksum offload - Link state information +- MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 02be26b..6fef725 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8c63853..79cb1e2 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a1bd49b..5cc9f3f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 4e02be2..6b06ee5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,50 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +static void +nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + uint32_t buffsz; + + dev = rxq->dev; + eth_dev = dev->eth_dev; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->qconf.mp); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + if (eth_dev->data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + dev->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + dev->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } +} + +static int +nix_recalc_mtu(struct rte_eth_dev *eth_dev) +{ + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_eth_rxq_sp *rxq; + uint16_t mtu; + int rc; + + rxq = ((struct cnxk_eth_rxq_sp *)data->rx_queues[0]) - 1; + /* Setup scatter mode if needed by jumbo */ + nix_enable_mseg_on_jumbo(rxq); + + /* Setup MTU based on max_rx_pkt_len */ + mtu = data->dev_conf.rxmode.max_rx_pkt_len - CNXK_NIX_L2_OVERHEAD + + CNXK_NIX_MAX_VTAG_ACT_SIZE; + + rc = cnxk_nix_mtu_set(eth_dev, mtu); + if (rc) + plt_err("Failed to set default MTU size, rc=%d", rc); + + return rc; +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -973,6 +1017,12 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; + if (eth_dev->data->nb_rx_queues != 0) { + rc = nix_recalc_mtu(eth_dev); + if (rc) + return rc; + } + /* Start rx queues */ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { rc = cnxk_nix_rx_queue_start(eth_dev, i); @@ -1017,6 +1067,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mtu_set = cnxk_nix_mtu_set, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 717a8d8..3838573 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -28,7 +28,9 @@ #define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) /* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ -#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + \ + RTE_ETHER_CRC_LEN + \ + CNXK_NIX_MAX_VTAG_ACT_SIZE) #define CNXK_NIX_RX_MIN_DESC 16 #define CNXK_NIX_RX_MIN_DESC_ALIGN 16 @@ -203,6 +205,7 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 87cf4ee..21b55c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -20,7 +20,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; - devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->max_mtu = devinfo->max_rx_pktlen - + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN); devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; devinfo->rx_offload_capa = dev->rx_offload_capa; @@ -98,3 +99,77 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) exit: return rc; } + +int +cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) +{ + uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + uint32_t buffsz; + + /* Check if MTU is within the allowed range */ + if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { + plt_err("MTU is lesser than minimum"); + goto exit; + } + + if ((frame_size - RTE_ETHER_CRC_LEN) > + ((uint32_t)roc_nix_max_pkt_len(nix))) { + plt_err("MTU is greater than maximum"); + goto exit; + } + + buffsz = data->min_rx_buf_size - RTE_PKTMBUF_HEADROOM; + old_frame_size = data->mtu + CNXK_NIX_L2_OVERHEAD; + + /* Refuse MTU that requires the support of scattered packets + * when this feature has not been enabled before. + */ + if (data->dev_started && frame_size > buffsz && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER)) { + plt_err("Scatter offload is not enabled for mtu"); + goto exit; + } + + /* Check <seg size> * <max_seg> >= max_frame */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) && + frame_size > (buffsz * CNXK_NIX_RX_NB_SEG_MAX)) { + plt_err("Greater than maximum supported packet length"); + goto exit; + } + + frame_size -= RTE_ETHER_CRC_LEN; + + /* Update mtu on Tx */ + rc = roc_nix_mac_mtu_set(nix, frame_size); + if (rc) { + plt_err("Failed to set MTU, rc=%d", rc); + goto exit; + } + + /* Sync same frame size on Rx */ + rc = roc_nix_mac_max_rx_len_set(nix, frame_size); + if (rc) { + /* Rollback to older mtu */ + roc_nix_mac_mtu_set(nix, + old_frame_size - RTE_ETHER_CRC_LEN); + plt_err("Failed to max Rx frame length, rc=%d", rc); + goto exit; + } + + frame_size += RTE_ETHER_CRC_LEN; + + if (frame_size > RTE_ETHER_MAX_LEN) + dev->rx_offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; + else + dev->rx_offloads &= ~DEV_RX_OFFLOAD_JUMBO_FRAME; + + /* Update max_rx_pkt_len */ + data->dev_conf.rxmode.max_rx_pkt_len = frame_size; + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 34/62] net/cnxk: add promiscuous mode enable and disable 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (32 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram ` (28 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Add device operations to enable and disable promisc mode for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 56 +++++++++++++++++++++++++++++++++++ 6 files changed, 63 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6cb90a7..364e511 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,6 +17,7 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Promiscuous mode - Jumbo frames - SR-IOV VF - Lock-free Tx queue diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 6fef725..9b2e163 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y TSO = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 79cb1e2..31471e0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -16,6 +16,7 @@ Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6b06ee5..1e58599 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1078,6 +1078,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, + .promiscuous_enable = cnxk_nix_promisc_enable, + .promiscuous_disable = cnxk_nix_promisc_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 3838573..73aef34 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -208,6 +208,8 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); +int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 21b55c4..6feb3a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -173,3 +173,59 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) exit: return rc; } + +int +cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, false); + return rc; + } + + return 0; +} + +int +cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, false); + if (rc < 0) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, false); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, true); + return rc; + } + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 35/62] net/cnxk: add DMAC filter support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (33 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram ` (27 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> DMAC filter support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 44 ++++++++++++++++++++++++++++++++--- 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 364e511..ce33f17 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -23,6 +23,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- MAC filtering - Inner and Outer Checksum offload - Link state information - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9b2e163..20d4d12 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 31471e0..e1de8ab 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1e58599..36d45b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1068,6 +1068,8 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, + .mac_addr_add = cnxk_nix_mac_addr_add, + .mac_addr_remove = cnxk_nix_mac_addr_del, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 73aef34..38ac654 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -139,6 +139,7 @@ struct cnxk_eth_dev { /* Max macfilter entries */ uint8_t max_mac_entries; + bool dmac_filter_enable; uint16_t flags; uint8_t ptype_disable; @@ -206,6 +207,10 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr, uint32_t index, + uint32_t pool); +void cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6feb3a9..fc60576 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -101,6 +101,43 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) } int +cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, + uint32_t index, uint32_t pool) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + PLT_SET_USED(index); + PLT_SET_USED(pool); + + rc = roc_nix_mac_addr_add(nix, addr->addr_bytes); + if (rc < 0) { + plt_err("Failed to add mac address, rc=%d", rc); + return rc; + } + + /* Enable promiscuous mode at NIX level */ + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} + +void +cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_mac_addr_del(nix, index); + if (rc) + plt_err("Failed to delete mac address, rc=%d", rc); +} + +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) { uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; @@ -212,8 +249,8 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (roc_nix_is_vf_or_sdp(nix)) return rc; - rc = roc_nix_npc_promisc_ena_dis(nix, false); - if (rc < 0) { + rc = roc_nix_npc_promisc_ena_dis(nix, dev->dmac_filter_enable); + if (rc) { plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, roc_error_msg_get(rc)); return rc; @@ -223,9 +260,10 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (rc) { plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, roc_error_msg_get(rc)); - roc_nix_npc_promisc_ena_dis(nix, true); + roc_nix_npc_promisc_ena_dis(nix, !dev->dmac_filter_enable); return rc; } + dev->dmac_filter_enable = false; return 0; } -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 36/62] net/cnxk: add all multicast enable/disable ethops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (34 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram ` (26 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> L2 multicast packets can be allowed or blocked. Patch implements corresponding ethops. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 17 +++++++++++++++++ 5 files changed, 23 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 20d4d12..b41af2d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e1de8ab..7fe8018 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 36d45b4..1a4f96b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1082,6 +1082,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, .promiscuous_enable = cnxk_nix_promisc_enable, .promiscuous_disable = cnxk_nix_promisc_disable, + .allmulticast_enable = cnxk_nix_allmulticast_enable, + .allmulticast_disable = cnxk_nix_allmulticast_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 38ac654..09031e9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -215,6 +215,8 @@ int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index fc60576..61ecbab 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -267,3 +267,20 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) dev->dmac_filter_enable = false; return 0; } + +int +cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, true, false); +} + +int +cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, false, + eth_dev->data->promiscuous); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 37/62] net/cnxk: add Rx/Tx burst mode get ops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (35 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram ` (25 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements ethdev operations to get Rx and Tx burst mode. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 127 ++++++++++++++++++++++++++++++++++ 6 files changed, 136 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b41af2d..298f167 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 7fe8018..a673cc1 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5cc9f3f..335d082 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1a4f96b..25451e1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1084,6 +1084,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .promiscuous_disable = cnxk_nix_promisc_disable, .allmulticast_enable = cnxk_nix_allmulticast_enable, .allmulticast_disable = cnxk_nix_allmulticast_disable, + .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, + .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 09031e9..481ede9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -219,6 +219,10 @@ int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); +int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 61ecbab..7ae961a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -72,6 +72,133 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) } int +cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } rx_offload_map[] = { + {DEV_RX_OFFLOAD_VLAN_STRIP, " VLAN Strip,"}, + {DEV_RX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_LRO, " TCP LRO,"}, + {DEV_RX_OFFLOAD_QINQ_STRIP, " QinQ VLAN Strip,"}, + {DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_MACSEC_STRIP, " MACsec Strip,"}, + {DEV_RX_OFFLOAD_HEADER_SPLIT, " Header Split,"}, + {DEV_RX_OFFLOAD_VLAN_FILTER, " VLAN Filter,"}, + {DEV_RX_OFFLOAD_VLAN_EXTEND, " VLAN Extend,"}, + {DEV_RX_OFFLOAD_JUMBO_FRAME, " Jumbo Frame,"}, + {DEV_RX_OFFLOAD_SCATTER, " Scattered,"}, + {DEV_RX_OFFLOAD_TIMESTAMP, " Timestamp,"}, + {DEV_RX_OFFLOAD_SECURITY, " Security,"}, + {DEV_RX_OFFLOAD_KEEP_CRC, " Keep CRC,"}, + {DEV_RX_OFFLOAD_SCTP_CKSUM, " SCTP,"}, + {DEV_RX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_RX_OFFLOAD_RSS_HASH, " RSS,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Rx Offloads:", + "Scalar, Rx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Rx offload info */ + for (i = 0; i < RTE_DIM(rx_offload_map); i++) { + if (dev->rx_offloads & rx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + rx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int +cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } tx_offload_map[] = { + {DEV_TX_OFFLOAD_VLAN_INSERT, " VLAN Insert,"}, + {DEV_TX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_TX_OFFLOAD_SCTP_CKSUM, " SCTP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_TSO, " TCP TSO,"}, + {DEV_TX_OFFLOAD_UDP_TSO, " UDP TSO,"}, + {DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_QINQ_INSERT, " QinQ VLAN Insert,"}, + {DEV_TX_OFFLOAD_VXLAN_TNL_TSO, " VXLAN Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GRE_TNL_TSO, " GRE Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IPIP_TNL_TSO, " IP-in-IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GENEVE_TNL_TSO, " Geneve Tunnel TSO,"}, + {DEV_TX_OFFLOAD_MACSEC_INSERT, " MACsec Insert,"}, + {DEV_TX_OFFLOAD_MT_LOCKFREE, " Multi Thread Lockless Tx,"}, + {DEV_TX_OFFLOAD_MULTI_SEGS, " Scattered,"}, + {DEV_TX_OFFLOAD_MBUF_FAST_FREE, " H/W MBUF Free,"}, + {DEV_TX_OFFLOAD_SECURITY, " Security,"}, + {DEV_TX_OFFLOAD_UDP_TNL_TSO, " UDP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IP_TNL_TSO, " IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP, " Timestamp,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Tx Offloads:", + "Scalar, Tx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Tx offload info */ + for (i = 0; i < RTE_DIM(tx_offload_map); i++) { + if (dev->tx_offloads & tx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + tx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (36 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-15 12:40 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram ` (24 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements set and get operations for flow control. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 74 +++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 95 +++++++++++++++++++++++++++++++++++ 6 files changed, 185 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ce33f17..96b2c5d 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -26,6 +26,7 @@ Features of the CNXK Ethdev PMD are: - MAC filtering - Inner and Outer Checksum offload - Link state information +- Link flow control - MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 298f167..afd0f01 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index a673cc1..4bd11ce 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 25451e1..29c551f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -81,6 +81,55 @@ nix_recalc_mtu(struct rte_eth_dev *eth_dev) return rc; } +static int +nix_init_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_conf = {0}; + int rc; + + /* Both Rx & Tx flow ctrl get enabled(RTE_FC_FULL) in HW + * by AF driver, update those info in PMD structure. + */ + rc = cnxk_nix_flow_ctrl_get(eth_dev, &fc_conf); + if (rc) + goto exit; + + fc->mode = fc_conf.mode; + fc->rx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_RX_PAUSE); + fc->tx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_TX_PAUSE); + +exit: + return rc; +} + +static int +nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_cfg = {0}; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + return 0; + + fc_cfg.mode = fc->mode; + + /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ + if (roc_model_is_cn96_Ax() && + (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { + fc_cfg.mode = + (fc_cfg.mode == RTE_FC_FULL || + fc_cfg.mode == RTE_FC_TX_PAUSE) ? + RTE_FC_TX_PAUSE : RTE_FC_NONE; + } + + return cnxk_nix_flow_ctrl_set(eth_dev, &fc_cfg); +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -657,6 +706,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct rte_eth_rxmode *rxmode = &conf->rxmode; struct rte_eth_txmode *txmode = &conf->txmode; char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix_fc_cfg fc_cfg = {0}; struct roc_nix *nix = &dev->nix; struct rte_ether_addr *ea; uint8_t nb_rxq, nb_txq; @@ -838,6 +888,21 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Init flow control configuration */ + fc_cfg.cq_cfg_valid = false; + fc_cfg.rxchan_cfg.enable = true; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } + + /* Update flow control configuration to PMD */ + rc = nix_init_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } /* * Restore queue config when reconfigure followed by * reconfigure and no queue configure invoked from application case. @@ -1037,6 +1102,13 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } + /* Update Flow control configuration */ + rc = nix_update_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to enable flow control. error code(%d)", rc); + return rc; + } + /* Enable Rx in NPC */ rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); if (rc) { @@ -1086,6 +1158,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .allmulticast_disable = cnxk_nix_allmulticast_disable, .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, + .flow_ctrl_get = cnxk_nix_flow_ctrl_get, + .flow_ctrl_set = cnxk_nix_flow_ctrl_set, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 481ede9..77139d0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -113,6 +113,12 @@ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) +struct cnxk_fc_cfg { + enum rte_eth_fc_mode mode; + uint8_t rx_pause; + uint8_t tx_pause; +}; + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -174,6 +180,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Flow control configuration */ + struct cnxk_fc_cfg fc_cfg; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -223,6 +232,10 @@ int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); +int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7ae961a..eac50a2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -199,6 +199,101 @@ cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, } int +cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum rte_eth_fc_mode mode_map[] = { + RTE_FC_NONE, RTE_FC_RX_PAUSE, + RTE_FC_TX_PAUSE, RTE_FC_FULL + }; + struct roc_nix *nix = &dev->nix; + int mode; + + mode = roc_nix_fc_mode_get(nix); + if (mode < 0) + return mode; + + memset(fc_conf, 0, sizeof(struct rte_eth_fc_conf)); + fc_conf->mode = mode_map[mode]; + return 0; +} + +int +cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum roc_nix_fc_mode mode_map[] = { + ROC_NIX_FC_NONE, ROC_NIX_FC_RX, + ROC_NIX_FC_TX, ROC_NIX_FC_FULL + }; + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct roc_nix *nix = &dev->nix; + struct roc_nix_fc_cfg fc_cfg; + struct cnxk_eth_rxq_sp *rxq; + uint8_t rx_pause, tx_pause; + struct roc_nix_cq *cq; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) { + plt_err("Flow control configuration is not allowed on VFs"); + return -ENOTSUP; + } + + if (fc_conf->high_water || fc_conf->low_water || fc_conf->pause_time || + fc_conf->mac_ctrl_frame_fwd || fc_conf->autoneg) { + plt_info("Only MODE configuration is supported"); + return -EINVAL; + } + + if (fc_conf->mode == fc->mode) + return 0; + + rx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_RX_PAUSE); + tx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_TX_PAUSE); + + /* Check if TX pause frame is already enabled or not */ + if (fc->tx_pause ^ tx_pause) { + if (roc_model_is_cn96_Ax() && data->dev_started) { + /* On Ax, CQ should be in disabled state + * while setting flow control configuration. + */ + plt_info("Stop the port=%d for setting flow control", + data->port_id); + return 0; + } + + for (i = 0; i < data->nb_rx_queues; i++) { + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + rxq = ((struct cnxk_eth_rxq_sp *) + data->rx_queues[i]) - 1; + cq = &dev->cqs[rxq->qid]; + fc_cfg.cq_cfg_valid = true; + fc_cfg.cq_cfg.enable = tx_pause; + fc_cfg.cq_cfg.rq = rxq->qid; + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) + return rc; + } + } + + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); + if (rc) + return rc; + + fc->rx_pause = rx_pause; + fc->tx_pause = tx_pause; + fc->mode = fc_conf->mode; + + return rc; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram @ 2021-06-15 12:40 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:40 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:37 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Sunil Kumar Kori <skori@marvell.com> > > Patch implements set and get operations for flow control. > > Signed-off-by: Sunil Kumar Kori <skori@marvell.com> > --- > + > + /* Check if TX pause frame is already enabled or not */ > + if (fc->tx_pause ^ tx_pause) { > + if (roc_model_is_cn96_Ax() && data->dev_started) { > + /* On Ax, CQ should be in disabled state > + * while setting flow control configuration. > + */ > + plt_info("Stop the port=%d for setting flow control", > + data->port_id); > + return 0; > + } > + > + for (i = 0; i < data->nb_rx_queues; i++) { From here: > + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); > + rxq = ((struct cnxk_eth_rxq_sp *) > + data->rx_queues[i]) - 1; > + cq = &dev->cqs[rxq->qid]; > + fc_cfg.cq_cfg_valid = true; > + fc_cfg.cq_cfg.enable = tx_pause; > + fc_cfg.cq_cfg.rq = rxq->qid; > + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; > + rc = roc_nix_fc_config_set(nix, &fc_cfg); > + if (rc) > + return rc; Better to move this separate static function. > + } > + } > + > + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); > + if (rc) > + return rc; > + > + fc->rx_pause = rx_pause; > + fc->tx_pause = tx_pause; > + fc->mode = fc_conf->mode; > + > + return rc; > +} > + > +int > cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) > { > struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 39/62] net/cnxk: add link up/down operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (37 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram ` (23 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements link up/down ethdev operations for cn9k and cn10k platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 4 +++- drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 47 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 29c551f..3be641a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -946,7 +946,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } -static int +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -1160,6 +1160,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, .flow_ctrl_get = cnxk_nix_flow_ctrl_get, .flow_ctrl_set = cnxk_nix_flow_ctrl_set, + .dev_set_link_up = cnxk_nix_set_link_up, + .dev_set_link_down = cnxk_nix_set_link_down, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 77139d0..6500433 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -236,6 +236,9 @@ int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); +int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); + int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, @@ -244,6 +247,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index eac50a2..37ba211 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -506,3 +506,50 @@ cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) return roc_nix_npc_mcast_config(&dev->nix, false, eth_dev->data->promiscuous); } + +int +cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + rc = roc_nix_mac_link_state_set(nix, true); + if (rc) + goto exit; + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + goto exit; + } + +exit: + return rc; +} + +int +cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_stop(eth_dev, i); + if (rc) + goto exit; + } + + rc = roc_nix_mac_link_state_set(nix, false); +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 40/62] net/cnxk: add EEPROM module info get operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (38 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram ` (22 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements eeprom module info get ethops for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 39 +++++++++++++++++++++++++++++++++++ 6 files changed, 48 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index afd0f01..b1e8641 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -31,6 +31,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4bd11ce..0f99634 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -29,6 +29,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 335d082..cecced9 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -26,6 +26,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3be641a..cf53bee 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1162,6 +1162,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .flow_ctrl_set = cnxk_nix_flow_ctrl_set, .dev_set_link_up = cnxk_nix_set_link_up, .dev_set_link_down = cnxk_nix_set_link_down, + .get_module_info = cnxk_nix_get_module_info, + .get_module_eeprom = cnxk_nix_get_module_eeprom, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 6500433..c4a562b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -238,6 +238,10 @@ int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); +int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo); +int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 37ba211..a1a963a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -553,3 +553,42 @@ cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) exit: return rc; } + +int +cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + modinfo->type = eeprom_info.sff_id; + modinfo->eeprom_len = ROC_NIX_EEPROM_SIZE; + return 0; +} + +int +cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + + if (!info->data || !info->length || + (info->offset + info->length > ROC_NIX_EEPROM_SIZE)) + return rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 41/62] net/cnxk: add Rx queue interrupt enable/disable ops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (39 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram ` (21 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Application may choose to enable/disable interrupts on Rx queues so that application can select its processing if no packets are available on queues for a longer period. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 7 files changed, 29 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 96b2c5d..6a001d9 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -30,6 +30,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Support Rx interrupt Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b1e8641..e5669f5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 0f99634..dff0c9b 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index cecced9..b950d2f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cf53bee..b980a72 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1164,6 +1164,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_set_link_down = cnxk_nix_set_link_down, .get_module_info = cnxk_nix_get_module_info, .get_module_eeprom = cnxk_nix_get_module_eeprom, + .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, + .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c4a562b..76e1049 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -242,6 +242,10 @@ int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, struct rte_eth_dev_module_info *modinfo); int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, struct rte_dev_eeprom_info *info); +int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); +int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index a1a963a..34d4a42 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -592,3 +592,22 @@ cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); return 0; } + +int +cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_enable(&dev->nix, rx_queue_id); + return 0; +} + +int +cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 42/62] net/cnxk: add validation API for mempool ops 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (40 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram ` (20 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> cn9k and cn10k supports platform specific mempool ops. This patch implements API to validate whether given mempool ops is supported or not. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 11 +++++++++++ 3 files changed, 13 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b980a72..57c30c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1166,6 +1166,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_module_eeprom = cnxk_nix_get_module_eeprom, .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, + .pool_ops_supported = cnxk_nix_pool_ops_supported, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 76e1049..0b501f6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -246,6 +246,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); +int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 34d4a42..5b8bc53 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -611,3 +611,14 @@ cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); return 0; } + +int +cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) +{ + RTE_SET_USED(eth_dev); + + if (!strcmp(pool, rte_mbuf_platform_mempool_ops())) + return 0; + + return -ENOTSUP; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 43/62] net/cnxk: add port/queue stats 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (41 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 44/62] net/cnxk: add xstats apis Nithin Dabilpuram ` (19 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> This patch implements regular port statistics and queue mapping set api to get queue statistics Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 8 ++++ drivers/net/cnxk/cnxk_stats.c | 85 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_stats.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6a001d9..c2a6fbb 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -25,6 +25,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - MAC filtering - Inner and Outer Checksum offload +- Port hardware statistics - Link state information - Link flow control - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e5669f5..40952a9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index dff0c9b..32035bb 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -30,6 +30,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b950d2f..8060a68 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -27,6 +27,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 57c30c4..f6eb9c9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1167,6 +1167,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, .pool_ops_supported = cnxk_nix_pool_ops_supported, + .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, + .stats_get = cnxk_nix_stats_get, + .stats_reset = cnxk_nix_stats_reset, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0b501f6..5075e7c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -188,6 +188,10 @@ struct cnxk_eth_dev { /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + + /* Per queue statistics counters */ + uint32_t txq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; + uint32_t rxq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; struct cnxk_eth_rxq_sp { @@ -271,6 +275,10 @@ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx); +int cnxk_nix_stats_reset(struct rte_eth_dev *dev); +int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c new file mode 100644 index 0000000..24bff0b --- /dev/null +++ b/drivers/net/cnxk/cnxk_stats.c @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +int +cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct roc_nix_stats nix_stats; + int rc = 0, i; + + rc = roc_nix_stats_get(nix, &nix_stats); + if (rc) + goto exit; + + stats->opackets = nix_stats.tx_ucast; + stats->opackets += nix_stats.tx_mcast; + stats->opackets += nix_stats.tx_bcast; + stats->oerrors = nix_stats.tx_drop; + stats->obytes = nix_stats.tx_octs; + + stats->ipackets = nix_stats.rx_ucast; + stats->ipackets += nix_stats.rx_mcast; + stats->ipackets += nix_stats.rx_bcast; + stats->imissed = nix_stats.rx_drop; + stats->ibytes = nix_stats.rx_octs; + stats->ierrors = nix_stats.rx_err; + + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) { + struct roc_nix_stats_queue qstats; + uint16_t qidx; + + if (dev->txq_stat_map[i] & (1U << 31)) { + qidx = dev->txq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 0, &qstats); + if (rc) + goto exit; + stats->q_opackets[i] = qstats.tx_pkts; + stats->q_obytes[i] = qstats.tx_octs; + stats->q_errors[i] = qstats.tx_drop_pkts; + } + + if (dev->rxq_stat_map[i] & (1U << 31)) { + qidx = dev->rxq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 1, &qstats); + if (rc) + goto exit; + stats->q_ipackets[i] = qstats.rx_pkts; + stats->q_ibytes[i] = qstats.rx_octs; + stats->q_errors[i] += qstats.rx_drop_pkts; + } + } +exit: + return rc; +} + +int +cnxk_nix_stats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_stats_reset(&dev->nix); +} + +int +cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (is_rx) { + if (queue_id >= dev->nb_rxq) + return -EINVAL; + dev->rxq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } else { + if (queue_id >= dev->nb_txq) + return -EINVAL; + dev->txq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } + + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 748a49a..e0ab9d6 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -12,7 +12,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', - 'cnxk_lookup.c') + 'cnxk_lookup.c', + 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 44/62] net/cnxk: add xstats apis 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (42 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram ` (18 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Initial implementation of xstats operations. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 5 ++ drivers/net/cnxk/cnxk_ethdev.h | 11 +++ drivers/net/cnxk/cnxk_stats.c | 132 ++++++++++++++++++++++++++++++++++ 6 files changed, 151 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 40952a9..192c15a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -34,6 +34,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 32035bb..e990480 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -32,6 +32,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8060a68..3a4417c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -29,6 +29,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f6eb9c9..aaf8719 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1170,6 +1170,11 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, .stats_get = cnxk_nix_stats_get, .stats_reset = cnxk_nix_stats_reset, + .xstats_get = cnxk_nix_xstats_get, + .xstats_get_names = cnxk_nix_xstats_get_names, + .xstats_reset = cnxk_nix_xstats_reset, + .xstats_get_by_id = cnxk_nix_xstats_get_by_id, + .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5075e7c..dd05c24 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -279,6 +279,17 @@ int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx); int cnxk_nix_stats_reset(struct rte_eth_dev *dev); int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); +int cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat *xstats, unsigned int n); +int cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit); +int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit); +int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n); +int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c index 24bff0b..ce9f9f4 100644 --- a/drivers/net/cnxk/cnxk_stats.c +++ b/drivers/net/cnxk/cnxk_stats.c @@ -83,3 +83,135 @@ cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, return 0; } + +int +cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat roc_xstats[n]; + int size, i; + + size = roc_nix_xstats_get(&dev->nix, roc_xstats, n); + + /* If requested array do not have space then return with count */ + if (size < 0 || size > (int)n) + return size; + + for (i = 0; i < size; i++) { + xstats[i].id = roc_xstats[i].id; + xstats[i].value = roc_xstats[i].value; + } + + return size; +} + +int +cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat_name roc_xstats_name[limit]; + struct roc_nix *nix = &dev->nix; + int size, i; + + if ((int)limit < roc_nix_num_xstats_get(nix) && xstats_names == NULL) + return roc_nix_num_xstats_get(nix); + + size = roc_nix_xstats_names_get(nix, roc_xstats_name, limit); + + for (i = 0; i < size; i++) + strlcpy(xstats_names[i].name, roc_xstats_name[i].name, + sizeof(xstats_names[i].name)); + + return size; +} + +int +cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat_name xnames[xstat_cnt]; + + if (limit < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (limit > xstat_cnt) + return -EINVAL; + + if (xstats_names == NULL) + return -ENOMEM; + + roc_nix_xstats_names_get(&dev->nix, xnames, limit); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + + strlcpy(xstats_names[i].name, xnames[ids[i]].name, + sizeof(xstats_names[i].name)); + } + + return limit; +} + +int +cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t xstat_cnt = roc_nix_num_xstats_get(&dev->nix), i; + struct roc_nix_xstat xstats[xstat_cnt]; + + if (n < xstat_cnt && ids == NULL) + return xstat_cnt; + + if (n > xstat_cnt) + return -EINVAL; + + if (values == NULL) + return -ENOMEM; + + roc_nix_xstats_get(&dev->nix, xstats, n); + + for (i = 0; i < xstat_cnt; i++) { + if (ids[i] >= xstat_cnt) + return -EINVAL; + values[i] = xstats[ids[i]].value; + } + + return n; +} + +int +cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0, i; + + rc = roc_nix_stats_reset(nix); + if (rc) + goto exit; + + /* Reset Rx Queues */ + for (i = 0; i < dev->nb_rxq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 1); + if (rc) + goto exit; + } + + /* Reset Tx Queues */ + for (i = 0; i < dev->nb_txq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 0); + if (rc) + goto exit; + } + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 45/62] net/cnxk: add rxq/txq info get operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (43 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 44/62] net/cnxk: add xstats apis Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram ` (17 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Initial apis to get default queue information. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index aaf8719..4372530 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1175,6 +1175,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .rxq_info_get = cnxk_nix_rxq_info_get, + .txq_info_get = cnxk_nix_txq_info_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index dd05c24..eeb6a53 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -290,6 +290,10 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo); +void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 5b8bc53..0bcba0c 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -622,3 +622,33 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } + +void +cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo) +{ + struct cnxk_eth_rxq_sp *rxq_sp = + ((struct cnxk_eth_rxq_sp *)eth_dev->data->rx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->mp = rxq_sp->qconf.mp; + qinfo->scattered_rx = eth_dev->data->scattered_rx; + qinfo->nb_desc = rxq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &rxq_sp->qconf.conf.rx, sizeof(qinfo->conf)); +} + +void +cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo) +{ + struct cnxk_eth_txq_sp *txq_sp = + ((struct cnxk_eth_txq_sp *)eth_dev->data->tx_queues[qid]) - 1; + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->nb_desc = txq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 46/62] net/cnxk: add device close and reset operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (44 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram ` (16 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements device close and reset operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 4372530..584aca3 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1137,6 +1137,9 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } +static int cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev); +static int cnxk_nix_dev_close(struct rte_eth_dev *eth_dev); + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, @@ -1148,6 +1151,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, .dev_stop = cnxk_nix_dev_stop, + .dev_close = cnxk_nix_dev_close, + .dev_reset = cnxk_nix_dev_reset, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, @@ -1288,7 +1293,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } static int -cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; @@ -1342,14 +1347,11 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* Check if mbox close is needed */ - if (!mbox_close) - return 0; - rc = roc_nix_dev_fini(nix); /* Can be freed later by PMD if NPA LF is in use */ if (rc == -EAGAIN) { - eth_dev->data->dev_private = NULL; + if (!reset) + eth_dev->data->dev_private = NULL; return 0; } else if (rc) { plt_err("Failed in nix dev fini, rc=%d", rc); @@ -1358,6 +1360,25 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) return rc; } +static int +cnxk_nix_dev_close(struct rte_eth_dev *eth_dev) +{ + cnxk_eth_dev_uninit(eth_dev, false); + return 0; +} + +static int +cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev) +{ + int rc; + + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + return cnxk_eth_dev_init(eth_dev); +} + int cnxk_nix_remove(struct rte_pci_device *pci_dev) { @@ -1368,7 +1389,7 @@ cnxk_nix_remove(struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (eth_dev) { /* Cleanup eth dev */ - rc = cnxk_eth_dev_uninit(eth_dev, true); + rc = cnxk_eth_dev_uninit(eth_dev, false); if (rc) return rc; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 47/62] net/cnxk: add pending Tx mbuf cleanup operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (45 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram ` (15 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are accumalated as a pending mbuf. Hence operation is NOP for cnxk platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 3 files changed, 12 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 584aca3..280ef42 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1182,6 +1182,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, + .tx_done_cleanup = cnxk_nix_tx_done_cleanup, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index eeb6a53..1ca52bc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -251,6 +251,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); +int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 0bcba0c..ff8afac 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -652,3 +652,13 @@ cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); } + +/* It is a NOP for cnxk as HW frees the buffer on xmit */ +int +cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) +{ + RTE_SET_USED(txq); + RTE_SET_USED(free_cnt); + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 48/62] net/cnxk: add support to configure npc 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (46 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram ` (14 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Kiran Kumar K <kirankumark@marvell.com> Adding support to configure NPC on device initialization. This involves reading the MKEX and initializing the necessary data. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 25 ++++++++++++++++++++++--- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 3 +++ 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 280ef42..b4e049b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -8,7 +8,8 @@ nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) { uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; - if (roc_nix_is_vf_or_sdp(&dev->nix)) + if (roc_nix_is_vf_or_sdp(&dev->nix) || + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; return capa; @@ -120,6 +121,7 @@ nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ if (roc_model_is_cn96_Ax() && + dev->npc.switch_header_type != ROC_PRIV_FLAGS_HIGIG && (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { fc_cfg.mode = (fc_cfg.mode == RTE_FC_FULL || @@ -419,8 +421,10 @@ cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, dev->ethdev_rss_hf = ethdev_rss; - if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + if (ethdev_rss & ETH_RSS_L2_PAYLOAD && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_LEN_90B) { flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + } if (ethdev_rss & ETH_RSS_C_VLAN) flowkey_cfg |= FLOW_KEY_TYPE_VLAN; @@ -820,11 +824,18 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) roc_nix_err_intr_ena_dis(nix, true); roc_nix_ras_intr_ena_dis(nix, true); - if (nix->rx_ptp_ena) { + if (nix->rx_ptp_ena && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { plt_err("Both PTP and switch header enabled"); goto free_nix_lf; } + rc = roc_nix_switch_hdr_set(nix, dev->npc.switch_header_type); + if (rc) { + plt_err("Failed to enable switch type nix_lf rc=%d", rc); + goto free_nix_lf; + } + /* Setup LSO if needed */ rc = nix_lso_fmt_setup(dev); if (rc) { @@ -1277,6 +1288,11 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->speed_capa = nix_get_speed_capa(dev); /* Initialize roc npc */ + dev->npc.roc_nix = nix; + rc = roc_npc_init(&dev->npc); + if (rc) + goto free_mac_addrs; + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, eth_dev->data->port_id, roc_nix_get_pf(nix), @@ -1310,6 +1326,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable and free rte_flow entries */ + roc_npc_fini(&dev->npc); + /* Disable link status events */ roc_nix_mac_link_event_start_stop(nix, false); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1ca52bc..e3b0bc1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -133,6 +133,9 @@ struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC NPC */ + struct roc_npc npc; + /* ROC RQs, SQs and CQs */ struct roc_nix_rq *rqs; struct roc_nix_sq *sqs; diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 4af2803..7fd06eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -150,6 +150,9 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->npc.flow_prealloc_size = flow_prealloc_size; + dev->npc.flow_max_priority = flow_max_priority; + dev->npc.switch_header_type = switch_header_type; return 0; exit: -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (47 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-15 12:45 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram ` (13 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Kiran Kumar K <kirankumark@marvell.com> Adding initial version of rte_flow support for cnxk family device. Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_flush, flow_query, flow_isolate. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- doc/guides/nics/cnxk.rst | 118 ++++++++++++++++ doc/guides/nics/features/cnxk.ini | 42 ++++++ drivers/net/cnxk/cnxk_rte_flow.c | 282 ++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 69 ++++++++++ drivers/net/cnxk/meson.build | 1 + 5 files changed, 512 insertions(+) create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index c2a6fbb..87401f0 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - MAC filtering +- Generic flow API - Inner and Outer Checksum offload - Port hardware statistics - Link state information @@ -222,3 +223,120 @@ Debugging Options +---+------------+-------------------------------------------------------+ | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | +---+------------+-------------------------------------------------------+ + +RTE Flow Support +---------------- + +The OCTEON CN9K/CN10K SoC family NIC has support for the following patterns and +actions. + +Patterns: + +.. _table_cnxk_supported_flow_item_types: + +.. table:: Item types + + +----+--------------------------------+ + | # | Pattern Type | + +====+================================+ + | 1 | RTE_FLOW_ITEM_TYPE_ETH | + +----+--------------------------------+ + | 2 | RTE_FLOW_ITEM_TYPE_VLAN | + +----+--------------------------------+ + | 3 | RTE_FLOW_ITEM_TYPE_E_TAG | + +----+--------------------------------+ + | 4 | RTE_FLOW_ITEM_TYPE_IPV4 | + +----+--------------------------------+ + | 5 | RTE_FLOW_ITEM_TYPE_IPV6 | + +----+--------------------------------+ + | 6 | RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4| + +----+--------------------------------+ + | 7 | RTE_FLOW_ITEM_TYPE_MPLS | + +----+--------------------------------+ + | 8 | RTE_FLOW_ITEM_TYPE_ICMP | + +----+--------------------------------+ + | 9 | RTE_FLOW_ITEM_TYPE_UDP | + +----+--------------------------------+ + | 10 | RTE_FLOW_ITEM_TYPE_TCP | + +----+--------------------------------+ + | 11 | RTE_FLOW_ITEM_TYPE_SCTP | + +----+--------------------------------+ + | 12 | RTE_FLOW_ITEM_TYPE_ESP | + +----+--------------------------------+ + | 13 | RTE_FLOW_ITEM_TYPE_GRE | + +----+--------------------------------+ + | 14 | RTE_FLOW_ITEM_TYPE_NVGRE | + +----+--------------------------------+ + | 15 | RTE_FLOW_ITEM_TYPE_VXLAN | + +----+--------------------------------+ + | 16 | RTE_FLOW_ITEM_TYPE_GTPC | + +----+--------------------------------+ + | 17 | RTE_FLOW_ITEM_TYPE_GTPU | + +----+--------------------------------+ + | 18 | RTE_FLOW_ITEM_TYPE_GENEVE | + +----+--------------------------------+ + | 19 | RTE_FLOW_ITEM_TYPE_VXLAN_GPE | + +----+--------------------------------+ + | 20 | RTE_FLOW_ITEM_TYPE_IPV6_EXT | + +----+--------------------------------+ + | 21 | RTE_FLOW_ITEM_TYPE_VOID | + +----+--------------------------------+ + | 22 | RTE_FLOW_ITEM_TYPE_ANY | + +----+--------------------------------+ + | 23 | RTE_FLOW_ITEM_TYPE_GRE_KEY | + +----+--------------------------------+ + | 24 | RTE_FLOW_ITEM_TYPE_HIGIG2 | + +----+--------------------------------+ + +.. note:: + + ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing + bits in the GRE header are equal to 0. + +Actions: + +.. _table_cnxk_supported_ingress_action_types: + +.. table:: Ingress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_VOID | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_MARK | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_FLAG | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 6 | RTE_FLOW_ACTION_TYPE_QUEUE | + +----+-----------------------------------------+ + | 7 | RTE_FLOW_ACTION_TYPE_RSS | + +----+-----------------------------------------+ + | 8 | RTE_FLOW_ACTION_TYPE_PF | + +----+-----------------------------------------+ + | 9 | RTE_FLOW_ACTION_TYPE_VF | + +----+-----------------------------------------+ + | 10 | RTE_FLOW_ACTION_TYPE_OF_POP_VLAN | + +----+-----------------------------------------+ + +.. _table_cnxk_supported_egress_action_types: + +.. table:: Egress action types + + +----+-----------------------------------------+ + | # | Action Type | + +====+=========================================+ + | 1 | RTE_FLOW_ACTION_TYPE_COUNT | + +----+-----------------------------------------+ + | 2 | RTE_FLOW_ACTION_TYPE_DROP | + +----+-----------------------------------------+ + | 3 | RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN | + +----+-----------------------------------------+ + | 4 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID | + +----+-----------------------------------------+ + | 5 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP | + +----+-----------------------------------------+ diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 192c15a..3c59494 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -39,3 +39,45 @@ Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y + +[rte_flow items] +any = Y +arp_eth_ipv4 = Y +esp = Y +eth = Y +e_tag = Y +geneve = Y +gre = Y +gre_key = Y +gtpc = Y +gtpu = Y +higig2 = Y +icmp = Y +ipv4 = Y +ipv6 = Y +ipv6_ext = Y +mpls = Y +nvgre = Y +raw = Y +sctp = Y +tcp = Y +udp = Y +vlan = Y +vxlan = Y +vxlan_gpe = Y + +[rte_flow actions] +count = Y +drop = Y +flag = Y +mark = Y +of_pop_vlan = Y +of_push_vlan = Y +of_set_vlan_pcp = Y +of_set_vlan_vid = Y +pf = Y +port_id = Y +queue = Y +rss = Y +security = Y +vf = Y diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c new file mode 100644 index 0000000..d0e7bdc --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -0,0 +1,282 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_rte_flow.h> + +static int +cnxk_map_actions(struct rte_eth_dev *dev, + const struct rte_flow_action actions[], + struct roc_npc_action in_actions[]) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + const struct rte_flow_action_count *act_count; + const struct rte_flow_action_queue *act_q; + int rq; + int i = 0; + + RTE_SET_USED(hw); + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VOID; + break; + + case RTE_FLOW_ACTION_TYPE_MARK: + in_actions[i].type = ROC_NPC_ACTION_TYPE_MARK; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_FLAG: + in_actions[i].type = ROC_NPC_ACTION_TYPE_FLAG; + break; + + case RTE_FLOW_ACTION_TYPE_COUNT: + act_count = (const struct rte_flow_action_count *) + actions->conf; + + if (act_count->shared == 1) { + plt_npc_dbg("Shared counter is not supported"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_COUNT; + break; + + case RTE_FLOW_ACTION_TYPE_DROP: + in_actions[i].type = ROC_NPC_ACTION_TYPE_DROP; + break; + + case RTE_FLOW_ACTION_TYPE_PF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_PF; + break; + + case RTE_FLOW_ACTION_TYPE_VF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VF; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_QUEUE: + act_q = (const struct rte_flow_action_queue *) + actions->conf; + rq = act_q->index; + if (rq >= dev->data->nb_rx_queues) { + plt_npc_dbg("Invalid queue index"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_QUEUE; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_RSS: + in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + break; + + case RTE_FLOW_ACTION_TYPE_SECURITY: + in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; + break; + default: + plt_npc_dbg("Action is not supported = %d", + actions->type); + goto err_exit; + } + i++; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_END; + return 0; + +err_exit: + return -EINVAL; +} + +static int +cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct roc_npc_attr *in_attr, + struct roc_npc_item_info in_pattern[], + struct roc_npc_action in_actions[]) +{ + int i = 0; + + in_attr->priority = attr->priority; + in_attr->ingress = attr->ingress; + in_attr->egress = attr->egress; + + while (pattern->type != RTE_FLOW_ITEM_TYPE_END) { + in_pattern[i].spec = pattern->spec; + in_pattern[i].last = pattern->last; + in_pattern[i].mask = pattern->mask; + in_pattern[i].type = term[pattern->type].item_type; + in_pattern[i].size = term[pattern->type].item_size; + pattern++; + i++; + } + in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; + + return cnxk_map_actions(dev, actions, in_actions); +} + +static int +cnxk_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow flow; + int rc; + + memset(&flow, 0, sizeof(flow)); + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return rc; + } + + return roc_npc_flow_parse(npc, &in_attr, in_pattern, in_actions, &flow); +} + +static struct rte_flow * +cnxk_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct roc_npc *npc = &hw->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow *flow; + int errcode; + int rc; + + rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return NULL; + } + + flow = roc_npc_flow_create(npc, &in_attr, in_pattern, in_actions, + &errcode); + if (errcode != 0) { + rte_flow_error_set(error, errcode, errcode, NULL, + roc_error_msg_get(errcode)); + return NULL; + } + + return (struct rte_flow *)flow; +} + +static int +cnxk_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_flow_destroy(npc, in_flow); + if (rc) + rte_flow_error_set(error, rc, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow Destroy failed"); + return rc; +} + +static int +cnxk_flow_flush(struct rte_eth_dev *dev, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + int rc; + + rc = roc_npc_mcam_free_all_resources(npc); + if (rc) { + rte_flow_error_set(error, EIO, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Failed to flush filter"); + return -rte_errno; + } + + return 0; +} + +static int +cnxk_flow_query(struct rte_eth_dev *dev, struct rte_flow *flow, + const struct rte_flow_action *action, void *data, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + struct rte_flow_query_count *query = data; + const char *errmsg = NULL; + int errcode = ENOTSUP; + int rc; + + if (action->type != RTE_FLOW_ACTION_TYPE_COUNT) { + errmsg = "Only COUNT is supported in query"; + goto err_exit; + } + + if (in_flow->ctr_id == NPC_COUNTER_NONE) { + errmsg = "Counter is not available"; + goto err_exit; + } + + rc = roc_npc_mcam_read_counter(npc, in_flow->ctr_id, &query->hits); + if (rc != 0) { + errcode = EIO; + errmsg = "Error reading flow counter"; + goto err_exit; + } + query->hits_set = 1; + query->bytes_set = 0; + + if (query->reset) + rc = roc_npc_mcam_clear_counter(npc, in_flow->ctr_id); + if (rc != 0) { + errcode = EIO; + errmsg = "Error clearing flow counter"; + goto err_exit; + } + + return 0; + +err_exit: + rte_flow_error_set(error, errcode, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, errmsg); + return -rte_errno; +} + +static int +cnxk_flow_isolate(struct rte_eth_dev *dev __rte_unused, int enable __rte_unused, + struct rte_flow_error *error) +{ + /* If we support, we need to un-install the default mcam + * entry for this port. + */ + + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow isolation not supported"); + + return -rte_errno; +} + +const struct rte_flow_ops cnxk_flow_ops = { + .validate = cnxk_flow_validate, + .create = cnxk_flow_create, + .destroy = cnxk_flow_destroy, + .flush = cnxk_flow_flush, + .query = cnxk_flow_query, + .isolate = cnxk_flow_isolate, +}; diff --git a/drivers/net/cnxk/cnxk_rte_flow.h b/drivers/net/cnxk/cnxk_rte_flow.h new file mode 100644 index 0000000..3740226 --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_RTE_FLOW_H__ +#define __CNXK_RTE_FLOW_H__ + +#include <rte_flow_driver.h> +#include <rte_malloc.h> + +#include "cnxk_ethdev.h" +#include "roc_api.h" +#include "roc_npc_priv.h" + +struct cnxk_rte_flow_term_info { + uint16_t item_type; + uint16_t item_size; +}; + +struct cnxk_rte_flow_term_info term[] = { + [RTE_FLOW_ITEM_TYPE_ETH] = {ROC_NPC_ITEM_TYPE_ETH, + sizeof(struct rte_flow_item_eth)}, + [RTE_FLOW_ITEM_TYPE_VLAN] = {ROC_NPC_ITEM_TYPE_VLAN, + sizeof(struct rte_flow_item_vlan)}, + [RTE_FLOW_ITEM_TYPE_E_TAG] = {ROC_NPC_ITEM_TYPE_E_TAG, + sizeof(struct rte_flow_item_e_tag)}, + [RTE_FLOW_ITEM_TYPE_IPV4] = {ROC_NPC_ITEM_TYPE_IPV4, + sizeof(struct rte_flow_item_ipv4)}, + [RTE_FLOW_ITEM_TYPE_IPV6] = {ROC_NPC_ITEM_TYPE_IPV6, + sizeof(struct rte_flow_item_ipv6)}, + [RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4] = {ROC_NPC_ITEM_TYPE_ARP_ETH_IPV4, + sizeof(struct rte_flow_item_arp_eth_ipv4)}, + [RTE_FLOW_ITEM_TYPE_MPLS] = {ROC_NPC_ITEM_TYPE_MPLS, + sizeof(struct rte_flow_item_mpls)}, + [RTE_FLOW_ITEM_TYPE_ICMP] = {ROC_NPC_ITEM_TYPE_ICMP, + sizeof(struct rte_flow_item_icmp)}, + [RTE_FLOW_ITEM_TYPE_UDP] = {ROC_NPC_ITEM_TYPE_UDP, + sizeof(struct rte_flow_item_udp)}, + [RTE_FLOW_ITEM_TYPE_TCP] = {ROC_NPC_ITEM_TYPE_TCP, + sizeof(struct rte_flow_item_tcp)}, + [RTE_FLOW_ITEM_TYPE_SCTP] = {ROC_NPC_ITEM_TYPE_SCTP, + sizeof(struct rte_flow_item_sctp)}, + [RTE_FLOW_ITEM_TYPE_ESP] = {ROC_NPC_ITEM_TYPE_ESP, + sizeof(struct rte_flow_item_esp)}, + [RTE_FLOW_ITEM_TYPE_GRE] = {ROC_NPC_ITEM_TYPE_GRE, + sizeof(struct rte_flow_item_gre)}, + [RTE_FLOW_ITEM_TYPE_NVGRE] = {ROC_NPC_ITEM_TYPE_NVGRE, + sizeof(struct rte_flow_item_nvgre)}, + [RTE_FLOW_ITEM_TYPE_VXLAN] = {ROC_NPC_ITEM_TYPE_VXLAN, + sizeof(struct rte_flow_item_vxlan)}, + [RTE_FLOW_ITEM_TYPE_GTPC] = {ROC_NPC_ITEM_TYPE_GTPC, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GTPU] = {ROC_NPC_ITEM_TYPE_GTPU, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GENEVE] = {ROC_NPC_ITEM_TYPE_GENEVE, + sizeof(struct rte_flow_item_geneve)}, + [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = {ROC_NPC_ITEM_TYPE_VXLAN_GPE, + sizeof(struct rte_flow_item_vxlan_gpe)}, + [RTE_FLOW_ITEM_TYPE_IPV6_EXT] = {ROC_NPC_ITEM_TYPE_IPV6_EXT, + sizeof(struct rte_flow_item_ipv6_ext)}, + [RTE_FLOW_ITEM_TYPE_VOID] = {ROC_NPC_ITEM_TYPE_VOID, 0}, + [RTE_FLOW_ITEM_TYPE_ANY] = {ROC_NPC_ITEM_TYPE_ANY, 0}, + [RTE_FLOW_ITEM_TYPE_GRE_KEY] = {ROC_NPC_ITEM_TYPE_GRE_KEY, + sizeof(uint32_t)}, + [RTE_FLOW_ITEM_TYPE_HIGIG2] = { + ROC_NPC_ITEM_TYPE_HIGIG2, + sizeof(struct rte_flow_item_higig2_hdr)} +}; + +#endif /* __CNXK_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index e0ab9d6..92ef8f0 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_rte_flow.c', 'cnxk_stats.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram @ 2021-06-15 12:45 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:45 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:39 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Kiran Kumar K <kirankumark@marvell.com> > > Adding initial version of rte_flow support for cnxk family device. > Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, > flow_flush, flow_query, flow_isolate. > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > --- > doc/guides/nics/cnxk.rst | 118 ++++++++++++++++ > doc/guides/nics/features/cnxk.ini | 42 ++++++ > drivers/net/cnxk/cnxk_rte_flow.c | 282 ++++++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_rte_flow.h | 69 ++++++++++ > drivers/net/cnxk/meson.build | 1 + > 5 files changed, 512 insertions(+) > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h > > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst > index c2a6fbb..87401f0 100644 > --- a/doc/guides/nics/cnxk.rst > +++ b/doc/guides/nics/cnxk.rst > @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: > - Multiple queues for TX and RX > - Receiver Side Scaling (RSS) > - MAC filtering > +- Generic flow API > - Inner and Outer Checksum offload > - Port hardware statistics > - Link state information > @@ -222,3 +223,120 @@ Debugging Options > +---+------------+-------------------------------------------------------+ > | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | > +---+------------+-------------------------------------------------------+ > + > +RTE Flow Support > +---------------- > + > +The OCTEON CN9K/CN10K SoC family NIC has support for the following patterns and > +actions. > + > +Patterns: > + > +.. _table_cnxk_supported_flow_item_types: > + > +.. table:: Item types > + > + +----+--------------------------------+ > + | # | Pattern Type | > + +====+================================+ > + | 1 | RTE_FLOW_ITEM_TYPE_ETH | > + +----+--------------------------------+ > + | 2 | RTE_FLOW_ITEM_TYPE_VLAN | > + +----+--------------------------------+ > + | 3 | RTE_FLOW_ITEM_TYPE_E_TAG | > + +----+--------------------------------+ > + | 4 | RTE_FLOW_ITEM_TYPE_IPV4 | > + +----+--------------------------------+ > + | 5 | RTE_FLOW_ITEM_TYPE_IPV6 | > + +----+--------------------------------+ > + | 6 | RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4| > + +----+--------------------------------+ > + | 7 | RTE_FLOW_ITEM_TYPE_MPLS | > + +----+--------------------------------+ > + | 8 | RTE_FLOW_ITEM_TYPE_ICMP | > + +----+--------------------------------+ > + | 9 | RTE_FLOW_ITEM_TYPE_UDP | > + +----+--------------------------------+ > + | 10 | RTE_FLOW_ITEM_TYPE_TCP | > + +----+--------------------------------+ > + | 11 | RTE_FLOW_ITEM_TYPE_SCTP | > + +----+--------------------------------+ > + | 12 | RTE_FLOW_ITEM_TYPE_ESP | > + +----+--------------------------------+ > + | 13 | RTE_FLOW_ITEM_TYPE_GRE | > + +----+--------------------------------+ > + | 14 | RTE_FLOW_ITEM_TYPE_NVGRE | > + +----+--------------------------------+ > + | 15 | RTE_FLOW_ITEM_TYPE_VXLAN | > + +----+--------------------------------+ > + | 16 | RTE_FLOW_ITEM_TYPE_GTPC | > + +----+--------------------------------+ > + | 17 | RTE_FLOW_ITEM_TYPE_GTPU | > + +----+--------------------------------+ > + | 18 | RTE_FLOW_ITEM_TYPE_GENEVE | > + +----+--------------------------------+ > + | 19 | RTE_FLOW_ITEM_TYPE_VXLAN_GPE | > + +----+--------------------------------+ > + | 20 | RTE_FLOW_ITEM_TYPE_IPV6_EXT | > + +----+--------------------------------+ > + | 21 | RTE_FLOW_ITEM_TYPE_VOID | > + +----+--------------------------------+ > + | 22 | RTE_FLOW_ITEM_TYPE_ANY | > + +----+--------------------------------+ > + | 23 | RTE_FLOW_ITEM_TYPE_GRE_KEY | > + +----+--------------------------------+ > + | 24 | RTE_FLOW_ITEM_TYPE_HIGIG2 | > + +----+--------------------------------+ > + > +.. note:: > + > + ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing > + bits in the GRE header are equal to 0. > + > +Actions: > + > +.. _table_cnxk_supported_ingress_action_types: > + > +.. table:: Ingress action types > + > + +----+-----------------------------------------+ > + | # | Action Type | > + +====+=========================================+ > + | 1 | RTE_FLOW_ACTION_TYPE_VOID | > + +----+-----------------------------------------+ > + | 2 | RTE_FLOW_ACTION_TYPE_MARK | > + +----+-----------------------------------------+ > + | 3 | RTE_FLOW_ACTION_TYPE_FLAG | > + +----+-----------------------------------------+ > + | 4 | RTE_FLOW_ACTION_TYPE_COUNT | > + +----+-----------------------------------------+ > + | 5 | RTE_FLOW_ACTION_TYPE_DROP | > + +----+-----------------------------------------+ > + | 6 | RTE_FLOW_ACTION_TYPE_QUEUE | > + +----+-----------------------------------------+ > + | 7 | RTE_FLOW_ACTION_TYPE_RSS | > + +----+-----------------------------------------+ > + | 8 | RTE_FLOW_ACTION_TYPE_PF | > + +----+-----------------------------------------+ > + | 9 | RTE_FLOW_ACTION_TYPE_VF | > + +----+-----------------------------------------+ > + | 10 | RTE_FLOW_ACTION_TYPE_OF_POP_VLAN | > + +----+-----------------------------------------+ > + > +.. _table_cnxk_supported_egress_action_types: > + > +.. table:: Egress action types > + > + +----+-----------------------------------------+ > + | # | Action Type | > + +====+=========================================+ > + | 1 | RTE_FLOW_ACTION_TYPE_COUNT | > + +----+-----------------------------------------+ > + | 2 | RTE_FLOW_ACTION_TYPE_DROP | > + +----+-----------------------------------------+ > + | 3 | RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN | > + +----+-----------------------------------------+ > + | 4 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID | > + +----+-----------------------------------------+ > + | 5 | RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP | > + +----+-----------------------------------------+ See below, > diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini > index 192c15a..3c59494 100644 > --- a/doc/guides/nics/features/cnxk.ini > +++ b/doc/guides/nics/features/cnxk.ini > @@ -39,3 +39,45 @@ Module EEPROM dump = Y > Linux = Y > ARMv8 = Y > Usage doc = Y > + > +[rte_flow items] > +any = Y > +arp_eth_ipv4 = Y > +esp = Y > +eth = Y > +e_tag = Y > +geneve = Y > +gre = Y > +gre_key = Y > +gtpc = Y > +gtpu = Y > +higig2 = Y > +icmp = Y > +ipv4 = Y > +ipv6 = Y > +ipv6_ext = Y > +mpls = Y > +nvgre = Y > +raw = Y > +sctp = Y > +tcp = Y > +udp = Y > +vlan = Y > +vxlan = Y > +vxlan_gpe = Y > + > +[rte_flow actions] > +count = Y > +drop = Y > +flag = Y > +mark = Y > +of_pop_vlan = Y > +of_push_vlan = Y > +of_set_vlan_pcp = Y > +of_set_vlan_vid = Y > +pf = Y > +port_id = Y > +queue = Y > +rss = Y > +security = Y > +vf = Y Now that we have this generic doc, The above driver-specific documentation can be removed. We can just keep limitations in the driver documentation. ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 50/62] net/cnxk: add flow ops get operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (48 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram ` (12 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> This patch adds flow ops get operation to enable rte_flow_ops. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 3 +++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++- drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 4 files changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index a69be4f..dddb085 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -785,6 +785,8 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, struct npc_flow_list *list; int rc; + npc->channel = roc_npc->channel; + flow = plt_zmalloc(sizeof(*flow), 0); if (flow == NULL) { *errcode = NPC_ERR_NO_MEM; diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b4e049b..8a5e65f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -790,6 +790,8 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto fail_configure; } + dev->npc.channel = roc_nix_get_base_chan(nix); + nb_rxq = data->nb_rx_queues; nb_txq = data->nb_tx_queues; rc = -ENOMEM; @@ -1194,6 +1196,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, + .flow_ops_get = cnxk_nix_flow_ops_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e3b0bc1..3d4c4ae 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -218,6 +218,8 @@ cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; +extern const struct rte_flow_ops cnxk_flow_ops; + /* Ops */ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); @@ -255,7 +257,8 @@ int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); - +int cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index ff8afac..50d1b5b 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -294,6 +294,16 @@ cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, } int +cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops) +{ + RTE_SET_USED(eth_dev); + + *ops = &cnxk_flow_ops; + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (49 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-15 12:47 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 52/62] net/cnxk: add get register operation Nithin Dabilpuram ` (11 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> Add callback to get ethdev firmware version. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 6 files changed, 25 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 3c59494..b9c11b6 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -35,6 +35,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e990480..6b9ce2d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -33,6 +33,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 3a4417c..5629d07 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -30,6 +30,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8a5e65f..d12e68d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1193,6 +1193,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .fw_version_get = cnxk_nix_fw_version_get, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 3d4c4ae..38ef0a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -297,6 +297,8 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +int cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size); void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo); void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 50d1b5b..dbc7498 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -633,6 +633,25 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } +int +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const char *str = roc_npc_profile_name_get(&dev->npc); + uint32_t size = strlen(str) + 1; + + if (fw_size > size) + fw_size = size; + + strlcpy(fw_version, str, fw_size); + + if (fw_size < size) + return size; + + return 0; +} + void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo) -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram @ 2021-06-15 12:47 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 12:47 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:39 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Satha Rao <skoteshwar@marvell.com> > > Add callback to get ethdev firmware version. > > Signed-off-by: Satha Rao <skoteshwar@marvell.com> > +int > +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, > + size_t fw_size) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + const char *str = roc_npc_profile_name_get(&dev->npc); > + uint32_t size = strlen(str) + 1; > + > + if (fw_size > size) > + fw_size = size; > + > + strlcpy(fw_version, str, fw_size); use rte_strlcpy instead. > + > + if (fw_size < size) > + return size; > + > + return 0; > +} > + > void > cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, > struct rte_eth_rxq_info *qinfo) > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 52/62] net/cnxk: add get register operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (50 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow Nithin Dabilpuram ` (10 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> With this patch implemented api to dump platform registers for debug purposes. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 25 +++++++++++++++++++++++++ 7 files changed, 34 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 87401f0..98bcb51 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -32,6 +32,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Debug utilities - Context dump and error interrupt support - Support Rx interrupt Prerequisites diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b9c11b6..861b7c1 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -37,6 +37,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6b9ce2d..65ee8ba 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -35,6 +35,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5629d07..00bde9b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -32,6 +32,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index d12e68d..a40625e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1198,6 +1198,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, + .get_reg = cnxk_nix_dev_get_reg, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 38ef0a9..e5a7e98 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -312,6 +312,10 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Debug */ +int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, + struct rte_dev_reg_info *regs); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index dbc7498..a15d87c 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -691,3 +691,28 @@ cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) return 0; } + +int +cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t *data = regs->data; + int rc = -ENOTSUP; + + if (data == NULL) { + rc = roc_nix_lf_get_reg_count(nix); + if (rc > 0) { + regs->length = rc; + regs->width = 8; + rc = 0; + } + return rc; + } + + if (!regs->length || + regs->length == (uint32_t)roc_nix_lf_get_reg_count(nix)) + return roc_nix_lf_reg_dump(nix, data); + + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (51 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 52/62] net/cnxk: add get register operation Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-15 14:38 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram ` (9 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> Added support for rss action in rte_flow code based on ROC. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/net/cnxk/cnxk_rte_flow.c | 72 +++++++++++++++++++++++++++++++++++----- 1 file changed, 64 insertions(+), 8 deletions(-) diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index d0e7bdc..8486e9e 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -4,15 +4,64 @@ #include <cnxk_rte_flow.h> static int -cnxk_map_actions(struct rte_eth_dev *dev, +npc_rss_action_validate(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_action *act) +{ + const struct rte_flow_action_rss *rss; + + rss = (const struct rte_flow_action_rss *)act->conf; + + if (attr->egress) { + plt_err("No support of RSS in egress"); + return -EINVAL; + } + + if (dev->data->dev_conf.rxmode.mq_mode != ETH_MQ_RX_RSS) { + plt_err("multi-queue mode is disabled"); + return -ENOTSUP; + } + + if (!rss || !rss->queue_num) { + plt_err("no valid queues"); + return -EINVAL; + } + + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT) { + plt_err("non-default RSS hash functions are not supported"); + return -ENOTSUP; + } + + if (rss->key_len && rss->key_len > ROC_NIX_RSS_KEY_LEN) { + plt_err("RSS hash key too large"); + return -ENOTSUP; + } + + return 0; +} + +static void +npc_rss_flowkey_get(struct cnxk_eth_dev *dev, + const struct roc_npc_action *rss_action, + uint32_t *flowkey_cfg) +{ + const struct roc_npc_action_rss *rss; + + rss = (const struct roc_npc_action_rss *)rss_action->conf; + + *flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss->types, rss->level); +} + +static int +cnxk_map_actions(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { struct cnxk_eth_dev *hw = dev->data->dev_private; const struct rte_flow_action_count *act_count; const struct rte_flow_action_queue *act_q; + int i = 0, rc = 0; int rq; - int i = 0; RTE_SET_USED(hw); @@ -68,7 +117,12 @@ cnxk_map_actions(struct rte_eth_dev *dev, break; case RTE_FLOW_ACTION_TYPE_RSS: + rc = npc_rss_action_validate(dev, attr, actions); + if (rc) + goto err_exit; in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + in_actions[i].conf = actions->conf; + npc_rss_flowkey_get(hw, &in_actions[i], flowkey_cfg); break; case RTE_FLOW_ACTION_TYPE_SECURITY: @@ -94,7 +148,7 @@ cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], struct roc_npc_attr *in_attr, struct roc_npc_item_info in_pattern[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { int i = 0; @@ -113,7 +167,7 @@ cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, } in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; - return cnxk_map_actions(dev, actions, in_actions); + return cnxk_map_actions(dev, attr, actions, in_actions, flowkey_cfg); } static int @@ -128,12 +182,13 @@ cnxk_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, struct roc_npc *npc = &hw->npc; struct roc_npc_attr in_attr; struct roc_npc_flow flow; + uint32_t flowkey_cfg = 0; int rc; memset(&flow, 0, sizeof(flow)); rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, &flowkey_cfg); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); @@ -155,11 +210,12 @@ cnxk_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, struct roc_npc *npc = &hw->npc; struct roc_npc_attr in_attr; struct roc_npc_flow *flow; - int errcode; + int errcode = 0; int rc; rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, + &npc->flowkey_cfg_state); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow Nithin Dabilpuram @ 2021-06-15 14:38 ` Jerin Jacob 0 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-15 14:38 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:39 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > From: Satheesh Paul <psatheesh@marvell.com> > > Added support for rss action in rte_flow code based on ROC. > > Signed-off-by: Satheesh Paul <psatheesh@marvell.com> > --- > drivers/net/cnxk/cnxk_rte_flow.c | 72 +++++++++++++++++++++++++++++++++++----- Looks like we missed updating doc/guides/nics/features/cnxk.ini for this rss action. > 1 file changed, 64 insertions(+), 8 deletions(-) > > diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c > index d0e7bdc..8486e9e 100644 > --- a/drivers/net/cnxk/cnxk_rte_flow.c > +++ b/drivers/net/cnxk/cnxk_rte_flow.c > @@ -4,15 +4,64 @@ > #include <cnxk_rte_flow.h> > > static int > -cnxk_map_actions(struct rte_eth_dev *dev, > +npc_rss_action_validate(struct rte_eth_dev *dev, > + const struct rte_flow_attr *attr, > + const struct rte_flow_action *act) > +{ > + const struct rte_flow_action_rss *rss; > + > + rss = (const struct rte_flow_action_rss *)act->conf; > + > + if (attr->egress) { > + plt_err("No support of RSS in egress"); > + return -EINVAL; > + } > + > + if (dev->data->dev_conf.rxmode.mq_mode != ETH_MQ_RX_RSS) { > + plt_err("multi-queue mode is disabled"); > + return -ENOTSUP; > + } > + > + if (!rss || !rss->queue_num) { > + plt_err("no valid queues"); > + return -EINVAL; > + } > + > + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT) { > + plt_err("non-default RSS hash functions are not supported"); > + return -ENOTSUP; > + } > + > + if (rss->key_len && rss->key_len > ROC_NIX_RSS_KEY_LEN) { > + plt_err("RSS hash key too large"); > + return -ENOTSUP; > + } > + > + return 0; > +} > + > +static void > +npc_rss_flowkey_get(struct cnxk_eth_dev *dev, > + const struct roc_npc_action *rss_action, > + uint32_t *flowkey_cfg) > +{ > + const struct roc_npc_action_rss *rss; > + > + rss = (const struct roc_npc_action_rss *)rss_action->conf; > + > + *flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss->types, rss->level); > +} > + > +static int > +cnxk_map_actions(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, > const struct rte_flow_action actions[], > - struct roc_npc_action in_actions[]) > + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) > { > struct cnxk_eth_dev *hw = dev->data->dev_private; > const struct rte_flow_action_count *act_count; > const struct rte_flow_action_queue *act_q; > + int i = 0, rc = 0; > int rq; > - int i = 0; > > RTE_SET_USED(hw); > > @@ -68,7 +117,12 @@ cnxk_map_actions(struct rte_eth_dev *dev, > break; > > case RTE_FLOW_ACTION_TYPE_RSS: > + rc = npc_rss_action_validate(dev, attr, actions); > + if (rc) > + goto err_exit; > in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; > + in_actions[i].conf = actions->conf; > + npc_rss_flowkey_get(hw, &in_actions[i], flowkey_cfg); > break; > > case RTE_FLOW_ACTION_TYPE_SECURITY: > @@ -94,7 +148,7 @@ cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, > const struct rte_flow_action actions[], > struct roc_npc_attr *in_attr, > struct roc_npc_item_info in_pattern[], > - struct roc_npc_action in_actions[]) > + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) > { > int i = 0; > > @@ -113,7 +167,7 @@ cnxk_map_flow_data(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, > } > in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; > > - return cnxk_map_actions(dev, actions, in_actions); > + return cnxk_map_actions(dev, attr, actions, in_actions, flowkey_cfg); > } > > static int > @@ -128,12 +182,13 @@ cnxk_flow_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, > struct roc_npc *npc = &hw->npc; > struct roc_npc_attr in_attr; > struct roc_npc_flow flow; > + uint32_t flowkey_cfg = 0; > int rc; > > memset(&flow, 0, sizeof(flow)); > > rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, > - in_pattern, in_actions); > + in_pattern, in_actions, &flowkey_cfg); > if (rc) { > rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, > NULL, "Failed to map flow data"); > @@ -155,11 +210,12 @@ cnxk_flow_create(struct rte_eth_dev *dev, const struct rte_flow_attr *attr, > struct roc_npc *npc = &hw->npc; > struct roc_npc_attr in_attr; > struct roc_npc_flow *flow; > - int errcode; > + int errcode = 0; > int rc; > > rc = cnxk_map_flow_data(dev, attr, pattern, actions, &in_attr, > - in_pattern, in_actions); > + in_pattern, in_actions, > + &npc->flowkey_cfg_state); > if (rc) { > rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, > NULL, "Failed to map flow data"); > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 54/62] net/cnxk: register callback to get PTP status 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (52 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram ` (8 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Once PTP status is changed at H/W i.e. enable/disable then it is propagated to user via registered callback. So corresponding callback is registered to get PTP status. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 87 ++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx.h | 1 + drivers/net/cnxk/cn9k_ethdev.c | 73 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 +- drivers/net/cnxk/cnxk_ethdev.h | 5 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 2 + 7 files changed, 166 insertions(+), 5 deletions(-) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index fa7b835..7f02b6b 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -266,6 +266,76 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn10k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq) - 1; + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn10k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cant be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -317,6 +387,7 @@ static int cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; int rc; if (RTE_CACHE_LINE_SIZE != 64) { @@ -337,15 +408,23 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (rc) return rc; + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - eth_dev = rte_eth_dev_allocated(pci_dev->device.name); - if (!eth_dev) - return -ENOENT; - /* Setup callbacks for secondary process */ cn10k_eth_set_tx_function(eth_dev); cn10k_eth_set_rx_function(eth_dev); + return 0; } + + dev = cnxk_eth_pmd_priv(eth_dev); + + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn10k_nix_ptp_info_update_cb); + return 0; } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 7bb9dd8..29ee0ac 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -12,6 +12,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index e47fa9a..798a03d 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -275,6 +275,76 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn9k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = ((struct cnxk_eth_rxq_sp *)rxq) - 1; + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn9k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cant be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -379,6 +449,9 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) dev->hwcap = 0; + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn9k_nix_ptp_info_update_cb); + /* Update HW erratas */ if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) dev->cq_min_4k = 1; diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index bc04f5c..f4b3282 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -13,6 +13,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a40625e..afb5b56 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -59,7 +59,7 @@ nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) } } -static int +int nix_recalc_mtu(struct rte_eth_dev *eth_dev) { struct rte_eth_dev_data *data = eth_dev->data; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e5a7e98..b070df7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -100,6 +100,7 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +#define CNXK_NIX_TIMESYNC_RX_OFFSET 8 #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -153,6 +154,7 @@ struct cnxk_eth_dev { uint16_t flags; uint8_t ptype_disable; bool scalar_ena; + bool ptp_en; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -316,6 +318,9 @@ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs); +/* Other private functions */ +int nix_recalc_mtu(struct rte_eth_dev *eth_dev); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index a15d87c..d437e2c 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -379,6 +379,8 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) int rc = -EINVAL; uint32_t buffsz; + frame_size += CNXK_NIX_TIMESYNC_RX_OFFSET * dev->ptp_en; + /* Check if MTU is within the allowed range */ if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { plt_err("MTU is lesser than minimum"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (53 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-08 12:04 ` Pavan Nikhilesh Bhagavatula 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram ` (7 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Base PTP timesync support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 31 +++++ drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_rx.c | 32 ++--- drivers/net/cnxk/cn10k_rx.h | 61 +++++++--- drivers/net/cnxk/cn10k_rx_mseg.c | 2 +- drivers/net/cnxk/cn10k_rx_vec.c | 5 +- drivers/net/cnxk/cn10k_tx.c | 28 +++-- drivers/net/cnxk/cn10k_tx.h | 192 +++++++++++++++++++++++------ drivers/net/cnxk/cn10k_tx_mseg.c | 2 +- drivers/net/cnxk/cn10k_tx_vec.c | 3 +- drivers/net/cnxk/cn9k_ethdev.c | 34 +++++- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_rx.c | 32 ++--- drivers/net/cnxk/cn9k_rx.h | 61 +++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 2 +- drivers/net/cnxk/cn9k_rx_vec.c | 5 +- drivers/net/cnxk/cn9k_tx.c | 28 +++-- drivers/net/cnxk/cn9k_tx.h | 253 ++++++++++++++++++++++++++++----------- drivers/net/cnxk/cn9k_tx_mseg.c | 2 +- drivers/net/cnxk/cn9k_tx_vec.c | 3 +- drivers/net/cnxk/cnxk_ethdev.c | 36 +++++- drivers/net/cnxk/cnxk_ethdev.h | 64 +++++++++- drivers/net/cnxk/cnxk_ptp.c | 169 ++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 24 files changed, 833 insertions(+), 215 deletions(-) create mode 100644 drivers/net/cnxk/cnxk_ptp.c diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 7f02b6b..5de7c7a 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -29,6 +29,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -94,6 +97,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -120,6 +126,7 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; union nix_send_hdr_w0_u send_hdr_w0; + struct nix_send_mem_s *send_mem; union nix_send_sg_s sg_w0; RTE_SET_USED(dev); @@ -135,6 +142,22 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr_w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *)(txq->cmd + 2); + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } } else { /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ send_hdr_w0.sizem1 = 1; @@ -219,6 +242,7 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -340,6 +364,7 @@ static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -347,6 +372,12 @@ cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 58c51ab..ebe71c7 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -30,6 +30,7 @@ struct cn10k_eth_rxq { uint32_t available; uint16_t data_off; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 0598111..c9744e2 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 29ee0ac..c09ccdf 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -7,6 +7,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -250,6 +252,10 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -487,27 +493,44 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c index 9d283f7..b67d21f 100644 --- a/drivers/net/cnxk/cn10k_rx_mseg.c +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c index 0fa079c..1330235 100644 --- a/drivers/net/cnxk/cn10k_rx_vec.c +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -5,12 +5,15 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ struct rte_mbuf **rx_pkts, \ uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index e6eb101..18694dc 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -24,12 +24,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -42,25 +43,25 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -68,7 +69,8 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 5d649e0..4232037 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -46,8 +48,12 @@ static __rte_always_inline int cn10k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline uint64_t @@ -339,6 +345,44 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); } +static __rte_always_inline void +cn10k_nix_xmit_prepare_tstamp(uintptr_t lmt_addr, const uint64_t *cmd, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + struct nix_send_ext_s *send_hdr_ext = + (struct nix_send_ext_s *)lmt_addr + 16; + uint64_t *lmt = (uint64_t *)lmt_addr; + uint16_t off = (no_segdw - 1) << 1; + struct nix_send_mem_s *send_mem; + + send_mem = (struct nix_send_mem_s *)(lmt + off); + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + send_hdr_ext->w0.tstmp = 1; + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + lmt[off] = cmd[2]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + send_mem->addr = (rte_iova_t)((uint64_t *)cmd[3]); + } +} + static __rte_always_inline uint16_t cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) { @@ -404,7 +448,7 @@ cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -441,6 +485,8 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); } @@ -525,6 +571,9 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, /* Store sg list directly on lmt line */ segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, flags); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); data128 |= (((__uint128_t)(segdw - 1)) << shft); shft += 3; @@ -1539,75 +1588,140 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c index 6ae6907..33f6754 100644 --- a/drivers/net/cnxk/cn10k_tx_mseg.c +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c index 42baeb5..7453f3b 100644 --- a/drivers/net/cnxk/cn10k_tx_vec.c +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 798a03d..130c8b4 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -29,6 +29,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -94,6 +97,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -120,10 +126,9 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; struct nix_send_hdr_s *send_hdr; + struct nix_send_mem_s *send_mem; union nix_send_sg_s *sg; - RTE_SET_USED(dev); - /* Initialize the fields based on basic single segment packet */ memset(&txq->cmd, 0, sizeof(txq->cmd)); @@ -134,6 +139,23 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr->w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *) + (txq->cmd + (send_hdr->w0.sizem1 << 1)); + send_mem->w0.cn9k.subdc = NIX_SUBDC_MEM; + send_mem->w0.cn9k.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } sg = (union nix_send_sg_s *)&txq->cmd[4]; } else { send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; @@ -217,6 +239,7 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -349,6 +372,7 @@ static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -356,6 +380,12 @@ cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index cd0938f..1429c27 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -28,6 +28,7 @@ struct cn9k_eth_rxq { uint32_t qmask; uint32_t available; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 01eb21f..a15428d 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index f4b3282..c5ad5db 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -8,6 +8,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -253,6 +255,10 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -489,27 +495,44 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c index 6ad8c1d..3b26962 100644 --- a/drivers/net/cnxk/cn9k_rx_mseg.c +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c index 997177f..b19c7f3 100644 --- a/drivers/net/cnxk/cn9k_rx_vec.c +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -5,10 +5,13 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 2ff9720..b802606 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -23,12 +23,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TS] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -41,25 +42,25 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -67,7 +68,8 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index c633008..ee5f0e9 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -46,8 +48,12 @@ static __rte_always_inline int cn9k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline void @@ -281,6 +287,41 @@ cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) } static __rte_always_inline void +cn9k_nix_xmit_prepare_tstamp(uint64_t *cmd, const uint64_t *send_mem_desc, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + struct nix_send_mem_s *send_mem; + uint16_t off = (no_segdw - 1) << 1; + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + + send_mem = (struct nix_send_mem_s *)(cmd + off); + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + cmd[off] = send_mem_desc[6]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.cn9k.alg = + NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + + send_mem->addr = (rte_iova_t)((uint64_t *)send_mem_desc[7] + + (is_ol_tstamp)); + } +} + +static __rte_always_inline void cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, const uint32_t flags) { @@ -378,7 +419,7 @@ cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -442,6 +483,8 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); } @@ -480,6 +523,9 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags); segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); } @@ -1410,75 +1456,140 @@ cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ -#define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ - NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ - L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ - OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ - OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ - VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ - VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ - VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ - VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ - NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ - NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ - NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ - NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ - NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ - NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ - TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ - TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ - TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ - TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ - TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ - TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ - TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ - TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ - TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c index 65c5f36..f3c427c 100644 --- a/drivers/net/cnxk/cn9k_tx_mseg.c +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c index 21ffc2c..a6e7c9e 100644 --- a/drivers/net/cnxk/cn9k_tx_vec.c +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index afb5b56..e501ab0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -150,7 +150,8 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) offsetof(struct rte_mbuf, data_off) != 6); mb_def.nb_segs = 1; - mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.data_off = RTE_PKTMBUF_HEADROOM + + (dev->ptp_en * CNXK_NIX_TIMESYNC_RX_OFFSET); mb_def.port = port_id; rte_mbuf_refcnt_set(&mb_def, 1); @@ -356,6 +357,18 @@ cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, eth_dev->data->rx_queues[qid] = rxq_sp + 1; eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + /* Calculating delta and freq mult between PTP HI clock and tsc. + * These are needed in deriving raw clock value from tsc counter. + * read_clock eth op returns raw clock value. + */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) { + rc = cnxk_nix_tsc_convert(dev); + if (rc) { + plt_err("Failed to calculate delta and freq mult"); + goto rq_fini; + } + } + return 0; rq_fini: rc |= roc_nix_rq_fini(rq); @@ -1095,7 +1108,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; - if (eth_dev->data->nb_rx_queues != 0) { + if (eth_dev->data->nb_rx_queues != 0 && !dev->ptp_en) { rc = nix_recalc_mtu(eth_dev); if (rc) return rc; @@ -1140,6 +1153,25 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) } } + /* Enable PTP if it is requested by the user or already + * enabled on PF owning this VF + */ + memset(&dev->tstamp, 0, sizeof(struct cnxk_timesync_info)); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) + cnxk_eth_dev_ops.timesync_enable(eth_dev); + else + cnxk_eth_dev_ops.timesync_disable(eth_dev); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) { + rc = rte_mbuf_dyn_rx_timestamp_register + (&dev->tstamp.tstamp_dynfield_offset, + &dev->tstamp.rx_tstamp_dynflag); + if (rc != 0) { + plt_err("Failed to register Rx timestamp field/flag"); + goto rx_disable; + } + } + cnxk_nix_toggle_flag_link_cfg(dev, false); return 0; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index b070df7..303d205 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -13,6 +13,7 @@ #include <rte_mbuf.h> #include <rte_mbuf_pool_ops.h> #include <rte_mempool.h> +#include <rte_time.h> #include "roc_api.h" @@ -75,7 +76,7 @@ (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ - DEV_RX_OFFLOAD_RSS_HASH) + DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP) #define RSS_IPV4_ENABLE \ (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ @@ -100,7 +101,10 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +/* Default cycle counter mask */ +#define CNXK_CYCLECOUNTER_MASK 0xffffffffffffffffULL #define CNXK_NIX_TIMESYNC_RX_OFFSET 8 + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -130,6 +134,16 @@ struct cnxk_eth_qconf { uint8_t valid; }; +struct cnxk_timesync_info { + uint64_t rx_tstamp_dynflag; + rte_iova_t tx_tstamp_iova; + uint64_t *tx_tstamp; + uint64_t rx_tstamp; + int tstamp_dynfield_offset; + uint8_t tx_ready; + uint8_t rx_ready; +} __plt_cache_aligned; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -188,6 +202,14 @@ struct cnxk_eth_dev { /* Flow control configuration */ struct cnxk_fc_cfg fc_cfg; + /* PTP Counters */ + struct cnxk_timesync_info tstamp; + struct rte_timecounter systime_tc; + struct rte_timecounter rx_tstamp_tc; + struct rte_timecounter tx_tstamp_tc; + double clk_freq_mult; + uint64_t clk_delta; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -272,6 +294,9 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -388,4 +413,41 @@ cnxk_nix_prefree_seg(struct rte_mbuf *m) return 1; } +static inline rte_mbuf_timestamp_t * +cnxk_nix_timestamp_dynfield(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *info) +{ + return RTE_MBUF_DYNFIELD(mbuf, info->tstamp_dynfield_offset, + rte_mbuf_timestamp_t *); +} + +static __rte_always_inline void +cnxk_nix_mbuf_to_tstamp(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *tstamp, bool ts_enable, + uint64_t *tstamp_ptr) +{ + if (ts_enable && + (mbuf->data_off == + RTE_PKTMBUF_HEADROOM + CNXK_NIX_TIMESYNC_RX_OFFSET)) { + mbuf->pkt_len -= CNXK_NIX_TIMESYNC_RX_OFFSET; + + /* Reading the rx timestamp inserted by CGX, viz at + * starting of the packet data. + */ + *cnxk_nix_timestamp_dynfield(mbuf, tstamp) = + rte_be_to_cpu_64(*tstamp_ptr); + /* PKT_RX_IEEE1588_TMST flag needs to be set only in case + * PTP packets are received. + */ + if (mbuf->packet_type == RTE_PTYPE_L2_ETHER_TIMESYNC) { + tstamp->rx_tstamp = + *cnxk_nix_timestamp_dynfield(mbuf, tstamp); + tstamp->rx_ready = 1; + mbuf->ol_flags |= PKT_RX_IEEE1588_PTP | + PKT_RX_IEEE1588_TMST | + tstamp->rx_tstamp_dynflag; + } + } +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c new file mode 100644 index 0000000..fc317965 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +/* This function calculates two parameters "clk_freq_mult" and + * "clk_delta" which is useful in deriving PTP HI clock from + * timestamp counter (tsc) value. + */ +int +cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) +{ + uint64_t ticks_base = 0, ticks = 0, tsc = 0, t_freq; + struct roc_nix *nix = &dev->nix; + int rc, val; + + /* Calculating the frequency at which PTP HI clock is running */ + rc = roc_nix_ptp_clock_read(nix, &ticks_base, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + rte_delay_ms(100); + + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + t_freq = (ticks - ticks_base) * 10; + + /* Calculating the freq multiplier viz the ratio between the + * frequency at which PTP HI clock works and tsc clock runs + */ + dev->clk_freq_mult = + (double)pow(10, floor(log10(t_freq))) / rte_get_timer_hz(); + + val = false; +#ifdef RTE_ARM_EAL_RDTSC_USE_PMU + val = true; +#endif + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, val); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + /* Calculating delta between PTP HI clock and tsc */ + dev->clk_delta = ((uint64_t)(ticks / dev->clk_freq_mult) - tsc); + +fail: + return rc; +} + +int +cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + struct roc_nix *nix = &dev->nix; + const struct rte_memzone *ts; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be enabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) { + plt_err("PTP cannot be enabled for VF/SDP/LBK"); + return -EINVAL; + } + + if (dev->ptp_en) + return rc; + + if (dev->ptype_disable) { + plt_err("Ptype offload is disabled, it should be enabled"); + return -EINVAL; + } + + if (dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { + plt_err("Both PTP and switch header cannot be enabled"); + return -EINVAL; + } + + /* Allocating a iova address for tx tstamp */ + ts = rte_eth_dma_zone_reserve(eth_dev, "cnxk_ts", 0, 128, 128, 0); + if (ts == NULL) { + plt_err("Failed to allocate mem for tx tstamp addr"); + return -ENOMEM; + } + + tstamp->tx_tstamp_iova = ts->iova; + tstamp->tx_tstamp = ts->addr; + + rc = rte_mbuf_dyn_rx_timestamp_register(&tstamp->tstamp_dynfield_offset, + &tstamp->rx_tstamp_dynflag); + if (rc) { + plt_err("Failed to register Rx timestamp field/flag"); + goto error; + } + + /* System time should be already on by default */ + memset(&dev->systime_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->rx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->tx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + + dev->systime_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->rx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->tx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + + dev->rx_offloads |= DEV_RX_OFFLOAD_TIMESTAMP; + + rc = roc_nix_ptp_rx_ena_dis(nix, true); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, true); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, false); + goto error; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) { + plt_err("Failed to set MTU size for ptp"); + goto error; + } + + return rc; + +error: + rte_eth_dma_zone_free(eth_dev, "cnxk_ts", 0); + dev->tstamp.tx_tstamp_iova = 0; + dev->tstamp.tx_tstamp = NULL; + return rc; +} + +int +cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t rx_offloads = DEV_RX_OFFLOAD_TIMESTAMP; + struct roc_nix *nix = &dev->nix; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be disabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) + return -EINVAL; + + if (!dev->ptp_en) + return rc; + + dev->rx_offloads &= ~rx_offloads; + + rc = roc_nix_ptp_rx_ena_dis(nix, false); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, false); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, true); + return rc; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) + plt_err("Failed to set MTU size for ptp"); + + return rc; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 92ef8f0..37f61a2 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_ptp.c', 'cnxk_rte_flow.c', 'cnxk_stats.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram @ 2021-06-08 12:04 ` Pavan Nikhilesh Bhagavatula 2021-06-09 11:06 ` Nithin Dabilpuram 0 siblings, 1 reply; 262+ messages in thread From: Pavan Nikhilesh Bhagavatula @ 2021-06-08 12:04 UTC (permalink / raw) To: Nithin Kumar Dabilpuram, dev Cc: Jerin Jacob Kollanukkaran, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Kiran Kumar Kokkilagadda, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil <snip> > >+static __rte_always_inline void >+cn10k_nix_xmit_prepare_tstamp(uintptr_t lmt_addr, const uint64_t >*cmd, >+ const uint64_t ol_flags, const uint16_t >no_segdw, >+ const uint16_t flags) >+{ >+ if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { >+ const uint8_t is_ol_tstamp = !(ol_flags & >PKT_TX_IEEE1588_TMST); >+ struct nix_send_ext_s *send_hdr_ext = >+ (struct nix_send_ext_s >*)lmt_addr + 16; >+ uint64_t *lmt = (uint64_t *)lmt_addr; >+ uint16_t off = (no_segdw - 1) << 1; >+ struct nix_send_mem_s *send_mem; >+ >+ send_mem = (struct nix_send_mem_s *)(lmt + off); >+ send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; >+ send_hdr_ext->w0.tstmp = 1; >+ if (flags & NIX_TX_MULTI_SEG_F) { >+ /* Retrieving the default desc values */ >+ lmt[off] = cmd[2]; >+ >+ /* Using compiler barier to avoid voilation of C >+ * aliasing rules. >+ */ >+ rte_compiler_barrier(); >+ } >+ >+ /* Packets for which PKT_TX_IEEE1588_TMST is not set, >tx tstamp >+ * should not be recorded, hence changing the alg type >to >+ * NIX_SENDMEMALG_SET and also changing send mem >addr field to >+ * next 8 bytes as it corrpt the actual tx tstamp >registered >+ * address. >+ */ >+ send_mem->w0.subdc = NIX_SUBDC_MEM; >+ send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP - >(is_ol_tstamp); >+ send_mem->addr = (rte_iova_t)((uint64_t *)cmd[3]); Missing address increment for non-tstmp packets here. >+ } >+} >+ <snip> > static __rte_always_inline void >+cn9k_nix_xmit_prepare_tstamp(uint64_t *cmd, const uint64_t >*send_mem_desc, >+ const uint64_t ol_flags, const uint16_t >no_segdw, >+ const uint16_t flags) >+{ >+ if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { >+ struct nix_send_mem_s *send_mem; >+ uint16_t off = (no_segdw - 1) << 1; >+ const uint8_t is_ol_tstamp = !(ol_flags & >PKT_TX_IEEE1588_TMST); >+ >+ send_mem = (struct nix_send_mem_s *)(cmd + off); >+ if (flags & NIX_TX_MULTI_SEG_F) { >+ /* Retrieving the default desc values */ >+ cmd[off] = send_mem_desc[6]; >+ >+ /* Using compiler barier to avoid voilation of C >+ * aliasing rules. >+ */ >+ rte_compiler_barrier(); >+ } >+ >+ /* Packets for which PKT_TX_IEEE1588_TMST is not set, >tx tstamp >+ * should not be recorded, hence changing the alg type >to >+ * NIX_SENDMEMALG_SET and also changing send mem >addr field to >+ * next 8 bytes as it corrpt the actual tx tstamp >registered >+ * address. >+ */ >+ send_mem->w0.cn9k.alg = >+ NIX_SENDMEMALG_SETTSTMP - >(is_ol_tstamp); >+ >+ send_mem->addr = (rte_iova_t)((uint64_t >*)send_mem_desc[7] + >+ (is_ol_tstamp)); Need to increment by at-least 8 bytes. >+ } >+} >+ <snip> ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support 2021-06-08 12:04 ` Pavan Nikhilesh Bhagavatula @ 2021-06-09 11:06 ` Nithin Dabilpuram 0 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-09 11:06 UTC (permalink / raw) To: Pavan Nikhilesh Bhagavatula Cc: dev, Jerin Jacob Kollanukkaran, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Kiran Kumar Kokkilagadda, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Tue, Jun 08, 2021 at 12:04:45PM +0000, Pavan Nikhilesh Bhagavatula wrote: > <snip> > > > > >+static __rte_always_inline void > >+cn10k_nix_xmit_prepare_tstamp(uintptr_t lmt_addr, const uint64_t > >*cmd, > >+ const uint64_t ol_flags, const uint16_t > >no_segdw, > >+ const uint16_t flags) > >+{ > >+ if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { > >+ const uint8_t is_ol_tstamp = !(ol_flags & > >PKT_TX_IEEE1588_TMST); > >+ struct nix_send_ext_s *send_hdr_ext = > >+ (struct nix_send_ext_s > >*)lmt_addr + 16; > >+ uint64_t *lmt = (uint64_t *)lmt_addr; > >+ uint16_t off = (no_segdw - 1) << 1; > >+ struct nix_send_mem_s *send_mem; > >+ > >+ send_mem = (struct nix_send_mem_s *)(lmt + off); > >+ send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; > >+ send_hdr_ext->w0.tstmp = 1; > >+ if (flags & NIX_TX_MULTI_SEG_F) { > >+ /* Retrieving the default desc values */ > >+ lmt[off] = cmd[2]; > >+ > >+ /* Using compiler barier to avoid voilation of C > >+ * aliasing rules. > >+ */ > >+ rte_compiler_barrier(); > >+ } > >+ > >+ /* Packets for which PKT_TX_IEEE1588_TMST is not set, > >tx tstamp > >+ * should not be recorded, hence changing the alg type > >to > >+ * NIX_SENDMEMALG_SET and also changing send mem > >addr field to > >+ * next 8 bytes as it corrpt the actual tx tstamp > >registered > >+ * address. > >+ */ > >+ send_mem->w0.subdc = NIX_SUBDC_MEM; > >+ send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP - > >(is_ol_tstamp); > >+ send_mem->addr = (rte_iova_t)((uint64_t *)cmd[3]); > > Missing address increment for non-tstmp packets here. Ack, will fix it. > > >+ } > >+} > >+ > > <snip> > > > static __rte_always_inline void > >+cn9k_nix_xmit_prepare_tstamp(uint64_t *cmd, const uint64_t > >*send_mem_desc, > >+ const uint64_t ol_flags, const uint16_t > >no_segdw, > >+ const uint16_t flags) > >+{ > >+ if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { > >+ struct nix_send_mem_s *send_mem; > >+ uint16_t off = (no_segdw - 1) << 1; > >+ const uint8_t is_ol_tstamp = !(ol_flags & > >PKT_TX_IEEE1588_TMST); > >+ > >+ send_mem = (struct nix_send_mem_s *)(cmd + off); > >+ if (flags & NIX_TX_MULTI_SEG_F) { > >+ /* Retrieving the default desc values */ > >+ cmd[off] = send_mem_desc[6]; > >+ > >+ /* Using compiler barier to avoid voilation of C > >+ * aliasing rules. > >+ */ > >+ rte_compiler_barrier(); > >+ } > >+ > >+ /* Packets for which PKT_TX_IEEE1588_TMST is not set, > >tx tstamp > >+ * should not be recorded, hence changing the alg type > >to > >+ * NIX_SENDMEMALG_SET and also changing send mem > >addr field to > >+ * next 8 bytes as it corrpt the actual tx tstamp > >registered > >+ * address. > >+ */ > >+ send_mem->w0.cn9k.alg = > >+ NIX_SENDMEMALG_SETTSTMP - > >(is_ol_tstamp); > >+ > >+ send_mem->addr = (rte_iova_t)((uint64_t > >*)send_mem_desc[7] + > >+ (is_ol_tstamp)); > > Need to increment by at-least 8 bytes. Ack. > > >+ } > >+} > >+ > > <snip> ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 56/62] net/cnxk: add timesync enable/disable operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (54 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram ` (6 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements timesync enable/disable operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 5de7c7a..d8d3d2d 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -361,6 +361,54 @@ cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn10k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn10k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -406,6 +454,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn10k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn10k_nix_timesync_disable; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 130c8b4..0819f67 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -369,6 +369,54 @@ cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn9k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn9k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -414,6 +462,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn9k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn9k_nix_timesync_disable; } static int -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 57/62] net/cnxk: add Rx/Tx timestamp read operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (55 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram ` (5 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements Rx/Tx timestamp read operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 +++++ drivers/net/cnxk/cnxk_ptp.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index e501ab0..68aaf27 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1231,6 +1231,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, .get_reg = cnxk_nix_dev_get_reg, + .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, + .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 303d205..c03dbc5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -296,6 +296,11 @@ int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, + uint32_t flags); +int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index fc317965..7b00f87 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,44 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, uint32_t flags) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + PLT_SET_USED(flags); + + if (!tstamp->rx_ready) + return -EINVAL; + + ns = rte_timecounter_update(&dev->rx_tstamp_tc, tstamp->rx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + tstamp->rx_ready = 0; + return 0; +} + +int +cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + if (*tstamp->tx_tstamp == 0) + return -EINVAL; + + ns = rte_timecounter_update(&dev->tx_tstamp_tc, *tstamp->tx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + *tstamp->tx_tstamp = 0; + rte_wmb(); + + return 0; +} + +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 58/62] net/cnxk: add time read/write/adjust operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (56 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 59/62] net/cnxk: add read clock operation Nithin Dabilpuram ` (4 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements read/write/adjust time operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ptp.c | 63 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 68aaf27..578f9b9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1233,6 +1233,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_reg = cnxk_nix_dev_get_reg, .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, + .timesync_read_time = cnxk_nix_timesync_read_time, + .timesync_write_time = cnxk_nix_timesync_write_time, + .timesync_adjust_time = cnxk_nix_timesync_adjust_time, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c03dbc5..326f189 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -301,6 +301,11 @@ int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, uint32_t flags); int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp); +int cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, + struct timespec *ts); +int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts); +int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 7b00f87..52f6eb1 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,69 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t clock, ns; + int rc; + + rc = roc_nix_ptp_clock_read(nix, &clock, NULL, false); + if (rc) + return rc; + + ns = rte_timecounter_update(&dev->systime_tc, clock); + *ts = rte_ns_to_timespec(ns); + return 0; +} + +int +cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t ns; + + ns = rte_timespec_to_ns(ts); + /* Set the time counters to a new value. */ + dev->systime_tc.nsec = ns; + dev->rx_tstamp_tc.nsec = ns; + dev->tx_tstamp_tc.nsec = ns; + + return 0; +} + +int +cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Adjust the frequent to make tics increments in 10^9 tics per sec */ + if (delta < ROC_NIX_PTP_FREQ_ADJUST && + delta > -ROC_NIX_PTP_FREQ_ADJUST) { + rc = roc_nix_ptp_sync_time_adjust(nix, delta); + if (rc) + return rc; + + /* Since the frequency of PTP comp register is tuned, delta and + * freq mult calculation for deriving PTP_HI from timestamp + * counter should be done again. + */ + rc = cnxk_nix_tsc_convert(dev); + if (rc) + plt_err("Failed to calculate delta and freq mult"); + } + + dev->systime_tc.nsec += delta; + dev->rx_tstamp_tc.nsec += delta; + dev->tx_tstamp_tc.nsec += delta; + + return 0; +} + +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp, uint32_t flags) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 59/62] net/cnxk: add read clock operation 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (57 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 60/62] net/cnxk: support for rte flow dev dump API Nithin Dabilpuram ` (3 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch implements read raw clock operation for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ptp.c | 17 +++++++++++++++++ 4 files changed, 21 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 861b7c1..0d76540 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Timesync = Y +Timestamp offload = Y Basic stats = Y Stats per queue = Y Extended stats = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 578f9b9..06cc039 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1236,6 +1236,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_read_time = cnxk_nix_timesync_read_time, .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, + .read_clock = cnxk_nix_read_clock, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 326f189..9b96904 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -307,6 +307,7 @@ int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, const struct timespec *ts); int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); +int cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 52f6eb1..449489f 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -4,6 +4,23 @@ #include "cnxk_ethdev.h" +int +cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* This API returns the raw PTP HI clock value. Since LFs do not + * have direct access to PTP registers and it requires mbox msg + * to AF for this value. In fastpath reading this value for every + * packet (which involes mbox call) becomes very expensive, hence + * we should be able to derive PTP HI clock value from tsc by + * using freq_mult and clk_delta calculated during configure stage. + */ + *clock = (rte_get_tsc_cycles() + dev->clk_delta) * dev->clk_freq_mult; + + return 0; +} + /* This function calculates two parameters "clk_freq_mult" and * "clk_delta" which is useful in deriving PTP HI clock from * timestamp counter (tsc) value. -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 60/62] net/cnxk: support for rte flow dev dump API 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (58 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 59/62] net/cnxk: add read clock operation Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 61/62] net/cnxk: added reta and rss_hash operations Nithin Dabilpuram ` (2 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satheesh Paul <psatheesh@marvell.com> Add support to dump hardware internal representation information of rte flow to file. Every flow rule added will be dumped in the below format. MCAM Index:1881 Interface :NIX-RX (0) Priority :1 NPC RX Action:0X00000000404001 ActionOp:NIX_RX_ACTIONOP_UCAST (1) PF_FUNC: 0X400 RQ Index:0X004 Match Id:0000 Flow Key Alg:0 NPC RX VTAG Action:0X00000000008100 VTAG0:relptr:0 lid:0X1 type:0 Patterns: NPC_PARSE_NIBBLE_CHAN:000 NPC_PARSE_NIBBLE_LA_LTYPE:LA_ETHER NPC_PARSE_NIBBLE_LB_LTYPE:NONE NPC_PARSE_NIBBLE_LC_LTYPE:LC_IP NPC_PARSE_NIBBLE_LD_LTYPE:LD_TCP NPC_PARSE_NIBBLE_LE_LTYPE:NONE LA_ETHER, hdr offset:0, len:0X6, key offset:0X8,\ Data:0X4AE124FC7FFF, Mask:0XFFFFFFFFFFFF LA_ETHER, hdr offset:0XC, len:0X2, key offset:0X4, Data:0XCA5A,\ Mask:0XFFFF LC_IP, hdr offset:0XC, len:0X8, key offset:0X10,\ Data:0X0A01010300000000, Mask:0XFFFFFFFF00000000 LD_TCP, hdr offset:0, len:0X4, key offset:0X18, Data:0X03450000,\ Mask:0XFFFF0000 MCAM Raw Data : DW0 :0000CA5A01202000 DW0_Mask:0000FFFF0FF0F000 DW1 :00004AE124FC7FFF DW1_Mask:0000FFFFFFFFFFFF DW2 :0A01010300000000 DW2_Mask:FFFFFFFF00000000 DW3 :0000000003450000 DW3_Mask:00000000FFFF0000 DW4 :0000000000000000 DW4_Mask:0000000000000000 DW5 :0000000000000000 DW5_Mask:0000000000000000 DW6 :0000000000000000 DW6_Mask:0000000000000000 ci: skip_checkpatch skip_checkformat Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/net/cnxk/cnxk_rte_flow.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index 8486e9e..453425e 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -328,6 +328,33 @@ cnxk_flow_isolate(struct rte_eth_dev *dev __rte_unused, int enable __rte_unused, return -rte_errno; } +static int +cnxk_flow_dev_dump(struct rte_eth_dev *dev, struct rte_flow *flow, + FILE *file, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *hw = dev->data->dev_private; + struct roc_npc *npc = &hw->npc; + + if (file == NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "Invalid file"); + return -rte_errno; + } + + if (flow != NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_HANDLE, + NULL, + "Invalid argument"); + return -EINVAL; + } + + roc_npc_flow_dump(file, npc); + + return 0; +} + const struct rte_flow_ops cnxk_flow_ops = { .validate = cnxk_flow_validate, .create = cnxk_flow_create, @@ -335,4 +362,5 @@ const struct rte_flow_ops cnxk_flow_ops = { .flush = cnxk_flow_flush, .query = cnxk_flow_query, .isolate = cnxk_flow_isolate, + .dev_dump = cnxk_flow_dev_dump, }; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 61/62] net/cnxk: added reta and rss_hash operations 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (59 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 60/62] net/cnxk: support for rte flow dev dump API Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 62/62] net/cnxk: add multicast filter support Nithin Dabilpuram 2021-06-14 3:27 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Satha Rao <skoteshwar@marvell.com> This patch will implement reta and rss_hash apis. Also added device argument to lock rx context. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 4 ++ drivers/net/cnxk/cnxk_ethdev.h | 10 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 121 +++++++++++++++++++++++++++++++++ 7 files changed, 145 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 0d76540..24803da 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 65ee8ba..b4b1169 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 00bde9b..fbc50a8 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -19,6 +19,8 @@ Queue start/stop = Y MTU update = Y TSO = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 06cc039..a250cff 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1237,6 +1237,10 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, .read_clock = cnxk_nix_read_clock, + .reta_update = cnxk_nix_reta_update, + .reta_query = cnxk_nix_reta_query, + .rss_hash_update = cnxk_nix_rss_hash_update, + .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 9b96904..62c88e6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -314,6 +314,16 @@ uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +int cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); +int cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); /* Link */ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 7fd06eb..c76b628 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -109,6 +109,7 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) #define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" #define CNXK_SWITCH_HEADER_TYPE "switch_header" #define CNXK_RSS_TAG_AS_XOR "tag_as_xor" +#define CNXK_LOCK_RX_CTX "lock_rx_ctx" int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) @@ -120,6 +121,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) uint16_t flow_max_priority = 3; uint16_t rss_tag_as_xor = 0; uint16_t scalar_enable = 0; + uint8_t lock_rx_ctx = 0; struct rte_kvargs *kvlist; if (devargs == NULL) @@ -143,6 +145,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) &parse_switch_header_type, &switch_header_type); rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, &rss_tag_as_xor); + rte_kvargs_process(kvlist, CNXK_LOCK_RX_CTX, &parse_flag, &lock_rx_ctx); rte_kvargs_free(kvlist); null_devargs: @@ -150,6 +153,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->nix.lock_rx_ctx = lock_rx_ctx; dev->npc.flow_prealloc_size = flow_prealloc_size; dev->npc.flow_max_priority = flow_max_priority; dev->npc.switch_header_type = switch_header_type; diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index d437e2c..c28512a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -718,3 +718,124 @@ cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) return rc; } + +int +cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int i, j, rc = -EINVAL, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta[idx] = reta_conf[i].reta[j]; + idx++; + } + } + + return roc_nix_rss_reta_set(nix, 0, reta); + +fail: + return rc; +} + +int +cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL, i, j, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + rc = roc_nix_rss_reta_get(nix, 0, reta); + if (rc) + goto fail; + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = reta[idx]; + idx++; + } + } + + return 0; + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + int rc = -EINVAL; + uint8_t alg_idx; + + if (rss_conf->rss_key && rss_conf->rss_key_len != ROC_NIX_RSS_KEY_LEN) { + plt_err("Hash key size mismatch %d vs %d", + rss_conf->rss_key_len, ROC_NIX_RSS_KEY_LEN); + goto fail; + } + + if (rss_conf->rss_key) + roc_nix_rss_key_set(nix, rss_conf->rss_key); + + rss_hash_level = ETH_RSS_LEVEL(rss_conf->rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + flowkey_cfg = + cnxk_rss_ethdev_to_nix(dev, rss_conf->rss_hf, rss_hash_level); + + rc = roc_nix_rss_flowkey_set(nix, &alg_idx, flowkey_cfg, + ROC_NIX_RSS_GROUP_DEFAULT, + ROC_NIX_RSS_MCAM_IDX_DEFAULT); + if (rc) { + plt_err("Failed to set RSS hash function rc=%d", rc); + return rc; + } + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (rss_conf->rss_key) + roc_nix_rss_key_get(&dev->nix, rss_conf->rss_key); + + rss_conf->rss_key_len = ROC_NIX_RSS_KEY_LEN; + rss_conf->rss_hf = dev->ethdev_rss_hf; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v2 62/62] net/cnxk: add multicast filter support 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (60 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 61/62] net/cnxk: added reta and rss_hash operations Nithin Dabilpuram @ 2021-06-07 17:59 ` Nithin Dabilpuram 2021-06-14 3:27 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-07 17:59 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar From: Sunil Kumar Kori <skori@marvell.com> Patch adds multicast filter support for cn9k and cn10k platforms. CGX DMAC filter table(32 entries) is divided among all LMACs connected to it i.e. if CGX has 4 LMACs then each LMAC can have up to 8 filters. If CGX has 1 LMAC then it can have up to 32 filters. Above mentioned filter table is used to install unicast and multicast DMAC address filters. Unicast filters are installed via rte_eth_dev_mac_addr_add API while multicast filters are installed via rte_eth_dev_set_mc_addr_list API. So in total, supported MAC filters are equal to DMAC filters plus mcast filters. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 63 +++++++++++++++++++++++++++++++++++ 5 files changed, 71 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 24803da..337ad24 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -22,6 +22,7 @@ TSO = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index b4b1169..4c3f78e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -21,6 +21,7 @@ MTU update = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a250cff..6a0b49c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1241,6 +1241,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .reta_query = cnxk_nix_reta_query, .rss_hash_update = cnxk_nix_rss_hash_update, .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, + .set_mc_addr_list = cnxk_nix_mc_addr_list_configure, }; static int @@ -1306,6 +1307,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->max_mac_entries = max_entries; + dev->dmac_filter_count = 1; /* Get mac address */ rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 62c88e6..a28591b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -162,6 +162,7 @@ struct cnxk_eth_dev { uint8_t configured; /* Max macfilter entries */ + uint8_t dmac_filter_count; uint8_t max_mac_entries; bool dmac_filter_enable; @@ -249,6 +250,9 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr); int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, uint32_t index, uint32_t pool); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index c28512a..63fe9a6 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -353,6 +353,7 @@ cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, roc_nix_npc_promisc_ena_dis(nix, true); dev->dmac_filter_enable = true; eth_dev->data->promiscuous = false; + dev->dmac_filter_count++; return 0; } @@ -367,6 +368,8 @@ cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) rc = roc_nix_mac_addr_del(nix, index); if (rc) plt_err("Failed to delete mac address, rc=%d", rc); + + dev->dmac_filter_count--; } int @@ -839,3 +842,63 @@ cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, return 0; } + +int +cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct rte_ether_addr null_mac_addr = {0}; + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc, index; + uint32_t i; + + /* All configured multicast filters should be flushed first */ + for (i = 0; i < dev->max_mac_entries; i++) { + if (rte_is_multicast_ether_addr(&data->mac_addrs[i])) { + rc = roc_nix_mac_addr_del(nix, i); + if (rc) { + plt_err("Failed to flush mcast address, rc=%d", + rc); + return rc; + } + + dev->dmac_filter_count--; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&null_mac_addr, + &data->mac_addrs[i]); + } + } + + if (!mc_addr_set || !nb_mc_addr) + return 0; + + /* Check for available space */ + if (nb_mc_addr > + ((uint32_t)(dev->max_mac_entries - dev->dmac_filter_count))) { + plt_err("No space is available to add multicast filters"); + return -ENOSPC; + } + + /* Multicast addresses are to be installed */ + for (i = 0; i < nb_mc_addr; i++) { + index = roc_nix_mac_addr_add(nix, mc_addr_set[i].addr_bytes); + if (index < 0) { + plt_err("Failed to add mcast mac address, rc=%d", + index); + return index; + } + + dev->dmac_filter_count++; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&mc_addr_set[i], &data->mac_addrs[index]); + } + + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (61 preceding siblings ...) 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 62/62] net/cnxk: add multicast filter support Nithin Dabilpuram @ 2021-06-14 3:27 ` Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-14 3:27 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil On Mon, Jun 7, 2021 at 11:33 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > driver when code is ready and 'net/octeontx2' will be deprecated. A couple of checkpatch and check-git-log issue to check and fix. Wrong headline format: net/cnxk: support for rss in rte_flow net/cnxk: added reta and rss_hash operations Wrong headline prefix: net/cnxk: add flow ops get operation Wrong headline case: "net/cnxk: added reta and rss_hash operations": reta --> RETA Wrong headline case: "net/cnxk: support for rss in rte_flow": rss --> RSS Is it candidate for Cc: stable@dpdk.org backport? common/cnxk: fix batch alloc completion poll logic common/cnxk: fix flow create on CN98xx CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #267: FILE: drivers/net/cnxk/cn9k_rx.h:195: + rte_prefetch_non_temporal( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #178: FILE: drivers/net/cnxk/cn9k_rx.h:397: + ol_flags0 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #181: FILE: drivers/net/cnxk/cn9k_rx.h:400: + ol_flags1 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #184: FILE: drivers/net/cnxk/cn9k_rx.h:403: + ol_flags2 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #187: FILE: drivers/net/cnxk/cn9k_rx.h:406: + ol_flags3 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #173: FILE: drivers/net/cnxk/cn9k_tx.h:89: + *oudplen = rte_cpu_to_be_16( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #268: FILE: drivers/net/cnxk/cn10k_rx.h:193: + rte_prefetch_non_temporal( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #190: FILE: drivers/net/cnxk/cn10k_rx.h:394: + ol_flags0 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #193: FILE: drivers/net/cnxk/cn10k_rx.h:397: + ol_flags1 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #196: FILE: drivers/net/cnxk/cn10k_rx.h:400: + ol_flags2 = nix_update_match_id( CHECK:OPEN_ENDED_LINE: Lines should not end with a '(' #199: FILE: drivers/net/cnxk/cn10k_rx.h:403: + ol_flags3 = nix_update_match_id( WARNING:STRLCPY: Prefer strscpy over strlcpy - see: https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/ #131: FILE: drivers/net/cnxk/cnxk_stats.c:125: + strlcpy(xstats_names[i].name, roc_xstats_name[i].name, WARNING:STRLCPY: Prefer strscpy over strlcpy - see: https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/ #161: FILE: drivers/net/cnxk/cnxk_stats.c:155: + strlcpy(xstats_names[i].name, xnames[ids[i]].name, total: 0 errors, 2 warnings, 0 checks, 184 lines checked ### net/cnxk: add ethdev firmware version get WARNING:STRLCPY: Prefer strscpy over strlcpy - see: https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/ #90: FILE: drivers/net/cnxk/cnxk_ethdev_ops.c:647: + strlcpy(fw_version, str, fw_size); WARNING:TYPO_SPELLING: 'cant' may be misspelled - perhaps 'can't'? #79: FILE: drivers/net/cnxk/cn10k_ethdev.c:327: + /* In case of VF, setting of MTU cant be done directly in this ^^^^ WARNING:TYPO_SPELLING: 'cant' may be misspelled - perhaps 'can't'? #208: FILE: drivers/net/cnxk/cn9k_ethdev.c:336: + /* In case of VF, setting of MTU cant be done directly in this ^^^^ total: 0 errors, 2 warnings, 0 checks, 248 lines checked > > Harman Kalra (1): > common/cnxk: allocate lmt region in userspace > > Jerin Jacob (7): > common/cnxk: fix batch alloc completion poll logic > net/cnxk: add Rx support for cn9k > net/cnxk: add Rx vector version for cn9k > net/cnxk: add Tx support for cn9k > net/cnxk: add Rx support for cn10k > net/cnxk: add Rx vector version for cn10k > net/cnxk: add Tx support for cn10k > > Kiran Kumar K (2): > net/cnxk: add support to configure npc > net/cnxk: add initial version of rte flow support > > Nithin Dabilpuram (17): > net/cnxk: add build infra and common probe > net/cnxk: add platform specific probe and remove > net/cnxk: add common devargs parsing function > net/cnxk: add common dev infos get support > net/cnxk: add device configuration operation > net/cnxk: add link status update support > net/cnxk: add Rx queue setup and release > net/cnxk: add Tx queue setup and release > net/cnxk: add packet type support > net/cnxk: add queue start and stop support > net/cnxk: add Rx multi-segmented version for cn9k > net/cnxk: add Tx multi-segment version for cn9k > net/cnxk: add Tx vector version for cn9k > net/cnxk: add Rx multi-segment version for cn10k > net/cnxk: add Tx multi-segment version for cn10k > net/cnxk: add Tx vector version for cn10k > net/cnxk: add device start and stop operations > > Satha Rao (8): > common/cnxk: add support to lock NIX RQ contexts > common/cnxk: add provision to enable RED on RQ > net/cnxk: add port/queue stats > net/cnxk: add xstats apis > net/cnxk: add rxq/txq info get operations > net/cnxk: add ethdev firmware version get > net/cnxk: add get register operation > net/cnxk: added reta and rss_hash operations > > Satheesh Paul (6): > common/cnxk: add support to dump flow entries > common/cnxk: support for mark and flag flow actions > common/cnxk: fix flow create on CN98xx > net/cnxk: add flow ops get operation > net/cnxk: support for rss in rte_flow > net/cnxk: support for rte flow dev dump API > > Srujana Challa (1): > common/cnxk: update Rx inline IPsec mbox message format > > Sunil Kumar Kori (20): > net/cnxk: add MAC address set ops > net/cnxk: add MTU set device operation > net/cnxk: add promiscuous mode enable and disable > net/cnxk: add DMAC filter support > net/cnxk: add all multicast enable/disable ethops > net/cnxk: add Rx/Tx burst mode get ops > net/cnxk: add flow ctrl set/get ops > net/cnxk: add link up/down operations > net/cnxk: add EEPROM module info get operations > net/cnxk: add Rx queue interrupt enable/disable ops > net/cnxk: add validation API for mempool ops > net/cnxk: add device close and reset operations > net/cnxk: add pending Tx mbuf cleanup operation > net/cnxk: register callback to get PTP status > net/cnxk: add base PTP timesync support > net/cnxk: add timesync enable/disable operations > net/cnxk: add Rx/Tx timestamp read operations > net/cnxk: add time read/write/adjust operations > net/cnxk: add read clock operation > net/cnxk: add multicast filter support > > -- > > v2: > - Fixed issue with flow validate and flow create for 98xx > - Fixed issue batch alloc logic > - Fix lmtline allocation to be cached > - Sync Inline IPSec Rx mbox with kernel > - Add support for mark and flag flow actions > - Add reta key and hash update ops > - Added PTP and multicast filter support > > MAINTAINERS | 3 + > doc/guides/nics/cnxk.rst | 343 ++++++ > doc/guides/nics/features/cnxk.ini | 90 ++ > doc/guides/nics/features/cnxk_vec.ini | 44 + > doc/guides/nics/features/cnxk_vf.ini | 40 + > doc/guides/nics/index.rst | 1 + > doc/guides/platform/cnxk.rst | 3 + > drivers/common/cnxk/hw/npc.h | 2 + > drivers/common/cnxk/meson.build | 1 + > drivers/common/cnxk/roc_api.h | 2 + > drivers/common/cnxk/roc_dev.c | 98 +- > drivers/common/cnxk/roc_dev_priv.h | 1 + > drivers/common/cnxk/roc_mbox.h | 6 + > drivers/common/cnxk/roc_model.h | 6 + > drivers/common/cnxk/roc_nix.h | 39 +- > drivers/common/cnxk/roc_nix_queue.c | 52 + > drivers/common/cnxk/roc_nix_rss.c | 51 +- > drivers/common/cnxk/roc_nix_tm_utils.c | 86 +- > drivers/common/cnxk/roc_npa.c | 10 +- > drivers/common/cnxk/roc_npa.h | 35 +- > drivers/common/cnxk/roc_npc.c | 41 +- > drivers/common/cnxk/roc_npc.h | 15 +- > drivers/common/cnxk/roc_npc_mcam_dump.c | 611 +++++++++++ > drivers/common/cnxk/roc_npc_priv.h | 2 +- > drivers/common/cnxk/roc_npc_utils.c | 4 + > drivers/common/cnxk/roc_platform.h | 13 + > drivers/common/cnxk/version.map | 5 + > drivers/net/cnxk/cn10k_ethdev.c | 534 ++++++++++ > drivers/net/cnxk/cn10k_ethdev.h | 40 + > drivers/net/cnxk/cn10k_rx.c | 78 ++ > drivers/net/cnxk/cn10k_rx.h | 546 ++++++++++ > drivers/net/cnxk/cn10k_rx_mseg.c | 17 + > drivers/net/cnxk/cn10k_rx_vec.c | 22 + > drivers/net/cnxk/cn10k_tx.c | 82 ++ > drivers/net/cnxk/cn10k_tx.h | 1737 +++++++++++++++++++++++++++++++ > drivers/net/cnxk/cn10k_tx_mseg.c | 25 + > drivers/net/cnxk/cn10k_tx_vec.c | 26 + > drivers/net/cnxk/cn9k_ethdev.c | 557 ++++++++++ > drivers/net/cnxk/cn9k_ethdev.h | 38 + > drivers/net/cnxk/cn9k_rx.c | 78 ++ > drivers/net/cnxk/cn9k_rx.h | 548 ++++++++++ > drivers/net/cnxk/cn9k_rx_mseg.c | 17 + > drivers/net/cnxk/cn9k_rx_vec.c | 20 + > drivers/net/cnxk/cn9k_tx.c | 81 ++ > drivers/net/cnxk/cn9k_tx.h | 1605 ++++++++++++++++++++++++++++ > drivers/net/cnxk/cn9k_tx_mseg.c | 25 + > drivers/net/cnxk/cn9k_tx_vec.c | 26 + > drivers/net/cnxk/cnxk_ethdev.c | 1511 +++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 478 +++++++++ > drivers/net/cnxk/cnxk_ethdev_devargs.c | 173 +++ > drivers/net/cnxk/cnxk_ethdev_ops.c | 904 ++++++++++++++++ > drivers/net/cnxk/cnxk_link.c | 113 ++ > drivers/net/cnxk/cnxk_lookup.c | 326 ++++++ > drivers/net/cnxk/cnxk_ptp.c | 287 +++++ > drivers/net/cnxk/cnxk_rte_flow.c | 366 +++++++ > drivers/net/cnxk/cnxk_rte_flow.h | 69 ++ > drivers/net/cnxk/cnxk_stats.c | 217 ++++ > drivers/net/cnxk/meson.build | 45 + > drivers/net/cnxk/version.map | 3 + > drivers/net/meson.build | 1 + > 60 files changed, 12116 insertions(+), 83 deletions(-) > create mode 100644 doc/guides/nics/cnxk.rst > create mode 100644 doc/guides/nics/features/cnxk.ini > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c > create mode 100644 drivers/net/cnxk/cn10k_ethdev.c > create mode 100644 drivers/net/cnxk/cn10k_ethdev.h > create mode 100644 drivers/net/cnxk/cn10k_rx.c > create mode 100644 drivers/net/cnxk/cn10k_rx.h > create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c > create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c > create mode 100644 drivers/net/cnxk/cn10k_tx.c > create mode 100644 drivers/net/cnxk/cn10k_tx.h > create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c > create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c > create mode 100644 drivers/net/cnxk/cn9k_ethdev.c > create mode 100644 drivers/net/cnxk/cn9k_ethdev.h > create mode 100644 drivers/net/cnxk/cn9k_rx.c > create mode 100644 drivers/net/cnxk/cn9k_rx.h > create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c > create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c > create mode 100644 drivers/net/cnxk/cn9k_tx.c > create mode 100644 drivers/net/cnxk/cn9k_tx.h > create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c > create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c > create mode 100644 drivers/net/cnxk/cnxk_link.c > create mode 100644 drivers/net/cnxk/cnxk_lookup.c > create mode 100644 drivers/net/cnxk/cnxk_ptp.c > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h > create mode 100644 drivers/net/cnxk/cnxk_stats.c > create mode 100644 drivers/net/cnxk/meson.build > create mode 100644 drivers/net/cnxk/version.map > > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 00/62] Marvell CNXK Ethdev Driver 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (44 preceding siblings ...) 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram ` (62 more replies) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram 46 siblings, 63 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' driver. In future, CN9K a.k.a octeontx2 will also be supported by same driver when code is ready and 'net/octeontx2' will be deprecated. Harman Kalra (1): common/cnxk: allocate lmt region in userspace Jerin Jacob (7): common/cnxk: fix batch alloc completion poll logic net/cnxk: add Rx support for cn9k net/cnxk: add Rx vector version for cn9k net/cnxk: add Tx support for cn9k net/cnxk: add Rx support for cn10k net/cnxk: add Rx vector version for cn10k net/cnxk: add Tx support for cn10k Kiran Kumar K (2): net/cnxk: add support to configure npc net/cnxk: add initial version of rte flow support Nithin Dabilpuram (17): net/cnxk: add build infra and common probe net/cnxk: add platform specific probe and remove net/cnxk: add common devargs parsing function net/cnxk: add common dev infos get support net/cnxk: add device configuration operation net/cnxk: add link status update support net/cnxk: add Rx queue setup and release net/cnxk: add Tx queue setup and release net/cnxk: add packet type support net/cnxk: add queue start and stop support net/cnxk: add Rx multi-segmented version for cn9k net/cnxk: add Tx multi-segment version for cn9k net/cnxk: add Tx vector version for cn9k net/cnxk: add Rx multi-segment version for cn10k net/cnxk: add Tx multi-segment version for cn10k net/cnxk: add Tx vector version for cn10k net/cnxk: add device start and stop operations Satha Rao (8): common/cnxk: add support to lock NIX RQ contexts common/cnxk: add provision to enable RED on RQ net/cnxk: add port/queue stats net/cnxk: add xstats apis net/cnxk: add rxq/txq info get operations net/cnxk: add ethdev firmware version get net/cnxk: add get register operation net/cnxk: added RETA and RSS hash operations Satheesh Paul (7): common/cnxk: add support to dump flow entries common/cnxk: support for mark and flag flow actions common/cnxk: support for VLAN push and pop flow actions common/cnxk: fix flow create on CN98xx net/cnxk: add flow ops get operation net/cnxk: support for RSS in rte flow net/cnxk: add marking and VLAN tagging support Sunil Kumar Kori (20): net/cnxk: add MAC address set ops net/cnxk: add MTU set device operation net/cnxk: add promiscuous mode enable and disable net/cnxk: add DMAC filter support net/cnxk: add all multicast enable/disable ethops net/cnxk: add Rx/Tx burst mode get ops net/cnxk: add flow ctrl set/get ops net/cnxk: add link up/down operations net/cnxk: add EEPROM module info get operations net/cnxk: add Rx queue interrupt enable/disable ops net/cnxk: add validation API for mempool ops net/cnxk: add device close and reset operations net/cnxk: add pending Tx mbuf cleanup operation net/cnxk: register callback to get PTP status net/cnxk: add base PTP timesync support net/cnxk: add timesync enable/disable operations net/cnxk: add Rx/Tx timestamp read operations net/cnxk: add time read/write/adjust operations net/cnxk: add read clock operation net/cnxk: add multicast filter support -- v3: - Updated release notes - Removed RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag and add support for queue stats in xstats - Fixed issue with LSO format indices - Removed mbox sync changes patch from this series - Fixed documentation issues - Removed repetitive code in fast path SIMD - Optimize cn10k LMTST logic - Make rte_flow_create implementation specific to handle VLAN Stripping and MARK actions/offloads - Use rte_atomic_thread_fence() instead of rte_rmb() - Handle other comments from Jerin. - Merged rte flow dump API patch to flow ops get patch - Added marking and vlan tagging support. - Fixed some checkpatch and git check log issues. v2: - Fixed issue with flow validate and flow create for 98xx - Fixed issue batch alloc logic - Fix lmtline allocation to be cached - Sync Inline IPSec Rx mbox with kernel - Add support for mark and flag flow actions - Add reta key and hash update ops - Added PTP and multicast filter support MAINTAINERS | 5 +- doc/guides/nics/cnxk.rst | 232 +++++ doc/guides/nics/features/cnxk.ini | 90 ++ doc/guides/nics/features/cnxk_vec.ini | 44 + doc/guides/nics/features/cnxk_vf.ini | 40 + doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 +- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 3 + drivers/common/cnxk/roc_model.h | 6 + drivers/common/cnxk/roc_nix.h | 39 +- drivers/common/cnxk/roc_nix_queue.c | 52 + drivers/common/cnxk/roc_nix_rss.c | 51 +- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +- drivers/common/cnxk/roc_npa.c | 10 +- drivers/common/cnxk/roc_npa.h | 35 +- drivers/common/cnxk/roc_npc.c | 296 +++++- drivers/common/cnxk/roc_npc.h | 39 +- drivers/common/cnxk/roc_npc_mcam.c | 2 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++ drivers/common/cnxk/roc_npc_priv.h | 3 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/roc_platform.h | 13 + drivers/common/cnxk/version.map | 7 + drivers/net/cnxk/cn10k_ethdev.c | 551 +++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 41 + drivers/net/cnxk/cn10k_rte_flow.c | 72 ++ drivers/net/cnxk/cn10k_rte_flow.h | 17 + drivers/net/cnxk/cn10k_rx.c | 79 ++ drivers/net/cnxk/cn10k_rx.h | 653 +++++++++++++ drivers/net/cnxk/cn10k_rx_mseg.c | 17 + drivers/net/cnxk/cn10k_rx_vec.c | 22 + drivers/net/cnxk/cn10k_tx.c | 82 ++ drivers/net/cnxk/cn10k_tx.h | 1605 +++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 + drivers/net/cnxk/cn10k_tx_vec.c | 26 + drivers/net/cnxk/cn9k_ethdev.c | 574 +++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 39 + drivers/net/cnxk/cn9k_rte_flow.c | 72 ++ drivers/net/cnxk/cn9k_rte_flow.h | 17 + drivers/net/cnxk/cn9k_rx.c | 79 ++ drivers/net/cnxk/cn9k_rx.h | 655 +++++++++++++ drivers/net/cnxk/cn9k_rx_mseg.c | 17 + drivers/net/cnxk/cn9k_rx_vec.c | 20 + drivers/net/cnxk/cn9k_tx.c | 81 ++ drivers/net/cnxk/cn9k_tx.h | 1436 +++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 + drivers/net/cnxk/cn9k_tx_vec.c | 26 + drivers/net/cnxk/cnxk_ethdev.c | 1528 +++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 495 ++++++++++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 173 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 910 ++++++++++++++++++ drivers/net/cnxk/cnxk_link.c | 113 +++ drivers/net/cnxk/cnxk_lookup.c | 326 +++++++ drivers/net/cnxk/cnxk_ptp.c | 287 ++++++ drivers/net/cnxk/cnxk_rte_flow.c | 433 +++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 27 + drivers/net/cnxk/cnxk_stats.c | 320 ++++++ drivers/net/cnxk/meson.build | 48 + drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 66 files changed, 12577 insertions(+), 99 deletions(-) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn10k_rte_flow.c create mode 100644 drivers/net/cnxk/cn10k_rte_flow.h create mode 100644 drivers/net/cnxk/cn10k_rx.c create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c create mode 100644 drivers/net/cnxk/cn10k_tx.c create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_rte_flow.c create mode 100644 drivers/net/cnxk/cn9k_rte_flow.h create mode 100644 drivers/net/cnxk/cn9k_rx.c create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c create mode 100644 drivers/net/cnxk/cn9k_tx.c create mode 100644 drivers/net/cnxk/cn9k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c create mode 100644 drivers/net/cnxk/cnxk_link.c create mode 100644 drivers/net/cnxk/cnxk_lookup.c create mode 100644 drivers/net/cnxk/cnxk_ptp.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h create mode 100644 drivers/net/cnxk/cnxk_stats.c create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 01/62] common/cnxk: add support to lock NIX RQ contexts 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram ` (61 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch will consider device argument to lock rss table in NIX. This patch also adds few misc fixes such as disabling NIX Tx vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ for Tx completions and TM related stats API. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 31 ++++++++++-- drivers/common/cnxk/roc_nix_queue.c | 2 + drivers/common/cnxk/roc_nix_rss.c | 51 ++++++++++++++++++-- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +++++++++++++++++++++++++++++++++- drivers/common/cnxk/roc_platform.h | 2 + drivers/common/cnxk/version.map | 1 + 6 files changed, 163 insertions(+), 10 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index b39f461..6d9ac10 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -85,10 +85,11 @@ struct roc_nix_eeprom_info { #define ROC_NIX_LF_RX_CFG_LEN_OL3 BIT_ULL(41) /* Group 0 will be used for RSS, 1 -7 will be used for npc_flow RSS action*/ -#define ROC_NIX_RSS_GROUP_DEFAULT 0 -#define ROC_NIX_RSS_GRPS 8 -#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 -#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_GROUP_DEFAULT 0 +#define ROC_NIX_RSS_GRPS 8 +#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 +#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_MCAM_IDX_DEFAULT (-1) #define ROC_NIX_DEFAULT_HW_FRS 1514 @@ -184,6 +185,7 @@ struct roc_nix_sq { enum roc_nix_sq_max_sqe_sz max_sqe_sz; uint32_t nb_desc; uint16_t qid; + bool sso_ena; /* End of Input parameters */ uint16_t sqes_per_sqb_log2; struct roc_nix *roc_nix; @@ -241,6 +243,8 @@ struct roc_nix { uint16_t max_sqb_count; enum roc_nix_rss_reta_sz reta_sz; bool enable_loop; + bool hw_vlan_ins; + uint8_t lock_rx_ctx; /* End of input parameters */ /* LMT line base for "Per Core Tx LMT line" mode*/ uintptr_t lmt_base; @@ -371,6 +375,22 @@ struct roc_nix_tm_shaper_profile { void (*free_fn)(void *profile); }; +enum roc_nix_tm_node_stats_type { + ROC_NIX_TM_NODE_PKTS_DROPPED, + ROC_NIX_TM_NODE_BYTES_DROPPED, + ROC_NIX_TM_NODE_GREEN_PKTS, + ROC_NIX_TM_NODE_GREEN_BYTES, + ROC_NIX_TM_NODE_YELLOW_PKTS, + ROC_NIX_TM_NODE_YELLOW_BYTES, + ROC_NIX_TM_NODE_RED_PKTS, + ROC_NIX_TM_NODE_RED_BYTES, + ROC_NIX_TM_NODE_STATS_MAX, +}; + +struct roc_nix_tm_node_stats { + uint64_t stats[ROC_NIX_TM_NODE_STATS_MAX]; +}; + int __roc_api roc_nix_tm_node_add(struct roc_nix *roc_nix, struct roc_nix_tm_node *roc_node); int __roc_api roc_nix_tm_node_delete(struct roc_nix *roc_nix, uint32_t node_id, @@ -408,6 +428,9 @@ roc_nix_tm_shaper_profile_get(struct roc_nix *roc_nix, uint32_t profile_id); struct roc_nix_tm_shaper_profile *__roc_api roc_nix_tm_shaper_profile_next( struct roc_nix *roc_nix, struct roc_nix_tm_shaper_profile *__prev); +int __roc_api roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, + uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *stats); /* * TM ratelimit tree API. */ diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index fbf7efa..1c62aa2 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -582,6 +582,7 @@ sq_cn9k_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); @@ -679,6 +680,7 @@ sq_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); diff --git a/drivers/common/cnxk/roc_nix_rss.c b/drivers/common/cnxk/roc_nix_rss.c index 2d7b84a..7de69aa 100644 --- a/drivers/common/cnxk/roc_nix_rss.c +++ b/drivers/common/cnxk/roc_nix_rss.c @@ -52,7 +52,7 @@ roc_nix_rss_key_get(struct roc_nix *roc_nix, uint8_t key[ROC_NIX_RSS_KEY_LEN]) static int nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_aq_enq_req *req; @@ -77,6 +77,27 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -88,7 +109,7 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, static int nix_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_cn10k_aq_enq_req *req; @@ -113,6 +134,27 @@ nix_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -133,9 +175,10 @@ roc_nix_rss_reta_set(struct roc_nix *roc_nix, uint8_t group, return NIX_ERR_PARAM; if (roc_model_is_cn9k()) - rc = nix_cn9k_rss_reta_set(nix, group, reta); + rc = nix_cn9k_rss_reta_set(nix, group, reta, + roc_nix->lock_rx_ctx); else - rc = nix_rss_reta_set(nix, group, reta); + rc = nix_rss_reta_set(nix, group, reta, roc_nix->lock_rx_ctx); if (rc) return rc; diff --git a/drivers/common/cnxk/roc_nix_tm_utils.c b/drivers/common/cnxk/roc_nix_tm_utils.c index 1d7dd68..6b9543e 100644 --- a/drivers/common/cnxk/roc_nix_tm_utils.c +++ b/drivers/common/cnxk/roc_nix_tm_utils.c @@ -409,6 +409,7 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, volatile uint64_t *reg, volatile uint64_t *regval, volatile uint64_t *regval_mask) { + struct roc_nix *roc_nix = nix_priv_to_roc_nix(nix); uint8_t k = 0, hw_lvl, parent_lvl; uint64_t parent = 0, child = 0; enum roc_nix_tm_tree tree; @@ -454,8 +455,11 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, reg[k] = NIX_AF_SMQX_CFG(schq); regval[k] = (BIT_ULL(50) | NIX_MIN_HW_FRS | ((nix->mtu & 0xFFFF) << 8)); - regval_mask[k] = - ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | GENMASK_ULL(23, 8)); + /* Maximum Vtag insertion size as a multiple of four bytes */ + if (roc_nix->hw_vlan_ins) + regval[k] |= (0x2ULL << 36); + regval_mask[k] = ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | + GENMASK_ULL(23, 8) | GENMASK_ULL(38, 36)); k++; /* Parent and schedule conf */ @@ -1000,3 +1004,81 @@ nix_tm_shaper_profile_free(struct nix_tm_shaper_profile *profile) (profile->free_fn)(profile); } + +int +roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *n_stats) +{ + struct nix *nix = roc_nix_to_nix_priv(roc_nix); + struct mbox *mbox = (&nix->dev)->mbox; + struct nix_txschq_config *req, *rsp; + struct nix_tm_node *node; + uint32_t schq; + int rc, i; + + node = nix_tm_node_search(nix, node_id, ROC_NIX_TM_USER); + if (!node) + return NIX_ERR_TM_INVALID_NODE; + + if (node->hw_lvl != NIX_TXSCH_LVL_TL1) + return NIX_ERR_OP_NOTSUP; + + schq = node->hw_id; + /* Skip fetch if not requested */ + if (!n_stats) + goto clear_stats; + + memset(n_stats, 0, sizeof(struct roc_nix_tm_node_stats)); + /* Check if node has HW resource */ + if (!(node->flags & NIX_TM_NODE_HWRES)) + return 0; + + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->read = 1; + req->lvl = NIX_TXSCH_LVL_TL1; + + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + rc = mbox_process_msg(mbox, (void **)&rsp); + if (rc) + return rc; + + /* Return stats */ + n_stats->stats[ROC_NIX_TM_NODE_PKTS_DROPPED] = rsp->regval[0]; + n_stats->stats[ROC_NIX_TM_NODE_BYTES_DROPPED] = rsp->regval[1]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_PKTS] = rsp->regval[2]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_BYTES] = rsp->regval[3]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_PKTS] = rsp->regval[4]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_BYTES] = rsp->regval[5]; + n_stats->stats[ROC_NIX_TM_NODE_RED_PKTS] = rsp->regval[6]; + n_stats->stats[ROC_NIX_TM_NODE_RED_BYTES] = rsp->regval[7]; + +clear_stats: + if (!clear) + return 0; + + /* Clear all the stats */ + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->lvl = NIX_TXSCH_LVL_TL1; + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + return mbox_process_msg(mbox, (void **)&rsp); +} diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 7864fa4..911ae15 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -127,6 +127,8 @@ #define plt_memzone_reserve_cache_align(name, sz) \ rte_memzone_reserve_aligned(name, sz, 0, 0, RTE_CACHE_LINE_SIZE) #define plt_memzone_free rte_memzone_free +#define plt_memzone_reserve_aligned(name, len, flags, align) \ + rte_memzone_reserve_aligned((name), (len), 0, (flags), (align)) #define plt_tsc_hz rte_get_tsc_hz #define plt_delay_ms rte_delay_ms diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 8e67c83..c39d76f 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -123,6 +123,7 @@ INTERNAL { roc_nix_tm_node_parent_update; roc_nix_tm_node_pkt_mode_update; roc_nix_tm_node_shaper_update; + roc_nix_tm_node_stats_get; roc_nix_tm_node_suspend_resume; roc_nix_tm_prealloc_res; roc_nix_tm_rlimit_sq; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 02/62] common/cnxk: fix batch alloc completion poll logic 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram ` (60 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Jerin Jacob <jerinj@marvell.com> The instruction generation was not correct due to fact that volatile suppose to use with ccode variable as well. Change the logic to use gcc atomic builtin to simplify and avoid explicit volatile from the code. Fixes: 81af26789316 ("common/cnxk: support NPA batch alloc/free") Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com> --- drivers/common/cnxk/roc_npa.c | 2 +- drivers/common/cnxk/roc_npa.h | 30 +++++++++++++++--------------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index f1e03b7..5ba6e81 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -236,7 +236,7 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Block size should be cache line aligned and in range of 128B-128KB */ if (block_size % ROC_ALIGN || block_size < 128 || - block_size > 128 * 1024) + block_size > ROC_NPA_MAX_BLOCK_SZ) return NPA_ERR_INVALID_BLOCK_SZ; pos = 0; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 89f5c6f..59d6223 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -8,6 +8,7 @@ #define ROC_AURA_ID_MASK (BIT_ULL(16) - 1) #define ROC_AURA_OP_LIMIT_MASK (BIT_ULL(36) - 1) +#define ROC_NPA_MAX_BLOCK_SZ (128 * 1024) #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 @@ -219,6 +220,17 @@ roc_npa_aura_batch_alloc_issue(uint64_t aura_handle, uint64_t *buf, return 0; } +static inline void +roc_npa_batch_alloc_wait(uint64_t *cache_line) +{ + /* Batch alloc status code is updated in bits [5:6] of the first word + * of the 128 byte cache line. + */ + while (((__atomic_load_n(cache_line, __ATOMIC_RELAXED) >> 5) & 0x3) == + ALLOC_CCODE_INVAL) + ; +} + static inline unsigned int roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) { @@ -231,17 +243,10 @@ roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int ccode; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); - + roc_npa_batch_alloc_wait(&aligned_buf[i]); count += status->count; } @@ -261,16 +266,11 @@ roc_npa_aura_batch_alloc_extract(uint64_t *buf, uint64_t *aligned_buf, /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int line_count, ccode; + int line_count; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); + roc_npa_batch_alloc_wait(&aligned_buf[i]); line_count = status->count; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 03/62] common/cnxk: add support to dump flow entries 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram ` (59 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add NPC support API to dump created flow entries. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_npc.c | 20 ++ drivers/common/cnxk/roc_npc.h | 12 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npc_priv.h | 2 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/version.map | 2 + 8 files changed, 652 insertions(+), 2 deletions(-) create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c diff --git a/drivers/common/cnxk/hw/npc.h b/drivers/common/cnxk/hw/npc.h index e0f06bf..68c5037 100644 --- a/drivers/common/cnxk/hw/npc.h +++ b/drivers/common/cnxk/hw/npc.h @@ -193,6 +193,7 @@ enum npc_kpu_lb_ltype { NPC_LT_LB_EXDSA, NPC_LT_LB_EXDSA_VLAN, NPC_LT_LB_FDSA, + NPC_LT_LB_VLAN_EXDSA, NPC_LT_LB_CUSTOM0 = 0xE, NPC_LT_LB_CUSTOM1 = 0xF, }; @@ -208,6 +209,7 @@ enum npc_kpu_lc_ltype { NPC_LT_LC_MPLS, NPC_LT_LC_NSH, NPC_LT_LC_FCOE, + NPC_LT_LC_NGIO, NPC_LT_LC_CUSTOM0 = 0xE, NPC_LT_LC_CUSTOM1 = 0xF, }; diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index 178bce7..e7ab79f 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -37,6 +37,7 @@ sources = files( 'roc_npa_irq.c', 'roc_npc.c', 'roc_npc_mcam.c', + 'roc_npc_mcam_dump.c', 'roc_npc_parse.c', 'roc_npc_utils.c', 'roc_platform.c', diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index abaef77..81c7fd9 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -870,3 +870,23 @@ roc_npc_flow_destroy(struct roc_npc *roc_npc, struct roc_npc_flow *flow) plt_free(flow); return 0; } + +void +roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + struct roc_npc_flow *flow_iter; + struct npc_flow_list *list; + uint32_t max_prio, i; + + max_prio = npc->flow_max_priority; + + for (i = 0; i < max_prio; i++) { + list = &npc->flow_list[i]; + + /* List in ascending order of mcam entries */ + TAILQ_FOREACH(flow_iter, list, next) { + roc_npc_flow_mcam_dump(file, roc_npc, flow_iter); + } + } +} diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 223c4ba..115bcd5 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -90,6 +90,11 @@ struct roc_npc_attr { uint32_t reserved : 30; /**< Reserved, must be zero. */ }; +struct roc_npc_flow_dump_data { + uint8_t lid; + uint16_t ltype; +}; + struct roc_npc_flow { uint8_t nix_intf; uint8_t enable; @@ -102,6 +107,9 @@ struct roc_npc_flow { uint64_t mcam_mask[ROC_NPC_MAX_MCAM_WIDTH_DWORDS]; uint64_t npc_action; uint64_t vtag_action; +#define ROC_NPC_MAX_FLOW_PATTERNS 32 + struct roc_npc_flow_dump_data dump_data[ROC_NPC_MAX_FLOW_PATTERNS]; + uint16_t num_patterns; TAILQ_ENTRY(roc_npc_flow) next; }; @@ -185,5 +193,7 @@ int __roc_api roc_npc_mcam_clear_counter(struct roc_npc *roc_npc, uint32_t ctr_id); int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); - +void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); +void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *mcam); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/roc_npc_mcam_dump.c b/drivers/common/cnxk/roc_npc_mcam_dump.c new file mode 100644 index 0000000..19b4901 --- /dev/null +++ b/drivers/common/cnxk/roc_npc_mcam_dump.c @@ -0,0 +1,611 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#define NPC_MAX_FIELD_NAME_SIZE 80 +#define NPC_RX_ACTIONOP_MASK GENMASK(3, 0) +#define NPC_RX_ACTION_PFFUNC_MASK GENMASK(19, 4) +#define NPC_RX_ACTION_INDEX_MASK GENMASK(39, 20) +#define NPC_RX_ACTION_MATCH_MASK GENMASK(55, 40) +#define NPC_RX_ACTION_FLOWKEY_MASK GENMASK(60, 56) + +#define NPC_TX_ACTION_INDEX_MASK GENMASK(31, 12) +#define NPC_TX_ACTION_MATCH_MASK GENMASK(47, 32) + +#define NIX_RX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_RX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_RX_VTAGACT_VTAG0_TYPE_MASK GENMASK(14, 12) +#define NIX_RX_VTAGACT_VTAG0_VALID_MASK BIT_ULL(15) + +#define NIX_RX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_RX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_RX_VTAGACT_VTAG1_TYPE_MASK GENMASK(46, 44) +#define NIX_RX_VTAGACT_VTAG1_VALID_MASK BIT_ULL(47) + +#define NIX_TX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_TX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_TX_VTAGACT_VTAG0_OP_MASK GENMASK(13, 12) +#define NIX_TX_VTAGACT_VTAG0_DEF_MASK GENMASK(25, 16) + +#define NIX_TX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_TX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_TX_VTAGACT_VTAG1_OP_MASK GENMASK(45, 44) +#define NIX_TX_VTAGACT_VTAG1_DEF_MASK GENMASK(57, 48) + +struct npc_rx_parse_nibble_s { + uint16_t chan : 3; + uint16_t errlev : 1; + uint16_t errcode : 2; + uint16_t l2l3bm : 1; + uint16_t laflags : 2; + uint16_t latype : 1; + uint16_t lbflags : 2; + uint16_t lbtype : 1; + uint16_t lcflags : 2; + uint16_t lctype : 1; + uint16_t ldflags : 2; + uint16_t ldtype : 1; + uint16_t leflags : 2; + uint16_t letype : 1; + uint16_t lfflags : 2; + uint16_t lftype : 1; + uint16_t lgflags : 2; + uint16_t lgtype : 1; + uint16_t lhflags : 2; + uint16_t lhtype : 1; +} __plt_packed; + +static const char *const intf_str[] = { + "NIX-RX", + "NIX-TX", +}; + +static const char *const ltype_str[NPC_MAX_LID][NPC_MAX_LT] = { + [NPC_LID_LA][0] = "NONE", + [NPC_LID_LA][NPC_LT_LA_ETHER] = "LA_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_ETHER] = "LA_IH_NIX_ETHER", + [NPC_LID_LA][NPC_LT_LA_HIGIG2_ETHER] = "LA_HIGIG2_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_HIGIG2_ETHER] = "LA_IH_NIX_HIGIG2_ETHER", + [NPC_LID_LB][0] = "NONE", + [NPC_LID_LB][NPC_LT_LB_CTAG] = "LB_CTAG", + [NPC_LID_LB][NPC_LT_LB_STAG_QINQ] = "LB_STAG_QINQ", + [NPC_LID_LB][NPC_LT_LB_ETAG] = "LB_ETAG", + [NPC_LID_LB][NPC_LT_LB_EXDSA] = "LB_EXDSA", + [NPC_LID_LB][NPC_LT_LB_VLAN_EXDSA] = "LB_VLAN_EXDSA", + [NPC_LID_LC][0] = "NONE", + [NPC_LID_LC][NPC_LT_LC_IP] = "LC_IP", + [NPC_LID_LC][NPC_LT_LC_IP6] = "LC_IP6", + [NPC_LID_LC][NPC_LT_LC_ARP] = "LC_ARP", + [NPC_LID_LC][NPC_LT_LC_IP6_EXT] = "LC_IP6_EXT", + [NPC_LID_LC][NPC_LT_LC_NGIO] = "LC_NGIO", + [NPC_LID_LD][0] = "NONE", + [NPC_LID_LD][NPC_LT_LD_ICMP] = "LD_ICMP", + [NPC_LID_LD][NPC_LT_LD_ICMP6] = "LD_ICMP6", + [NPC_LID_LD][NPC_LT_LD_UDP] = "LD_UDP", + [NPC_LID_LD][NPC_LT_LD_TCP] = "LD_TCP", + [NPC_LID_LD][NPC_LT_LD_SCTP] = "LD_SCTP", + [NPC_LID_LD][NPC_LT_LD_GRE] = "LD_GRE", + [NPC_LID_LD][NPC_LT_LD_NVGRE] = "LD_NVGRE", + [NPC_LID_LE][0] = "NONE", + [NPC_LID_LE][NPC_LT_LE_VXLAN] = "LE_VXLAN", + [NPC_LID_LE][NPC_LT_LE_ESP] = "LE_ESP", + [NPC_LID_LE][NPC_LT_LE_GTPC] = "LE_GTPC", + [NPC_LID_LE][NPC_LT_LE_GTPU] = "LE_GTPU", + [NPC_LID_LE][NPC_LT_LE_GENEVE] = "LE_GENEVE", + [NPC_LID_LE][NPC_LT_LE_VXLANGPE] = "LE_VXLANGPE", + [NPC_LID_LF][0] = "NONE", + [NPC_LID_LF][NPC_LT_LF_TU_ETHER] = "LF_TU_ETHER", + [NPC_LID_LG][0] = "NONE", + [NPC_LID_LG][NPC_LT_LG_TU_IP] = "LG_TU_IP", + [NPC_LID_LG][NPC_LT_LG_TU_IP6] = "LG_TU_IP6", + [NPC_LID_LH][0] = "NONE", + [NPC_LID_LH][NPC_LT_LH_TU_UDP] = "LH_TU_UDP", + [NPC_LID_LH][NPC_LT_LH_TU_TCP] = "LH_TU_TCP", + [NPC_LID_LH][NPC_LT_LH_TU_SCTP] = "LH_TU_SCTP", + [NPC_LID_LH][NPC_LT_LH_TU_ESP] = "LH_TU_ESP", +}; + +static uint16_t +npc_get_nibbles(struct roc_npc_flow *flow, uint16_t size, uint32_t bit_offset) +{ + uint32_t byte_index, noffset; + uint16_t data, mask; + uint8_t *bytes; + + bytes = (uint8_t *)flow->mcam_data; + mask = (1ULL << (size * 4)) - 1; + byte_index = bit_offset / 8; + noffset = bit_offset % 8; + data = *(unaligned_uint16_t *)&bytes[byte_index]; + data >>= noffset; + data &= mask; + + return data; +} + +static void +npc_flow_print_parse_nibbles(FILE *file, struct roc_npc_flow *flow, + uint64_t parse_nibbles) +{ + struct npc_rx_parse_nibble_s *rx_parse; + uint32_t data, offset = 0; + + rx_parse = (struct npc_rx_parse_nibble_s *)&parse_nibbles; + + if (rx_parse->chan) { + data = npc_get_nibbles(flow, 3, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_CHAN:%#03X\n", data); + offset += 12; + } + + if (rx_parse->errlev) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRLEV:%#X\n", data); + offset += 4; + } + + if (rx_parse->errcode) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRCODE:%#02X\n", data); + offset += 8; + } + + if (rx_parse->l2l3bm) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_L2L3_BCAST:%#X\n", data); + offset += 4; + } + + if (rx_parse->latype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_LTYPE:%s\n", + ltype_str[NPC_LID_LA][data]); + offset += 4; + } + + if (rx_parse->laflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lbtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_LTYPE:%s\n", + ltype_str[NPC_LID_LB][data]); + offset += 4; + } + + if (rx_parse->lbflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lctype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_LTYPE:%s\n", + ltype_str[NPC_LID_LC][data]); + offset += 4; + } + + if (rx_parse->lcflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->ldtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_LTYPE:%s\n", + ltype_str[NPC_LID_LD][data]); + offset += 4; + } + + if (rx_parse->ldflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->letype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_LTYPE:%s\n", + ltype_str[NPC_LID_LE][data]); + offset += 4; + } + + if (rx_parse->leflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lftype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_LTYPE:%s\n", + ltype_str[NPC_LID_LF][data]); + offset += 4; + } + + if (rx_parse->lfflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lgtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_LTYPE:%s\n", + ltype_str[NPC_LID_LG][data]); + offset += 4; + } + + if (rx_parse->lgflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lhtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_LTYPE:%s\n", + ltype_str[NPC_LID_LH][data]); + offset += 4; + } + + if (rx_parse->lhflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_FLAGS:%#02X\n", data); + } +} + +static void +npc_flow_print_xtractinfo(FILE *file, struct npc_xtract_info *lfinfo, + struct roc_npc_flow *flow, int lid, int lt) +{ + uint8_t *datastart, *maskstart; + int i; + + datastart = (uint8_t *)&flow->mcam_data + lfinfo->key_off; + maskstart = (uint8_t *)&flow->mcam_mask + lfinfo->key_off; + + fprintf(file, "\t%s, hdr offset:%#X, len:%#X, key offset:%#X, ", + ltype_str[lid][lt], lfinfo->hdr_off, lfinfo->len, + lfinfo->key_off); + + fprintf(file, "Data:0X"); + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", datastart[i]); + + fprintf(file, ", Mask:0X"); + + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", maskstart[i]); + + fprintf(file, "\n"); +} + +static void +npc_flow_print_item(FILE *file, struct npc *npc, struct npc_xtract_info *xinfo, + struct roc_npc_flow *flow, int intf, int lid, int lt, + int ld) +{ + struct npc_xtract_info *lflags_info; + int i, lf_cfg = 0; + + npc_flow_print_xtractinfo(file, xinfo, flow, lid, lt); + + if (xinfo->flags_enable) { + lf_cfg = npc->prx_lfcfg[ld].i; + + if (lf_cfg != lid) + return; + + for (i = 0; i < NPC_MAX_LFL; i++) { + lflags_info = npc->prx_fxcfg[intf][ld][i].xtract; + + npc_flow_print_xtractinfo(file, lflags_info, flow, lid, + lt); + } + } +} + +static void +npc_flow_dump_patterns(FILE *file, struct npc *npc, struct roc_npc_flow *flow) +{ + struct npc_lid_lt_xtract_info *lt_xinfo; + struct npc_xtract_info *xinfo; + uint32_t intf, lid, ld, i; + uint64_t parse_nibbles; + uint16_t ltype; + + intf = flow->nix_intf; + parse_nibbles = npc->keyx_supp_nmask[intf]; + npc_flow_print_parse_nibbles(file, flow, parse_nibbles); + + for (i = 0; i < flow->num_patterns; i++) { + lid = flow->dump_data[i].lid; + ltype = flow->dump_data[i].ltype; + lt_xinfo = &npc->prx_dxcfg[intf][lid][ltype]; + + for (ld = 0; ld < NPC_MAX_LD; ld++) { + xinfo = <_xinfo->xtract[ld]; + if (!xinfo->enable) + continue; + npc_flow_print_item(file, npc, xinfo, flow, intf, lid, + ltype, ld); + } + } +} + +static void +npc_flow_dump_tx_action(FILE *file, uint64_t npc_action) +{ + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + uint32_t tx_op, index, match_id; + + tx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (tx_op) { + case NIX_TX_ACTIONOP_DROP: + fprintf(file, "NIX_TX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_TX_ACTIONOP_UCAST_DEFAULT: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_DEFAULT); + break; + case NIX_TX_ACTIONOP_UCAST_CHAN: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_CHAN); + plt_strlcpy(index_name, + "Transmit Channel:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_MCAST: + fprintf(file, "NIX_TX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_MCAST); + plt_strlcpy(index_name, + "Multicast Table Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_DROP_VIOL: + fprintf(file, "NIX_TX_ACTIONOP_DROP_VIOL (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_DROP_VIOL); + break; + default: + plt_err("Unknown NIX_TX_ACTIONOP found"); + return; + } + + index = ((npc_action & NPC_TX_ACTION_INDEX_MASK) >> 12) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_TX_ACTION_MATCH_MASK) >> 32) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); +} + +static void +npc_flow_dump_rx_action(FILE *file, uint64_t npc_action) +{ + uint32_t rx_op, pf_func, index, match_id, flowkey_alg; + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + + rx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (rx_op) { + case NIX_RX_ACTIONOP_DROP: + fprintf(file, "NIX_RX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_RX_ACTIONOP_UCAST: + fprintf(file, "NIX_RX_ACTIONOP_UCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST); + plt_strlcpy(index_name, "RQ Index", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_UCAST_IPSEC: + fprintf(file, "NIX_RX_ACTIONOP_UCAST_IPSEC (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST_IPSEC); + plt_strlcpy(index_name, "RQ Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_MCAST: + fprintf(file, "NIX_RX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MCAST); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_RSS: + fprintf(file, "NIX_RX_ACTIONOP_RSS (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_RSS); + plt_strlcpy(index_name, "RSS Group Index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_PF_FUNC_DROP: + fprintf(file, "NIX_RX_ACTIONOP_PF_FUNC_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_PF_FUNC_DROP); + break; + case NIX_RX_ACTIONOP_MIRROR: + fprintf(file, "NIX_RX_ACTIONOP_MIRROR (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MIRROR); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + default: + plt_err("Unknown NIX_RX_ACTIONOP found"); + return; + } + + pf_func = ((npc_action & NPC_RX_ACTION_PFFUNC_MASK) >> 4) & + GENMASK(15, 0); + + fprintf(file, "\tPF_FUNC: %#04X\n", pf_func); + + index = ((npc_action & NPC_RX_ACTION_INDEX_MASK) >> 20) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_RX_ACTION_MATCH_MASK) >> 40) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); + + flowkey_alg = ((npc_action & NPC_RX_ACTION_FLOWKEY_MASK) >> 56) & + GENMASK(4, 0); + + fprintf(file, "\tFlow Key Alg:%#X\n", flowkey_alg); +} + +static void +npc_flow_dump_parsed_action(FILE *file, uint64_t npc_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX Action:%#016lX\n", npc_action); + npc_flow_dump_rx_action(file, npc_action); + } else { + fprintf(file, "NPC TX Action:%#016lX\n", npc_action); + npc_flow_dump_tx_action(file, npc_action); + } +} + +static void +npc_flow_dump_rx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t type, lid, relptr; + + if (vtag_action & NIX_RX_VTAGACT_VTAG0_VALID_MASK) { + relptr = vtag_action & NIX_RX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG0_TYPE_MASK) >> 12) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG0:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } + + if (vtag_action & NIX_RX_VTAGACT_VTAG1_VALID_MASK) { + relptr = ((vtag_action & NIX_RX_VTAGACT_VTAG1_RELPTR_MASK) >> + 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG1_TYPE_MASK) >> 44) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } +} + +static void +npc_get_vtag_opname(uint32_t op, char *opname, int len) +{ + switch (op) { + case 0x0: + plt_strlcpy(opname, "NOP", len - 1); + break; + case 0x1: + plt_strlcpy(opname, "INSERT", len - 1); + break; + case 0x2: + plt_strlcpy(opname, "REPLACE", len - 1); + break; + default: + plt_err("Unknown vtag op found"); + break; + } +} + +static void +npc_flow_dump_tx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t relptr, lid, op, vtag_def; + char opname[10]; + + relptr = vtag_action & NIX_TX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG0_OP_MASK) >> 12) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG0_DEF_MASK) >> 16) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG0 relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); + + relptr = ((vtag_action & NIX_TX_VTAGACT_VTAG1_RELPTR_MASK) >> 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG1_OP_MASK) >> 44) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG1_DEF_MASK) >> 48) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); +} + +static void +npc_flow_dump_vtag_action(FILE *file, uint64_t vtag_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_rx_vtag_action(file, vtag_action); + } else { + fprintf(file, "NPC TX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_tx_vtag_action(file, vtag_action); + } +} + +void +roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *flow) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + bool is_rx = 0; + int i; + + fprintf(file, "MCAM Index:%d\n", flow->mcam_id); + fprintf(file, "Interface :%s (%d)\n", intf_str[flow->nix_intf], + flow->nix_intf); + fprintf(file, "Priority :%d\n", flow->priority); + + if (flow->nix_intf == NIX_INTF_RX) + is_rx = 1; + + npc_flow_dump_parsed_action(file, flow->npc_action, is_rx); + npc_flow_dump_vtag_action(file, flow->vtag_action, is_rx); + fprintf(file, "Patterns:\n"); + npc_flow_dump_patterns(file, npc, flow); + + fprintf(file, "MCAM Raw Data :\n"); + + for (i = 0; i < ROC_NPC_MAX_MCAM_WIDTH_DWORDS; i++) { + fprintf(file, "\tDW%d :%016lX\n", i, flow->mcam_data[i]); + fprintf(file, "\tDW%d_Mask:%016lX\n", i, flow->mcam_mask[i]); + } + + fprintf(file, "\n"); +} diff --git a/drivers/common/cnxk/roc_npc_priv.h b/drivers/common/cnxk/roc_npc_priv.h index 8bc5bac..961583b 100644 --- a/drivers/common/cnxk/roc_npc_priv.h +++ b/drivers/common/cnxk/roc_npc_priv.h @@ -47,7 +47,7 @@ #define NPC_RVUPF_MAX_9XXX 0x10 /* HRM: RVU_PRIV_CONST */ #define NPC_RVUPF_MAX_10XX 0x20 /* HRM: RVU_PRIV_CONST */ #define NPC_NIXLF_MAX 0x80 /* HRM: NIX_AF_CONST2 */ -#define NPC_MCAME_PER_PF 2 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ +#define NPC_MCAME_PER_PF 3 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ #define NPC_MCAME_PER_LF 1 /* DRV: RSVD_MCAM_ENTRIES_PER_NIXLF */ #define NPC_MCAME_RESVD_9XXX \ (NPC_NIXLF_MAX * NPC_MCAME_PER_LF + \ diff --git a/drivers/common/cnxk/roc_npc_utils.c b/drivers/common/cnxk/roc_npc_utils.c index 1fb8973..5c97588 100644 --- a/drivers/common/cnxk/roc_npc_utils.c +++ b/drivers/common/cnxk/roc_npc_utils.c @@ -206,6 +206,7 @@ npc_update_parse_state(struct npc_parse_state *pst, uint8_t flags) { struct npc_lid_lt_xtract_info *xinfo; + struct roc_npc_flow_dump_data *dump; struct npc_xtract_info *lfinfo; int intf, lf_cfg; int i, j, rc = 0; @@ -248,6 +249,9 @@ npc_update_parse_state(struct npc_parse_state *pst, } done: + dump = &pst->flow->dump_data[pst->flow->num_patterns++]; + dump->lid = lid; + dump->ltype = lt; pst->pattern++; return 0; } diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index c39d76f..a11ba4d 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -160,6 +160,8 @@ INTERNAL { roc_npc_fini; roc_npc_flow_create; roc_npc_flow_destroy; + roc_npc_flow_dump; + roc_npc_flow_mcam_dump; roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 04/62] common/cnxk: support for mark and flag flow actions 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (2 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram ` (58 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add roc API to get mark action. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 17 +++++++++++++++++ drivers/common/cnxk/roc_npc.h | 3 +++ drivers/common/cnxk/version.map | 2 ++ 3 files changed, 22 insertions(+) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 81c7fd9..e6a5036 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -757,6 +757,23 @@ npc_rss_action_program(struct roc_npc *roc_npc, return 0; } +int +roc_npc_mark_actions_get(struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + return npc->mark_actions; +} + +int +roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + npc->mark_actions -= count; + return npc->mark_actions; +} + struct roc_npc_flow * roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, const struct roc_npc_item_info pattern[], diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 115bcd5..cf6f732 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -196,4 +196,7 @@ int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, struct roc_npc_flow *mcam); +int __roc_api roc_npc_mark_actions_get(struct roc_npc *roc_npc); +int __roc_api roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, + uint32_t count); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index a11ba4d..554459b 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -165,6 +165,8 @@ INTERNAL { roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; + roc_npc_mark_actions_get; + roc_npc_mark_actions_sub_return; roc_npc_mcam_alloc_entries; roc_npc_mcam_alloc_entry; roc_npc_mcam_clear_counter; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 05/62] common/cnxk: allocate lmt region in userspace 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (3 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram ` (57 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Harman Kalra <hkalra@marvell.com> As per the new LMTST design, userspace shall allocate lmt region, setup the DMA translation and share the IOVA with kernel via MBOX. Kernel will convert this IOVA to physical memory and update the LMT table entry with the same. With this new design also shared mode (i.e. all pci funcs sharing the LMT region allocated by primary/base pci func) is intact. Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 ++++++++++++++++++-------------------- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 3 ++ drivers/common/cnxk/roc_platform.h | 11 +++++ 5 files changed, 63 insertions(+), 52 deletions(-) diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h index 67f5d13..32e383c 100644 --- a/drivers/common/cnxk/roc_api.h +++ b/drivers/common/cnxk/roc_api.h @@ -24,6 +24,8 @@ /* Platform definition */ #include "roc_platform.h" +#define ROC_LMT_LINE_SZ 128 +#define ROC_NUM_LMT_LINES 2048 #define ROC_LMT_LINES_PER_CORE_LOG2 5 #define ROC_LMT_LINE_SIZE_LOG2 7 #define ROC_LMT_BASE_PER_CORE_LOG2 \ diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index a39acc9..adff779 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -915,43 +915,30 @@ dev_vf_mbase_put(struct plt_pci_device *pci_dev, uintptr_t vf_mbase) mbox_mem_unmap((void *)vf_mbase, MBOX_SIZE * pci_dev->max_vfs); } -static uint16_t -dev_pf_total_vfs(struct plt_pci_device *pci_dev) -{ - uint16_t total_vfs = 0; - int sriov_pos, rc; - - sriov_pos = - plt_pci_find_ext_capability(pci_dev, ROC_PCI_EXT_CAP_ID_SRIOV); - if (sriov_pos <= 0) { - plt_warn("Unable to find SRIOV cap, rc=%d", sriov_pos); - return 0; - } - - rc = plt_pci_read_config(pci_dev, &total_vfs, 2, - sriov_pos + ROC_PCI_SRIOV_TOTAL_VF); - if (rc < 0) { - plt_warn("Unable to read SRIOV cap, rc=%d", rc); - return 0; - } - - return total_vfs; -} - static int -dev_setup_shared_lmt_region(struct mbox *mbox) +dev_setup_shared_lmt_region(struct mbox *mbox, bool valid_iova, uint64_t iova) { struct lmtst_tbl_setup_req *req; req = mbox_alloc_msg_lmtst_tbl_setup(mbox); - req->pcifunc = idev_lmt_pffunc_get(); + /* This pcifunc is defined with primary pcifunc whose LMT address + * will be shared. If call contains valid IOVA, following pcifunc + * field is of no use. + */ + req->pcifunc = valid_iova ? 0 : idev_lmt_pffunc_get(); + req->use_local_lmt_region = valid_iova; + req->lmt_iova = iova; return mbox_process(mbox); } +/* Total no of lines * size of each lmtline */ +#define LMT_REGION_SIZE (ROC_NUM_LMT_LINES * ROC_LMT_LINE_SZ) static int -dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) +dev_lmt_setup(struct dev *dev) { + char name[PLT_MEMZONE_NAMESIZE]; + const struct plt_memzone *mz; struct idev_cfg *idev; int rc; @@ -965,8 +952,11 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) /* Set common lmt region from second pf_func onwards. */ if (!dev->disable_shared_lmt && idev_lmt_pffunc_get() && dev->pf_func != idev_lmt_pffunc_get()) { - rc = dev_setup_shared_lmt_region(dev->mbox); + rc = dev_setup_shared_lmt_region(dev->mbox, false, 0); if (!rc) { + /* On success, updating lmt base of secondary pf_funcs + * with primary pf_func's lmt base. + */ dev->lmt_base = roc_idev_lmt_base_addr_get(); return rc; } @@ -975,34 +965,30 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) dev->pf_func, rc); } - if (dev_is_vf(dev)) { - /* VF BAR4 should always be sufficient enough to - * hold LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (RVU_LMT_LINE_MAX * RVU_LMT_SZ)) { - plt_err("Not enough bar4 space for lmt lines"); - return -EFAULT; - } + /* Allocating memory for LMT region */ + sprintf(name, "LMT_MAP%x", dev->pf_func); - dev->lmt_base = dev->bar4; - } else { - uint64_t bar4_mbox_sz = MBOX_SIZE; - - /* PF BAR4 should always be sufficient enough to - * hold PF-AF MBOX + PF-VF MBOX + LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (bar4_mbox_sz + (RVU_LMT_LINE_MAX * RVU_LMT_SZ))) { - plt_err("Not enough bar4 space for lmt lines and mbox"); - return -EFAULT; - } + /* Setting alignment to ensure correct masking for resetting to lmt base + * of a core after all lmt lines under that core are used. + * Alignment value LMT_REGION_SIZE to handle the case where all lines + * are used by 1 core. + */ + mz = plt_lmt_region_reserve_aligned(name, LMT_REGION_SIZE, + LMT_REGION_SIZE); + if (!mz) { + plt_err("Memory alloc failed: %s", strerror(errno)); + goto fail; + } - /* LMT base is just after total VF MBOX area */ - bar4_mbox_sz += (MBOX_SIZE * dev_pf_total_vfs(pci_dev)); - dev->lmt_base = dev->bar4 + bar4_mbox_sz; + /* Share the IOVA address with Kernel */ + rc = dev_setup_shared_lmt_region(dev->mbox, true, mz->iova); + if (rc) { + errno = rc; + goto free; } + dev->lmt_base = mz->iova; + dev->lmt_mz = mz; /* Base LMT address should be chosen from only those pci funcs which * participate in LMT shared mode. */ @@ -1016,6 +1002,10 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) } return 0; +free: + plt_memzone_free(mz); +fail: + return -errno; } int @@ -1130,7 +1120,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) goto iounmap; /* Setup LMT line base */ - rc = dev_lmt_setup(pci_dev, dev); + rc = dev_lmt_setup(dev); if (rc) goto iounmap; @@ -1161,6 +1151,10 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) /* Clear references to this pci dev */ npa_lf_fini(); + /* Releasing memory allocated for lmt region */ + if (dev->lmt_mz) + plt_memzone_free(dev->lmt_mz); + mbox_unregister_irq(pci_dev, dev); if (!dev_is_vf(dev)) diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h index 910cfb6..7ee604e 100644 --- a/drivers/common/cnxk/roc_dev_priv.h +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -84,6 +84,7 @@ struct dev { struct dev_ops *ops; void *roc_nix; bool disable_shared_lmt; /* false(default): shared lmt mode enabled */ + const struct plt_memzone *lmt_mz; } __plt_cache_aligned; struct npa { diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h index f6b11b6..9c529d7 100644 --- a/drivers/common/cnxk/roc_mbox.h +++ b/drivers/common/cnxk/roc_mbox.h @@ -403,6 +403,9 @@ struct lmtst_tbl_setup_req { uint64_t __io dis_line_pref : 1; uint64_t __io ssow_pf_func : 13; uint16_t __io pcifunc; + uint8_t __io use_local_lmt_region; + uint64_t __io lmt_iova; + uint64_t __io rsvd[2]; /* Future use */ }; /* CGX mbox message formats */ diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 911ae15..be58625 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -194,4 +194,15 @@ int roc_plt_init(void); typedef int (*roc_plt_init_cb_t)(void); int __roc_api roc_plt_init_cb_register(roc_plt_init_cb_t cb); +static inline const void * +plt_lmt_region_reserve_aligned(const char *name, size_t len, uint32_t align) +{ + /* To ensure returned memory is physically contiguous, bounding + * the start and end address in 2M range. + */ + return rte_memzone_reserve_bounded(name, len, SOCKET_ID_ANY, + RTE_MEMZONE_IOVA_CONTIG, + align, RTE_PGSIZE_2M); +} + #endif /* _ROC_PLATFORM_H_ */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 06/62] common/cnxk: add provision to enable RED on RQ 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (4 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 07/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram ` (56 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Send RED pass/drop levels based on rq configurations to kernel. Fixed the aura and pool shift value calculation. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 8 ++++++ drivers/common/cnxk/roc_nix_queue.c | 50 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npa.c | 8 ++++-- drivers/common/cnxk/roc_npa.h | 5 ++++ 4 files changed, 69 insertions(+), 2 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index 6d9ac10..bb69027 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -161,6 +161,14 @@ struct roc_nix_rq { uint32_t vwqe_max_sz_exp; uint64_t vwqe_wait_tmo; uint64_t vwqe_aura_handle; + /* Average LPB aura level drop threshold for RED */ + uint8_t red_drop; + /* Average LPB aura level pass threshold for RED */ + uint8_t red_pass; + /* Average SPB aura level drop threshold for RED */ + uint8_t spb_red_drop; + /* Average SPB aura level pass threshold for RED */ + uint8_t spb_red_pass; /* End of Input parameters */ struct roc_nix *roc_nix; }; diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index 1c62aa2..0604e7a 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -119,6 +119,15 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_aura_pass = rq->spb_red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + + aq->rq.spb_aura_drop = rq->spb_red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -155,6 +164,14 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + } } return 0; @@ -244,6 +261,23 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_pool_pass = rq->red_pass; + aq->rq.spb_aura_pass = rq->red_pass; + aq->rq.lpb_pool_pass = rq->red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + aq->rq.wqe_pool_pass = rq->red_pass; + aq->rq.xqe_pass = rq->red_pass; + + aq->rq.spb_pool_drop = rq->red_drop; + aq->rq.spb_aura_drop = rq->red_drop; + aq->rq.lpb_pool_drop = rq->red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + aq->rq.wqe_pool_drop = rq->red_drop; + aq->rq.xqe_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -296,6 +330,22 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_pool_pass = ~aq->rq_mask.spb_pool_pass; + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_pool_pass = ~aq->rq_mask.lpb_pool_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + aq->rq_mask.wqe_pool_pass = ~aq->rq_mask.wqe_pool_pass; + aq->rq_mask.xqe_pass = ~aq->rq_mask.xqe_pass; + + aq->rq_mask.spb_pool_drop = ~aq->rq_mask.spb_pool_drop; + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_pool_drop = ~aq->rq_mask.lpb_pool_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + aq->rq_mask.wqe_pool_drop = ~aq->rq_mask.wqe_pool_drop; + aq->rq_mask.xqe_drop = ~aq->rq_mask.xqe_drop; + } } return 0; diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index 5ba6e81..d064d12 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -278,13 +278,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Update aura fields */ aura->pool_addr = pool_id; /* AF will translate to associated poolctx */ aura->ena = 1; - aura->shift = __builtin_clz(block_count) - 8; + aura->shift = plt_log2_u32(block_count); + aura->shift = aura->shift < 8 ? 0 : aura->shift - 8; aura->limit = block_count; aura->pool_caching = 1; aura->err_int_ena = BIT(NPA_AURA_ERR_INT_AURA_ADD_OVER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_ADD_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_FREE_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_POOL_DIS); + aura->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ aura->err_qint_idx = aura_id % lf->qints; @@ -293,13 +295,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, pool->ena = 1; pool->buf_size = block_size / ROC_ALIGN; pool->stack_max_pages = stack_size; - pool->shift = __builtin_clz(block_count) - 8; + pool->shift = plt_log2_u32(block_count); + pool->shift = pool->shift < 8 ? 0 : pool->shift - 8; pool->ptr_start = 0; pool->ptr_end = ~0; pool->stack_caching = 1; pool->err_int_ena = BIT(NPA_POOL_ERR_INT_OVFLS); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_RANGE); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_PERR); + pool->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ pool->err_qint_idx = pool_id % lf->qints; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 59d6223..3fc6192 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -12,6 +12,11 @@ #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 +/* This value controls how much of the present average resource level is used to + * calculate the new resource level. + */ +#define ROC_NPA_AVG_CONT 0xE0 + /* 16 CASP instructions can be outstanding in CN9k, but we use only 15 * outstanding CASPs as we run out of registers. */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 07/62] common/cnxk: support for VLAN push and pop flow actions 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (5 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram ` (55 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add roc API to configure VLAN tag addition and removal. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 255 +++++++++++++++++++++++++++++++++++-- drivers/common/cnxk/roc_npc.h | 24 ++++ drivers/common/cnxk/roc_npc_mcam.c | 2 +- drivers/common/cnxk/roc_npc_priv.h | 1 + drivers/common/cnxk/version.map | 2 + 5 files changed, 269 insertions(+), 15 deletions(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index e6a5036..bb55f3d 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -6,6 +6,23 @@ #include "roc_priv.h" int +roc_npc_vtag_actions_get(struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + return npc->vtag_actions; +} + +int +roc_npc_vtag_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + npc->vtag_actions -= count; + return npc->vtag_actions; +} + +int roc_npc_mcam_free_counter(struct roc_npc *roc_npc, uint16_t ctr_id) { struct npc *npc = roc_npc_to_npc_priv(roc_npc); @@ -330,6 +347,7 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, const struct roc_npc_action_mark *act_mark; const struct roc_npc_action_queue *act_q; const struct roc_npc_action_vf *vf_act; + bool vlan_insert_action = false; int sel_act, req_act = 0; uint16_t pf_func, vf_id; int errcode = 0; @@ -417,25 +435,69 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, req_act |= ROC_NPC_ACTION_TYPE_SEC; rq = 0; break; + case ROC_NPC_ACTION_TYPE_VLAN_STRIP: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_STRIP; + break; + case ROC_NPC_ACTION_TYPE_VLAN_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_INSERT; + break; + case ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT; + break; + case ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT; + break; default: errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; } } + if (req_act & (ROC_NPC_ACTION_TYPE_VLAN_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT)) + vlan_insert_action = true; + + if ((req_act & (ROC_NPC_ACTION_TYPE_VLAN_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT)) == + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT) { + plt_err("PCP insert action can't be supported alone"); + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } + + /* Both STRIP and INSERT actions are not supported */ + if (vlan_insert_action && (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP)) { + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } + /* Check if actions specified are compatible */ if (attr->egress) { - /* Only DROP/COUNT is supported */ - if (!(req_act & ROC_NPC_ACTION_TYPE_DROP)) { + if (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP) { + plt_err("VLAN pop action is not supported on Egress"); errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; - } else if (req_act & ~(ROC_NPC_ACTION_TYPE_DROP | - ROC_NPC_ACTION_TYPE_COUNT)) { + } + + if (req_act & ROC_NPC_ACTION_TYPE_DROP) { + flow->npc_action = NIX_TX_ACTIONOP_DROP; + } else if ((req_act & ROC_NPC_ACTION_TYPE_COUNT) || + vlan_insert_action) { + flow->npc_action = NIX_TX_ACTIONOP_UCAST_DEFAULT; + } else { + plt_err("Unsupported action for egress"); errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; } - flow->npc_action = NIX_TX_ACTIONOP_DROP; + goto set_pf_func; + } else { + if (vlan_insert_action) { + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } } /* We have already verified the attr, this is ingress. @@ -463,6 +525,13 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, goto err_exit; } + if (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP) + npc->vtag_actions++; + + /* Only VLAN action is provided */ + if (req_act == ROC_NPC_ACTION_TYPE_VLAN_STRIP) + flow->npc_action = NIX_RX_ACTIONOP_UCAST; + /* Set NIX_RX_ACTIONOP */ if (req_act & (ROC_NPC_ACTION_TYPE_PF | ROC_NPC_ACTION_TYPE_VF)) { flow->npc_action = NIX_RX_ACTIONOP_UCAST; @@ -774,6 +843,161 @@ roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) return npc->mark_actions; } +static int +npc_vtag_cfg_delete(struct roc_npc *roc_npc, struct roc_npc_flow *flow) +{ + struct roc_nix *roc_nix = roc_npc->roc_nix; + struct nix_vtag_config *vtag_cfg; + struct nix_vtag_config_rsp *rsp; + struct mbox *mbox; + struct nix *nix; + int rc = 0; + + union { + uint64_t reg; + struct nix_tx_vtag_action_s act; + } tx_vtag_action; + + nix = roc_nix_to_nix_priv(roc_nix); + mbox = (&nix->dev)->mbox; + + tx_vtag_action.reg = flow->vtag_action; + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_TX; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->tx.vtag0_idx = tx_vtag_action.act.vtag0_def; + vtag_cfg->tx.free_vtag0 = true; + + rc = mbox_process_msg(mbox, (void *)&rsp); + if (rc) + return rc; + + return 0; +} + +static int +npc_vtag_action_program(struct roc_npc *roc_npc, + const struct roc_npc_action actions[], + struct roc_npc_flow *flow) +{ + uint16_t vlan_id = 0, vlan_ethtype = ROC_ETHER_TYPE_VLAN; + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + struct roc_nix *roc_nix = roc_npc->roc_nix; + struct nix_vtag_config *vtag_cfg; + struct nix_vtag_config_rsp *rsp; + uint64_t rx_vtag_action = 0; + uint8_t vlan_pcp = 0; + struct mbox *mbox; + struct nix *nix; + int rc; + + union { + uint64_t reg; + struct nix_tx_vtag_action_s act; + } tx_vtag_action; + + nix = roc_nix_to_nix_priv(roc_nix); + mbox = (&nix->dev)->mbox; + + flow->vtag_insert_enabled = false; + + for (; actions->type != ROC_NPC_ACTION_TYPE_END; actions++) { + if (actions->type == ROC_NPC_ACTION_TYPE_VLAN_STRIP) { + if (npc->vtag_actions == 1) { + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_RX; + vtag_cfg->rx.strip_vtag = 1; + /* Always capture */ + vtag_cfg->rx.capture_vtag = 1; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->rx.vtag_type = 0; + + rc = mbox_process(mbox); + if (rc) + return rc; + } + + rx_vtag_action |= (NIX_RX_VTAGACTION_VTAG_VALID << 15); + rx_vtag_action |= ((uint64_t)NPC_LID_LB << 8); + rx_vtag_action |= NIX_RX_VTAGACTION_VTAG0_RELPTR; + flow->vtag_action = rx_vtag_action; + } else if (actions->type == ROC_NPC_ACTION_TYPE_VLAN_INSERT) { + const struct roc_npc_action_of_set_vlan_vid *vtag = + (const struct roc_npc_action_of_set_vlan_vid *) + actions->conf; + vlan_id = plt_be_to_cpu_16(vtag->vlan_vid); + if (vlan_id > 0xfff) { + plt_err("Invalid vlan_id for set vlan action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } else if (actions->type == + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT) { + const struct roc_npc_action_of_push_vlan *ethtype = + (const struct roc_npc_action_of_push_vlan *) + actions->conf; + vlan_ethtype = plt_be_to_cpu_16(ethtype->ethertype); + if (vlan_ethtype != ROC_ETHER_TYPE_VLAN && + vlan_ethtype != ROC_ETHER_TYPE_QINQ) { + plt_err("Invalid ethtype specified for push" + " vlan action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } else if (actions->type == + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT) { + const struct roc_npc_action_of_set_vlan_pcp *pcp = + (const struct roc_npc_action_of_set_vlan_pcp *) + actions->conf; + vlan_pcp = pcp->vlan_pcp; + if (vlan_pcp > 0x7) { + plt_err("Invalid PCP value for pcp action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } + } + + if (flow->vtag_insert_enabled) { + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_TX; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->tx.vtag0 = + ((vlan_ethtype << 16) | (vlan_pcp << 13) | vlan_id); + + vtag_cfg->tx.cfg_vtag0 = 1; + rc = mbox_process_msg(mbox, (void *)&rsp); + if (rc) + return rc; + + if (rsp->vtag0_idx < 0) { + plt_err("Failed to config TX VTAG action"); + return -EINVAL; + } + + tx_vtag_action.reg = 0; + tx_vtag_action.act.vtag0_def = rsp->vtag0_idx; + tx_vtag_action.act.vtag0_lid = NPC_LID_LA; + tx_vtag_action.act.vtag0_op = NIX_TX_VTAGOP_INSERT; + tx_vtag_action.act.vtag0_relptr = + NIX_TX_VTAGACTION_VTAG0_RELPTR; + flow->vtag_action = tx_vtag_action.reg; + } + return 0; +} + struct roc_npc_flow * roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, const struct roc_npc_item_info pattern[], @@ -798,6 +1022,12 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, goto err_exit; } + rc = npc_vtag_action_program(roc_npc, actions, flow); + if (rc != 0) { + *errcode = rc; + goto err_exit; + } + parse_state.is_vf = !roc_nix_is_pf(roc_npc->roc_nix); rc = npc_program_mcam(npc, &parse_state, 1); @@ -858,23 +1088,20 @@ roc_npc_flow_destroy(struct roc_npc *roc_npc, struct roc_npc_flow *flow) { struct npc *npc = roc_npc_to_npc_priv(roc_npc); struct plt_bitmap *bmap; - uint16_t match_id; int rc; - match_id = (flow->npc_action >> NPC_RX_ACT_MATCH_OFFSET) & - NPC_RX_ACT_MATCH_MASK; - - if (match_id && match_id < NPC_ACTION_FLAG_DEFAULT) { - if (npc->mark_actions == 0) - return NPC_ERR_PARAM; - } - rc = npc_rss_group_free(npc, flow); if (rc != 0) { plt_err("Failed to free rss action rc = %d", rc); return rc; } + if (flow->vtag_insert_enabled) { + rc = npc_vtag_cfg_delete(roc_npc, flow); + if (rc != 0) + return rc; + } + rc = npc_mcam_free_entry(npc, flow->mcam_id); if (rc != 0) return rc; diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index cf6f732..2c0a536 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -62,6 +62,10 @@ enum roc_npc_action_type { ROC_NPC_ACTION_TYPE_COUNT = (1 << 9), ROC_NPC_ACTION_TYPE_PF = (1 << 10), ROC_NPC_ACTION_TYPE_VF = (1 << 11), + ROC_NPC_ACTION_TYPE_VLAN_STRIP = (1 << 12), + ROC_NPC_ACTION_TYPE_VLAN_INSERT = (1 << 13), + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT = (1 << 14), + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT = (1 << 15), }; struct roc_npc_action { @@ -83,6 +87,18 @@ struct roc_npc_action_queue { uint16_t index; /**< Queue index to use. */ }; +struct roc_npc_action_of_push_vlan { + uint16_t ethertype; /**< EtherType. */ +}; + +struct roc_npc_action_of_set_vlan_vid { + uint16_t vlan_vid; /**< VLAN id. */ +}; + +struct roc_npc_action_of_set_vlan_pcp { + uint8_t vlan_pcp; /**< VLAN priority. */ +}; + struct roc_npc_attr { uint32_t priority; /**< Rule priority level within group. */ uint32_t ingress : 1; /**< Rule applies to ingress traffic. */ @@ -107,6 +123,7 @@ struct roc_npc_flow { uint64_t mcam_mask[ROC_NPC_MAX_MCAM_WIDTH_DWORDS]; uint64_t npc_action; uint64_t vtag_action; + bool vtag_insert_enabled; #define ROC_NPC_MAX_FLOW_PATTERNS 32 struct roc_npc_flow_dump_data dump_data[ROC_NPC_MAX_FLOW_PATTERNS]; uint16_t num_patterns; @@ -138,6 +155,10 @@ enum roc_npc_intf { ROC_NPC_INTF_MAX = 2, }; +enum flow_vtag_cfg_dir { VTAG_TX, VTAG_RX }; +#define ROC_ETHER_TYPE_VLAN 0x8100 /**< IEEE 802.1Q VLAN tagging. */ +#define ROC_ETHER_TYPE_QINQ 0x88A8 /**< IEEE 802.1ad QinQ tagging. */ + struct roc_npc { struct roc_nix *roc_nix; uint8_t switch_header_type; @@ -199,4 +220,7 @@ void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, int __roc_api roc_npc_mark_actions_get(struct roc_npc *roc_npc); int __roc_api roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count); +int __roc_api roc_npc_vtag_actions_get(struct roc_npc *roc_npc); +int __roc_api roc_npc_vtag_actions_sub_return(struct roc_npc *roc_npc, + uint32_t count); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/roc_npc_mcam.c b/drivers/common/cnxk/roc_npc_mcam.c index ff0676d..8ccaaad 100644 --- a/drivers/common/cnxk/roc_npc_mcam.c +++ b/drivers/common/cnxk/roc_npc_mcam.c @@ -546,7 +546,7 @@ npc_mcam_alloc_and_write(struct npc *npc, struct roc_npc_flow *flow, * * Second approach is used now. */ - req->entry_data.vtag_action = 0ULL; + req->entry_data.vtag_action = flow->vtag_action; for (idx = 0; idx < ROC_NPC_MAX_MCAM_WIDTH_DWORDS; idx++) { req->entry_data.kw[idx] = flow->mcam_data[idx]; diff --git a/drivers/common/cnxk/roc_npc_priv.h b/drivers/common/cnxk/roc_npc_priv.h index 961583b..484c3ae 100644 --- a/drivers/common/cnxk/roc_npc_priv.h +++ b/drivers/common/cnxk/roc_npc_priv.h @@ -350,6 +350,7 @@ struct npc { uint16_t flow_max_priority; /* Max priority for flow */ uint16_t switch_header_type; /* Suppprted switch header type */ uint32_t mark_actions; /* Number of mark actions */ + uint32_t vtag_actions; /* vtag insert/strip actions */ uint16_t pf_func; /* pf_func of device */ npc_dxcfg_t prx_dxcfg; /* intf, lid, lt, extract */ npc_fxcfg_t prx_fxcfg; /* Flag extract */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 554459b..8a5c839 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -167,6 +167,8 @@ INTERNAL { roc_npc_init; roc_npc_mark_actions_get; roc_npc_mark_actions_sub_return; + roc_npc_vtag_actions_get; + roc_npc_vtag_actions_sub_return; roc_npc_mcam_alloc_entries; roc_npc_mcam_alloc_entry; roc_npc_mcam_clear_counter; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 08/62] common/cnxk: fix flow create on CN98xx 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (6 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 07/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram ` (54 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> CN96xx and CN98xx have 4096 and 16384 MCAM entries respectively. Aligning the code with the same numbers. Fixes: a07f7ced436d ("common/cnxk: add NPC init and fini") Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_model.h | 6 ++++++ drivers/common/cnxk/roc_npc.c | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_model.h b/drivers/common/cnxk/roc_model.h index fb774ac..d6ef459 100644 --- a/drivers/common/cnxk/roc_model.h +++ b/drivers/common/cnxk/roc_model.h @@ -88,6 +88,12 @@ roc_model_is_cn10k(void) } static inline uint64_t +roc_model_is_cn98xx(void) +{ + return (roc_model->flag & ROC_MODEL_CN98xx_A0); +} + +static inline uint64_t roc_model_is_cn96_A0(void) { return roc_model->flag & ROC_MODEL_CN96xx_A0; diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index bb55f3d..8a76823 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -118,7 +118,7 @@ npc_mcam_tot_entries(void) /* FIXME: change to reading in AF from NPC_AF_CONST1/2 * MCAM_BANK_DEPTH(_EXT) * MCAM_BANKS */ - if (roc_model_is_cn10k()) + if (roc_model_is_cn10k() || roc_model_is_cn98xx()) return 16 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 4096 */ else return 4 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 1024 */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 09/62] net/cnxk: add build infra and common probe 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (7 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram ` (53 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add build infrastructure and common probe and remove for cnxk driver which is used by both CN10K and CN9K SoC. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- MAINTAINERS | 5 +- doc/guides/nics/cnxk.rst | 29 +++++ doc/guides/nics/features/cnxk.ini | 9 ++ doc/guides/nics/features/cnxk_vec.ini | 9 ++ doc/guides/nics/features/cnxk_vf.ini | 9 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + doc/guides/rel_notes/release_21_08.rst | 5 + drivers/net/cnxk/cnxk_ethdev.c | 218 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ drivers/net/cnxk/meson.build | 14 +++ drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 13 files changed, 362 insertions(+), 1 deletion(-) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16..b39a1c2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -745,8 +745,11 @@ M: Kiran Kumar K <kirankumark@marvell.com> M: Sunil Kumar Kori <skori@marvell.com> M: Satha Rao <skoteshwar@marvell.com> T: git://dpdk.org/next/dpdk-next-net-mrvl -F: drivers/common/cnxk/ +F: doc/guides/nics/cnxk.rst +F: doc/guides/nics/features/cnxk*.ini F: doc/guides/platform/cnxk.rst +F: drivers/common/cnxk/ +F: drivers/net/cnxk/ Marvell mvpp2 M: Liron Himi <lironh@marvell.com> diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst new file mode 100644 index 0000000..ca21842 --- /dev/null +++ b/doc/guides/nics/cnxk.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK Poll Mode driver +===================== + +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** +SoC family as well as for their virtual functions (VF) in SR-IOV context. + +More information can be found at `Marvell Official Website +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. + +Features +-------- + +Features of the CNXK Ethdev PMD are: + +Prerequisites +------------- + +See :doc:`../platform/cnxk` for setup information. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini new file mode 100644 index 0000000..2c23464 --- /dev/null +++ b/doc/guides/nics/features/cnxk.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini new file mode 100644 index 0000000..de78516 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vec' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini new file mode 100644 index 0000000..9c96351 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 799697c..c1a04d9 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -19,6 +19,7 @@ Network Interface Controller Drivers axgbe bnx2x bnxt + cnxk cxgbe dpaa dpaa2 diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index cebb3d0..b506c11 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -142,6 +142,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in cnxk SoC. +#. **Ethdev Driver** + See :doc:`../nics/cnxk` for NIX Ethdev driver information. + #. **Mempool Driver** See :doc:`../mempool/cnxk` for NPA mempool driver information. diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index a6ecfdf..31e49e1 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added support for Marvell CN10K SoC ethernet device.** + + * Added net/cnxk driver which provides the support for the integrated ethernet + device. + Removed Items ------------- diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c new file mode 100644 index 0000000..589b0da --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -0,0 +1,218 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_ethdev.h> + +/* CNXK platform independent eth dev ops */ +struct eth_dev_ops cnxk_eth_dev_ops; + +static int +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct rte_pci_device *pci_dev; + int rc, max_entries; + + eth_dev->dev_ops = &cnxk_eth_dev_ops; + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + rte_eth_copy_pci_info(eth_dev, pci_dev); + + /* Initialize base roc nix */ + nix->pci_dev = pci_dev; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to initialize roc nix rc=%d", rc); + goto error; + } + + dev->eth_dev = eth_dev; + + /* For vfs, returned max_entries will be 0. but to keep default mac + * address, one entry must be allocated. so setting up to 1. + */ + if (roc_nix_is_vf_or_sdp(nix)) + max_entries = 1; + else + max_entries = roc_nix_mac_max_entries_get(nix); + + if (max_entries <= 0) { + plt_err("Failed to get max entries for mac addr"); + rc = -ENOTSUP; + goto dev_fini; + } + + eth_dev->data->mac_addrs = + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + plt_err("Failed to allocate memory for mac addr"); + rc = -ENOMEM; + goto dev_fini; + } + + dev->max_mac_entries = max_entries; + + /* Get mac address */ + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to get mac addr, rc=%d", rc); + goto free_mac_addrs; + } + + /* Update the mac address */ + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); + + if (!roc_nix_is_vf_or_sdp(nix)) { + /* Sync same MAC address to CGX/RPM table */ + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to set mac addr, rc=%d", rc); + goto free_mac_addrs; + } + } + + /* Initialize roc npc */ + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, + eth_dev->data->port_id, roc_nix_get_pf(nix), + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, + dev->rx_offload_capa, dev->tx_offload_capa); + return 0; + +free_mac_addrs: + rte_free(eth_dev->data->mac_addrs); +dev_fini: + roc_nix_dev_fini(nix); +error: + plt_err("Failed to init nix eth_dev rc=%d", rc); + return rc; +} + +static int +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct roc_nix *nix = &dev->nix; + int rc, i; + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + roc_nix_npc_rx_ena_dis(nix, false); + + /* Free up SQs */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); + eth_dev->data->tx_queues[i] = NULL; + } + eth_dev->data->nb_tx_queues = 0; + + /* Free up RQ's and CQ's */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); + eth_dev->data->rx_queues[i] = NULL; + } + eth_dev->data->nb_rx_queues = 0; + + /* Free tm resources */ + roc_nix_tm_fini(nix); + + /* Unregister queue irqs */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister cq irqs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Free nix lf resources */ + rc = roc_nix_lf_free(nix); + if (rc) + plt_err("Failed to free nix lf, rc=%d", rc); + + rte_free(eth_dev->data->mac_addrs); + eth_dev->data->mac_addrs = NULL; + + /* Check if mbox close is needed */ + if (!mbox_close) + return 0; + + rc = roc_nix_dev_fini(nix); + /* Can be freed later by PMD if NPA LF is in use */ + if (rc == -EAGAIN) { + eth_dev->data->dev_private = NULL; + return 0; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + } + + return rc; +} + +int +cnxk_nix_remove(struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct roc_nix *nix; + int rc = -EINVAL; + + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (eth_dev) { + /* Cleanup eth dev */ + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + rte_eth_dev_release_port(eth_dev); + } + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (nix->pci_dev != pci_dev) + return 0; + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("%s: common resource in use by other devices", + pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + /* Free device pointer as rte_ethdev does not have it anymore */ + rte_free(nix); +exit: + return rc; +} + +int +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + int rc; + + RTE_SET_USED(pci_drv); + + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), + cnxk_eth_dev_init); + + /* On error on secondary, recheck if port exists in primary or + * in mid of detach state. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) + if (!rte_eth_dev_allocated(pci_dev->device.name)) + return 0; + return rc; +} diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h new file mode 100644 index 0000000..0460d1e --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_ETHDEV_H__ +#define __CNXK_ETHDEV_H__ + +#include <math.h> +#include <stdint.h> + +#include <ethdev_driver.h> +#include <ethdev_pci.h> + +#include "roc_api.h" + +#define CNXK_ETH_DEV_PMD_VERSION "1.0" + +struct cnxk_eth_dev { + /* ROC NIX */ + struct roc_nix nix; + + /* Max macfilter entries */ + uint8_t max_mac_entries; + + uint16_t flags; + + /* Pointer back to rte */ + struct rte_eth_dev *eth_dev; + + /* HW capabilities / Limitations */ + union { + uint64_t hwcap; + }; + + /* Rx and Tx offload capabilities */ + uint64_t rx_offload_capa; + uint64_t tx_offload_capa; + uint32_t speed_capa; + + /* Default mac address */ + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; +}; + +static inline struct cnxk_eth_dev * +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) +{ + return eth_dev->data->dev_private; +} + +/* Common ethdev ops */ +extern struct eth_dev_ops cnxk_eth_dev_ops; + +/* Ops */ +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); +int cnxk_nix_remove(struct rte_pci_device *pci_dev); + +#endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build new file mode 100644 index 0000000..7dd4bca --- /dev/null +++ b/drivers/net/cnxk/meson.build @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit' + subdir_done() +endif + +sources = files('cnxk_ethdev.c') + +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] +deps += ['common_cnxk', 'mempool_cnxk'] diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map new file mode 100644 index 0000000..ee80c51 --- /dev/null +++ b/drivers/net/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index c8b5ce2..5b066fd 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -12,6 +12,7 @@ drivers = [ 'bnx2x', 'bnxt', 'bonding', + 'cnxk', 'cxgbe', 'dpaa', 'dpaa2', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 10/62] net/cnxk: add platform specific probe and remove 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (8 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram ` (52 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add platform specific probe and remove callbacks for CN9K and CN10K which use common probe and remove functions. Register ethdev driver for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ drivers/net/cnxk/meson.build | 5 +++ 7 files changed, 230 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c new file mode 100644 index 0000000..ff8ce31 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn10k_ethdev.h" + +static int +cn10k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 64) { + plt_err("Driver not compiled for CN10K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + } + return 0; +} + +static const struct rte_pci_id cn10k_pci_nix_map[] = { + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn10k_pci_nix = { + .id_table = cn10k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn10k_nix_probe, + .remove = cn10k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h new file mode 100644 index 0000000..1bf4a65 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_ETHDEV_H__ +#define __CN10K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c new file mode 100644 index 0000000..701dc12 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn9k_ethdev.h" + +static int +cn9k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 128) { + plt_err("Driver not compiled for CN9K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + + dev = cnxk_eth_pmd_priv(eth_dev); + /* Update capabilities already set for TSO. + * TSO not supported for earlier chip revisions + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_GRE_TNL_TSO); + + /* 50G and 100G to be supported for board version C0 + * and above of CN9K. + */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) { + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; + } + + dev->hwcap = 0; + + /* Update HW erratas */ + if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) + dev->cq_min_4k = 1; + return 0; +} + +static const struct rte_pci_id cn9k_pci_nix_map[] = { + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn9k_pci_nix = { + .id_table = cn9k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn9k_nix_probe, + .remove = cn9k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h new file mode 100644 index 0000000..15d9397 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_ETHDEV_H__ +#define __CN9K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 589b0da..526c19b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -3,6 +3,40 @@ */ #include <cnxk_ethdev.h> +static inline uint64_t +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) +{ + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; + + return capa; +} + +static inline uint64_t +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) +{ + RTE_SET_USED(dev); + return CNXK_NIX_TX_OFFLOAD_CAPA; +} + +static inline uint32_t +nix_get_speed_capa(struct cnxk_eth_dev *dev) +{ + uint32_t speed_capa; + + /* Auto negotiation disabled */ + speed_capa = ETH_LINK_SPEED_FIXED; + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; + } + + return speed_capa; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops; @@ -76,6 +110,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } } + /* Union of all capabilities supported by CNXK. + * Platform specific capabilities will be + * updated later. + */ + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); + dev->speed_capa = nix_get_speed_capa(dev); + /* Initialize roc npc */ plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0460d1e..ba2bfcd 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -14,6 +14,22 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +#define CNXK_NIX_TX_OFFLOAD_CAPA \ + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ + DEV_TX_OFFLOAD_IPV4_CKSUM) + +#define CNXK_NIX_RX_OFFLOAD_CAPA \ + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_RX_OFFLOAD_RSS_HASH) + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -28,6 +44,9 @@ struct cnxk_eth_dev { /* HW capabilities / Limitations */ union { + struct { + uint64_t cq_min_4k : 1; + }; uint64_t hwcap; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7dd4bca..089e4fc 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,5 +10,10 @@ endif sources = files('cnxk_ethdev.c') +# CN9K +sources += files('cn9k_ethdev.c') +# CN10K +sources += files('cn10k_ethdev.c') + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 11/62] net/cnxk: add common devargs parsing function 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (9 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram ` (51 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add various devargs parsing command line arguments parsing functions supported by CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 7 ++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ca21842..6652e17 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -27,3 +27,97 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. + +Runtime Config Options +---------------------- + +- ``Rx&Tx scalar mode enable`` (default ``0``) + + PMD supports both scalar and vector mode, it may be selected at runtime + using ``scalar_enable`` ``devargs`` parameter. + +- ``RSS reta size`` (default ``64``) + + RSS redirection table size may be configured during runtime using ``reta_size`` + ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,reta_size=256 + + With the above configuration, reta table of size 256 is populated. + +- ``Flow priority levels`` (default ``3``) + + RTE Flow priority levels can be configured during runtime using + ``flow_max_priority`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_max_priority=10 + + With the above configuration, priority level was set to 10 (0-9). Max + priority level supported is 32. + +- ``Reserve Flow entries`` (default ``8``) + + RTE flow entries can be pre allocated and the size of pre allocation can be + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_prealloc_size=4 + + With the above configuration, pre alloc size was set to 4. Max pre alloc + size supported is 32. + +- ``Max SQB buffer count`` (default ``512``) + + Send queue descriptor buffer count may be limited during runtime using + ``max_sqb_count`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,max_sqb_count=64 + + With the above configuration, each send queue's descriptor buffer count is + limited to a maximum of 64 buffers. + +- ``Switch header enable`` (default ``none``) + + A port can be configured to a specific switch header type by using + ``switch_header`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,switch_header="higig2" + + With the above configuration, higig2 will be enabled on that port and the + traffic on this port should be higig2 traffic only. Supported switch header + types are "higig2", "dsa", "chlen90b" and "chlen24b". + +- ``RSS tag as XOR`` (default ``0``) + + The HW gives two options to configure the RSS adder i.e + + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` + + * ``rss_adder<7:0> = flow_tag<7:0>`` + + Latter one aligns with standard NIC behavior vs former one is a legacy + RSS adder scheme used in OCTEON TX2 products. + + By default, the driver runs in the latter mode. + Setting this flag to 1 to select the legacy mode. + + For example to select the legacy mode(RSS tag adder as XOR):: + + -a 0002:02:00.0,tag_as_xor=1 + + + +.. note:: + + Above devarg parameters are configurable per device, user needs to pass the + parameters to all the PCIe devices if application requires to configure on + all the ethdev ports. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 526c19b..109fd35 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -57,6 +57,13 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); rte_eth_copy_pci_info(eth_dev, pci_dev); + /* Parse devargs string */ + rc = cnxk_ethdev_parse_devargs(eth_dev->device->devargs, dev); + if (rc) { + plt_err("Failed to parse devargs rc=%d", rc); + goto error; + } + /* Initialize base roc nix */ nix->pci_dev = pci_dev; rc = roc_nix_dev_init(nix); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ba2bfcd..97e3a15 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -9,11 +9,15 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> +#include <rte_kvargs.h> #include "roc_api.h" #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Max supported SQB count */ +#define CNXK_NIX_TX_MAX_SQB 512 + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -38,6 +42,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + bool scalar_ena; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -73,4 +78,8 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +/* Devargs */ +int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, + struct cnxk_eth_dev *dev); + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c new file mode 100644 index 0000000..4af2803 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <inttypes.h> +#include <math.h> + +#include "cnxk_ethdev.h" + +static int +parse_flow_max_priority(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the max priority to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flow_prealloc_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the prealloc size to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_reta_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val <= ETH_RSS_RETA_SIZE_64) + val = ROC_NIX_RSS_RETA_SZ_64; + else if (val > ETH_RSS_RETA_SIZE_64 && val <= ETH_RSS_RETA_SIZE_128) + val = ROC_NIX_RSS_RETA_SZ_128; + else if (val > ETH_RSS_RETA_SIZE_128 && val <= ETH_RSS_RETA_SIZE_256) + val = ROC_NIX_RSS_RETA_SZ_256; + else + val = ROC_NIX_RSS_RETA_SZ_64; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flag(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + *(uint16_t *)extra_args = atoi(value); + + return 0; +} + +static int +parse_sqb_count(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_switch_header_type(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + if (strcmp(value, "higig2") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_HIGIG; + + if (strcmp(value, "dsa") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_EDSA; + + if (strcmp(value, "chlen90b") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_LEN_90B; + return 0; +} + +#define CNXK_RSS_RETA_SIZE "reta_size" +#define CNXK_SCL_ENABLE "scalar_enable" +#define CNXK_MAX_SQB_COUNT "max_sqb_count" +#define CNXK_FLOW_PREALLOC_SIZE "flow_prealloc_size" +#define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" +#define CNXK_SWITCH_HEADER_TYPE "switch_header" +#define CNXK_RSS_TAG_AS_XOR "tag_as_xor" + +int +cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) +{ + uint16_t reta_sz = ROC_NIX_RSS_RETA_SZ_64; + uint16_t sqb_count = CNXK_NIX_TX_MAX_SQB; + uint16_t flow_prealloc_size = 8; + uint16_t switch_header_type = 0; + uint16_t flow_max_priority = 3; + uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto null_devargs; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_RSS_RETA_SIZE, &parse_reta_size, + &reta_sz); + rte_kvargs_process(kvlist, CNXK_SCL_ENABLE, &parse_flag, + &scalar_enable); + rte_kvargs_process(kvlist, CNXK_MAX_SQB_COUNT, &parse_sqb_count, + &sqb_count); + rte_kvargs_process(kvlist, CNXK_FLOW_PREALLOC_SIZE, + &parse_flow_prealloc_size, &flow_prealloc_size); + rte_kvargs_process(kvlist, CNXK_FLOW_MAX_PRIORITY, + &parse_flow_max_priority, &flow_max_priority); + rte_kvargs_process(kvlist, CNXK_SWITCH_HEADER_TYPE, + &parse_switch_header_type, &switch_header_type); + rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, + &rss_tag_as_xor); + rte_kvargs_free(kvlist); + +null_devargs: + dev->scalar_ena = !!scalar_enable; + dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; + dev->nix.max_sqb_count = sqb_count; + dev->nix.reta_sz = reta_sz; + return 0; + +exit: + return -EINVAL; +} + +RTE_PMD_REGISTER_PARAM_STRING(net_cnxk, + CNXK_RSS_RETA_SIZE "=<64|128|256>" + CNXK_SCL_ENABLE "=1" + CNXK_MAX_SQB_COUNT "=<8-512>" + CNXK_FLOW_PREALLOC_SIZE "=<1-32>" + CNXK_FLOW_MAX_PRIORITY "=<1-32>" + CNXK_SWITCH_HEADER_TYPE "=<higig2|dsa|chlen90b>" + CNXK_RSS_TAG_AS_XOR "=1"); diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 089e4fc..e7e43f0 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -8,7 +8,8 @@ if not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_ethdev.c') +sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_devargs.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 12/62] net/cnxk: add common dev infos get support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (10 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram ` (50 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add support to retrieve dev infos get for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 3 ++ doc/guides/nics/features/cnxk.ini | 4 ++ doc/guides/nics/features/cnxk_vec.ini | 4 ++ doc/guides/nics/features/cnxk_vf.ini | 3 ++ drivers/net/cnxk/cnxk_ethdev.c | 4 +- drivers/net/cnxk/cnxk_ethdev.h | 33 ++++++++++++++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 71 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 8 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6652e17..6bd410b 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,9 @@ Features Features of the CNXK Ethdev PMD are: +- SR-IOV VF +- Lock-free Tx queue + Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c23464..b426340 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index de78516..292ac1e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 9c96351..bc2eb8a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 109fd35..066e01c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -38,7 +38,9 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) } /* CNXK platform independent eth dev ops */ -struct eth_dev_ops cnxk_eth_dev_ops; +struct eth_dev_ops cnxk_eth_dev_ops = { + .dev_infos_get = cnxk_nix_info_get, +}; static int cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 97e3a15..8d9a7e0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,9 +15,40 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* VLAN tag inserted by NIX_TX_VTAG_ACTION. + * In Tx space is always reserved for this in FRS. + */ +#define CNXK_NIX_MAX_VTAG_INS 2 +#define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) + +/* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) + +#define CNXK_NIX_RX_MIN_DESC 16 +#define CNXK_NIX_RX_MIN_DESC_ALIGN 16 +#define CNXK_NIX_RX_NB_SEG_MAX 6 +#define CNXK_NIX_RX_DEFAULT_RING_SZ 4096 /* Max supported SQB count */ #define CNXK_NIX_TX_MAX_SQB 512 +/* If PTP is enabled additional SEND MEM DESC is required which + * takes 2 words, hence max 7 iova address are possible + */ +#if defined(RTE_LIBRTE_IEEE1588) +#define CNXK_NIX_TX_NB_SEG_MAX 7 +#else +#define CNXK_NIX_TX_NB_SEG_MAX 9 +#endif + +#define CNXK_NIX_RSS_L3_L4_SRC_DST \ + (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ + ETH_RSS_L4_DST_ONLY) + +#define CNXK_NIX_RSS_OFFLOAD \ + (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | \ + ETH_RSS_SCTP | ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \ + CNXK_NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK | ETH_RSS_C_VLAN) + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -77,6 +108,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_info *dev_info); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c new file mode 100644 index 0000000..4a45956 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <cnxk_ethdev.h> + +int +cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) +{ + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int max_rx_pktlen; + + max_rx_pktlen = (roc_nix_max_pkt_len(&dev->nix) + RTE_ETHER_CRC_LEN - + CNXK_NIX_MAX_VTAG_ACT_SIZE); + + devinfo->min_rx_bufsize = NIX_MIN_HW_FRS + RTE_ETHER_CRC_LEN; + devinfo->max_rx_pktlen = max_rx_pktlen; + devinfo->max_rx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_mac_addrs = dev->max_mac_entries; + devinfo->max_vfs = pci_dev->max_vfs; + devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; + + devinfo->rx_offload_capa = dev->rx_offload_capa; + devinfo->tx_offload_capa = dev->tx_offload_capa; + devinfo->rx_queue_offload_capa = 0; + devinfo->tx_queue_offload_capa = 0; + + devinfo->reta_size = dev->nix.reta_sz; + devinfo->hash_key_size = ROC_NIX_RSS_KEY_LEN; + devinfo->flow_type_rss_offloads = CNXK_NIX_RSS_OFFLOAD; + + devinfo->default_rxconf = (struct rte_eth_rxconf){ + .rx_drop_en = 0, + .offloads = 0, + }; + + devinfo->default_txconf = (struct rte_eth_txconf){ + .offloads = 0, + }; + + devinfo->default_rxportconf = (struct rte_eth_dev_portconf){ + .ring_size = CNXK_NIX_RX_DEFAULT_RING_SZ, + }; + + devinfo->rx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = CNXK_NIX_RX_MIN_DESC, + .nb_align = CNXK_NIX_RX_MIN_DESC_ALIGN, + .nb_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + }; + devinfo->rx_desc_lim.nb_max = + RTE_ALIGN_MUL_FLOOR(devinfo->rx_desc_lim.nb_max, + CNXK_NIX_RX_MIN_DESC_ALIGN); + + devinfo->tx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = 1, + .nb_align = 1, + .nb_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + }; + + devinfo->speed_capa = dev->speed_capa; + devinfo->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index e7e43f0..8495732 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -9,6 +9,7 @@ if not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 13/62] net/cnxk: add device configuration operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (11 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 14/62] net/cnxk: add link status update support Nithin Dabilpuram ` (49 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add device configuration op for CN9K and CN10K. Most of the device configuration is common between two platforms except for some supported offloads. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 2 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cn10k_ethdev.c | 34 +++ drivers/net/cnxk/cn9k_ethdev.c | 45 +++ drivers/net/cnxk/cnxk_ethdev.c | 556 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 85 ++++++ 8 files changed, 728 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6bd410b..0c2ea89 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -18,6 +18,8 @@ Features of the CNXK Ethdev PMD are: - SR-IOV VF - Lock-free Tx queue +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b426340..96dba2a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 292ac1e..616991c 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index bc2eb8a..a0bd2f1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index ff8ce31..d971bbd 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,38 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; +} + +static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -26,6 +58,8 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 701dc12..7f3e910 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_conf *conf = ð_dev->data->dev_conf; + struct rte_eth_txmode *txmode = &conf->txmode; + int rc; + + /* Platform specific checks */ + if ((roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) && + (txmode->offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) && + ((txmode->offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) || + (txmode->offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM))) { + plt_err("Outer IP and SCTP checksum unsupported"); + return -EINVAL; + } + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; +} + +static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -27,6 +70,8 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 066e01c..14e28fe 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,555 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint32_t +cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level) +{ + uint32_t flow_key_type[RSS_MAX_LEVELS][6] = { + {FLOW_KEY_TYPE_IPV4, FLOW_KEY_TYPE_IPV6, FLOW_KEY_TYPE_TCP, + FLOW_KEY_TYPE_UDP, FLOW_KEY_TYPE_SCTP, FLOW_KEY_TYPE_ETH_DMAC}, + {FLOW_KEY_TYPE_INNR_IPV4, FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_INNR_TCP, FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_INNR_SCTP, FLOW_KEY_TYPE_INNR_ETH_DMAC}, + {FLOW_KEY_TYPE_IPV4 | FLOW_KEY_TYPE_INNR_IPV4, + FLOW_KEY_TYPE_IPV6 | FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_TCP | FLOW_KEY_TYPE_INNR_TCP, + FLOW_KEY_TYPE_UDP | FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_SCTP | FLOW_KEY_TYPE_INNR_SCTP, + FLOW_KEY_TYPE_ETH_DMAC | FLOW_KEY_TYPE_INNR_ETH_DMAC} + }; + uint32_t flowkey_cfg = 0; + + dev->ethdev_rss_hf = ethdev_rss; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + + if (ethdev_rss & ETH_RSS_C_VLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VLAN; + + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC; + + if (ethdev_rss & ETH_RSS_L3_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST; + + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC; + + if (ethdev_rss & ETH_RSS_L4_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST; + + if (ethdev_rss & RSS_IPV4_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX]; + + if (ethdev_rss & RSS_IPV6_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV6_INDEX]; + + if (ethdev_rss & ETH_RSS_TCP) + flowkey_cfg |= flow_key_type[rss_level][RSS_TCP_INDEX]; + + if (ethdev_rss & ETH_RSS_UDP) + flowkey_cfg |= flow_key_type[rss_level][RSS_UDP_INDEX]; + + if (ethdev_rss & ETH_RSS_SCTP) + flowkey_cfg |= flow_key_type[rss_level][RSS_SCTP_INDEX]; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= flow_key_type[rss_level][RSS_DMAC_INDEX]; + + if (ethdev_rss & RSS_IPV6_EX_ENABLE) + flowkey_cfg |= FLOW_KEY_TYPE_IPV6_EXT; + + if (ethdev_rss & ETH_RSS_PORT) + flowkey_cfg |= FLOW_KEY_TYPE_PORT; + + if (ethdev_rss & ETH_RSS_NVGRE) + flowkey_cfg |= FLOW_KEY_TYPE_NVGRE; + + if (ethdev_rss & ETH_RSS_VXLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VXLAN; + + if (ethdev_rss & ETH_RSS_GENEVE) + flowkey_cfg |= FLOW_KEY_TYPE_GENEVE; + + if (ethdev_rss & ETH_RSS_GTPU) + flowkey_cfg |= FLOW_KEY_TYPE_GTPU; + + return flowkey_cfg; +} + +static void +nix_free_queue_mem(struct cnxk_eth_dev *dev) +{ + plt_free(dev->rqs); + plt_free(dev->cqs); + plt_free(dev->sqs); + dev->rqs = NULL; + dev->cqs = NULL; + dev->sqs = NULL; +} + +static int +nix_rss_default_setup(struct cnxk_eth_dev *dev) +{ + struct rte_eth_dev *eth_dev = dev->eth_dev; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + uint64_t rss_hf; + + rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_hash_level = ETH_RSS_LEVEL(rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + + flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level); + return roc_nix_rss_default_setup(&dev->nix, flowkey_cfg); +} + +static int +nix_store_queue_cfg_and_then_release(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = NULL; + struct cnxk_eth_qconf *rx_qconf = NULL; + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_txq_sp *txq_sp; + int i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + tx_qconf = malloc(nb_txq * sizeof(*tx_qconf)); + if (tx_qconf == NULL) { + plt_err("Failed to allocate memory for tx_qconf"); + goto fail; + } + + rx_qconf = malloc(nb_rxq * sizeof(*rx_qconf)); + if (rx_qconf == NULL) { + plt_err("Failed to allocate memory for rx_qconf"); + goto fail; + } + + txq = eth_dev->data->tx_queues; + for (i = 0; i < nb_txq; i++) { + if (txq[i] == NULL) { + tx_qconf[i].valid = false; + plt_info("txq[%d] is already released", i); + continue; + } + txq_sp = cnxk_eth_txq_to_sp(txq[i]); + memcpy(&tx_qconf[i], &txq_sp->qconf, sizeof(*tx_qconf)); + tx_qconf[i].valid = true; + dev_ops->tx_queue_release(txq[i]); + eth_dev->data->tx_queues[i] = NULL; + } + + rxq = eth_dev->data->rx_queues; + for (i = 0; i < nb_rxq; i++) { + if (rxq[i] == NULL) { + rx_qconf[i].valid = false; + plt_info("rxq[%d] is already released", i); + continue; + } + rxq_sp = cnxk_eth_rxq_to_sp(rxq[i]); + memcpy(&rx_qconf[i], &rxq_sp->qconf, sizeof(*rx_qconf)); + rx_qconf[i].valid = true; + dev_ops->rx_queue_release(rxq[i]); + eth_dev->data->rx_queues[i] = NULL; + } + + dev->tx_qconf = tx_qconf; + dev->rx_qconf = rx_qconf; + return 0; + +fail: + free(tx_qconf); + free(rx_qconf); + return -ENOMEM; +} + +static int +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; + int rc, i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + rc = -ENOMEM; + /* Setup tx & rx queues with previous configuration so + * that the queues can be functional in cases like ports + * are started without re configuring queues. + * + * Usual re config sequence is like below: + * port_configure() { + * if(reconfigure) { + * queue_release() + * queue_setup() + * } + * queue_configure() { + * queue_release() + * queue_setup() + * } + * } + * port_start() + * + * In some application's control path, queue_configure() would + * NOT be invoked for TXQs/RXQs in port_configure(). + * In such cases, queues can be functional after start as the + * queues are already setup in port_configure(). + */ + for (i = 0; i < nb_txq; i++) { + if (!tx_qconf[i].valid) + continue; + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, + &tx_qconf[i].conf.tx); + if (rc) { + plt_err("Failed to setup tx queue rc=%d", rc); + txq = eth_dev->data->tx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->tx_queue_release(txq[i]); + goto fail; + } + } + + free(tx_qconf); + tx_qconf = NULL; + + for (i = 0; i < nb_rxq; i++) { + if (!rx_qconf[i].valid) + continue; + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, + &rx_qconf[i].conf.rx, + rx_qconf[i].mp); + if (rc) { + plt_err("Failed to setup rx queue rc=%d", rc); + rxq = eth_dev->data->rx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->rx_queue_release(rxq[i]); + goto tx_queue_release; + } + } + + free(rx_qconf); + rx_qconf = NULL; + + return 0; + +tx_queue_release: + txq = eth_dev->data->tx_queues; + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_release(txq[i]); +fail: + if (tx_qconf) + free(tx_qconf); + if (rx_qconf) + free(rx_qconf); + + return rc; +} + +static uint16_t +nix_eth_nop_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + RTE_SET_USED(queue); + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + return 0; +} + +static void +nix_set_nop_rxtx_function(struct rte_eth_dev *eth_dev) +{ + /* These dummy functions are required for supporting + * some applications which reconfigure queues without + * stopping tx burst and rx burst threads(eg kni app) + * When the queues context is saved, txq/rxqs are released + * which caused app crash since rx/tx burst is still + * on different lcores + */ + eth_dev->tx_pkt_burst = nix_eth_nop_burst; + eth_dev->rx_pkt_burst = nix_eth_nop_burst; + rte_mb(); +} + +static int +nix_lso_fmt_setup(struct cnxk_eth_dev *dev) +{ + uint8_t udp_tun[ROC_NIX_LSO_TUN_MAX]; + uint8_t tun[ROC_NIX_LSO_TUN_MAX]; + struct roc_nix *nix = &dev->nix; + int rc; + + /* Nothing much to do if offload is not enabled */ + if (!(dev->tx_offloads & + (DEV_TX_OFFLOAD_TCP_TSO | DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO))) + return 0; + + /* Setup LSO formats in AF. Its a no-op if other ethdev has + * already set it up + */ + rc = roc_nix_lso_fmt_setup(nix); + if (rc) + return rc; + + roc_nix_lso_fmt_get(nix, udp_tun, tun); + dev->lso_tun_fmt = ((uint64_t)tun[ROC_NIX_LSO_TUN_V4V4] | + (uint64_t)tun[ROC_NIX_LSO_TUN_V4V6] << 8 | + (uint64_t)tun[ROC_NIX_LSO_TUN_V6V4] << 16 | + (uint64_t)tun[ROC_NIX_LSO_TUN_V6V6] << 24); + + dev->lso_tun_fmt |= ((uint64_t)udp_tun[ROC_NIX_LSO_TUN_V4V4] << 32 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V4V6] << 40 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V6V4] << 48 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V6V6] << 56); + return 0; +} + +int +cnxk_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + struct rte_eth_txmode *txmode = &conf->txmode; + char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix *nix = &dev->nix; + struct rte_ether_addr *ea; + uint8_t nb_rxq, nb_txq; + uint64_t rx_cfg; + void *qs; + int rc; + + rc = -EINVAL; + + /* Sanity checks */ + if (rte_eal_has_hugepages() == 0) { + plt_err("Huge page is not configured"); + goto fail_configure; + } + + if (conf->dcb_capability_en == 1) { + plt_err("dcb enable is not supported"); + goto fail_configure; + } + + if (conf->fdir_conf.mode != RTE_FDIR_MODE_NONE) { + plt_err("Flow director is not supported"); + goto fail_configure; + } + + if (rxmode->mq_mode != ETH_MQ_RX_NONE && + rxmode->mq_mode != ETH_MQ_RX_RSS) { + plt_err("Unsupported mq rx mode %d", rxmode->mq_mode); + goto fail_configure; + } + + if (txmode->mq_mode != ETH_MQ_TX_NONE) { + plt_err("Unsupported mq tx mode %d", txmode->mq_mode); + goto fail_configure; + } + + /* Free the resources allocated from the previous configure */ + if (dev->configured == 1) { + /* Unregister queue irq's */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister CQ irqs if present */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Set no-op functions */ + nix_set_nop_rxtx_function(eth_dev); + /* Store queue config for later */ + rc = nix_store_queue_cfg_and_then_release(eth_dev); + if (rc) + goto fail_configure; + roc_nix_tm_fini(nix); + roc_nix_lf_free(nix); + } + + dev->rx_offloads = rxmode->offloads; + dev->tx_offloads = txmode->offloads; + + /* Prepare rx cfg */ + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) { + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_OL4; + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_IL4; + } + rx_cfg |= (ROC_NIX_LF_RX_CFG_DROP_RE | ROC_NIX_LF_RX_CFG_L2_LEN_ERR | + ROC_NIX_LF_RX_CFG_LEN_IL4 | ROC_NIX_LF_RX_CFG_LEN_IL3 | + ROC_NIX_LF_RX_CFG_LEN_OL4 | ROC_NIX_LF_RX_CFG_LEN_OL3); + + nb_rxq = RTE_MAX(data->nb_rx_queues, 1); + nb_txq = RTE_MAX(data->nb_tx_queues, 1); + + /* Alloc a nix lf */ + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("Failed to init nix_lf rc=%d", rc); + goto fail_configure; + } + + nb_rxq = data->nb_rx_queues; + nb_txq = data->nb_tx_queues; + rc = -ENOMEM; + if (nb_rxq) { + /* Allocate memory for roc rq's and cq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_rq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc rqs"); + goto free_nix_lf; + } + dev->rqs = qs; + + qs = plt_zmalloc(sizeof(struct roc_nix_cq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc cqs"); + goto free_nix_lf; + } + dev->cqs = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_sq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc sqs"); + goto free_nix_lf; + } + dev->sqs = qs; + } + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + if (nix->rx_ptp_ena) { + plt_err("Both PTP and switch header enabled"); + goto free_nix_lf; + } + + /* Setup LSO if needed */ + rc = nix_lso_fmt_setup(dev); + if (rc) { + plt_err("Failed to setup nix lso format fields, rc=%d", rc); + goto free_nix_lf; + } + + /* Configure RSS */ + rc = nix_rss_default_setup(dev); + if (rc) { + plt_err("Failed to configure rss rc=%d", rc); + goto free_nix_lf; + } + + /* Init the default TM scheduler hierarchy */ + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("Failed to init traffic manager, rc=%d", rc); + goto free_nix_lf; + } + + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("Failed to enable default tm hierarchy, rc=%d", rc); + goto tm_fini; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Register cq IRQs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) { + if (eth_dev->data->nb_rx_queues > dev->nix.cints) { + plt_err("Rx interrupt cannot be enabled, rxq > %d", + dev->nix.cints); + goto q_irq_fini; + } + /* Rx interrupt feature cannot work with vector mode because, + * vector mode does not process packets unless min 4 pkts are + * received, while cq interrupts are generated even for 1 pkt + * in the CQ. + */ + dev->scalar_ena = true; + + rc = roc_nix_register_cq_irqs(nix); + if (rc) { + plt_err("Failed to register CQ interrupts rc=%d", rc); + goto q_irq_fini; + } + } + + /* Configure loop back mode */ + rc = roc_nix_mac_loopback_enable(nix, + eth_dev->data->dev_conf.lpbk_mode); + if (rc) { + plt_err("Failed to configure cgx loop back mode rc=%d", rc); + goto cq_fini; + } + + /* + * Restore queue config when reconfigure followed by + * reconfigure and no queue configure invoked from application case. + */ + if (dev->configured == 1) { + rc = nix_restore_queue_cfg(eth_dev); + if (rc) + goto cq_fini; + } + + /* Update the mac address */ + ea = eth_dev->data->mac_addrs; + memcpy(ea, dev->mac_addr, RTE_ETHER_ADDR_LEN); + if (rte_is_zero_ether_addr(ea)) + rte_eth_random_addr((uint8_t *)ea); + + rte_ether_format_addr(ea_fmt, RTE_ETHER_ADDR_FMT_SIZE, ea); + + plt_nix_dbg("Configured port%d mac=%s nb_rxq=%d nb_txq=%d" + " rx_offloads=0x%" PRIx64 " tx_offloads=0x%" PRIx64 "", + eth_dev->data->port_id, ea_fmt, nb_rxq, nb_txq, + dev->rx_offloads, dev->tx_offloads); + + /* All good */ + dev->configured = 1; + dev->nb_rxq = data->nb_rx_queues; + dev->nb_txq = data->nb_tx_queues; + return 0; + +cq_fini: + roc_nix_unregister_cq_irqs(nix); +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_nix_lf: + nix_free_queue_mem(dev); + rc |= roc_nix_lf_free(nix); +fail_configure: + dev->configured = 0; + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, @@ -75,6 +624,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->eth_dev = eth_dev; + dev->configured = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. @@ -156,6 +706,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; + /* Clear the flag since we are closing down */ + dev->configured = 0; + roc_nix_npc_rx_ena_dis(nix, false); /* Free up SQs */ @@ -182,6 +735,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (eth_dev->data->dev_conf.intr_conf.rxq) roc_nix_unregister_cq_irqs(nix); + /* Free ROC RQ's, SQ's and CQ's memory */ + nix_free_queue_mem(dev); + /* Free nix lf resources */ rc = roc_nix_lf_free(nix); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8d9a7e0..291f5f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -65,10 +65,50 @@ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ DEV_RX_OFFLOAD_RSS_HASH) +#define RSS_IPV4_ENABLE \ + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV4_TCP | ETH_RSS_NONFRAG_IPV4_SCTP) + +#define RSS_IPV6_ENABLE \ + (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_UDP | \ + ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_NONFRAG_IPV6_SCTP) + +#define RSS_IPV6_EX_ENABLE \ + (ETH_RSS_IPV6_EX | ETH_RSS_IPV6_TCP_EX | ETH_RSS_IPV6_UDP_EX) + +#define RSS_MAX_LEVELS 3 + +#define RSS_IPV4_INDEX 0 +#define RSS_IPV6_INDEX 1 +#define RSS_TCP_INDEX 2 +#define RSS_UDP_INDEX 3 +#define RSS_SCTP_INDEX 4 +#define RSS_DMAC_INDEX 5 + +struct cnxk_eth_qconf { + union { + struct rte_eth_txconf tx; + struct rte_eth_rxconf rx; + } conf; + struct rte_mempool *mp; + uint16_t nb_desc; + uint8_t valid; +}; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC RQs, SQs and CQs */ + struct roc_nix_rq *rqs; + struct roc_nix_sq *sqs; + struct roc_nix_cq *cqs; + + /* Configured queue count */ + uint16_t nb_rxq; + uint16_t nb_txq; + uint8_t configured; + /* Max macfilter entries */ uint8_t max_mac_entries; @@ -90,17 +130,57 @@ struct cnxk_eth_dev { uint64_t rx_offload_capa; uint64_t tx_offload_capa; uint32_t speed_capa; + /* Configured Rx and Tx offloads */ + uint64_t rx_offloads; + uint64_t tx_offloads; + /* Platform specific offload flags */ + uint16_t rx_offload_flags; + uint16_t tx_offload_flags; + + /* ETHDEV RSS HF bitmask */ + uint64_t ethdev_rss_hf; + + /* Saved qconf before lf realloc */ + struct cnxk_eth_qconf *tx_qconf; + struct cnxk_eth_qconf *rx_qconf; /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + + /* LSO Tunnel format indices */ + uint64_t lso_tun_fmt; }; +struct cnxk_eth_rxq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + +struct cnxk_eth_txq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + static inline struct cnxk_eth_dev * cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) { return eth_dev->data->dev_private; } +static inline struct cnxk_eth_rxq_sp * +cnxk_eth_rxq_to_sp(void *__rxq) +{ + return ((struct cnxk_eth_rxq_sp *)__rxq) - 1; +} + +static inline struct cnxk_eth_txq_sp * +cnxk_eth_txq_to_sp(void *__txq) +{ + return ((struct cnxk_eth_txq_sp *)__txq) - 1; +} + /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; @@ -110,6 +190,11 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_configure(struct rte_eth_dev *eth_dev); + +/* RSS */ +uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 14/62] net/cnxk: add link status update support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (12 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram ` (48 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add link status update callback to get current link status. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 7 +++ drivers/net/cnxk/cnxk_ethdev.h | 8 +++ drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_link.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 0c2ea89..7bf6cf5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -20,6 +20,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Link state information Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 96dba2a..affbbd9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 616991c..836cc9f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a0bd2f1..29bb24f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 14e28fe..002afdc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -589,6 +589,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, + .link_update = cnxk_nix_link_update, }; static int @@ -623,6 +624,9 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) goto error; } + /* Register up msg callbacks */ + roc_nix_mac_link_cb_register(nix, cnxk_eth_dev_link_status_cb); + dev->eth_dev = eth_dev; dev->configured = 0; @@ -711,6 +715,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable link status events */ + roc_nix_mac_link_event_start_stop(nix, false); + /* Free up SQs */ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 291f5f9..daa87af 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,6 +15,9 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Used for struct cnxk_eth_dev::flags */ +#define CNXK_LINK_CFG_IN_PROGRESS_F BIT_ULL(0) + /* VLAN tag inserted by NIX_TX_VTAG_ACTION. * In Tx space is always reserved for this in FRS. */ @@ -196,6 +199,11 @@ int cnxk_nix_configure(struct rte_eth_dev *eth_dev); uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +/* Link */ +void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, + struct roc_nix_link_info *link); +int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c new file mode 100644 index 0000000..b0273e7 --- /dev/null +++ b/drivers/net/cnxk/cnxk_link.c @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +static inline int +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) +{ + uint16_t wait = 1000; + + do { + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) + break; + wait--; + rte_delay_ms(1); + } while (wait); + + return wait ? 0 : -1; +} + +static void +nix_link_status_print(struct rte_eth_dev *eth_dev, struct rte_eth_link *link) +{ + if (link && link->link_status) + plt_info("Port %d: Link Up - speed %u Mbps - %s", + (int)(eth_dev->data->port_id), + (uint32_t)link->link_speed, + link->link_duplex == ETH_LINK_FULL_DUPLEX + ? "full-duplex" + : "half-duplex"); + else + plt_info("Port %d: Link Down", (int)(eth_dev->data->port_id)); +} + +void +cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_link eth_link; + struct rte_eth_dev *eth_dev; + + if (!link || !nix) + return; + + eth_dev = dev->eth_dev; + if (!eth_dev || !eth_dev->data->dev_conf.intr_conf.lsc) + return; + + if (nix_wait_for_link_cfg(dev)) { + plt_err("Timeout waiting for link_cfg to complete"); + return; + } + + eth_link.link_status = link->status; + eth_link.link_speed = link->speed; + eth_link.link_autoneg = ETH_LINK_AUTONEG; + eth_link.link_duplex = link->full_duplex; + + /* Print link info */ + nix_link_status_print(eth_dev, ð_link); + + /* Update link info */ + rte_eth_linkstatus_set(eth_dev, ð_link); + + /* Set the flag and execute application callbacks */ + rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); +} + +int +cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_link_info info; + struct rte_eth_link link; + int rc; + + RTE_SET_USED(wait_to_complete); + memset(&link, 0, sizeof(struct rte_eth_link)); + + if (roc_nix_is_sdp(&dev->nix)) + return 0; + + if (roc_nix_is_lbk(&dev->nix)) { + link.link_status = ETH_LINK_UP; + link.link_speed = ETH_SPEED_NUM_100G; + link.link_autoneg = ETH_LINK_FIXED; + link.link_duplex = ETH_LINK_FULL_DUPLEX; + } else { + rc = roc_nix_mac_link_info_get(&dev->nix, &info); + if (rc) + return rc; + link.link_status = info.status; + link.link_speed = info.speed; + link.link_autoneg = ETH_LINK_AUTONEG; + if (info.full_duplex) + link.link_duplex = info.full_duplex; + } + + return rte_eth_linkstatus_set(eth_dev, &link); +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 8495732..1ac3d08 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,7 +10,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', - 'cnxk_ethdev_devargs.c') + 'cnxk_ethdev_devargs.c', + 'cnxk_link.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 15/62] net/cnxk: add Rx queue setup and release 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (13 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 14/62] net/cnxk: add link status update support Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 16/62] net/cnxk: add Tx " Nithin Dabilpuram ` (47 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d971bbd..b87c4e5 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 7f3e910..ed9a813 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 002afdc..10e3ead 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -590,6 +761,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index daa87af..4a7c2ca 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> #include <rte_kvargs.h> +#include <rte_mbuf.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> #include "roc_api.h" @@ -194,6 +197,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 16/62] net/cnxk: add Tx queue setup and release 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (14 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 17/62] net/cnxk: add packet type support Nithin Dabilpuram ` (46 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram aDD tx queue setup and release for CN9K and CN10K. Release is common while setup is platform dependent due to differences in fast path Tx queue structures. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 72 +++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 13 +++++ drivers/net/cnxk/cn10k_tx.h | 13 +++++ drivers/net/cnxk/cn9k_ethdev.c | 70 +++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 11 ++++ drivers/net/cnxk/cn9k_tx.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev.c | 98 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++ 11 files changed, 296 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx.h diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index a9d2b03..462d7c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6a8ca1f..09e0d3a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index f761638..4a93a35 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -10,6 +10,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index b87c4e5..454c8ca 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,77 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + union nix_send_hdr_w0_u send_hdr_w0; + union nix_send_sg_s sg_w0; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + send_hdr_w0.u = 0; + sg_w0.u = 0; + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr_w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + } else { + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr_w0.sizem1 = 1; + } + + send_hdr_w0.sq = qid; + sg_w0.subdc = NIX_SUBDC_SG; + sg_w0.segs = 1; + sg_w0.ld_type = NIX_SENDLDTYPE_LDD; + + txq->send_hdr_w0 = send_hdr_w0.u; + txq->sg_w0 = sg_w0.u; + + rte_wmb(); +} + +static int +cn10k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + /* Store lmt base in tx queue for easy access */ + txq->lmt_base = dev->nix.lmt_base; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + txq->lso_tun_fmt = dev->lso_tun_fmt; + return 0; +} static int cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -76,6 +147,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 08e11bb..18deb95 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,6 +6,19 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_txq { + uint64_t send_hdr_w0; + uint64_t sg_w0; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + uintptr_t lmt_base; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t cmd[4]; + uint64_t lso_tun_fmt; +} __plt_cache_aligned; + struct cn10k_eth_rxq { uint64_t mbuf_initializer; uintptr_t desc; diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h new file mode 100644 index 0000000..39d4755 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_TX_H__ +#define __CN10K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index ed9a813..a8ac2c5 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,75 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr->w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + sg = (union nix_send_sg_s *)&txq->cmd[4]; + } else { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr->w0.sizem1 = 1; + sg = (union nix_send_sg_s *)&txq->cmd[2]; + } + + send_hdr->w0.sq = qid; + sg->subdc = NIX_SUBDC_SG; + sg->segs = 1; + sg->ld_type = NIX_SENDLDTYPE_LDD; + + rte_wmb(); +} + +static int +cn9k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + txq->lmt_addr = sq->lmt_addr; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + txq->lso_tun_fmt = dev->lso_tun_fmt; + return 0; +} static int cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -87,6 +156,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 6384609..de635fa 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,6 +6,17 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_txq { + uint64_t cmd[8]; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + void *lmt_addr; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t lso_tun_fmt; +} __plt_cache_aligned; + struct cn9k_eth_rxq { uint64_t mbuf_initializer; uint64_t data_off; diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h new file mode 100644 index 0000000..bb6379b --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_TX_H__ +#define __CN9K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 10e3ead..0a11081 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -66,6 +66,103 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) return *tmp; } +static inline uint8_t +nix_sq_max_sqe_sz(struct cnxk_eth_dev *dev) +{ + /* + * Maximum three segments can be supported with W8, Choose + * NIX_MAXSQESZ_W16 for multi segment offload. + */ + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + return NIX_MAXSQESZ_W16; + else + return NIX_MAXSQESZ_W8; +} + +int +cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_txq_sp *txq_sp; + struct roc_nix_sq *sq; + size_t txq_sz; + int rc; + + /* Free memory prior to re-allocation if needed. */ + if (eth_dev->data->tx_queues[qid] != NULL) { + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->tx_queue_release(eth_dev->data->tx_queues[qid]); + eth_dev->data->tx_queues[qid] = NULL; + } + + /* Setup ROC SQ */ + sq = &dev->sqs[qid]; + sq->qid = qid; + sq->nb_desc = nb_desc; + sq->max_sqe_sz = nix_sq_max_sqe_sz(dev); + + rc = roc_nix_sq_init(&dev->nix, sq); + if (rc) { + plt_err("Failed to init sq=%d, rc=%d", qid, rc); + return rc; + } + + rc = -ENOMEM; + txq_sz = sizeof(struct cnxk_eth_txq_sp) + fp_tx_q_sz; + txq_sp = plt_zmalloc(txq_sz, PLT_CACHE_LINE_SIZE); + if (!txq_sp) { + plt_err("Failed to alloc tx queue mem"); + rc |= roc_nix_sq_fini(sq); + return rc; + } + + txq_sp->dev = dev; + txq_sp->qid = qid; + txq_sp->qconf.conf.tx = *tx_conf; + txq_sp->qconf.nb_desc = nb_desc; + + plt_nix_dbg("sq=%d fc=%p offload=0x%" PRIx64 " lmt_addr=%p" + " nb_sqb_bufs=%d sqes_per_sqb_log2=%d", + qid, sq->fc, dev->tx_offloads, sq->lmt_addr, + sq->nb_sqb_bufs, sq->sqes_per_sqb_log2); + + /* Store start of fast path area */ + eth_dev->data->tx_queues[qid] = txq_sp + 1; + eth_dev->data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + +static void +cnxk_nix_tx_queue_release(void *txq) +{ + struct cnxk_eth_txq_sp *txq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_sq *sq; + uint16_t qid; + int rc; + + if (!txq) + return; + + txq_sp = cnxk_eth_txq_to_sp(txq); + dev = txq_sp->dev; + qid = txq_sp->qid; + + plt_nix_dbg("Releasing txq %u", qid); + + /* Cleanup ROC SQ */ + sq = &dev->sqs[qid]; + rc = roc_nix_sq_fini(sq); + if (rc) + plt_err("Failed to cleanup sq, rc=%d", rc); + + /* Finally free */ + plt_free(txq_sp); +} + int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, @@ -761,6 +858,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4a7c2ca..ef8e408 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -197,6 +197,9 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf); int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 17/62] net/cnxk: add packet type support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (15 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 16/62] net/cnxk: add Tx " Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram ` (45 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add support for packet type lookup on Rx to translate HW specific types to RTE_PTYPE_* defines Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 21 +++ drivers/net/cnxk/cn10k_rx.h | 11 ++ drivers/net/cnxk/cn9k_ethdev.c | 21 +++ drivers/net/cnxk/cn9k_rx.h | 12 ++ drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 14 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 12 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cnxk_lookup.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 7bf6cf5..8bc85c0 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,7 @@ Features Features of the CNXK Ethdev PMD are: +- Packet type information - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 462d7c4..503582c 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 09e0d3a..9ad225a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4a93a35..8c93ba7 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -13,6 +13,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 454c8ca..f79d03c 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rx.h" #include "cn10k_tx.h" +static int +cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, uint16_t qid) @@ -114,6 +131,9 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -149,6 +169,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h new file mode 100644 index 0000000..d3d1661 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_RX_H__ +#define __CN10K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index a8ac2c5..b5a4bd7 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rx.h" #include "cn9k_tx.h" +static int +cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, uint16_t qid) @@ -112,6 +129,9 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -158,6 +178,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h new file mode 100644 index 0000000..95a1e69 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef __CN9K_RX_H__ +#define __CN9K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 0a11081..6a0bd50 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -860,6 +860,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; static int @@ -899,6 +900,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->eth_dev = eth_dev; dev->configured = 0; + dev->ptype_disable = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ef8e408..b23df4a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,15 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ \ + ((PTYPE_NON_TUNNEL_ARRAY_SZ + PTYPE_TUNNEL_ARRAY_SZ) * sizeof(uint16_t)) +/* Fastpath lookup */ +#define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -119,6 +128,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + uint8_t ptype_disable; bool scalar_ena; /* Pointer back to rte */ @@ -216,6 +226,10 @@ void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +/* Lookup configuration */ +const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); +void *cnxk_nix_fastpath_lookup_mem_get(void); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_lookup.c b/drivers/net/cnxk/cnxk_lookup.c new file mode 100644 index 0000000..0152ad9 --- /dev/null +++ b/drivers/net/cnxk/cnxk_lookup.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <rte_common.h> +#include <rte_memzone.h> + +#include "cnxk_ethdev.h" + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) * sizeof(uint32_t)) + +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ + SA_TBL_SZ) +const uint32_t * +cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) +{ + RTE_SET_USED(eth_dev); + + static const uint32_t ptypes[] = { + RTE_PTYPE_L2_ETHER_QINQ, /* LB */ + RTE_PTYPE_L2_ETHER_VLAN, /* LB */ + RTE_PTYPE_L2_ETHER_TIMESYNC, /* LB */ + RTE_PTYPE_L2_ETHER_ARP, /* LC */ + RTE_PTYPE_L2_ETHER_NSH, /* LC */ + RTE_PTYPE_L2_ETHER_FCOE, /* LC */ + RTE_PTYPE_L2_ETHER_MPLS, /* LC */ + RTE_PTYPE_L3_IPV4, /* LC */ + RTE_PTYPE_L3_IPV4_EXT, /* LC */ + RTE_PTYPE_L3_IPV6, /* LC */ + RTE_PTYPE_L3_IPV6_EXT, /* LC */ + RTE_PTYPE_L4_TCP, /* LD */ + RTE_PTYPE_L4_UDP, /* LD */ + RTE_PTYPE_L4_SCTP, /* LD */ + RTE_PTYPE_L4_ICMP, /* LD */ + RTE_PTYPE_L4_IGMP, /* LD */ + RTE_PTYPE_TUNNEL_GRE, /* LD */ + RTE_PTYPE_TUNNEL_ESP, /* LD */ + RTE_PTYPE_TUNNEL_NVGRE, /* LD */ + RTE_PTYPE_TUNNEL_VXLAN, /* LE */ + RTE_PTYPE_TUNNEL_GENEVE, /* LE */ + RTE_PTYPE_TUNNEL_GTPC, /* LE */ + RTE_PTYPE_TUNNEL_GTPU, /* LE */ + RTE_PTYPE_TUNNEL_VXLAN_GPE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_GRE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_UDP, /* LE */ + RTE_PTYPE_INNER_L2_ETHER, /* LF */ + RTE_PTYPE_INNER_L3_IPV4, /* LG */ + RTE_PTYPE_INNER_L3_IPV6, /* LG */ + RTE_PTYPE_INNER_L4_TCP, /* LH */ + RTE_PTYPE_INNER_L4_UDP, /* LH */ + RTE_PTYPE_INNER_L4_SCTP, /* LH */ + RTE_PTYPE_INNER_L4_ICMP, /* LH */ + RTE_PTYPE_UNKNOWN, + }; + + return ptypes; +} + +/* + * +------------------ +------------------ + + * | | IL4 | IL3| IL2 | TU | L4 | L3 | L2 | + * +-------------------+-------------------+ + * + * +-------------------+------------------ + + * | | LH | LG | LF | LE | LD | LC | LB | + * +-------------------+-------------------+ + * + * ptype [LE - LD - LC - LB] = TU - L4 - L3 - T2 + * ptype_tunnel[LH - LG - LF] = IL4 - IL3 - IL2 - TU + * + */ +static void +nix_create_non_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lb, lc, ld, le; + uint16_t val; + uint32_t idx; + + for (idx = 0; idx < PTYPE_NON_TUNNEL_ARRAY_SZ; idx++) { + lb = idx & 0xF; + lc = (idx & 0xF0) >> 4; + ld = (idx & 0xF00) >> 8; + le = (idx & 0xF000) >> 12; + val = RTE_PTYPE_UNKNOWN; + + switch (lb) { + case NPC_LT_LB_STAG_QINQ: + val |= RTE_PTYPE_L2_ETHER_QINQ; + break; + case NPC_LT_LB_CTAG: + val |= RTE_PTYPE_L2_ETHER_VLAN; + break; + } + + switch (lc) { + case NPC_LT_LC_ARP: + val |= RTE_PTYPE_L2_ETHER_ARP; + break; + case NPC_LT_LC_NSH: + val |= RTE_PTYPE_L2_ETHER_NSH; + break; + case NPC_LT_LC_FCOE: + val |= RTE_PTYPE_L2_ETHER_FCOE; + break; + case NPC_LT_LC_MPLS: + val |= RTE_PTYPE_L2_ETHER_MPLS; + break; + case NPC_LT_LC_IP: + val |= RTE_PTYPE_L3_IPV4; + break; + case NPC_LT_LC_IP_OPT: + val |= RTE_PTYPE_L3_IPV4_EXT; + break; + case NPC_LT_LC_IP6: + val |= RTE_PTYPE_L3_IPV6; + break; + case NPC_LT_LC_IP6_EXT: + val |= RTE_PTYPE_L3_IPV6_EXT; + break; + case NPC_LT_LC_PTP: + val |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + } + + switch (ld) { + case NPC_LT_LD_TCP: + val |= RTE_PTYPE_L4_TCP; + break; + case NPC_LT_LD_UDP: + val |= RTE_PTYPE_L4_UDP; + break; + case NPC_LT_LD_SCTP: + val |= RTE_PTYPE_L4_SCTP; + break; + case NPC_LT_LD_ICMP: + case NPC_LT_LD_ICMP6: + val |= RTE_PTYPE_L4_ICMP; + break; + case NPC_LT_LD_IGMP: + val |= RTE_PTYPE_L4_IGMP; + break; + case NPC_LT_LD_GRE: + val |= RTE_PTYPE_TUNNEL_GRE; + break; + case NPC_LT_LD_NVGRE: + val |= RTE_PTYPE_TUNNEL_NVGRE; + break; + } + + switch (le) { + case NPC_LT_LE_VXLAN: + val |= RTE_PTYPE_TUNNEL_VXLAN; + break; + case NPC_LT_LE_ESP: + val |= RTE_PTYPE_TUNNEL_ESP; + break; + case NPC_LT_LE_VXLANGPE: + val |= RTE_PTYPE_TUNNEL_VXLAN_GPE; + break; + case NPC_LT_LE_GENEVE: + val |= RTE_PTYPE_TUNNEL_GENEVE; + break; + case NPC_LT_LE_GTPC: + val |= RTE_PTYPE_TUNNEL_GTPC; + break; + case NPC_LT_LE_GTPU: + val |= RTE_PTYPE_TUNNEL_GTPU; + break; + case NPC_LT_LE_TU_MPLS_IN_GRE: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + break; + case NPC_LT_LE_TU_MPLS_IN_UDP: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_UDP; + break; + } + ptype[idx] = val; + } +} + +#define TU_SHIFT(x) ((x) >> PTYPE_NON_TUNNEL_WIDTH) +static void +nix_create_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lf, lg, lh; + uint16_t val; + uint32_t idx; + + /* Skip non tunnel ptype array memory */ + ptype = ptype + PTYPE_NON_TUNNEL_ARRAY_SZ; + + for (idx = 0; idx < PTYPE_TUNNEL_ARRAY_SZ; idx++) { + lf = idx & 0xF; + lg = (idx & 0xF0) >> 4; + lh = (idx & 0xF00) >> 8; + val = RTE_PTYPE_UNKNOWN; + + switch (lf) { + case NPC_LT_LF_TU_ETHER: + val |= TU_SHIFT(RTE_PTYPE_INNER_L2_ETHER); + break; + } + switch (lg) { + case NPC_LT_LG_TU_IP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV4); + break; + case NPC_LT_LG_TU_IP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV6); + break; + } + switch (lh) { + case NPC_LT_LH_TU_TCP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_TCP); + break; + case NPC_LT_LH_TU_UDP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_UDP); + break; + case NPC_LT_LH_TU_SCTP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_SCTP); + break; + case NPC_LT_LH_TU_ICMP: + case NPC_LT_LH_TU_ICMP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_ICMP); + break; + } + + ptype[idx] = val; + } +} + +static void +nix_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)((uint8_t *)mem + PTYPE_ARRAY_SZ); + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errlev = idx & 0xf; + errcode = (idx & 0xff0) >> 4; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case NPC_ERRLEV_RE: + /* Mark all errors as BAD checksum errors + * including Outer L2 length mismatch error + */ + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LC: + if (errcode == NPC_EC_OIP4_CSUM || + errcode == NPC_EC_IP_FRAG_OFFSET_1) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_OUTER_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LG: + if (errcode == NPC_EC_IIP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case NPC_ERRLEV_NIX: + if (errcode == NIX_RX_PERRCODE_OL4_CHK || + errcode == NIX_RX_PERRCODE_OL4_LEN || + errcode == NIX_RX_PERRCODE_OL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL4_CHK || + errcode == NIX_RX_PERRCODE_IL4_LEN || + errcode == NIX_RX_PERRCODE_IL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL3_LEN || + errcode == NIX_RX_PERRCODE_OL3_LEN) { + val |= PKT_RX_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + } + ol_flags[idx] = val; + } +} + +void * +cnxk_nix_fastpath_lookup_mem_get(void) +{ + const char name[] = CNXK_NIX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, SOCKET_ID_ANY, + 0, ROC_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the ptype array lookup memory */ + nix_create_non_tunnel_ptype_array(mem); + nix_create_tunnel_ptype_array(mem); + /* Form the rx ol_flags based on errcode */ + nix_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1ac3d08..5bc0bb5 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -11,7 +11,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', - 'cnxk_link.c') + 'cnxk_link.c', + 'cnxk_lookup.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 18/62] net/cnxk: add queue start and stop support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (16 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 17/62] net/cnxk: add packet type support Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram ` (44 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx/Tx queue start and stop callbacks for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 16 ++++++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++++++ drivers/net/cnxk/cnxk_ethdev.c | 92 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 1 + 7 files changed, 128 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 503582c..712f8d5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 9ad225a..82f2af0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8c93ba7..61fed11 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index f79d03c..d70ab00 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -138,6 +138,21 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn10k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn10k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -169,6 +184,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index b5a4bd7..1dab096 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -136,6 +136,21 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn9k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn9k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -178,6 +193,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6a0bd50..98e0ba0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -854,12 +854,104 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } +static int +cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc = -EINVAL; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, true); + if (rc) { + plt_err("Failed to enable sq aura fc, txq=%u, rc=%d", qid, rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +int +cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, false); + if (rc) { + plt_err("Failed to disable sqb aura fc, txq=%u, rc=%d", qid, + rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, true); + if (rc) { + plt_err("Failed to enable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, false); + if (rc) { + plt_err("Failed to disable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .tx_queue_start = cnxk_nix_tx_queue_start, + .rx_queue_start = cnxk_nix_rx_queue_start, + .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index b23df4a..5a52489 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -214,6 +214,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 19/62] net/cnxk: add Rx support for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (17 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram ` (43 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Jerin Jacob <jerinj@marvell.com> Add Rx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 3 + drivers/net/cnxk/cn9k_rx.c | 46 ++++++++ drivers/net/cnxk/cn9k_rx.h | 237 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/meson.build | 3 +- 5 files changed, 291 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_rx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index de635fa..bd41d3c 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -31,4 +31,7 @@ struct cn9k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c new file mode 100644 index 0000000..a4297f9 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 95a1e69..92f3c7c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -7,6 +7,243 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->cn9k.pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = + nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn9k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5a52489..a6f5d36 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,9 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +/* Default mark value used when none is provided. */ +#define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 5bc0bb5..7a44001 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -15,7 +15,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_lookup.c') # CN9K -sources += files('cn9k_ethdev.c') +sources += files('cn9k_ethdev.c', + 'cn9k_rx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 20/62] net/cnxk: add Rx multi-segmented version for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (18 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram @ 2021-06-18 10:36 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (42 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:36 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 17 ++++++++++++ drivers/net/cnxk/cn9k_rx.h | 60 ++++++++++++++++++++++++++++++++++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 17 ++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/meson.build | 3 ++- 5 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index a4297f9..87a62c9 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -40,7 +42,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 92f3c7c..49f80ce 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -104,6 +104,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -138,8 +185,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,8 +290,11 @@ R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c new file mode 100644 index 0000000..6ad8c1d --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a6f5d36..333a54c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7a44001..5bfab21 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -16,7 +16,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', - 'cn9k_rx.c') + 'cn9k_rx.c', + 'cn9k_rx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 21/62] net/cnxk: add Rx vector version for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (19 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 22/62] net/cnxk: add Tx support " Nithin Dabilpuram ` (41 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 13 ++- drivers/net/cnxk/cn9k_rx.h | 221 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx_vec.c | 17 ++++ drivers/net/cnxk/meson.build | 11 +- 4 files changed, 260 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 87a62c9..01eb21f 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -50,7 +50,18 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 49f80ce..bc04f5c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -6,6 +6,7 @@ #define __CN9K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -266,6 +267,223 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn9k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -295,6 +513,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c new file mode 100644 index 0000000..997177f --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 5bfab21..9aba7d4 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -17,9 +17,18 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', 'cn9k_rx.c', - 'cn9k_rx_mseg.c') + 'cn9k_rx_mseg.c', + 'cn9k_rx_vec.c') # CN10K sources += files('cn10k_ethdev.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] + +# Allow implicit vector conversions +extra_flags = ['-flax-vector-conversions'] +foreach flag: extra_flags + if cc.has_argument(flag) + cflags += flag + endif +endforeach -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 22/62] net/cnxk: add Tx support for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (20 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (40 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_tx.c | 53 ++++++ drivers/net/cnxk/cn9k_tx.h | 419 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 71 +++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 546 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index bd41d3c..b92f3fc 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -33,5 +33,6 @@ struct cn9k_eth_rxq { /* Rx and Tx routines */ void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c new file mode 100644 index 0000000..a0b022a --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.c @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags);\ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index bb6379b..7acecc6 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,10 +4,429 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn9k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags, + const uint64_t lso_tun_fmt) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + uint8_t shift = is_udp_tun ? 32 : 0; + + shift += (!!(ol_flags & PKT_TX_OUTER_IPV6) << 4); + shift += (!!(ol_flags & PKT_TX_IPV6) << 3); + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format = (lso_tun_fmt >> shift); + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(++sg) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, + const uint32_t flags) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_prep_lmt(uint64_t *cmd, void *lmt_addr, const uint32_t flags) +{ + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeor(io_addr); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeorl(io_addr); +} + +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint64_t lso_tun_fmt; + uint16_t i; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + lso_tun_fmt = txq->lso_tun_fmt; + + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 333a54c..58cc6b7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -103,6 +103,10 @@ /* Fastpath lookup */ #define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" +#define CNXK_NIX_UDP_TUN_BITMASK \ + ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ + (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -241,4 +245,71 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Inlines */ +static __rte_always_inline uint64_t +cnxk_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +cnxk_nix_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 9aba7d4..6c2cd13 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -18,7 +18,8 @@ sources = files('cnxk_ethdev.c', sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', - 'cn9k_rx_vec.c') + 'cn9k_rx_vec.c', + 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 23/62] net/cnxk: add Tx multi-segment version for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (21 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 22/62] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (39 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst multi-segment version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 14 ++++ drivers/net/cnxk/cn9k_tx.h | 150 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 +++++++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index a0b022a..8f1d5f5 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,6 +21,7 @@ NIX_TX_FASTPATH_MODES #undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -39,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, @@ -47,7 +50,18 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 7acecc6..d9aa406 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -311,6 +311,111 @@ cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) } static __rte_always_inline uint16_t +cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Commit changes to mbuf */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + rte_io_wmb(); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_prep_lmt(uint64_t *cmd, void *lmt_addr, uint16_t segdw) +{ + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, rte_iova_t io_addr, + uint16_t segdw) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_io_wmb(); + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline uint16_t cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -349,6 +454,48 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint64_t lso_tun_fmt; + uint16_t segdw; + uint64_t i; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + lso_tun_fmt = txq->lso_tun_fmt; + + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -424,6 +571,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c new file mode 100644 index 0000000..65c5f36 --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58cc6b7..276e569 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -46,6 +46,10 @@ #define CNXK_NIX_TX_NB_SEG_MAX 9 #endif +#define CNXK_NIX_TX_MSEG_SG_DWORDS \ + ((RTE_ALIGN_MUL_CEIL(CNXK_NIX_TX_NB_SEG_MAX, 3) / 3) + \ + CNXK_NIX_TX_NB_SEG_MAX) + #define CNXK_NIX_RSS_L3_L4_SRC_DST \ (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ ETH_RSS_L4_DST_ONLY) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 6c2cd13..a3cd200 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -19,7 +19,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', - 'cn9k_tx.c') + 'cn9k_tx.c', + 'cn9k_tx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 24/62] net/cnxk: add Tx vector version for cn9k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (22 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram ` (38 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst vector version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 16 +- drivers/net/cnxk/cn9k_tx.h | 743 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 784 insertions(+), 3 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 8f1d5f5..2ff9720 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,7 +21,6 @@ NIX_TX_FASTPATH_MODES #undef T - static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -58,7 +57,20 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index d9aa406..7b0d536 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -495,6 +497,744 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64x2_t cmd0[NIX_DESCS_PER_LOOP], cmd1[NIX_DESCS_PER_LOOP]; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn9k_eth_txq *txq = tx_queue; + uint64_t *lmt_addr = txq->lmt_addr; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64_t lmt_status, i; + uint16_t pkts_left; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->cmd[2]); + sgdesc23_w0 = sgdesc01_w0; + + for (i = 0; i < pkts; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + /* Move mbuf to point to pool_id. */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, pool_id)); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf0)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf1)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf2)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf3)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } else { + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + /* Mark mempool object as "put" since + * it is freed by NIX + */ + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + rte_io_wmb(); +#endif + } + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd0[0] = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd0[1] = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd0[2] = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd0[3] = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + cmd1[0] = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[1] = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[2] = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd1[3] = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + do { + vst1q_u64(lmt_addr, cmd0[0]); + vst1q_u64(lmt_addr + 2, cmd1[0]); + vst1q_u64(lmt_addr + 4, cmd0[1]); + vst1q_u64(lmt_addr + 6, cmd1[1]); + vst1q_u64(lmt_addr + 8, cmd0[2]); + vst1q_u64(lmt_addr + 10, cmd1[2]); + vst1q_u64(lmt_addr + 12, cmd0[3]); + vst1q_u64(lmt_addr + 14, cmd1[3]); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + if (unlikely(pkts_left)) + pkts += cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts_left, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F @@ -574,6 +1314,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c new file mode 100644 index 0000000..21ffc2c --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index a3cd200..534c4bd 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -20,7 +20,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', 'cn9k_tx.c', - 'cn9k_tx_mseg.c') + 'cn9k_tx_mseg.c', + 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 25/62] net/cnxk: add Rx support for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (23 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram ` (37 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst support for CN10K SoC. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.h | 3 + drivers/net/cnxk/cn10k_rx.c | 45 ++++++++ drivers/net/cnxk/cn10k_rx.h | 236 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.c diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 18deb95..596985f 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -33,4 +33,7 @@ struct cn10k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c new file mode 100644 index 0000000..8b422d0 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.c @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index d3d1661..01c9d29 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -6,6 +6,242 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn10k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 534c4bd..1fcc211 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -23,7 +23,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_mseg.c', 'cn9k_tx_vec.c') # CN10K -sources += files('cn10k_ethdev.c') +sources += files('cn10k_ethdev.c', + 'cn10k_rx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 26/62] net/cnxk: add Rx multi-segment version for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (24 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (36 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- doc/guides/nics/cnxk.rst | 2 ++ doc/guides/nics/features/cnxk.ini | 2 ++ doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 2 ++ drivers/net/cnxk/cn10k_rx.c | 20 +++++++++++- drivers/net/cnxk/cn10k_rx.h | 57 +++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx_mseg.c | 17 +++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 100 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 8bc85c0..fd7f2dd 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,11 +17,13 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Jumbo frames - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - Link state information +- Scatter-Gather IO support Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 712f8d5..23564b7 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 82f2af0..421048d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 61fed11..e901fa2 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 8b422d0..ce2cfee 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -10,7 +10,7 @@ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ - } + } \ NIX_RX_FASTPATH_MODES #undef R @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, @@ -40,6 +42,22 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 01c9d29..c667c9a 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -103,6 +103,52 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + ((rx->desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -136,8 +182,12 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,6 +289,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c new file mode 100644 index 0000000..9d283f7 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1fcc211..20bb00b 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -24,7 +24,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c', - 'cn10k_rx.c') + 'cn10k_rx.c', + 'cn10k_rx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 27/62] net/cnxk: add Rx vector version for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (25 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 28/62] net/cnxk: add Tx support " Nithin Dabilpuram ` (35 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + drivers/net/cnxk/cn10k_rx.c | 13 ++- drivers/net/cnxk/cn10k_rx.h | 222 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_rx_vec.c | 19 ++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 256 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index fd7f2dd..481bc7e 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Link state information - Scatter-Gather IO support +- Vector Poll mode driver Prerequisites ------------- diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index ce2cfee..0598111 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -50,7 +50,18 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index c667c9a..7bb9dd8 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -5,6 +5,7 @@ #define __CN10K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -263,6 +264,224 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn10k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -292,6 +511,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c new file mode 100644 index 0000000..0fa079c --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 20bb00b..6748e9c 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -25,7 +25,8 @@ sources += files('cn9k_ethdev.c', # CN10K sources += files('cn10k_ethdev.c', 'cn10k_rx.c', - 'cn10k_rx_mseg.c') + 'cn10k_rx_mseg.c', + 'cn10k_rx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 28/62] net/cnxk: add Tx support for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (26 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (34 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 7 + doc/guides/nics/features/cnxk_vec.ini | 6 + doc/guides/nics/features/cnxk_vf.ini | 7 + drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_tx.c | 54 ++++ drivers/net/cnxk/cn10k_tx.h | 491 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 7 +- 8 files changed, 571 insertions(+), 3 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_tx.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 481bc7e..17da141 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -22,6 +22,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Inner and Outer Checksum offload - Link state information - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 23564b7..02be26b 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,11 +12,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 421048d..8c63853 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,10 +12,16 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index e901fa2..a1bd49b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,11 +11,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 596985f..d39ca31 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -35,5 +35,6 @@ struct cn10k_eth_rxq { /* Rx and Tx routines */ void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c new file mode 100644 index 0000000..13c605f --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ + flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 39d4755..c54fbfe 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,10 +4,501 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn10k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_data(const uint16_t flags) +{ + const uint64_t dw_m1 = cn10k_nix_tx_ext_subs(flags) + 1; + uint64_t data; + + /* This will be moved to addr area */ + data = dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + +static __rte_always_inline void +cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, + const uint16_t flags) +{ + /* Send hdr */ + cmd[0] = txq->send_hdr_w0; + cmd[1] = 0; + cmd += 2; + + /* Send ext if present */ + if (flags & NIX_TX_NEED_EXT_HDR) { + *(__uint128_t *)cmd = *(const __uint128_t *)txq->cmd; + cmd += 2; + } + + /* Send sg */ + cmd[0] = txq->sg_w0; + cmd[1] = 0; +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, + const uint16_t flags, const uint64_t lso_tun_fmt) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + uint8_t shift = is_udp_tun ? 32 : 0; + + shift += (!!(ol_flags & PKT_TX_OUTER_IPV6) << 4); + shift += (!!(ol_flags & PKT_TX_IPV6) << 3); + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format = (lso_tun_fmt >> shift); + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(sg + 1) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } + + /* With minimal offloads, 'cmd' being local could be optimized out to + * registers. In other cases, 'cmd' will be in stack. Intent is + * 'cmd' stores content from txq->cmd which is copied only once. + */ + *((struct nix_send_hdr_s *)lmt_addr) = *send_hdr; + lmt_addr += 16; + if (flags & NIX_TX_NEED_EXT_HDR) { + *((struct nix_send_ext_s *)lmt_addr) = *send_hdr_ext; + lmt_addr += 16; + } + /* In case of multi-seg, sg template is stored here */ + *((union nix_send_sg_s *)lmt_addr) = *sg; + *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); +} + +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + uintptr_t pa, lmt_addr = txq->lmt_base; + uint16_t lmt_id, burst, left, i; + uint64_t lso_tun_fmt; + uint64_t data; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + /* Get cmd skeleton */ + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + if (flags & NIX_TX_OFFLOAD_TSO_F) + lso_tun_fmt = txq->lso_tun_fmt; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, + lso_tun_fmt); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 6748e9c..b1ba824 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -26,13 +26,14 @@ sources += files('cn9k_ethdev.c', sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', - 'cn10k_rx_vec.c') + 'cn10k_rx_vec.c', + 'cn10k_tx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -# Allow implicit vector conversions -extra_flags = ['-flax-vector-conversions'] +# Allow implicit vector conversions and strict aliasing warning +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] foreach flag: extra_flags if cc.has_argument(flag) cflags += flag -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 29/62] net/cnxk: add Tx multi-segment version for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (27 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 28/62] net/cnxk: add Tx support " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (33 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 18 ++++- drivers/net/cnxk/cn10k_tx.h | 171 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 ++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 215 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 13c605f..9803002 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -40,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, @@ -48,7 +50,21 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index c54fbfe..63e9848 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -339,6 +339,77 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, } static __rte_always_inline uint16_t +cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Mark mempool object as "put" since it is freed by NIX + */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline uint16_t cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -421,6 +492,103 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; + const rte_iova_t io_addr = txq->io_addr; + uint16_t segdw, lmt_id, burst, left, i; + uint64_t data0, data1; + uint64_t lso_tun_fmt; + __uint128_t data128; + uint16_t shft; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + if (flags & NIX_TX_OFFLOAD_TSO_F) + lso_tun_fmt = txq->lso_tun_fmt; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + shft = 16; + data128 = 0; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, + lso_tun_fmt); + /* Store sg list directly on lmt line */ + segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, + flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + data128 |= (((__uint128_t)(segdw - 1)) << shft); + shft += 3; + } + + data0 = (uint64_t)data128; + data1 = (uint64_t)(data128 >> 64); + /* Make data0 similar to data1 */ + data0 >>= 16; + /* Trigger LMTST */ + if (burst > 16) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= (15ULL << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + + pa1 = io_addr | (data1 & 0x7) << 4; + data1 &= ~0x7ULL; + data1 <<= 16; + data1 |= ((uint64_t)(burst - 17)) << 12; + data1 |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data1, pa1); + } else if (burst) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= ((burst - 1) << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -496,6 +664,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c new file mode 100644 index 0000000..6ae6907 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz)]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b1ba824..21e5676 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -27,7 +27,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', - 'cn10k_tx.c') + 'cn10k_tx.c', + 'cn10k_tx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 30/62] net/cnxk: add Tx vector version for cn10k 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (28 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram ` (32 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst vector version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 10 + drivers/net/cnxk/cn10k_tx.h | 815 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 852 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 9803002..e6eb101 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -58,10 +58,20 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + if (dev->scalar_ena || (dev->tx_offload_flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 63e9848..b74df10 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -38,6 +40,9 @@ } \ } while (0) +#define LMT_OFF(lmt_addr, lmt_num, offset) \ + (void *)((lmt_addr) + ((lmt_num) << ROC_LMT_LINE_SIZE_LOG2) + (offset)) + /* Function to determine no of tx subdesc required in case ext * sub desc is enabled. */ @@ -48,6 +53,14 @@ cn10k_nix_tx_ext_subs(const uint16_t flags) (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; } +static __rte_always_inline uint8_t +cn10k_nix_pkts_per_vec_brst(const uint16_t flags) +{ + RTE_SET_USED(flags); + /* We can pack up to 4 packets per LMTLINE if there are no offloads. */ + return 4 << ROC_LMT_LINES_PER_CORE_LOG2; +} + static __rte_always_inline uint64_t cn10k_nix_tx_steor_data(const uint16_t flags) { @@ -76,6 +89,35 @@ cn10k_nix_tx_steor_data(const uint16_t flags) return data; } +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_vec_data(const uint16_t flags) +{ + const uint64_t dw_m1 = 0x7; + uint64_t data; + + RTE_SET_USED(flags); + /* This will be moved to addr area */ + data = dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + static __rte_always_inline void cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, const uint16_t flags) @@ -589,6 +631,776 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64x2_t cmd0[NIX_DESCS_PER_LOOP], cmd1[NIX_DESCS_PER_LOOP]; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3, data, pa; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint16_t left, scalar, burst, i, lmt_id; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t laddr = txq->lmt_base; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint8_t lnum; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + senddesc01_w0 = vld1q_dup_u64(&txq->send_hdr_w0); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->sg_w0); + sgdesc23_w0 = sgdesc01_w0; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(laddr, lmt_id); + left = pkts; +again: + /* Number of packets to prepare depends on offloads enabled. */ + burst = left > cn10k_nix_pkts_per_vec_brst(flags) ? + cn10k_nix_pkts_per_vec_brst(flags) : + left; + lnum = 0; + for (i = 0; i < burst; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + /* Move mbuf to point to pool_id. */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, pool_id)); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf0)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf1)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf2)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf3)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + } else { + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + /* Mark mempool object as "put" since + * it is freed by NIX + */ + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + } + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd0[0] = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd0[1] = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd0[2] = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd0[3] = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + cmd1[0] = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[1] = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[2] = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd1[3] = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Store the prepared send desc to LMT lines */ + vst1q_u64(LMT_OFF(laddr, lnum, 0), cmd0[0]); + vst1q_u64(LMT_OFF(laddr, lnum, 16), cmd1[0]); + vst1q_u64(LMT_OFF(laddr, lnum, 32), cmd0[1]); + vst1q_u64(LMT_OFF(laddr, lnum, 48), cmd1[1]); + vst1q_u64(LMT_OFF(laddr, lnum, 64), cmd0[2]); + vst1q_u64(LMT_OFF(laddr, lnum, 80), cmd1[2]); + vst1q_u64(LMT_OFF(laddr, lnum, 96), cmd0[3]); + vst1q_u64(LMT_OFF(laddr, lnum, 112), cmd1[3]); + lnum += 1; + + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + /* Trigger LMTST */ + if (lnum > 16) { + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(lnum - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (lnum) { + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(lnum - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) + goto again; + + if (unlikely(scalar)) + pkts += cn10k_nix_xmit_pkts(tx_queue, tx_pkts, scalar, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -667,6 +1479,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c new file mode 100644 index 0000000..42baeb5 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 21e5676..8f32dc7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -28,7 +28,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', 'cn10k_tx.c', - 'cn10k_tx_mseg.c') + 'cn10k_tx_mseg.c', + 'cn10k_tx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 31/62] net/cnxk: add device start and stop operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (29 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram ` (31 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add device start and stop operation callbacks for CN9K and CN10K. Device stop is common for both platforms while device start as some platform dependent portion where the platform specific offload flags are recomputed and the right Rx/Tx burst function is chosen. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 84 ++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 127 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 90 ++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 2 + drivers/net/cnxk/cnxk_link.c | 11 ++++ 6 files changed, 438 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 17da141..15911ee 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -39,6 +39,58 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1 + EAL: Detected 4 lcore(s) + EAL: Detected 1 NUMA nodes + EAL: Multi-process socket /var/run/dpdk/rte/mp_socket + EAL: Selected IOVA mode 'VA' + EAL: No available hugepages reported in hugepages-16777216kB + EAL: No available hugepages reported in hugepages-2048kB + EAL: Probing VFIO support... + EAL: VFIO support initialized + EAL: using IOMMU type 1 (Type 1) + [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98 + EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0) + PMD: RoC Model: cn10k + EAL: No legacy callbacks, legacy socket not created + testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0 + testpmd: preferred mempool ops selected: cn10k_mempool_ops + Configuring Port 0 (socket 0) + PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex + + Port 0: link state change event + Port 0: 96:D4:99:72:A5:BF + Checking link statuses... + Done + No commandline core given, start packet forwarding + io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native + Logical Core 3 (socket 0) forwards packets on 1 streams: + RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 + + io packet forwarding packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=1 + port 0: RX queue number: 1 Tx queue number: 1 + Rx offloads=0x0 Tx offloads=0x10000 + RX queue: 0 + RX desc=4096 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + RX Offloads=0x0 + TX queue: 0 + TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX offloads=0x0 - TX RS bit threshold=0 + Press enter to exit + Runtime Config Options ---------------------- @@ -132,3 +184,35 @@ Runtime Config Options Above devarg parameters are configurable per device, user needs to pass the parameters to all the PCIe devices if application requires to configure on all the ethdev ports. + +Limitations +----------- + +``mempool_cnxk`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager. +``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler +as it is performance wise most effective way for packet allocation and Tx buffer +recycling on OCTEON TX2 SoC platform. + +CRC stripping +~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by +the host interface irrespective of the offload configuration. + +Debugging Options +----------------- + +.. _table_cnxk_ethdev_debug_options: + +.. table:: cnxk ethdev debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NIX | --log-level='pmd\.net.cnxk,8' | + +---+------------+-------------------------------------------------------+ + | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | + +---+------------+-------------------------------------------------------+ diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d70ab00..5ff36bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -5,6 +5,98 @@ #include "cn10k_rx.h" #include "cn10k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn10k_eth_set_rx_function(eth_dev); return 0; } @@ -163,6 +256,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -170,6 +267,28 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -185,6 +304,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } @@ -222,6 +342,10 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (!eth_dev) return -ENOENT; + + /* Setup callbacks for secondary process */ + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); } return 0; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 1dab096..847d24a 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -5,6 +5,98 @@ #include "cn9k_rx.h" #include "cn9k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn9k_eth_set_rx_function(eth_dev); return 0; } @@ -172,6 +265,10 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -179,6 +276,28 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -194,6 +313,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } @@ -233,6 +353,13 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (!eth_dev) return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* Setup callbacks for secondary process */ + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; + } + dev = cnxk_eth_pmd_priv(eth_dev); /* Update capabilities already set for TSO. * TSO not supported for earlier chip revisions diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 98e0ba0..beb07ae 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -943,12 +943,102 @@ cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) return rc; } +static int +cnxk_nix_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct rte_mbuf *rx_pkts[32]; + int count, i, j, rc; + void *rxq; + + /* Disable switch hdr pkind */ + roc_nix_switch_hdr_set(&dev->nix, 0); + + /* Stop link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) + roc_nix_mac_link_event_start_stop(&dev->nix, false); + + /* Disable Rx via NPC */ + roc_nix_npc_rx_ena_dis(&dev->nix, false); + + /* Stop rx queues and free up pkts pending */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = dev_ops->rx_queue_stop(eth_dev, i); + if (rc) + continue; + + rxq = eth_dev->data->rx_queues[i]; + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + while (count) { + for (j = 0; j < count; j++) + rte_pktmbuf_free(rx_pkts[j]); + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + } + } + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_stop(eth_dev, i); + + return 0; +} + +int +cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc, i; + + /* Start rx queues */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = cnxk_nix_rx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Enable Rx in NPC */ + rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); + if (rc) { + plt_err("Failed to enable NPC rx %d", rc); + return rc; + } + + cnxk_nix_toggle_flag_link_cfg(dev, true); + + /* Start link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) { + rc = roc_nix_mac_link_event_start_stop(&dev->nix, true); + if (rc) { + plt_err("Failed to start cgx link event %d", rc); + goto rx_disable; + } + } + + cnxk_nix_toggle_flag_link_cfg(dev, false); + + return 0; + +rx_disable: + roc_nix_npc_rx_ena_dis(&dev->nix, false); + cnxk_nix_toggle_flag_link_cfg(dev, false); + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_stop = cnxk_nix_dev_stop, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 276e569..50c75e1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -229,6 +229,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); +int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -237,6 +238,7 @@ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); /* Link */ +void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c index b0273e7..caf35ee 100644 --- a/drivers/net/cnxk/cnxk_link.c +++ b/drivers/net/cnxk/cnxk_link.c @@ -4,6 +4,17 @@ #include "cnxk_ethdev.h" +void +cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set) +{ + if (set) + dev->flags |= CNXK_LINK_CFG_IN_PROGRESS_F; + else + dev->flags &= ~CNXK_LINK_CFG_IN_PROGRESS_F; + + rte_wmb(); +} + static inline int nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 32/62] net/cnxk: add MAC address set ops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (30 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram ` (30 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Default mac address set operation is implemented for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index beb07ae..8908016 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1034,6 +1034,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 50c75e1..a5380a5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -218,6 +218,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4a45956..87cf4ee 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -69,3 +69,32 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; return 0; } + +int +cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Update mac address at NPC */ + rc = roc_nix_npc_mac_addr_set(nix, addr->addr_bytes); + if (rc) + goto exit; + + /* Update mac address at CGX for PFs only */ + if (!roc_nix_is_vf_or_sdp(nix)) { + rc = roc_nix_mac_addr_set(nix, addr->addr_bytes); + if (rc) { + /* Rollback to previous mac address */ + roc_nix_npc_mac_addr_set(nix, dev->mac_addr); + goto exit; + } + } + + /* Update mac address to cnxk ethernet device */ + rte_memcpy(dev->mac_addr, addr->addr_bytes, RTE_ETHER_ADDR_LEN); + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 33/62] net/cnxk: add MTU set device operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (31 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram ` (29 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> This Patch implements mtu set dev op for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 51 +++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 77 ++++++++++++++++++++++++++++++++++- 7 files changed, 135 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 15911ee..d34d3fa 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Inner and Outer Checksum offload - Link state information +- MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 02be26b..6fef725 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8c63853..79cb1e2 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a1bd49b..5cc9f3f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8908016..8c5a1f6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,50 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +static void +nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + uint32_t buffsz; + + dev = rxq->dev; + eth_dev = dev->eth_dev; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->qconf.mp); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + if (eth_dev->data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + dev->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + dev->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } +} + +static int +nix_recalc_mtu(struct rte_eth_dev *eth_dev) +{ + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_eth_rxq_sp *rxq; + uint16_t mtu; + int rc; + + rxq = ((struct cnxk_eth_rxq_sp *)data->rx_queues[0]) - 1; + /* Setup scatter mode if needed by jumbo */ + nix_enable_mseg_on_jumbo(rxq); + + /* Setup MTU based on max_rx_pkt_len */ + mtu = data->dev_conf.rxmode.max_rx_pkt_len - CNXK_NIX_L2_OVERHEAD + + CNXK_NIX_MAX_VTAG_ACT_SIZE; + + rc = cnxk_nix_mtu_set(eth_dev, mtu); + if (rc) + plt_err("Failed to set default MTU size, rc=%d", rc); + + return rc; +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -990,6 +1034,12 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; + if (eth_dev->data->nb_rx_queues != 0) { + rc = nix_recalc_mtu(eth_dev); + if (rc) + return rc; + } + /* Start rx queues */ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { rc = cnxk_nix_rx_queue_start(eth_dev, i); @@ -1034,6 +1084,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mtu_set = cnxk_nix_mtu_set, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a5380a5..c216dd5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -28,7 +28,9 @@ #define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) /* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ -#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + \ + RTE_ETHER_CRC_LEN + \ + CNXK_NIX_MAX_VTAG_ACT_SIZE) #define CNXK_NIX_RX_MIN_DESC 16 #define CNXK_NIX_RX_MIN_DESC_ALIGN 16 @@ -218,6 +220,7 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 87cf4ee..21b55c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -20,7 +20,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; - devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->max_mtu = devinfo->max_rx_pktlen - + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN); devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; devinfo->rx_offload_capa = dev->rx_offload_capa; @@ -98,3 +99,77 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) exit: return rc; } + +int +cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) +{ + uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + uint32_t buffsz; + + /* Check if MTU is within the allowed range */ + if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { + plt_err("MTU is lesser than minimum"); + goto exit; + } + + if ((frame_size - RTE_ETHER_CRC_LEN) > + ((uint32_t)roc_nix_max_pkt_len(nix))) { + plt_err("MTU is greater than maximum"); + goto exit; + } + + buffsz = data->min_rx_buf_size - RTE_PKTMBUF_HEADROOM; + old_frame_size = data->mtu + CNXK_NIX_L2_OVERHEAD; + + /* Refuse MTU that requires the support of scattered packets + * when this feature has not been enabled before. + */ + if (data->dev_started && frame_size > buffsz && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER)) { + plt_err("Scatter offload is not enabled for mtu"); + goto exit; + } + + /* Check <seg size> * <max_seg> >= max_frame */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) && + frame_size > (buffsz * CNXK_NIX_RX_NB_SEG_MAX)) { + plt_err("Greater than maximum supported packet length"); + goto exit; + } + + frame_size -= RTE_ETHER_CRC_LEN; + + /* Update mtu on Tx */ + rc = roc_nix_mac_mtu_set(nix, frame_size); + if (rc) { + plt_err("Failed to set MTU, rc=%d", rc); + goto exit; + } + + /* Sync same frame size on Rx */ + rc = roc_nix_mac_max_rx_len_set(nix, frame_size); + if (rc) { + /* Rollback to older mtu */ + roc_nix_mac_mtu_set(nix, + old_frame_size - RTE_ETHER_CRC_LEN); + plt_err("Failed to max Rx frame length, rc=%d", rc); + goto exit; + } + + frame_size += RTE_ETHER_CRC_LEN; + + if (frame_size > RTE_ETHER_MAX_LEN) + dev->rx_offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; + else + dev->rx_offloads &= ~DEV_RX_OFFLOAD_JUMBO_FRAME; + + /* Update max_rx_pkt_len */ + data->dev_conf.rxmode.max_rx_pkt_len = frame_size; + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 34/62] net/cnxk: add promiscuous mode enable and disable 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (32 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram ` (28 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Add device operations to enable and disable promisc mode for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 56 +++++++++++++++++++++++++++++++++++ 6 files changed, 63 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index d34d3fa..87ceda5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,6 +17,7 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Promiscuous mode - Jumbo frames - SR-IOV VF - Lock-free Tx queue diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 6fef725..9b2e163 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y TSO = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 79cb1e2..31471e0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -16,6 +16,7 @@ Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 8c5a1f6..d3dbe54 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1095,6 +1095,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, + .promiscuous_enable = cnxk_nix_promisc_enable, + .promiscuous_disable = cnxk_nix_promisc_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c216dd5..2fce20e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -223,6 +223,8 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); +int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 21b55c4..6feb3a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -173,3 +173,59 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) exit: return rc; } + +int +cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, false); + return rc; + } + + return 0; +} + +int +cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, false); + if (rc < 0) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, false); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, true); + return rc; + } + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 35/62] net/cnxk: add DMAC filter support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (33 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram ` (27 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> DMAC filter support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 44 ++++++++++++++++++++++++++++++++--- 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 87ceda5..af7141f 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -23,6 +23,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- MAC filtering - Inner and Outer Checksum offload - Link state information - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9b2e163..20d4d12 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 31471e0..e1de8ab 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index d3dbe54..a813c08 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1085,6 +1085,8 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, + .mac_addr_add = cnxk_nix_mac_addr_add, + .mac_addr_remove = cnxk_nix_mac_addr_del, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 2fce20e..ab94b99 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -139,6 +139,7 @@ struct cnxk_eth_dev { /* Max macfilter entries */ uint8_t max_mac_entries; + bool dmac_filter_enable; uint16_t flags; uint8_t ptype_disable; @@ -221,6 +222,10 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr, uint32_t index, + uint32_t pool); +void cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6feb3a9..fc60576 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -101,6 +101,43 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) } int +cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, + uint32_t index, uint32_t pool) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + PLT_SET_USED(index); + PLT_SET_USED(pool); + + rc = roc_nix_mac_addr_add(nix, addr->addr_bytes); + if (rc < 0) { + plt_err("Failed to add mac address, rc=%d", rc); + return rc; + } + + /* Enable promiscuous mode at NIX level */ + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} + +void +cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_mac_addr_del(nix, index); + if (rc) + plt_err("Failed to delete mac address, rc=%d", rc); +} + +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) { uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; @@ -212,8 +249,8 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (roc_nix_is_vf_or_sdp(nix)) return rc; - rc = roc_nix_npc_promisc_ena_dis(nix, false); - if (rc < 0) { + rc = roc_nix_npc_promisc_ena_dis(nix, dev->dmac_filter_enable); + if (rc) { plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, roc_error_msg_get(rc)); return rc; @@ -223,9 +260,10 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (rc) { plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, roc_error_msg_get(rc)); - roc_nix_npc_promisc_ena_dis(nix, true); + roc_nix_npc_promisc_ena_dis(nix, !dev->dmac_filter_enable); return rc; } + dev->dmac_filter_enable = false; return 0; } -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 36/62] net/cnxk: add all multicast enable/disable ethops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (34 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram ` (26 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> L2 multicast packets can be allowed or blocked. Patch implements corresponding ethops. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 17 +++++++++++++++++ 5 files changed, 23 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 20d4d12..b41af2d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e1de8ab..7fe8018 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a813c08..f3d5a9d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1099,6 +1099,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, .promiscuous_enable = cnxk_nix_promisc_enable, .promiscuous_disable = cnxk_nix_promisc_disable, + .allmulticast_enable = cnxk_nix_allmulticast_enable, + .allmulticast_disable = cnxk_nix_allmulticast_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ab94b99..70bc374 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -230,6 +230,8 @@ int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index fc60576..61ecbab 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -267,3 +267,20 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) dev->dmac_filter_enable = false; return 0; } + +int +cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, true, false); +} + +int +cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, false, + eth_dev->data->promiscuous); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 37/62] net/cnxk: add Rx/Tx burst mode get ops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (35 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram ` (25 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements ethdev operations to get Rx and Tx burst mode. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 127 ++++++++++++++++++++++++++++++++++ 6 files changed, 136 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b41af2d..298f167 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 7fe8018..a673cc1 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5cc9f3f..335d082 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f3d5a9d..4ec0dfb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1101,6 +1101,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .promiscuous_disable = cnxk_nix_promisc_disable, .allmulticast_enable = cnxk_nix_allmulticast_enable, .allmulticast_disable = cnxk_nix_allmulticast_disable, + .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, + .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 70bc374..aea0005 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -234,6 +234,10 @@ int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); +int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 61ecbab..7ae961a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -72,6 +72,133 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) } int +cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } rx_offload_map[] = { + {DEV_RX_OFFLOAD_VLAN_STRIP, " VLAN Strip,"}, + {DEV_RX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_LRO, " TCP LRO,"}, + {DEV_RX_OFFLOAD_QINQ_STRIP, " QinQ VLAN Strip,"}, + {DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_MACSEC_STRIP, " MACsec Strip,"}, + {DEV_RX_OFFLOAD_HEADER_SPLIT, " Header Split,"}, + {DEV_RX_OFFLOAD_VLAN_FILTER, " VLAN Filter,"}, + {DEV_RX_OFFLOAD_VLAN_EXTEND, " VLAN Extend,"}, + {DEV_RX_OFFLOAD_JUMBO_FRAME, " Jumbo Frame,"}, + {DEV_RX_OFFLOAD_SCATTER, " Scattered,"}, + {DEV_RX_OFFLOAD_TIMESTAMP, " Timestamp,"}, + {DEV_RX_OFFLOAD_SECURITY, " Security,"}, + {DEV_RX_OFFLOAD_KEEP_CRC, " Keep CRC,"}, + {DEV_RX_OFFLOAD_SCTP_CKSUM, " SCTP,"}, + {DEV_RX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_RX_OFFLOAD_RSS_HASH, " RSS,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Rx Offloads:", + "Scalar, Rx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Rx offload info */ + for (i = 0; i < RTE_DIM(rx_offload_map); i++) { + if (dev->rx_offloads & rx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + rx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int +cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } tx_offload_map[] = { + {DEV_TX_OFFLOAD_VLAN_INSERT, " VLAN Insert,"}, + {DEV_TX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_TX_OFFLOAD_SCTP_CKSUM, " SCTP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_TSO, " TCP TSO,"}, + {DEV_TX_OFFLOAD_UDP_TSO, " UDP TSO,"}, + {DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_QINQ_INSERT, " QinQ VLAN Insert,"}, + {DEV_TX_OFFLOAD_VXLAN_TNL_TSO, " VXLAN Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GRE_TNL_TSO, " GRE Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IPIP_TNL_TSO, " IP-in-IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GENEVE_TNL_TSO, " Geneve Tunnel TSO,"}, + {DEV_TX_OFFLOAD_MACSEC_INSERT, " MACsec Insert,"}, + {DEV_TX_OFFLOAD_MT_LOCKFREE, " Multi Thread Lockless Tx,"}, + {DEV_TX_OFFLOAD_MULTI_SEGS, " Scattered,"}, + {DEV_TX_OFFLOAD_MBUF_FAST_FREE, " H/W MBUF Free,"}, + {DEV_TX_OFFLOAD_SECURITY, " Security,"}, + {DEV_TX_OFFLOAD_UDP_TNL_TSO, " UDP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IP_TNL_TSO, " IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP, " Timestamp,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Tx Offloads:", + "Scalar, Tx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Tx offload info */ + for (i = 0; i < RTE_DIM(tx_offload_map); i++) { + if (dev->tx_offloads & tx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + tx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 38/62] net/cnxk: add flow ctrl set/get ops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (36 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram ` (24 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements set and get operations for flow control. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 74 +++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 101 ++++++++++++++++++++++++++++++++++ 6 files changed, 191 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index af7141f..88ca313 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -26,6 +26,7 @@ Features of the CNXK Ethdev PMD are: - MAC filtering - Inner and Outer Checksum offload - Link state information +- Link flow control - MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 298f167..afd0f01 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index a673cc1..4bd11ce 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 4ec0dfb..4fe9c1c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -81,6 +81,55 @@ nix_recalc_mtu(struct rte_eth_dev *eth_dev) return rc; } +static int +nix_init_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_conf = {0}; + int rc; + + /* Both Rx & Tx flow ctrl get enabled(RTE_FC_FULL) in HW + * by AF driver, update those info in PMD structure. + */ + rc = cnxk_nix_flow_ctrl_get(eth_dev, &fc_conf); + if (rc) + goto exit; + + fc->mode = fc_conf.mode; + fc->rx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_RX_PAUSE); + fc->tx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_TX_PAUSE); + +exit: + return rc; +} + +static int +nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_cfg = {0}; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + return 0; + + fc_cfg.mode = fc->mode; + + /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ + if (roc_model_is_cn96_Ax() && + (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { + fc_cfg.mode = + (fc_cfg.mode == RTE_FC_FULL || + fc_cfg.mode == RTE_FC_TX_PAUSE) ? + RTE_FC_TX_PAUSE : RTE_FC_NONE; + } + + return cnxk_nix_flow_ctrl_set(eth_dev, &fc_cfg); +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -674,6 +723,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct rte_eth_rxmode *rxmode = &conf->rxmode; struct rte_eth_txmode *txmode = &conf->txmode; char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix_fc_cfg fc_cfg = {0}; struct roc_nix *nix = &dev->nix; struct rte_ether_addr *ea; uint8_t nb_rxq, nb_txq; @@ -855,6 +905,21 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Init flow control configuration */ + fc_cfg.cq_cfg_valid = false; + fc_cfg.rxchan_cfg.enable = true; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } + + /* Update flow control configuration to PMD */ + rc = nix_init_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } /* * Restore queue config when reconfigure followed by * reconfigure and no queue configure invoked from application case. @@ -1054,6 +1119,13 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } + /* Update Flow control configuration */ + rc = nix_update_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to enable flow control. error code(%d)", rc); + return rc; + } + /* Enable Rx in NPC */ rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); if (rc) { @@ -1103,6 +1175,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .allmulticast_disable = cnxk_nix_allmulticast_disable, .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, + .flow_ctrl_get = cnxk_nix_flow_ctrl_get, + .flow_ctrl_set = cnxk_nix_flow_ctrl_set, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index aea0005..e788a42 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -113,6 +113,12 @@ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) +struct cnxk_fc_cfg { + enum rte_eth_fc_mode mode; + uint8_t rx_pause; + uint8_t tx_pause; +}; + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -174,6 +180,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Flow control configuration */ + struct cnxk_fc_cfg fc_cfg; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -238,6 +247,10 @@ int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); +int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7ae961a..420c928 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -199,6 +199,107 @@ cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, } int +cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum rte_eth_fc_mode mode_map[] = { + RTE_FC_NONE, RTE_FC_RX_PAUSE, + RTE_FC_TX_PAUSE, RTE_FC_FULL + }; + struct roc_nix *nix = &dev->nix; + int mode; + + mode = roc_nix_fc_mode_get(nix); + if (mode < 0) + return mode; + + memset(fc_conf, 0, sizeof(struct rte_eth_fc_conf)); + fc_conf->mode = mode_map[mode]; + return 0; +} + +static int +nix_fc_cq_config_set(struct cnxk_eth_dev *dev, uint16_t qid, bool enable) +{ + struct roc_nix *nix = &dev->nix; + struct roc_nix_fc_cfg fc_cfg; + struct roc_nix_cq *cq; + + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + cq = &dev->cqs[qid]; + fc_cfg.cq_cfg_valid = true; + fc_cfg.cq_cfg.enable = enable; + fc_cfg.cq_cfg.rq = qid; + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; + + return roc_nix_fc_config_set(nix, &fc_cfg); +} + +int +cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum roc_nix_fc_mode mode_map[] = { + ROC_NIX_FC_NONE, ROC_NIX_FC_RX, + ROC_NIX_FC_TX, ROC_NIX_FC_FULL + }; + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct roc_nix *nix = &dev->nix; + uint8_t rx_pause, tx_pause; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) { + plt_err("Flow control configuration is not allowed on VFs"); + return -ENOTSUP; + } + + if (fc_conf->high_water || fc_conf->low_water || fc_conf->pause_time || + fc_conf->mac_ctrl_frame_fwd || fc_conf->autoneg) { + plt_info("Only MODE configuration is supported"); + return -EINVAL; + } + + if (fc_conf->mode == fc->mode) + return 0; + + rx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_RX_PAUSE); + tx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_TX_PAUSE); + + /* Check if TX pause frame is already enabled or not */ + if (fc->tx_pause ^ tx_pause) { + if (roc_model_is_cn96_Ax() && data->dev_started) { + /* On Ax, CQ should be in disabled state + * while setting flow control configuration. + */ + plt_info("Stop the port=%d for setting flow control", + data->port_id); + return 0; + } + + for (i = 0; i < data->nb_rx_queues; i++) { + rc = nix_fc_cq_config_set(dev, i, tx_pause); + if (rc) + return rc; + } + } + + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); + if (rc) + return rc; + + fc->rx_pause = rx_pause; + fc->tx_pause = tx_pause; + fc->mode = fc_conf->mode; + + return rc; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 39/62] net/cnxk: add link up/down operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (37 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram ` (23 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements link up/down ethdev operations for cn9k and cn10k platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 4 +++- drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 47 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 4fe9c1c..c6072c1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -963,7 +963,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } -static int +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -1177,6 +1177,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, .flow_ctrl_get = cnxk_nix_flow_ctrl_get, .flow_ctrl_set = cnxk_nix_flow_ctrl_set, + .dev_set_link_up = cnxk_nix_set_link_up, + .dev_set_link_down = cnxk_nix_set_link_down, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e788a42..5e982f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -251,6 +251,9 @@ int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); +int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); + int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, @@ -259,6 +262,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 420c928..b58d21e 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -512,3 +512,50 @@ cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) return roc_nix_npc_mcast_config(&dev->nix, false, eth_dev->data->promiscuous); } + +int +cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + rc = roc_nix_mac_link_state_set(nix, true); + if (rc) + goto exit; + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + goto exit; + } + +exit: + return rc; +} + +int +cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_stop(eth_dev, i); + if (rc) + goto exit; + } + + rc = roc_nix_mac_link_state_set(nix, false); +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 40/62] net/cnxk: add EEPROM module info get operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (38 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram ` (22 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements eeprom module info get ethops for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 39 +++++++++++++++++++++++++++++++++++ 6 files changed, 48 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index afd0f01..b1e8641 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -31,6 +31,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4bd11ce..0f99634 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -29,6 +29,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 335d082..cecced9 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -26,6 +26,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index c6072c1..2e34867 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1179,6 +1179,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .flow_ctrl_set = cnxk_nix_flow_ctrl_set, .dev_set_link_up = cnxk_nix_set_link_up, .dev_set_link_down = cnxk_nix_set_link_down, + .get_module_info = cnxk_nix_get_module_info, + .get_module_eeprom = cnxk_nix_get_module_eeprom, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5e982f9..083af29 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -253,6 +253,10 @@ int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); +int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo); +int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index b58d21e..6e83a63 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -559,3 +559,42 @@ cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) exit: return rc; } + +int +cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + modinfo->type = eeprom_info.sff_id; + modinfo->eeprom_len = ROC_NIX_EEPROM_SIZE; + return 0; +} + +int +cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + + if (!info->data || !info->length || + (info->offset + info->length > ROC_NIX_EEPROM_SIZE)) + return rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 41/62] net/cnxk: add Rx queue interrupt enable/disable ops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (39 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram ` (21 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Application may choose to enable/disable interrupts on Rx queues so that application can select its processing if no packets are available on queues for a longer period. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 7 files changed, 29 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 88ca313..6678486 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -30,6 +30,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Support Rx interrupt Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b1e8641..e5669f5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 0f99634..dff0c9b 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index cecced9..b950d2f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 2e34867..fd8d1fb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1181,6 +1181,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_set_link_down = cnxk_nix_set_link_down, .get_module_info = cnxk_nix_get_module_info, .get_module_eeprom = cnxk_nix_get_module_eeprom, + .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, + .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 083af29..a01b72a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -257,6 +257,10 @@ int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, struct rte_eth_dev_module_info *modinfo); int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, struct rte_dev_eeprom_info *info); +int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); +int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6e83a63..6399090 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -598,3 +598,22 @@ cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); return 0; } + +int +cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_enable(&dev->nix, rx_queue_id); + return 0; +} + +int +cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 42/62] net/cnxk: add validation API for mempool ops 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (40 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram ` (20 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> cn9k and cn10k supports platform specific mempool ops. This patch implements API to validate whether given mempool ops is supported or not. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 11 +++++++++++ 3 files changed, 13 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index fd8d1fb..627ac32 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1183,6 +1183,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_module_eeprom = cnxk_nix_get_module_eeprom, .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, + .pool_ops_supported = cnxk_nix_pool_ops_supported, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a01b72a..e6dac95 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -261,6 +261,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); +int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6399090..ddfaffa 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -617,3 +617,14 @@ cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); return 0; } + +int +cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) +{ + RTE_SET_USED(eth_dev); + + if (!strcmp(pool, rte_mbuf_platform_mempool_ops())) + return 0; + + return -ENOTSUP; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 43/62] net/cnxk: add port/queue stats 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (41 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 44/62] net/cnxk: add xstats apis Nithin Dabilpuram ` (19 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch implements regular port statistics and queue mapping set api to get queue statistics Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 8 ++++ drivers/net/cnxk/cnxk_stats.c | 85 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_stats.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6678486..e875e55 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -25,6 +25,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - MAC filtering - Inner and Outer Checksum offload +- Port hardware statistics - Link state information - Link flow control - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e5669f5..40952a9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index dff0c9b..32035bb 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -30,6 +30,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b950d2f..8060a68 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -27,6 +27,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 627ac32..ecc7f13 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1184,6 +1184,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, .pool_ops_supported = cnxk_nix_pool_ops_supported, + .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, + .stats_get = cnxk_nix_stats_get, + .stats_reset = cnxk_nix_stats_reset, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e6dac95..860cfe1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -191,6 +191,10 @@ struct cnxk_eth_dev { /* LSO Tunnel format indices */ uint64_t lso_tun_fmt; + + /* Per queue statistics counters */ + uint32_t txq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; + uint32_t rxq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; struct cnxk_eth_rxq_sp { @@ -286,6 +290,10 @@ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx); +int cnxk_nix_stats_reset(struct rte_eth_dev *dev); +int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c new file mode 100644 index 0000000..24bff0b --- /dev/null +++ b/drivers/net/cnxk/cnxk_stats.c @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +int +cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct roc_nix_stats nix_stats; + int rc = 0, i; + + rc = roc_nix_stats_get(nix, &nix_stats); + if (rc) + goto exit; + + stats->opackets = nix_stats.tx_ucast; + stats->opackets += nix_stats.tx_mcast; + stats->opackets += nix_stats.tx_bcast; + stats->oerrors = nix_stats.tx_drop; + stats->obytes = nix_stats.tx_octs; + + stats->ipackets = nix_stats.rx_ucast; + stats->ipackets += nix_stats.rx_mcast; + stats->ipackets += nix_stats.rx_bcast; + stats->imissed = nix_stats.rx_drop; + stats->ibytes = nix_stats.rx_octs; + stats->ierrors = nix_stats.rx_err; + + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) { + struct roc_nix_stats_queue qstats; + uint16_t qidx; + + if (dev->txq_stat_map[i] & (1U << 31)) { + qidx = dev->txq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 0, &qstats); + if (rc) + goto exit; + stats->q_opackets[i] = qstats.tx_pkts; + stats->q_obytes[i] = qstats.tx_octs; + stats->q_errors[i] = qstats.tx_drop_pkts; + } + + if (dev->rxq_stat_map[i] & (1U << 31)) { + qidx = dev->rxq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 1, &qstats); + if (rc) + goto exit; + stats->q_ipackets[i] = qstats.rx_pkts; + stats->q_ibytes[i] = qstats.rx_octs; + stats->q_errors[i] += qstats.rx_drop_pkts; + } + } +exit: + return rc; +} + +int +cnxk_nix_stats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_stats_reset(&dev->nix); +} + +int +cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (is_rx) { + if (queue_id >= dev->nb_rxq) + return -EINVAL; + dev->rxq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } else { + if (queue_id >= dev->nb_txq) + return -EINVAL; + dev->txq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } + + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 8f32dc7..7d711a2 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -12,7 +12,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', - 'cnxk_lookup.c') + 'cnxk_lookup.c', + 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 44/62] net/cnxk: add xstats apis 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (42 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram ` (18 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Initial implementation of xstats operations. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 5 + drivers/net/cnxk/cnxk_ethdev.h | 11 ++ drivers/net/cnxk/cnxk_stats.c | 235 ++++++++++++++++++++++++++++++++++ 6 files changed, 254 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 40952a9..192c15a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -34,6 +34,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 32035bb..e990480 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -32,6 +32,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8060a68..3a4417c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -29,6 +29,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ecc7f13..27b92f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1187,6 +1187,11 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, .stats_get = cnxk_nix_stats_get, .stats_reset = cnxk_nix_stats_reset, + .xstats_get = cnxk_nix_xstats_get, + .xstats_get_names = cnxk_nix_xstats_get_names, + .xstats_reset = cnxk_nix_xstats_reset, + .xstats_get_by_id = cnxk_nix_xstats_get_by_id, + .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 860cfe1..5a0dc13 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -294,6 +294,17 @@ int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx); int cnxk_nix_stats_reset(struct rte_eth_dev *dev); int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); +int cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat *xstats, unsigned int n); +int cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit); +int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit); +int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n); +int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c index 24bff0b..19bab21 100644 --- a/drivers/net/cnxk/cnxk_stats.c +++ b/drivers/net/cnxk/cnxk_stats.c @@ -4,6 +4,9 @@ #include "cnxk_ethdev.h" +#define CNXK_NB_RXQ_STATS 5 +#define CNXK_NB_TXQ_STATS 4 + int cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) { @@ -83,3 +86,235 @@ cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, return 0; } + +int +cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat roc_xstats[n]; + struct roc_nix *nix = &dev->nix; + int roc_size, q, idx = 0, size; + + roc_size = roc_nix_xstats_get(nix, roc_xstats, n); + + if (roc_size < 0) + return roc_size; + + /* Per Queue statistics also returned as part of xstats */ + size = roc_size + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + + /* If requested array do not have space then return with count */ + if (size > (int)n || xstats == NULL) + return size; + + for (idx = 0; idx < roc_size; idx++) { + xstats[idx].id = roc_xstats[idx].id; + xstats[idx].value = roc_xstats[idx].value; + } + for (q = 0; q < dev->nb_rxq; q++) { + struct roc_nix_stats_queue qstats; + int rc; + + rc = roc_nix_stats_queue_get(nix, q, 1, &qstats); + if (rc) + return rc; + + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_drop_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_drop_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_error_pkts; + idx++; + } + for (q = 0; q < dev->nb_txq; q++) { + struct roc_nix_stats_queue qstats; + int rc; + + rc = roc_nix_stats_queue_get(nix, q, 0, &qstats); + if (rc) + return rc; + + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_drop_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_drop_octs; + idx++; + } + + return size; +} + +int +cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat_name roc_xstats_name[limit]; + struct roc_nix *nix = &dev->nix; + int roc_size, size, i, q; + + roc_size = roc_nix_num_xstats_get(nix); + /* Per Queue statistics also returned as part of xstats */ + size = roc_size + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + + if (xstats_names == NULL) + return size; + + if ((int)limit < size && xstats_names != NULL) + return size; + + roc_size = roc_nix_xstats_names_get(nix, roc_xstats_name, limit); + + for (i = 0; i < roc_size; i++) + rte_strscpy(xstats_names[i].name, roc_xstats_name[i].name, + sizeof(xstats_names[i].name)); + + for (q = 0; q < dev->nb_rxq; q++) { + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_drop_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_drop_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_err_pkts", q); + i++; + } + + for (q = 0; q < dev->nb_txq; q++) { + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_drop_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_drop_octs", q); + i++; + } + + return size; +} + +int +cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t nix_cnt = roc_nix_num_xstats_get(&dev->nix); + uint32_t stat_cnt = nix_cnt + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + struct rte_eth_xstat_name xnames[stat_cnt]; + uint32_t i; + + if (limit < stat_cnt && ids == NULL) + return stat_cnt; + + if (limit > stat_cnt) + return -EINVAL; + + if (xstats_names == NULL) + return -ENOMEM; + + cnxk_nix_xstats_get_names(eth_dev, xnames, stat_cnt); + + for (i = 0; i < limit; i++) { + if (ids[i] >= stat_cnt) + return -EINVAL; + + rte_strscpy(xstats_names[i].name, xnames[ids[i]].name, + sizeof(xstats_names[i].name)); + } + + return limit; +} + +int +cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t nix_cnt = roc_nix_num_xstats_get(&dev->nix); + uint32_t stat_cnt = nix_cnt + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + struct rte_eth_xstat xstats[stat_cnt]; + uint32_t i; + + if (n < stat_cnt && ids == NULL) + return stat_cnt; + + if (n > stat_cnt) + return -EINVAL; + + if (values == NULL) + return -ENOMEM; + + cnxk_nix_xstats_get(eth_dev, xstats, stat_cnt); + + for (i = 0; i < n; i++) { + if (ids[i] >= stat_cnt) + return -EINVAL; + values[i] = xstats[ids[i]].value; + } + + return n; +} + +int +cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0, i; + + rc = roc_nix_stats_reset(nix); + if (rc) + goto exit; + + /* Reset Rx Queues */ + for (i = 0; i < dev->nb_rxq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 1); + if (rc) + goto exit; + } + + /* Reset Tx Queues */ + for (i = 0; i < dev->nb_txq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 0); + if (rc) + goto exit; + } + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 45/62] net/cnxk: add rxq/txq info get operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (43 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 44/62] net/cnxk: add xstats apis Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram ` (17 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Initial apis to get default queue information. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 27b92f9..288e069 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1192,6 +1192,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .rxq_info_get = cnxk_nix_rxq_info_get, + .txq_info_get = cnxk_nix_txq_info_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5a0dc13..32977eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -305,6 +305,10 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo); +void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index ddfaffa..55c26c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -628,3 +628,33 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } + +void +cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo) +{ + void *rxq = eth_dev->data->rx_queues[qid]; + struct cnxk_eth_rxq_sp *rxq_sp = cnxk_eth_rxq_to_sp(rxq); + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->mp = rxq_sp->qconf.mp; + qinfo->scattered_rx = eth_dev->data->scattered_rx; + qinfo->nb_desc = rxq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &rxq_sp->qconf.conf.rx, sizeof(qinfo->conf)); +} + +void +cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo) +{ + void *txq = eth_dev->data->tx_queues[qid]; + struct cnxk_eth_txq_sp *txq_sp = cnxk_eth_txq_to_sp(txq); + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->nb_desc = txq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 46/62] net/cnxk: add device close and reset operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (44 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram ` (16 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements device close and reset operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 288e069..3929e7e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1154,6 +1154,9 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } +static int cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev); +static int cnxk_nix_dev_close(struct rte_eth_dev *eth_dev); + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, @@ -1165,6 +1168,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, .dev_stop = cnxk_nix_dev_stop, + .dev_close = cnxk_nix_dev_close, + .dev_reset = cnxk_nix_dev_reset, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, @@ -1304,7 +1309,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } static int -cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; @@ -1358,14 +1363,11 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* Check if mbox close is needed */ - if (!mbox_close) - return 0; - rc = roc_nix_dev_fini(nix); /* Can be freed later by PMD if NPA LF is in use */ if (rc == -EAGAIN) { - eth_dev->data->dev_private = NULL; + if (!reset) + eth_dev->data->dev_private = NULL; return 0; } else if (rc) { plt_err("Failed in nix dev fini, rc=%d", rc); @@ -1374,6 +1376,25 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) return rc; } +static int +cnxk_nix_dev_close(struct rte_eth_dev *eth_dev) +{ + cnxk_eth_dev_uninit(eth_dev, false); + return 0; +} + +static int +cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev) +{ + int rc; + + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + return cnxk_eth_dev_init(eth_dev); +} + int cnxk_nix_remove(struct rte_pci_device *pci_dev) { @@ -1384,7 +1405,7 @@ cnxk_nix_remove(struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (eth_dev) { /* Cleanup eth dev */ - rc = cnxk_eth_dev_uninit(eth_dev, true); + rc = cnxk_eth_dev_uninit(eth_dev, false); if (rc) return rc; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 47/62] net/cnxk: add pending Tx mbuf cleanup operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (45 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram ` (15 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are accumalated as a pending mbuf. Hence operation is NOP for cnxk platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 3 files changed, 12 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3929e7e..f2df0ba 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1199,6 +1199,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, + .tx_done_cleanup = cnxk_nix_tx_done_cleanup, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 32977eb..1c4403c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -266,6 +266,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); +int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 55c26c4..ce94eb6 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -658,3 +658,13 @@ cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); } + +/* It is a NOP for cnxk as HW frees the buffer on xmit */ +int +cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) +{ + RTE_SET_USED(txq); + RTE_SET_USED(free_cnt); + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 48/62] net/cnxk: add support to configure npc 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (46 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram ` (14 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Kiran Kumar K <kirankumark@marvell.com> Adding support to configure NPC on device initialization. This involves reading the MKEX and initializing the necessary data. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 25 ++++++++++++++++++++++--- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 3 +++ 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f2df0ba..d1a7f2e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -8,7 +8,8 @@ nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) { uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; - if (roc_nix_is_vf_or_sdp(&dev->nix)) + if (roc_nix_is_vf_or_sdp(&dev->nix) || + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; return capa; @@ -120,6 +121,7 @@ nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ if (roc_model_is_cn96_Ax() && + dev->npc.switch_header_type != ROC_PRIV_FLAGS_HIGIG && (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { fc_cfg.mode = (fc_cfg.mode == RTE_FC_FULL || @@ -419,8 +421,10 @@ cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, dev->ethdev_rss_hf = ethdev_rss; - if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + if (ethdev_rss & ETH_RSS_L2_PAYLOAD && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_LEN_90B) { flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + } if (ethdev_rss & ETH_RSS_C_VLAN) flowkey_cfg |= FLOW_KEY_TYPE_VLAN; @@ -837,11 +841,18 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) roc_nix_err_intr_ena_dis(nix, true); roc_nix_ras_intr_ena_dis(nix, true); - if (nix->rx_ptp_ena) { + if (nix->rx_ptp_ena && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { plt_err("Both PTP and switch header enabled"); goto free_nix_lf; } + rc = roc_nix_switch_hdr_set(nix, dev->npc.switch_header_type); + if (rc) { + plt_err("Failed to enable switch type nix_lf rc=%d", rc); + goto free_nix_lf; + } + /* Setup LSO if needed */ rc = nix_lso_fmt_setup(dev); if (rc) { @@ -1293,6 +1304,11 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->speed_capa = nix_get_speed_capa(dev); /* Initialize roc npc */ + dev->npc.roc_nix = nix; + rc = roc_npc_init(&dev->npc); + if (rc) + goto free_mac_addrs; + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, eth_dev->data->port_id, roc_nix_get_pf(nix), @@ -1326,6 +1342,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable and free rte_flow entries */ + roc_npc_fini(&dev->npc); + /* Disable link status events */ roc_nix_mac_link_event_start_stop(nix, false); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1c4403c..4879ef9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -133,6 +133,9 @@ struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC NPC */ + struct roc_npc npc; + /* ROC RQs, SQs and CQs */ struct roc_nix_rq *rqs; struct roc_nix_sq *sqs; diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 4af2803..7fd06eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -150,6 +150,9 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->npc.flow_prealloc_size = flow_prealloc_size; + dev->npc.flow_max_priority = flow_max_priority; + dev->npc.switch_header_type = switch_header_type; return 0; exit: -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 49/62] net/cnxk: add initial version of rte flow support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (47 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram ` (13 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Kiran Kumar K <kirankumark@marvell.com> Adding initial version of rte_flow support for cnxk family device. Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_flush, flow_query, flow_isolate. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- doc/guides/nics/cnxk.rst | 7 + doc/guides/nics/features/cnxk.ini | 36 +++++ drivers/net/cnxk/cn10k_ethdev.c | 16 ++ drivers/net/cnxk/cn10k_rte_flow.c | 30 ++++ drivers/net/cnxk/cn10k_rte_flow.h | 17 ++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++ drivers/net/cnxk/cn9k_rte_flow.c | 30 ++++ drivers/net/cnxk/cn9k_rte_flow.h | 17 ++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/cnxk_rte_flow.c | 330 ++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 27 ++++ drivers/net/cnxk/meson.build | 3 + 12 files changed, 532 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_rte_flow.c create mode 100644 drivers/net/cnxk/cn10k_rte_flow.h create mode 100644 drivers/net/cnxk/cn9k_rte_flow.c create mode 100644 drivers/net/cnxk/cn9k_rte_flow.h create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index e875e55..cc71b22 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - MAC filtering +- Generic flow API - Inner and Outer Checksum offload - Port hardware statistics - Link state information @@ -208,6 +209,12 @@ CRC stripping The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by the host interface irrespective of the offload configuration. +RTE Flow GRE support +~~~~~~~~~~~~~~~~~~~~ + +- ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing + bits in the GRE header are equal to 0. + Debugging Options ----------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 192c15a..e2da5dd 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -39,3 +39,39 @@ Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y + +[rte_flow items] +any = Y +arp_eth_ipv4 = Y +esp = Y +eth = Y +e_tag = Y +geneve = Y +gre = Y +gre_key = Y +gtpc = Y +gtpu = Y +higig2 = Y +icmp = Y +ipv4 = Y +ipv6 = Y +ipv6_ext = Y +mpls = Y +nvgre = Y +raw = Y +sctp = Y +tcp = Y +udp = Y +vlan = Y +vxlan = Y +vxlan_gpe = Y + +[rte_flow actions] +count = Y +drop = Y +flag = Y +pf = Y +port_id = Y +queue = Y +security = Y +vf = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 5ff36bb..0396ff6 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rte_flow.h" #include "cn10k_rx.h" #include "cn10k_tx.h" @@ -308,6 +309,20 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } +static void +npc_flow_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_flow_ops.create = cn10k_flow_create; + cnxk_flow_ops.destroy = cn10k_flow_destroy; +} + static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { @@ -332,6 +347,7 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } nix_eth_dev_ops_override(); + npc_flow_ops_override(); /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); diff --git a/drivers/net/cnxk/cn10k_rte_flow.c b/drivers/net/cnxk/cn10k_rte_flow.c new file mode 100644 index 0000000..65893cc --- /dev/null +++ b/drivers/net/cnxk/cn10k_rte_flow.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#include <cnxk_rte_flow.h> +#include "cn10k_rte_flow.h" +#include "cn10k_ethdev.h" + +struct rte_flow * +cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow; + + flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); + if (!flow) + return NULL; + + return (struct rte_flow *)flow; +} + +int +cn10k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + + return cnxk_flow_destroy(eth_dev, flow, error); +} diff --git a/drivers/net/cnxk/cn10k_rte_flow.h b/drivers/net/cnxk/cn10k_rte_flow.h new file mode 100644 index 0000000..f64fcf2 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rte_flow.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#ifndef __CN10K_RTE_FLOW_H__ +#define __CN10K_RTE_FLOW_H__ + +#include <rte_flow_driver.h> + +struct rte_flow *cn10k_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cn10k_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error); + +#endif /* __CN10K_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 847d24a..5cc0db9 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rte_flow.h" #include "cn9k_rx.h" #include "cn9k_tx.h" @@ -317,6 +318,20 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } +static void +npc_flow_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_flow_ops.create = cn9k_flow_create; + cnxk_flow_ops.destroy = cn9k_flow_destroy; +} + static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { @@ -342,6 +357,7 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } nix_eth_dev_ops_override(); + npc_flow_ops_override(); /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); diff --git a/drivers/net/cnxk/cn9k_rte_flow.c b/drivers/net/cnxk/cn9k_rte_flow.c new file mode 100644 index 0000000..24e1b92 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rte_flow.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#include <cnxk_rte_flow.h> +#include "cn9k_ethdev.h" +#include "cn9k_rte_flow.h" + +struct rte_flow * +cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow; + + flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); + if (!flow) + return NULL; + + return (struct rte_flow *)flow; +} + +int +cn9k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + + return cnxk_flow_destroy(eth_dev, flow, error); +} diff --git a/drivers/net/cnxk/cn9k_rte_flow.h b/drivers/net/cnxk/cn9k_rte_flow.h new file mode 100644 index 0000000..43d59e1 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rte_flow.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#ifndef __CN9K_RTE_FLOW_H__ +#define __CN9K_RTE_FLOW_H__ + +#include <rte_flow_driver.h> + +struct rte_flow *cn9k_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cn9k_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error); + +#endif /* __CN9K_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4879ef9..b7869b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -233,6 +233,9 @@ cnxk_eth_txq_to_sp(void *__txq) /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; +/* Common flow ops */ +extern struct rte_flow_ops cnxk_flow_ops; + /* Ops */ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c new file mode 100644 index 0000000..1695d4f --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -0,0 +1,330 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_rte_flow.h> + +const struct cnxk_rte_flow_term_info term[] = { + [RTE_FLOW_ITEM_TYPE_ETH] = {ROC_NPC_ITEM_TYPE_ETH, + sizeof(struct rte_flow_item_eth)}, + [RTE_FLOW_ITEM_TYPE_VLAN] = {ROC_NPC_ITEM_TYPE_VLAN, + sizeof(struct rte_flow_item_vlan)}, + [RTE_FLOW_ITEM_TYPE_E_TAG] = {ROC_NPC_ITEM_TYPE_E_TAG, + sizeof(struct rte_flow_item_e_tag)}, + [RTE_FLOW_ITEM_TYPE_IPV4] = {ROC_NPC_ITEM_TYPE_IPV4, + sizeof(struct rte_flow_item_ipv4)}, + [RTE_FLOW_ITEM_TYPE_IPV6] = {ROC_NPC_ITEM_TYPE_IPV6, + sizeof(struct rte_flow_item_ipv6)}, + [RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4] = { + ROC_NPC_ITEM_TYPE_ARP_ETH_IPV4, + sizeof(struct rte_flow_item_arp_eth_ipv4)}, + [RTE_FLOW_ITEM_TYPE_MPLS] = {ROC_NPC_ITEM_TYPE_MPLS, + sizeof(struct rte_flow_item_mpls)}, + [RTE_FLOW_ITEM_TYPE_ICMP] = {ROC_NPC_ITEM_TYPE_ICMP, + sizeof(struct rte_flow_item_icmp)}, + [RTE_FLOW_ITEM_TYPE_UDP] = {ROC_NPC_ITEM_TYPE_UDP, + sizeof(struct rte_flow_item_udp)}, + [RTE_FLOW_ITEM_TYPE_TCP] = {ROC_NPC_ITEM_TYPE_TCP, + sizeof(struct rte_flow_item_tcp)}, + [RTE_FLOW_ITEM_TYPE_SCTP] = {ROC_NPC_ITEM_TYPE_SCTP, + sizeof(struct rte_flow_item_sctp)}, + [RTE_FLOW_ITEM_TYPE_ESP] = {ROC_NPC_ITEM_TYPE_ESP, + sizeof(struct rte_flow_item_esp)}, + [RTE_FLOW_ITEM_TYPE_GRE] = {ROC_NPC_ITEM_TYPE_GRE, + sizeof(struct rte_flow_item_gre)}, + [RTE_FLOW_ITEM_TYPE_NVGRE] = {ROC_NPC_ITEM_TYPE_NVGRE, + sizeof(struct rte_flow_item_nvgre)}, + [RTE_FLOW_ITEM_TYPE_VXLAN] = {ROC_NPC_ITEM_TYPE_VXLAN, + sizeof(struct rte_flow_item_vxlan)}, + [RTE_FLOW_ITEM_TYPE_GTPC] = {ROC_NPC_ITEM_TYPE_GTPC, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GTPU] = {ROC_NPC_ITEM_TYPE_GTPU, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GENEVE] = {ROC_NPC_ITEM_TYPE_GENEVE, + sizeof(struct rte_flow_item_geneve)}, + [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = { + ROC_NPC_ITEM_TYPE_VXLAN_GPE, + sizeof(struct rte_flow_item_vxlan_gpe)}, + [RTE_FLOW_ITEM_TYPE_IPV6_EXT] = {ROC_NPC_ITEM_TYPE_IPV6_EXT, + sizeof(struct rte_flow_item_ipv6_ext)}, + [RTE_FLOW_ITEM_TYPE_VOID] = {ROC_NPC_ITEM_TYPE_VOID, 0}, + [RTE_FLOW_ITEM_TYPE_ANY] = {ROC_NPC_ITEM_TYPE_ANY, 0}, + [RTE_FLOW_ITEM_TYPE_GRE_KEY] = {ROC_NPC_ITEM_TYPE_GRE_KEY, + sizeof(uint32_t)}, + [RTE_FLOW_ITEM_TYPE_HIGIG2] = { + ROC_NPC_ITEM_TYPE_HIGIG2, + sizeof(struct rte_flow_item_higig2_hdr)} +}; + +static int +cnxk_map_actions(struct rte_eth_dev *eth_dev, + const struct rte_flow_action actions[], + struct roc_npc_action in_actions[]) +{ + const struct rte_flow_action_count *act_count; + const struct rte_flow_action_queue *act_q; + int rq; + int i = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VOID; + break; + + case RTE_FLOW_ACTION_TYPE_MARK: + in_actions[i].type = ROC_NPC_ACTION_TYPE_MARK; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_FLAG: + in_actions[i].type = ROC_NPC_ACTION_TYPE_FLAG; + break; + + case RTE_FLOW_ACTION_TYPE_COUNT: + act_count = (const struct rte_flow_action_count *) + actions->conf; + + if (act_count->shared == 1) { + plt_npc_dbg("Shared counter is not supported"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_COUNT; + break; + + case RTE_FLOW_ACTION_TYPE_DROP: + in_actions[i].type = ROC_NPC_ACTION_TYPE_DROP; + break; + + case RTE_FLOW_ACTION_TYPE_PF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_PF; + break; + + case RTE_FLOW_ACTION_TYPE_VF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VF; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_QUEUE: + act_q = (const struct rte_flow_action_queue *) + actions->conf; + rq = act_q->index; + if (rq >= eth_dev->data->nb_rx_queues) { + plt_npc_dbg("Invalid queue index"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_QUEUE; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_RSS: + in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + break; + + case RTE_FLOW_ACTION_TYPE_SECURITY: + in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; + break; + default: + plt_npc_dbg("Action is not supported = %d", + actions->type); + goto err_exit; + } + i++; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_END; + return 0; + +err_exit: + return -EINVAL; +} + +static int +cnxk_map_flow_data(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct roc_npc_attr *in_attr, + struct roc_npc_item_info in_pattern[], + struct roc_npc_action in_actions[]) +{ + int i = 0; + + in_attr->priority = attr->priority; + in_attr->ingress = attr->ingress; + in_attr->egress = attr->egress; + + while (pattern->type != RTE_FLOW_ITEM_TYPE_END) { + in_pattern[i].spec = pattern->spec; + in_pattern[i].last = pattern->last; + in_pattern[i].mask = pattern->mask; + in_pattern[i].type = term[pattern->type].item_type; + in_pattern[i].size = term[pattern->type].item_size; + pattern++; + i++; + } + in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; + + return cnxk_map_actions(eth_dev, actions, in_actions); +} + +static int +cnxk_flow_validate(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow flow; + int rc; + + memset(&flow, 0, sizeof(flow)); + + rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return rc; + } + + return roc_npc_flow_parse(npc, &in_attr, in_pattern, in_actions, &flow); +} + +struct roc_npc_flow * +cnxk_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct roc_npc *npc = &dev->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow *flow; + int errcode; + int rc; + + rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return NULL; + } + + flow = roc_npc_flow_create(npc, &in_attr, in_pattern, in_actions, + &errcode); + if (errcode != 0) { + rte_flow_error_set(error, errcode, errcode, NULL, + roc_error_msg_get(errcode)); + return NULL; + } + + return flow; +} + +int +cnxk_flow_destroy(struct rte_eth_dev *eth_dev, struct roc_npc_flow *flow, + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + int rc; + + rc = roc_npc_flow_destroy(npc, flow); + if (rc) + rte_flow_error_set(error, rc, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow Destroy failed"); + return rc; +} + +static int +cnxk_flow_flush(struct rte_eth_dev *eth_dev, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + int rc; + + rc = roc_npc_mcam_free_all_resources(npc); + if (rc) { + rte_flow_error_set(error, EIO, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Failed to flush filter"); + return -rte_errno; + } + + return 0; +} + +static int +cnxk_flow_query(struct rte_eth_dev *eth_dev, struct rte_flow *flow, + const struct rte_flow_action *action, void *data, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + struct rte_flow_query_count *query = data; + const char *errmsg = NULL; + int errcode = ENOTSUP; + int rc; + + if (action->type != RTE_FLOW_ACTION_TYPE_COUNT) { + errmsg = "Only COUNT is supported in query"; + goto err_exit; + } + + if (in_flow->ctr_id == NPC_COUNTER_NONE) { + errmsg = "Counter is not available"; + goto err_exit; + } + + rc = roc_npc_mcam_read_counter(npc, in_flow->ctr_id, &query->hits); + if (rc != 0) { + errcode = EIO; + errmsg = "Error reading flow counter"; + goto err_exit; + } + query->hits_set = 1; + query->bytes_set = 0; + + if (query->reset) + rc = roc_npc_mcam_clear_counter(npc, in_flow->ctr_id); + if (rc != 0) { + errcode = EIO; + errmsg = "Error clearing flow counter"; + goto err_exit; + } + + return 0; + +err_exit: + rte_flow_error_set(error, errcode, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, errmsg); + return -rte_errno; +} + +static int +cnxk_flow_isolate(struct rte_eth_dev *eth_dev __rte_unused, + int enable __rte_unused, struct rte_flow_error *error) +{ + /* If we support, we need to un-install the default mcam + * entry for this port. + */ + + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow isolation not supported"); + + return -rte_errno; +} + +struct rte_flow_ops cnxk_flow_ops = { + .validate = cnxk_flow_validate, + .flush = cnxk_flow_flush, + .query = cnxk_flow_query, + .isolate = cnxk_flow_isolate, +}; diff --git a/drivers/net/cnxk/cnxk_rte_flow.h b/drivers/net/cnxk/cnxk_rte_flow.h new file mode 100644 index 0000000..bb23629 --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_RTE_FLOW_H__ +#define __CNXK_RTE_FLOW_H__ + +#include <rte_flow_driver.h> +#include <rte_malloc.h> + +#include "cnxk_ethdev.h" +#include "roc_api.h" +#include "roc_npc_priv.h" + +struct cnxk_rte_flow_term_info { + uint16_t item_type; + uint16_t item_size; +}; + +struct roc_npc_flow *cnxk_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cnxk_flow_destroy(struct rte_eth_dev *dev, struct roc_npc_flow *flow, + struct rte_flow_error *error); + +#endif /* __CNXK_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7d711a2..df953fd 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,10 +13,12 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_rte_flow.c', 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', + 'cn9k_rte_flow.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', @@ -25,6 +27,7 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c', + 'cn10k_rte_flow.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 50/62] net/cnxk: add flow ops get operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (48 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram ` (12 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> This patch adds flow ops get operation to enable rte_flow_ops.A This patch also add support for flow dev dump API. Every flow rule added will be dumped in the below format. MCAM Index:1881 Interface :NIX-RX (0) Priority :1 NPC RX Action:0X00000000404001 ActionOp:NIX_RX_ACTIONOP_UCAST (1) PF_FUNC: 0X400 RQ Index:0X004 Match Id:0000 Flow Key Alg:0 NPC RX VTAG Action:0X00000000008100 VTAG0:relptr:0 lid:0X1 type:0 Patterns: NPC_PARSE_NIBBLE_CHAN:000 NPC_PARSE_NIBBLE_LA_LTYPE:LA_ETHER NPC_PARSE_NIBBLE_LB_LTYPE:NONE NPC_PARSE_NIBBLE_LC_LTYPE:LC_IP NPC_PARSE_NIBBLE_LD_LTYPE:LD_TCP NPC_PARSE_NIBBLE_LE_LTYPE:NONE LA_ETHER, hdr offset:0, len:0X6, key offset:0X8,\ Data:0X4AE124FC7FFF, Mask:0XFFFFFFFFFFFF LA_ETHER, hdr offset:0XC, len:0X2, key offset:0X4, Data:0XCA5A,\ Mask:0XFFFF LC_IP, hdr offset:0XC, len:0X8, key offset:0X10,\ Data:0X0A01010300000000, Mask:0XFFFFFFFF00000000 LD_TCP, hdr offset:0, len:0X4, key offset:0X18, Data:0X03450000,\ Mask:0XFFFF0000 MCAM Raw Data : DW0 :0000CA5A01202000 DW0_Mask:0000FFFF0FF0F000 DW1 :00004AE124FC7FFF DW1_Mask:0000FFFFFFFFFFFF DW2 :0A01010300000000 DW2_Mask:FFFFFFFF00000000 DW3 :0000000003450000 DW3_Mask:00000000FFFF0000 DW4 :0000000000000000 DW4_Mask:0000000000000000 DW5 :0000000000000000 DW5_Mask:0000000000000000 DW6 :0000000000000000 DW6_Mask:0000000000000000 Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 3 +++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ drivers/net/cnxk/cnxk_rte_flow.c | 28 ++++++++++++++++++++++++++++ 5 files changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 8a76823..aff4eef 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -1009,6 +1009,8 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, struct npc_flow_list *list; int rc; + npc->channel = roc_npc->channel; + flow = plt_zmalloc(sizeof(*flow), 0); if (flow == NULL) { *errcode = NPC_ERR_NO_MEM; diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index d1a7f2e..a330875 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -807,6 +807,8 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto fail_configure; } + dev->npc.channel = roc_nix_get_base_chan(nix); + nb_rxq = data->nb_rx_queues; nb_txq = data->nb_tx_queues; rc = -ENOMEM; @@ -1211,6 +1213,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, + .flow_ops_get = cnxk_nix_flow_ops_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index b7869b4..07c87ea 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -273,7 +273,8 @@ int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); - +int cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index ce94eb6..0b62911 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -300,6 +300,16 @@ cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, } int +cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops) +{ + RTE_SET_USED(eth_dev); + + *ops = &cnxk_flow_ops; + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index 1695d4f..2dee19e 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -322,9 +322,37 @@ cnxk_flow_isolate(struct rte_eth_dev *eth_dev __rte_unused, return -rte_errno; } +static int +cnxk_flow_dev_dump(struct rte_eth_dev *eth_dev, struct rte_flow *flow, + FILE *file, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + + if (file == NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "Invalid file"); + return -rte_errno; + } + + if (flow != NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_HANDLE, + NULL, + "Invalid argument"); + return -EINVAL; + } + + roc_npc_flow_dump(file, npc); + + return 0; +} + struct rte_flow_ops cnxk_flow_ops = { .validate = cnxk_flow_validate, .flush = cnxk_flow_flush, .query = cnxk_flow_query, .isolate = cnxk_flow_isolate, + .dev_dump = cnxk_flow_dev_dump, }; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 51/62] net/cnxk: add ethdev firmware version get 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (49 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 52/62] net/cnxk: add get register operation Nithin Dabilpuram ` (11 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Add callback to get ethdev firmware version. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 6 files changed, 25 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e2da5dd..d43b1c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -35,6 +35,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e990480..6b9ce2d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -33,6 +33,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 3a4417c..5629d07 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -30,6 +30,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a330875..284c8f4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1210,6 +1210,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .fw_version_get = cnxk_nix_fw_version_get, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 07c87ea..a25be8a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -313,6 +313,8 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +int cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size); void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo); void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 0b62911..4e863e2 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -639,6 +639,25 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } +int +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const char *str = roc_npc_profile_name_get(&dev->npc); + uint32_t size = strlen(str) + 1; + + if (fw_size > size) + fw_size = size; + + rte_strlcpy(fw_version, str, fw_size); + + if (fw_size < size) + return size; + + return 0; +} + void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo) -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 52/62] net/cnxk: add get register operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (50 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram ` (10 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> With this patch implemented api to dump platform registers for debug purposes. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 25 +++++++++++++++++++++++++ 7 files changed, 34 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index cc71b22..e561321 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -32,6 +32,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Debug utilities - Context dump and error interrupt support - Support Rx interrupt Prerequisites diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index d43b1c4..246caf9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -37,6 +37,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6b9ce2d..65ee8ba 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -35,6 +35,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5629d07..00bde9b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -32,6 +32,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 284c8f4..cb0dd45 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1215,6 +1215,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, + .get_reg = cnxk_nix_dev_get_reg, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a25be8a..d51a6d7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -328,6 +328,10 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Debug */ +int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, + struct rte_dev_reg_info *regs); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4e863e2..27d5dc0 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -697,3 +697,28 @@ cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) return 0; } + +int +cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t *data = regs->data; + int rc = -ENOTSUP; + + if (data == NULL) { + rc = roc_nix_lf_get_reg_count(nix); + if (rc > 0) { + regs->length = rc; + regs->width = 8; + rc = 0; + } + return rc; + } + + if (!regs->length || + regs->length == (uint32_t)roc_nix_lf_get_reg_count(nix)) + return roc_nix_lf_reg_dump(nix, data); + + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 53/62] net/cnxk: support for RSS in rte flow 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (51 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 52/62] net/cnxk: add get register operation Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram ` (9 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Added support for RSS action in rte flow code based on ROC. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + drivers/net/cnxk/cnxk_rte_flow.c | 74 ++++++++++++++++++++++++++++++++++----- 2 files changed, 67 insertions(+), 8 deletions(-) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 246caf9..eba4107 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -75,5 +75,6 @@ flag = Y pf = Y port_id = Y queue = Y +rss = Y security = Y vf = Y diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index 2dee19e..c61618a 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -56,14 +56,64 @@ const struct cnxk_rte_flow_term_info term[] = { }; static int -cnxk_map_actions(struct rte_eth_dev *eth_dev, +npc_rss_action_validate(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_action *act) +{ + const struct rte_flow_action_rss *rss; + + rss = (const struct rte_flow_action_rss *)act->conf; + + if (attr->egress) { + plt_err("No support of RSS in egress"); + return -EINVAL; + } + + if (eth_dev->data->dev_conf.rxmode.mq_mode != ETH_MQ_RX_RSS) { + plt_err("multi-queue mode is disabled"); + return -ENOTSUP; + } + + if (!rss || !rss->queue_num) { + plt_err("no valid queues"); + return -EINVAL; + } + + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT) { + plt_err("non-default RSS hash functions are not supported"); + return -ENOTSUP; + } + + if (rss->key_len && rss->key_len > ROC_NIX_RSS_KEY_LEN) { + plt_err("RSS hash key too large"); + return -ENOTSUP; + } + + return 0; +} + +static void +npc_rss_flowkey_get(struct cnxk_eth_dev *eth_dev, + const struct roc_npc_action *rss_action, + uint32_t *flowkey_cfg) +{ + const struct roc_npc_action_rss *rss; + + rss = (const struct roc_npc_action_rss *)rss_action->conf; + + *flowkey_cfg = cnxk_rss_ethdev_to_nix(eth_dev, rss->types, rss->level); +} + +static int +cnxk_map_actions(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct rte_flow_action_count *act_count; const struct rte_flow_action_queue *act_q; + int i = 0, rc = 0; int rq; - int i = 0; for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { switch (actions->type) { @@ -117,7 +167,12 @@ cnxk_map_actions(struct rte_eth_dev *eth_dev, break; case RTE_FLOW_ACTION_TYPE_RSS: + rc = npc_rss_action_validate(eth_dev, attr, actions); + if (rc) + goto err_exit; in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + in_actions[i].conf = actions->conf; + npc_rss_flowkey_get(dev, &in_actions[i], flowkey_cfg); break; case RTE_FLOW_ACTION_TYPE_SECURITY: @@ -144,7 +199,7 @@ cnxk_map_flow_data(struct rte_eth_dev *eth_dev, const struct rte_flow_action actions[], struct roc_npc_attr *in_attr, struct roc_npc_item_info in_pattern[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { int i = 0; @@ -163,7 +218,8 @@ cnxk_map_flow_data(struct rte_eth_dev *eth_dev, } in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; - return cnxk_map_actions(eth_dev, actions, in_actions); + return cnxk_map_actions(eth_dev, attr, actions, in_actions, + flowkey_cfg); } static int @@ -179,12 +235,13 @@ cnxk_flow_validate(struct rte_eth_dev *eth_dev, struct roc_npc *npc = &dev->npc; struct roc_npc_attr in_attr; struct roc_npc_flow flow; + uint32_t flowkey_cfg = 0; int rc; memset(&flow, 0, sizeof(flow)); rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, &flowkey_cfg); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); @@ -206,11 +263,12 @@ cnxk_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, struct roc_npc *npc = &dev->npc; struct roc_npc_attr in_attr; struct roc_npc_flow *flow; - int errcode; + int errcode = 0; int rc; rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, + &npc->flowkey_cfg_state); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 54/62] net/cnxk: register callback to get PTP status 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (52 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram ` (8 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Once PTP status is changed at H/W i.e. enable/disable then it is propagated to user via registered callback. So corresponding callback is registered to get PTP status. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 87 ++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx.h | 1 + drivers/net/cnxk/cn9k_ethdev.c | 73 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 +- drivers/net/cnxk/cnxk_ethdev.h | 5 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 2 + 7 files changed, 166 insertions(+), 5 deletions(-) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 0396ff6..bddb7fb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -268,6 +268,76 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn10k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn10k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cannot be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -333,6 +403,7 @@ static int cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; int rc; if (RTE_CACHE_LINE_SIZE != 64) { @@ -354,15 +425,23 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (rc) return rc; + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - eth_dev = rte_eth_dev_allocated(pci_dev->device.name); - if (!eth_dev) - return -ENOENT; - /* Setup callbacks for secondary process */ cn10k_eth_set_tx_function(eth_dev); cn10k_eth_set_rx_function(eth_dev); + return 0; } + + dev = cnxk_eth_pmd_priv(eth_dev); + + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn10k_nix_ptp_info_update_cb); + return 0; } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 7bb9dd8..29ee0ac 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -12,6 +12,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 5cc0db9..fc3054d 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -277,6 +277,76 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn9k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn9k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cannot be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -396,6 +466,9 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) dev->hwcap = 0; + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn9k_nix_ptp_info_update_cb); + /* Update HW erratas */ if (roc_model_is_cn96_A0() || roc_model_is_cn95_A0()) dev->cq_min_4k = 1; diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index bc04f5c..f4b3282 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -13,6 +13,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cb0dd45..094f1cb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -59,7 +59,7 @@ nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) } } -static int +int nix_recalc_mtu(struct rte_eth_dev *eth_dev) { struct rte_eth_dev_data *data = eth_dev->data; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index d51a6d7..1c41dcb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -100,6 +100,7 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +#define CNXK_NIX_TIMESYNC_RX_OFFSET 8 #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -153,6 +154,7 @@ struct cnxk_eth_dev { uint16_t flags; uint8_t ptype_disable; bool scalar_ena; + bool ptp_en; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -332,6 +334,9 @@ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs); +/* Other private functions */ +int nix_recalc_mtu(struct rte_eth_dev *eth_dev); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 27d5dc0..30222e6 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -385,6 +385,8 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) int rc = -EINVAL; uint32_t buffsz; + frame_size += CNXK_NIX_TIMESYNC_RX_OFFSET * dev->ptp_en; + /* Check if MTU is within the allowed range */ if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { plt_err("MTU is lesser than minimum"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 55/62] net/cnxk: add base PTP timesync support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (53 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram ` (7 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Base PTP timesync support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 31 +++++ drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_rx.c | 32 ++--- drivers/net/cnxk/cn10k_rx.h | 61 +++++++--- drivers/net/cnxk/cn10k_rx_mseg.c | 2 +- drivers/net/cnxk/cn10k_rx_vec.c | 5 +- drivers/net/cnxk/cn10k_tx.c | 28 +++-- drivers/net/cnxk/cn10k_tx.h | 193 +++++++++++++++++++++++------ drivers/net/cnxk/cn10k_tx_mseg.c | 2 +- drivers/net/cnxk/cn10k_tx_vec.c | 3 +- drivers/net/cnxk/cn9k_ethdev.c | 34 +++++- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_rx.c | 32 ++--- drivers/net/cnxk/cn9k_rx.h | 61 +++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 2 +- drivers/net/cnxk/cn9k_rx_vec.c | 5 +- drivers/net/cnxk/cn9k_tx.c | 28 +++-- drivers/net/cnxk/cn9k_tx.h | 253 ++++++++++++++++++++++++++++----------- drivers/net/cnxk/cn9k_tx_mseg.c | 2 +- drivers/net/cnxk/cn9k_tx_vec.c | 3 +- drivers/net/cnxk/cnxk_ethdev.c | 36 +++++- drivers/net/cnxk/cnxk_ethdev.h | 64 +++++++++- drivers/net/cnxk/cnxk_ptp.c | 169 ++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 24 files changed, 834 insertions(+), 215 deletions(-) create mode 100644 drivers/net/cnxk/cnxk_ptp.c diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index bddb7fb..5e0de13 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -30,6 +30,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -95,6 +98,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -121,6 +127,7 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; union nix_send_hdr_w0_u send_hdr_w0; + struct nix_send_mem_s *send_mem; union nix_send_sg_s sg_w0; RTE_SET_USED(dev); @@ -136,6 +143,22 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr_w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *)(txq->cmd + 2); + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } } else { /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ send_hdr_w0.sizem1 = 1; @@ -221,6 +244,7 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -342,6 +366,7 @@ static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -349,6 +374,12 @@ cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index d39ca31..8b6e0f2 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -31,6 +31,7 @@ struct cn10k_eth_rxq { uint32_t available; uint16_t data_off; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 0598111..c9744e2 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 29ee0ac..c09ccdf 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -7,6 +7,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -250,6 +252,10 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -487,27 +493,44 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c index 9d283f7..b67d21f 100644 --- a/drivers/net/cnxk/cn10k_rx_mseg.c +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c index 0fa079c..1330235 100644 --- a/drivers/net/cnxk/cn10k_rx_vec.c +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -5,12 +5,15 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ struct rte_mbuf **rx_pkts, \ uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index e6eb101..18694dc 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -24,12 +24,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -42,25 +43,25 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -68,7 +69,8 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index b74df10..8b1446f 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -49,8 +51,12 @@ static __rte_always_inline int cn10k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline uint8_t @@ -380,6 +386,45 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); } +static __rte_always_inline void +cn10k_nix_xmit_prepare_tstamp(uintptr_t lmt_addr, const uint64_t *cmd, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + struct nix_send_ext_s *send_hdr_ext = + (struct nix_send_ext_s *)lmt_addr + 16; + uint64_t *lmt = (uint64_t *)lmt_addr; + uint16_t off = (no_segdw - 1) << 1; + struct nix_send_mem_s *send_mem; + + send_mem = (struct nix_send_mem_s *)(lmt + off); + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + send_hdr_ext->w0.tstmp = 1; + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + lmt[off] = cmd[2]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + send_mem->addr = + (rte_iova_t)(((uint64_t *)cmd[3]) + is_ol_tstamp); + } +} + static __rte_always_inline uint16_t cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) { @@ -445,7 +490,7 @@ cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -487,6 +532,8 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, lso_tun_fmt); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); } @@ -576,6 +623,9 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, /* Store sg list directly on lmt line */ segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, flags); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); data128 |= (((__uint128_t)(segdw - 1)) << shft); shft += 3; @@ -1406,75 +1456,140 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c index 6ae6907..33f6754 100644 --- a/drivers/net/cnxk/cn10k_tx_mseg.c +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c index 42baeb5..7453f3b 100644 --- a/drivers/net/cnxk/cn10k_tx_vec.c +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index fc3054d..924b747 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -30,6 +30,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -95,6 +98,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -121,10 +127,9 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; struct nix_send_hdr_s *send_hdr; + struct nix_send_mem_s *send_mem; union nix_send_sg_s *sg; - RTE_SET_USED(dev); - /* Initialize the fields based on basic single segment packet */ memset(&txq->cmd, 0, sizeof(txq->cmd)); @@ -135,6 +140,23 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr->w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *) + (txq->cmd + (send_hdr->w0.sizem1 << 1)); + send_mem->w0.cn9k.subdc = NIX_SUBDC_MEM; + send_mem->w0.cn9k.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } sg = (union nix_send_sg_s *)&txq->cmd[4]; } else { send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; @@ -219,6 +241,7 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -351,6 +374,7 @@ static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -358,6 +382,12 @@ cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index b92f3fc..f481b3a 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -29,6 +29,7 @@ struct cn9k_eth_rxq { uint32_t qmask; uint32_t available; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 01eb21f..a15428d 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index f4b3282..c5ad5db 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -8,6 +8,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -253,6 +255,10 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -489,27 +495,44 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c index 6ad8c1d..3b26962 100644 --- a/drivers/net/cnxk/cn9k_rx_mseg.c +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c index 997177f..b19c7f3 100644 --- a/drivers/net/cnxk/cn9k_rx_vec.c +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -5,10 +5,13 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 2ff9720..b802606 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -23,12 +23,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TS] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -41,25 +42,25 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -67,7 +68,8 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 7b0d536..1899d66 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -46,8 +48,12 @@ static __rte_always_inline int cn9k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline void @@ -283,6 +289,41 @@ cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags, } static __rte_always_inline void +cn9k_nix_xmit_prepare_tstamp(uint64_t *cmd, const uint64_t *send_mem_desc, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + struct nix_send_mem_s *send_mem; + uint16_t off = (no_segdw - 1) << 1; + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + + send_mem = (struct nix_send_mem_s *)(cmd + off); + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + cmd[off] = send_mem_desc[6]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.cn9k.alg = + NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + + send_mem->addr = (rte_iova_t)((uint64_t *)send_mem_desc[7] + + (is_ol_tstamp)); + } +} + +static __rte_always_inline void cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, const uint32_t flags) { @@ -380,7 +421,7 @@ cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -447,6 +488,8 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); } @@ -488,6 +531,9 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); } @@ -1241,75 +1287,140 @@ cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ -#define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ - NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ - L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ - OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ - OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ - VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ - VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ - VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ - VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ - NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ - NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ - NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ - NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ - NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ - NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ - TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ - TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ - TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ - TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ - TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ - TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ - TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ - TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ - TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c index 65c5f36..f3c427c 100644 --- a/drivers/net/cnxk/cn9k_tx_mseg.c +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c index 21ffc2c..a6e7c9e 100644 --- a/drivers/net/cnxk/cn9k_tx_vec.c +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 094f1cb..71d567a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -150,7 +150,8 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) offsetof(struct rte_mbuf, data_off) != 6); mb_def.nb_segs = 1; - mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.data_off = RTE_PKTMBUF_HEADROOM + + (dev->ptp_en * CNXK_NIX_TIMESYNC_RX_OFFSET); mb_def.port = port_id; rte_mbuf_refcnt_set(&mb_def, 1); @@ -356,6 +357,18 @@ cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, eth_dev->data->rx_queues[qid] = rxq_sp + 1; eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + /* Calculating delta and freq mult between PTP HI clock and tsc. + * These are needed in deriving raw clock value from tsc counter. + * read_clock eth op returns raw clock value. + */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) { + rc = cnxk_nix_tsc_convert(dev); + if (rc) { + plt_err("Failed to calculate delta and freq mult"); + goto rq_fini; + } + } + return 0; rq_fini: rc |= roc_nix_rq_fini(rq); @@ -1112,7 +1125,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; - if (eth_dev->data->nb_rx_queues != 0) { + if (eth_dev->data->nb_rx_queues != 0 && !dev->ptp_en) { rc = nix_recalc_mtu(eth_dev); if (rc) return rc; @@ -1157,6 +1170,25 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) } } + /* Enable PTP if it is requested by the user or already + * enabled on PF owning this VF + */ + memset(&dev->tstamp, 0, sizeof(struct cnxk_timesync_info)); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) + cnxk_eth_dev_ops.timesync_enable(eth_dev); + else + cnxk_eth_dev_ops.timesync_disable(eth_dev); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) { + rc = rte_mbuf_dyn_rx_timestamp_register + (&dev->tstamp.tstamp_dynfield_offset, + &dev->tstamp.rx_tstamp_dynflag); + if (rc != 0) { + plt_err("Failed to register Rx timestamp field/flag"); + goto rx_disable; + } + } + cnxk_nix_toggle_flag_link_cfg(dev, false); return 0; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1c41dcb..de6d533 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -13,6 +13,7 @@ #include <rte_mbuf.h> #include <rte_mbuf_pool_ops.h> #include <rte_mempool.h> +#include <rte_time.h> #include "roc_api.h" @@ -75,7 +76,7 @@ (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ - DEV_RX_OFFLOAD_RSS_HASH) + DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP) #define RSS_IPV4_ENABLE \ (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ @@ -100,7 +101,10 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +/* Default cycle counter mask */ +#define CNXK_CYCLECOUNTER_MASK 0xffffffffffffffffULL #define CNXK_NIX_TIMESYNC_RX_OFFSET 8 + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -130,6 +134,16 @@ struct cnxk_eth_qconf { uint8_t valid; }; +struct cnxk_timesync_info { + uint64_t rx_tstamp_dynflag; + rte_iova_t tx_tstamp_iova; + uint64_t *tx_tstamp; + uint64_t rx_tstamp; + int tstamp_dynfield_offset; + uint8_t tx_ready; + uint8_t rx_ready; +} __plt_cache_aligned; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -188,6 +202,14 @@ struct cnxk_eth_dev { /* Flow control configuration */ struct cnxk_fc_cfg fc_cfg; + /* PTP Counters */ + struct cnxk_timesync_info tstamp; + struct rte_timecounter systime_tc; + struct rte_timecounter rx_tstamp_tc; + struct rte_timecounter tx_tstamp_tc; + double clk_freq_mult; + uint64_t clk_delta; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -288,6 +310,9 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -404,4 +429,41 @@ cnxk_nix_prefree_seg(struct rte_mbuf *m) return 1; } +static inline rte_mbuf_timestamp_t * +cnxk_nix_timestamp_dynfield(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *info) +{ + return RTE_MBUF_DYNFIELD(mbuf, info->tstamp_dynfield_offset, + rte_mbuf_timestamp_t *); +} + +static __rte_always_inline void +cnxk_nix_mbuf_to_tstamp(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *tstamp, bool ts_enable, + uint64_t *tstamp_ptr) +{ + if (ts_enable && + (mbuf->data_off == + RTE_PKTMBUF_HEADROOM + CNXK_NIX_TIMESYNC_RX_OFFSET)) { + mbuf->pkt_len -= CNXK_NIX_TIMESYNC_RX_OFFSET; + + /* Reading the rx timestamp inserted by CGX, viz at + * starting of the packet data. + */ + *cnxk_nix_timestamp_dynfield(mbuf, tstamp) = + rte_be_to_cpu_64(*tstamp_ptr); + /* PKT_RX_IEEE1588_TMST flag needs to be set only in case + * PTP packets are received. + */ + if (mbuf->packet_type == RTE_PTYPE_L2_ETHER_TIMESYNC) { + tstamp->rx_tstamp = + *cnxk_nix_timestamp_dynfield(mbuf, tstamp); + tstamp->rx_ready = 1; + mbuf->ol_flags |= PKT_RX_IEEE1588_PTP | + PKT_RX_IEEE1588_TMST | + tstamp->rx_tstamp_dynflag; + } + } +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c new file mode 100644 index 0000000..fc317965 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +/* This function calculates two parameters "clk_freq_mult" and + * "clk_delta" which is useful in deriving PTP HI clock from + * timestamp counter (tsc) value. + */ +int +cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) +{ + uint64_t ticks_base = 0, ticks = 0, tsc = 0, t_freq; + struct roc_nix *nix = &dev->nix; + int rc, val; + + /* Calculating the frequency at which PTP HI clock is running */ + rc = roc_nix_ptp_clock_read(nix, &ticks_base, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + rte_delay_ms(100); + + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + t_freq = (ticks - ticks_base) * 10; + + /* Calculating the freq multiplier viz the ratio between the + * frequency at which PTP HI clock works and tsc clock runs + */ + dev->clk_freq_mult = + (double)pow(10, floor(log10(t_freq))) / rte_get_timer_hz(); + + val = false; +#ifdef RTE_ARM_EAL_RDTSC_USE_PMU + val = true; +#endif + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, val); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + /* Calculating delta between PTP HI clock and tsc */ + dev->clk_delta = ((uint64_t)(ticks / dev->clk_freq_mult) - tsc); + +fail: + return rc; +} + +int +cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + struct roc_nix *nix = &dev->nix; + const struct rte_memzone *ts; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be enabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) { + plt_err("PTP cannot be enabled for VF/SDP/LBK"); + return -EINVAL; + } + + if (dev->ptp_en) + return rc; + + if (dev->ptype_disable) { + plt_err("Ptype offload is disabled, it should be enabled"); + return -EINVAL; + } + + if (dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { + plt_err("Both PTP and switch header cannot be enabled"); + return -EINVAL; + } + + /* Allocating a iova address for tx tstamp */ + ts = rte_eth_dma_zone_reserve(eth_dev, "cnxk_ts", 0, 128, 128, 0); + if (ts == NULL) { + plt_err("Failed to allocate mem for tx tstamp addr"); + return -ENOMEM; + } + + tstamp->tx_tstamp_iova = ts->iova; + tstamp->tx_tstamp = ts->addr; + + rc = rte_mbuf_dyn_rx_timestamp_register(&tstamp->tstamp_dynfield_offset, + &tstamp->rx_tstamp_dynflag); + if (rc) { + plt_err("Failed to register Rx timestamp field/flag"); + goto error; + } + + /* System time should be already on by default */ + memset(&dev->systime_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->rx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->tx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + + dev->systime_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->rx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->tx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + + dev->rx_offloads |= DEV_RX_OFFLOAD_TIMESTAMP; + + rc = roc_nix_ptp_rx_ena_dis(nix, true); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, true); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, false); + goto error; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) { + plt_err("Failed to set MTU size for ptp"); + goto error; + } + + return rc; + +error: + rte_eth_dma_zone_free(eth_dev, "cnxk_ts", 0); + dev->tstamp.tx_tstamp_iova = 0; + dev->tstamp.tx_tstamp = NULL; + return rc; +} + +int +cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t rx_offloads = DEV_RX_OFFLOAD_TIMESTAMP; + struct roc_nix *nix = &dev->nix; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be disabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) + return -EINVAL; + + if (!dev->ptp_en) + return rc; + + dev->rx_offloads &= ~rx_offloads; + + rc = roc_nix_ptp_rx_ena_dis(nix, false); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, false); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, true); + return rc; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) + plt_err("Failed to set MTU size for ptp"); + + return rc; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index df953fd..2071d0d 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_ptp.c', 'cnxk_rte_flow.c', 'cnxk_stats.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 56/62] net/cnxk: add timesync enable/disable operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (54 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram ` (6 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements timesync enable/disable operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 5e0de13..b079edb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -363,6 +363,54 @@ cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn10k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn10k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -408,6 +456,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn10k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn10k_nix_timesync_disable; } static void diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 924b747..107a540 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -371,6 +371,54 @@ cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn9k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn9k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -416,6 +464,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn9k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn9k_nix_timesync_disable; } static void -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 57/62] net/cnxk: add Rx/Tx timestamp read operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (55 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram ` (5 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements Rx/Tx timestamp read operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 +++++ drivers/net/cnxk/cnxk_ptp.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 71d567a..0bef942 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1248,6 +1248,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, .get_reg = cnxk_nix_dev_get_reg, + .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, + .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index de6d533..76df84a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -312,6 +312,11 @@ int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, + uint32_t flags); +int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index fc317965..7b00f87 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,44 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, uint32_t flags) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + PLT_SET_USED(flags); + + if (!tstamp->rx_ready) + return -EINVAL; + + ns = rte_timecounter_update(&dev->rx_tstamp_tc, tstamp->rx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + tstamp->rx_ready = 0; + return 0; +} + +int +cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + if (*tstamp->tx_tstamp == 0) + return -EINVAL; + + ns = rte_timecounter_update(&dev->tx_tstamp_tc, *tstamp->tx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + *tstamp->tx_tstamp = 0; + rte_wmb(); + + return 0; +} + +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 58/62] net/cnxk: add time read/write/adjust operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (56 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 59/62] net/cnxk: add read clock operation Nithin Dabilpuram ` (4 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements read/write/adjust time operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ptp.c | 63 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 0bef942..cac8ddf 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1250,6 +1250,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_reg = cnxk_nix_dev_get_reg, .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, + .timesync_read_time = cnxk_nix_timesync_read_time, + .timesync_write_time = cnxk_nix_timesync_write_time, + .timesync_adjust_time = cnxk_nix_timesync_adjust_time, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 76df84a..4214365 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -317,6 +317,11 @@ int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, uint32_t flags); int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp); +int cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, + struct timespec *ts); +int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts); +int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 7b00f87..52f6eb1 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,69 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t clock, ns; + int rc; + + rc = roc_nix_ptp_clock_read(nix, &clock, NULL, false); + if (rc) + return rc; + + ns = rte_timecounter_update(&dev->systime_tc, clock); + *ts = rte_ns_to_timespec(ns); + return 0; +} + +int +cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t ns; + + ns = rte_timespec_to_ns(ts); + /* Set the time counters to a new value. */ + dev->systime_tc.nsec = ns; + dev->rx_tstamp_tc.nsec = ns; + dev->tx_tstamp_tc.nsec = ns; + + return 0; +} + +int +cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Adjust the frequent to make tics increments in 10^9 tics per sec */ + if (delta < ROC_NIX_PTP_FREQ_ADJUST && + delta > -ROC_NIX_PTP_FREQ_ADJUST) { + rc = roc_nix_ptp_sync_time_adjust(nix, delta); + if (rc) + return rc; + + /* Since the frequency of PTP comp register is tuned, delta and + * freq mult calculation for deriving PTP_HI from timestamp + * counter should be done again. + */ + rc = cnxk_nix_tsc_convert(dev); + if (rc) + plt_err("Failed to calculate delta and freq mult"); + } + + dev->systime_tc.nsec += delta; + dev->rx_tstamp_tc.nsec += delta; + dev->tx_tstamp_tc.nsec += delta; + + return 0; +} + +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp, uint32_t flags) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 59/62] net/cnxk: add read clock operation 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (57 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram ` (3 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements read raw clock operation for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ptp.c | 17 +++++++++++++++++ 4 files changed, 21 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index eba4107..5874531 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Timesync = Y +Timestamp offload = Y Basic stats = Y Stats per queue = Y Extended stats = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cac8ddf..e1373cf 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1253,6 +1253,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_read_time = cnxk_nix_timesync_read_time, .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, + .read_clock = cnxk_nix_read_clock, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4214365..fa6f16f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -323,6 +323,7 @@ int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, const struct timespec *ts); int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); +int cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 52f6eb1..449489f 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -4,6 +4,23 @@ #include "cnxk_ethdev.h" +int +cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* This API returns the raw PTP HI clock value. Since LFs do not + * have direct access to PTP registers and it requires mbox msg + * to AF for this value. In fastpath reading this value for every + * packet (which involes mbox call) becomes very expensive, hence + * we should be able to derive PTP HI clock value from tsc by + * using freq_mult and clk_delta calculated during configure stage. + */ + *clock = (rte_get_tsc_cycles() + dev->clk_delta) * dev->clk_freq_mult; + + return 0; +} + /* This function calculates two parameters "clk_freq_mult" and * "clk_delta" which is useful in deriving PTP HI clock from * timestamp counter (tsc) value. -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 60/62] net/cnxk: added RETA and RSS hash operations 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (58 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 59/62] net/cnxk: add read clock operation Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 61/62] net/cnxk: add multicast filter support Nithin Dabilpuram ` (2 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch will implement RETA and RSS hash apis. Also added device argument to lock rx context. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 4 ++ drivers/net/cnxk/cnxk_ethdev.h | 10 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 121 +++++++++++++++++++++++++++++++++ 7 files changed, 145 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 5874531..9945af9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 65ee8ba..b4b1169 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 00bde9b..fbc50a8 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -19,6 +19,8 @@ Queue start/stop = Y MTU update = Y TSO = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index e1373cf..f49c9e6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1254,6 +1254,10 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, .read_clock = cnxk_nix_read_clock, + .reta_update = cnxk_nix_reta_update, + .reta_query = cnxk_nix_reta_query, + .rss_hash_update = cnxk_nix_rss_hash_update, + .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index fa6f16f..cd08e3a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -330,6 +330,16 @@ uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +int cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); +int cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); /* Link */ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 7fd06eb..c76b628 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -109,6 +109,7 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) #define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" #define CNXK_SWITCH_HEADER_TYPE "switch_header" #define CNXK_RSS_TAG_AS_XOR "tag_as_xor" +#define CNXK_LOCK_RX_CTX "lock_rx_ctx" int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) @@ -120,6 +121,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) uint16_t flow_max_priority = 3; uint16_t rss_tag_as_xor = 0; uint16_t scalar_enable = 0; + uint8_t lock_rx_ctx = 0; struct rte_kvargs *kvlist; if (devargs == NULL) @@ -143,6 +145,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) &parse_switch_header_type, &switch_header_type); rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, &rss_tag_as_xor); + rte_kvargs_process(kvlist, CNXK_LOCK_RX_CTX, &parse_flag, &lock_rx_ctx); rte_kvargs_free(kvlist); null_devargs: @@ -150,6 +153,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->nix.lock_rx_ctx = lock_rx_ctx; dev->npc.flow_prealloc_size = flow_prealloc_size; dev->npc.flow_max_priority = flow_max_priority; dev->npc.switch_header_type = switch_header_type; diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 30222e6..28b49ad 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -724,3 +724,124 @@ cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) return rc; } + +int +cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int i, j, rc = -EINVAL, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta[idx] = reta_conf[i].reta[j]; + idx++; + } + } + + return roc_nix_rss_reta_set(nix, 0, reta); + +fail: + return rc; +} + +int +cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL, i, j, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + rc = roc_nix_rss_reta_get(nix, 0, reta); + if (rc) + goto fail; + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = reta[idx]; + idx++; + } + } + + return 0; + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + int rc = -EINVAL; + uint8_t alg_idx; + + if (rss_conf->rss_key && rss_conf->rss_key_len != ROC_NIX_RSS_KEY_LEN) { + plt_err("Hash key size mismatch %d vs %d", + rss_conf->rss_key_len, ROC_NIX_RSS_KEY_LEN); + goto fail; + } + + if (rss_conf->rss_key) + roc_nix_rss_key_set(nix, rss_conf->rss_key); + + rss_hash_level = ETH_RSS_LEVEL(rss_conf->rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + flowkey_cfg = + cnxk_rss_ethdev_to_nix(dev, rss_conf->rss_hf, rss_hash_level); + + rc = roc_nix_rss_flowkey_set(nix, &alg_idx, flowkey_cfg, + ROC_NIX_RSS_GROUP_DEFAULT, + ROC_NIX_RSS_MCAM_IDX_DEFAULT); + if (rc) { + plt_err("Failed to set RSS hash function rc=%d", rc); + return rc; + } + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (rss_conf->rss_key) + roc_nix_rss_key_get(&dev->nix, rss_conf->rss_key); + + rss_conf->rss_key_len = ROC_NIX_RSS_KEY_LEN; + rss_conf->rss_hf = dev->ethdev_rss_hf; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 61/62] net/cnxk: add multicast filter support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (59 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 62/62] net/cnxk: add marking and VLAN tagging support Nithin Dabilpuram 2021-06-21 13:41 ` [dpdk-dev] [PATCH v3 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch adds multicast filter support for cn9k and cn10k platforms. CGX DMAC filter table(32 entries) is divided among all LMACs connected to it i.e. if CGX has 4 LMACs then each LMAC can have up to 8 filters. If CGX has 1 LMAC then it can have up to 32 filters. Above mentioned filter table is used to install unicast and multicast DMAC address filters. Unicast filters are installed via rte_eth_dev_mac_addr_add API while multicast filters are installed via rte_eth_dev_set_mc_addr_list API. So in total, supported MAC filters are equal to DMAC filters plus mcast filters. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 63 +++++++++++++++++++++++++++++++++++ 5 files changed, 71 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9945af9..c686ceb 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -22,6 +22,7 @@ TSO = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index b4b1169..4c3f78e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -21,6 +21,7 @@ MTU update = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index f49c9e6..1edea35 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1258,6 +1258,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .reta_query = cnxk_nix_reta_query, .rss_hash_update = cnxk_nix_rss_hash_update, .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, + .set_mc_addr_list = cnxk_nix_mc_addr_list_configure, }; static int @@ -1322,6 +1323,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->max_mac_entries = max_entries; + dev->dmac_filter_count = 1; /* Get mac address */ rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index cd08e3a..c07e988 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -162,6 +162,7 @@ struct cnxk_eth_dev { uint8_t configured; /* Max macfilter entries */ + uint8_t dmac_filter_count; uint8_t max_mac_entries; bool dmac_filter_enable; @@ -265,6 +266,9 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr); int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, uint32_t index, uint32_t pool); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 28b49ad..3be678a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -359,6 +359,7 @@ cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, roc_nix_npc_promisc_ena_dis(nix, true); dev->dmac_filter_enable = true; eth_dev->data->promiscuous = false; + dev->dmac_filter_count++; return 0; } @@ -373,6 +374,8 @@ cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) rc = roc_nix_mac_addr_del(nix, index); if (rc) plt_err("Failed to delete mac address, rc=%d", rc); + + dev->dmac_filter_count--; } int @@ -845,3 +848,63 @@ cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, return 0; } + +int +cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct rte_ether_addr null_mac_addr = {0}; + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc, index; + uint32_t i; + + /* All configured multicast filters should be flushed first */ + for (i = 0; i < dev->max_mac_entries; i++) { + if (rte_is_multicast_ether_addr(&data->mac_addrs[i])) { + rc = roc_nix_mac_addr_del(nix, i); + if (rc) { + plt_err("Failed to flush mcast address, rc=%d", + rc); + return rc; + } + + dev->dmac_filter_count--; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&null_mac_addr, + &data->mac_addrs[i]); + } + } + + if (!mc_addr_set || !nb_mc_addr) + return 0; + + /* Check for available space */ + if (nb_mc_addr > + ((uint32_t)(dev->max_mac_entries - dev->dmac_filter_count))) { + plt_err("No space is available to add multicast filters"); + return -ENOSPC; + } + + /* Multicast addresses are to be installed */ + for (i = 0; i < nb_mc_addr; i++) { + index = roc_nix_mac_addr_add(nix, mc_addr_set[i].addr_bytes); + if (index < 0) { + plt_err("Failed to add mcast mac address, rc=%d", + index); + return index; + } + + dev->dmac_filter_count++; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&mc_addr_set[i], &data->mac_addrs[index]); + } + + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v3 62/62] net/cnxk: add marking and VLAN tagging support 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (60 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 61/62] net/cnxk: add multicast filter support Nithin Dabilpuram @ 2021-06-18 10:37 ` Nithin Dabilpuram 2021-06-21 13:41 ` [dpdk-dev] [PATCH v3 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-18 10:37 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> This patch adds support for mark, flag, VLAN pop and push flow actions. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- doc/guides/nics/features/cnxk.ini | 5 ++ drivers/net/cnxk/cn10k_rte_flow.c | 42 +++++++++ drivers/net/cnxk/cn10k_rx.c | 27 +++--- drivers/net/cnxk/cn10k_rx.h | 175 ++++++++++++++++++++++++++++++-------- drivers/net/cnxk/cn10k_rx_mseg.c | 2 +- drivers/net/cnxk/cn10k_rx_vec.c | 2 +- drivers/net/cnxk/cn9k_rte_flow.c | 42 +++++++++ drivers/net/cnxk/cn9k_rx.c | 27 +++--- drivers/net/cnxk/cn9k_rx.h | 175 ++++++++++++++++++++++++++++++-------- drivers/net/cnxk/cn9k_rx_mseg.c | 2 +- drivers/net/cnxk/cn9k_rx_vec.c | 2 +- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 3 +- drivers/net/cnxk/cnxk_rte_flow.c | 17 ++++ 14 files changed, 423 insertions(+), 99 deletions(-) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index c686ceb..337ad24 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -77,6 +77,11 @@ vxlan_gpe = Y count = Y drop = Y flag = Y +mark = Y +of_pop_vlan = Y +of_push_vlan = Y +of_set_vlan_pcp = Y +of_set_vlan_vid = Y pf = Y port_id = Y queue = Y diff --git a/drivers/net/cnxk/cn10k_rte_flow.c b/drivers/net/cnxk/cn10k_rte_flow.c index 65893cc..b04de6a 100644 --- a/drivers/net/cnxk/cn10k_rte_flow.c +++ b/drivers/net/cnxk/cn10k_rte_flow.c @@ -4,6 +4,7 @@ #include <cnxk_rte_flow.h> #include "cn10k_rte_flow.h" #include "cn10k_ethdev.h" +#include "cn10k_rx.h" struct rte_flow * cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, @@ -11,12 +12,29 @@ cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], struct rte_flow_error *error) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; struct roc_npc_flow *flow; flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); if (!flow) return NULL; + mark_actions = roc_npc_mark_actions_get(npc); + + if (mark_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn10k_eth_set_rx_function(eth_dev); + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + + if (vtag_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn10k_eth_set_rx_function(eth_dev); + } + return (struct rte_flow *)flow; } @@ -25,6 +43,30 @@ cn10k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, struct rte_flow_error *error) { struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; + + mark_actions = roc_npc_mark_actions_get(npc); + if (mark_actions) { + mark_actions = roc_npc_mark_actions_sub_return(npc, 1); + if (mark_actions == 0) { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn10k_eth_set_rx_function(eth_dev); + } + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + if (vtag_actions) { + if (flow->nix_intf == ROC_NPC_INTF_RX) { + vtag_actions = roc_npc_vtag_actions_sub_return(npc, 1); + if (vtag_actions == 0) { + dev->rx_offload_flags &= + ~NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn10k_eth_set_rx_function(eth_dev); + } + } + } return cnxk_flow_destroy(eth_dev, flow, error); } diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index c9744e2..5c956c0 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [VLAN] [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] @@ -35,25 +36,25 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R @@ -73,6 +74,6 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index c09ccdf..1cc37cb 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -15,6 +15,7 @@ #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) +#define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(5) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -179,6 +180,17 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + if (flag & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + if (rx->vtag0_gone) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + mbuf->vlan_tci = rx->vtag0_tci; + } + if (rx->vtag1_gone) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = rx->vtag1_tci; + } + } + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); @@ -273,6 +285,28 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, #if defined(RTE_ARCH_ARM64) +static __rte_always_inline uint64_t +nix_vlan_update(const uint64_t w2, uint64_t ol_flags, uint8x16_t *f) +{ + if (w2 & BIT_ULL(21) /* vtag0_gone */) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + *f = vsetq_lane_u16((uint16_t)(w2 >> 32), *f, 5); + } + + return ol_flags; +} + +static __rte_always_inline uint64_t +nix_qinq_update(const uint64_t w2, uint64_t ol_flags, struct rte_mbuf *mbuf) +{ + if (w2 & BIT_ULL(23) /* vtag1_gone */) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = (uint16_t)(w2 >> 48); + } + + return ol_flags; +} + static __rte_always_inline uint16_t cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, const uint16_t flags) @@ -397,6 +431,23 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); } + if (flags & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + uint64_t cq0_w2 = *(uint64_t *)(cq0 + CQE_SZ(0) + 16); + uint64_t cq1_w2 = *(uint64_t *)(cq0 + CQE_SZ(1) + 16); + uint64_t cq2_w2 = *(uint64_t *)(cq0 + CQE_SZ(2) + 16); + uint64_t cq3_w2 = *(uint64_t *)(cq0 + CQE_SZ(3) + 16); + + ol_flags0 = nix_vlan_update(cq0_w2, ol_flags0, &f0); + ol_flags1 = nix_vlan_update(cq1_w2, ol_flags1, &f1); + ol_flags2 = nix_vlan_update(cq2_w2, ol_flags2, &f2); + ol_flags3 = nix_vlan_update(cq3_w2, ol_flags3, &f3); + + ol_flags0 = nix_qinq_update(cq0_w2, ol_flags0, mbuf0); + ol_flags1 = nix_qinq_update(cq1_w2, ol_flags1, mbuf1); + ol_flags2 = nix_qinq_update(cq2_w2, ol_flags2, mbuf2); + ol_flags3 = nix_qinq_update(cq3_w2, ol_flags3, mbuf3); + } + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { ol_flags0 = nix_update_match_id( *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, @@ -494,43 +545,99 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F -/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +/* [RX_VLAN_F] [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ -R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts, 1, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ -R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ -R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ -R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ -R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ -R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ -R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ -R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ -R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) +R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan, 1, 0, 0, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 1, 0, 0, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 1, 0, 0, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 1, 0, 0, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 1, 0, 0, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 1, 0, 0, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 1, 0, 0, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_mark, 1, 0, 1, 0, 0, 0, RX_VLAN_F | MARK_F) \ +R(vlan_mark_rss, 1, 0, 1, 0, 0, 1, RX_VLAN_F | MARK_F | RSS_F) \ +R(vlan_mark_ptype, 1, 0, 1, 0, 1, 0, RX_VLAN_F | MARK_F | PTYPE_F)\ +R(vlan_mark_ptype_rss, 1, 0, 1, 0, 1, 1, \ + RX_VLAN_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_mark_cksum, 1, 0, 1, 1, 0, 0, RX_VLAN_F | MARK_F | CKSUM_F)\ +R(vlan_mark_cksum_rss, 1, 0, 1, 1, 0, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_mark_cksum_ptype, 1, 0, 1, 1, 1, 0, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_mark_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts, 1, 1, 0, 0, 0, 0, RX_VLAN_F | TS_F) \ +R(vlan_ts_rss, 1, 1, 0, 0, 0, 1, RX_VLAN_F | TS_F | RSS_F) \ +R(vlan_ts_ptype, 1, 1, 0, 0, 1, 0, RX_VLAN_F | TS_F | PTYPE_F) \ +R(vlan_ts_ptype_rss, 1, 1, 0, 0, 1, 1, \ + RX_VLAN_F | TS_F | PTYPE_F | RSS_F) \ +R(vlan_ts_cksum, 1, 1, 0, 1, 0, 0, RX_VLAN_F | TS_F | CKSUM_F) \ +R(vlan_ts_cksum_rss, 1, 1, 0, 1, 0, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | RSS_F) \ +R(vlan_ts_cksum_ptype, 1, 1, 0, 1, 1, 0, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark, 1, 1, 1, 0, 0, 0, RX_VLAN_F | TS_F | MARK_F) \ +R(vlan_ts_mark_rss, 1, 1, 1, 0, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | RSS_F) \ +R(vlan_ts_mark_ptype, 1, 1, 1, 0, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F) \ +R(vlan_ts_mark_ptype_rss, 1, 1, 1, 0, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark_cksum, 1, 1, 1, 1, 0, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F) \ +R(vlan_ts_mark_cksum_rss, 1, 1, 1, 1, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_ts_mark_cksum_ptype, 1, 1, 1, 1, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c index b67d21f..3340771 100644 --- a/drivers/net/cnxk/cn10k_rx_mseg.c +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c index 1330235..65ffa97 100644 --- a/drivers/net/cnxk/cn10k_rx_vec.c +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ struct rte_mbuf **rx_pkts, \ diff --git a/drivers/net/cnxk/cn9k_rte_flow.c b/drivers/net/cnxk/cn9k_rte_flow.c index 24e1b92..b94d29e 100644 --- a/drivers/net/cnxk/cn9k_rte_flow.c +++ b/drivers/net/cnxk/cn9k_rte_flow.c @@ -4,6 +4,7 @@ #include <cnxk_rte_flow.h> #include "cn9k_ethdev.h" #include "cn9k_rte_flow.h" +#include "cn9k_rx.h" struct rte_flow * cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, @@ -11,12 +12,29 @@ cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], struct rte_flow_error *error) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; struct roc_npc_flow *flow; flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); if (!flow) return NULL; + mark_actions = roc_npc_mark_actions_get(npc); + + if (mark_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn9k_eth_set_rx_function(eth_dev); + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + + if (vtag_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn9k_eth_set_rx_function(eth_dev); + } + return (struct rte_flow *)flow; } @@ -25,6 +43,30 @@ cn9k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, struct rte_flow_error *error) { struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; + + mark_actions = roc_npc_mark_actions_get(npc); + if (mark_actions) { + mark_actions = roc_npc_mark_actions_sub_return(npc, 1); + if (mark_actions == 0) { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn9k_eth_set_rx_function(eth_dev); + } + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + if (vtag_actions) { + if (flow->nix_intf == ROC_NPC_INTF_RX) { + vtag_actions = roc_npc_vtag_actions_sub_return(npc, 1); + if (vtag_actions == 0) { + dev->rx_offload_flags &= + ~NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn9k_eth_set_rx_function(eth_dev); + } + } + } return cnxk_flow_destroy(eth_dev, flow, error); } diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index a15428d..0acedd0 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] @@ -35,25 +36,25 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R @@ -73,6 +74,6 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index c5ad5db..10ef5c6 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -16,6 +16,7 @@ #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) +#define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(5) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -181,6 +182,17 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + if (flag & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + if (rx->cn9k.vtag0_gone) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + mbuf->vlan_tci = rx->cn9k.vtag0_tci; + } + if (rx->cn9k.vtag1_gone) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = rx->cn9k.vtag1_tci; + } + } + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); @@ -276,6 +288,28 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, #if defined(RTE_ARCH_ARM64) +static __rte_always_inline uint64_t +nix_vlan_update(const uint64_t w2, uint64_t ol_flags, uint8x16_t *f) +{ + if (w2 & BIT_ULL(21) /* vtag0_gone */) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + *f = vsetq_lane_u16((uint16_t)(w2 >> 32), *f, 5); + } + + return ol_flags; +} + +static __rte_always_inline uint64_t +nix_qinq_update(const uint64_t w2, uint64_t ol_flags, struct rte_mbuf *mbuf) +{ + if (w2 & BIT_ULL(23) /* vtag1_gone */) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = (uint16_t)(w2 >> 48); + } + + return ol_flags; +} + static __rte_always_inline uint16_t cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, const uint16_t flags) @@ -400,6 +434,23 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); } + if (flags & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + uint64_t cq0_w2 = *(uint64_t *)(cq0 + CQE_SZ(0) + 16); + uint64_t cq1_w2 = *(uint64_t *)(cq0 + CQE_SZ(1) + 16); + uint64_t cq2_w2 = *(uint64_t *)(cq0 + CQE_SZ(2) + 16); + uint64_t cq3_w2 = *(uint64_t *)(cq0 + CQE_SZ(3) + 16); + + ol_flags0 = nix_vlan_update(cq0_w2, ol_flags0, &f0); + ol_flags1 = nix_vlan_update(cq1_w2, ol_flags1, &f1); + ol_flags2 = nix_vlan_update(cq2_w2, ol_flags2, &f2); + ol_flags3 = nix_vlan_update(cq3_w2, ol_flags3, &f3); + + ol_flags0 = nix_qinq_update(cq0_w2, ol_flags0, mbuf0); + ol_flags1 = nix_qinq_update(cq1_w2, ol_flags1, mbuf1); + ol_flags2 = nix_qinq_update(cq2_w2, ol_flags2, mbuf2); + ol_flags3 = nix_qinq_update(cq3_w2, ol_flags3, mbuf3); + } + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { ol_flags0 = nix_update_match_id( *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, @@ -496,43 +547,99 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F -/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +/* [RX_VLAN_F] [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ -R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts, 1, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ -R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ -R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ -R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ -R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ -R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ -R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ -R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ -R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) +R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan, 1, 0, 0, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 1, 0, 0, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 1, 0, 0, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 1, 0, 0, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 1, 0, 0, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 1, 0, 0, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 1, 0, 0, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_mark, 1, 0, 1, 0, 0, 0, RX_VLAN_F | MARK_F) \ +R(vlan_mark_rss, 1, 0, 1, 0, 0, 1, RX_VLAN_F | MARK_F | RSS_F) \ +R(vlan_mark_ptype, 1, 0, 1, 0, 1, 0, RX_VLAN_F | MARK_F | PTYPE_F)\ +R(vlan_mark_ptype_rss, 1, 0, 1, 0, 1, 1, \ + RX_VLAN_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_mark_cksum, 1, 0, 1, 1, 0, 0, RX_VLAN_F | MARK_F | CKSUM_F)\ +R(vlan_mark_cksum_rss, 1, 0, 1, 1, 0, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_mark_cksum_ptype, 1, 0, 1, 1, 1, 0, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_mark_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts, 1, 1, 0, 0, 0, 0, RX_VLAN_F | TS_F) \ +R(vlan_ts_rss, 1, 1, 0, 0, 0, 1, RX_VLAN_F | TS_F | RSS_F) \ +R(vlan_ts_ptype, 1, 1, 0, 0, 1, 0, RX_VLAN_F | TS_F | PTYPE_F) \ +R(vlan_ts_ptype_rss, 1, 1, 0, 0, 1, 1, \ + RX_VLAN_F | TS_F | PTYPE_F | RSS_F) \ +R(vlan_ts_cksum, 1, 1, 0, 1, 0, 0, RX_VLAN_F | TS_F | CKSUM_F) \ +R(vlan_ts_cksum_rss, 1, 1, 0, 1, 0, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | RSS_F) \ +R(vlan_ts_cksum_ptype, 1, 1, 0, 1, 1, 0, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark, 1, 1, 1, 0, 0, 0, RX_VLAN_F | TS_F | MARK_F) \ +R(vlan_ts_mark_rss, 1, 1, 1, 0, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | RSS_F) \ +R(vlan_ts_mark_ptype, 1, 1, 1, 0, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F) \ +R(vlan_ts_mark_ptype_rss, 1, 1, 1, 0, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark_cksum, 1, 1, 1, 1, 0, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F) \ +R(vlan_ts_mark_cksum_rss, 1, 1, 1, 1, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_ts_mark_cksum_ptype, 1, 1, 1, 1, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c index 3b26962..d7e19b1 100644 --- a/drivers/net/cnxk/cn9k_rx_mseg.c +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c index b19c7f3..e61c222 100644 --- a/drivers/net/cnxk/cn9k_rx_vec.c +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1edea35..93d7ec7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1287,6 +1287,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) /* Initialize base roc nix */ nix->pci_dev = pci_dev; + nix->hw_vlan_ins = true; rc = roc_nix_dev_init(nix); if (rc) { plt_err("Failed to initialize roc nix rc=%d", rc); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c07e988..67b1f42 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -76,7 +76,8 @@ (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ - DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP) + DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP | \ + DEV_RX_OFFLOAD_VLAN_STRIP) #define RSS_IPV4_ENABLE \ (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index c61618a..213125b 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -178,6 +178,23 @@ cnxk_map_actions(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, case RTE_FLOW_ACTION_TYPE_SECURITY: in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; break; + case RTE_FLOW_ACTION_TYPE_OF_POP_VLAN: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VLAN_STRIP; + break; + case RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VLAN_INSERT; + in_actions[i].conf = actions->conf; + break; + case RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN: + in_actions[i].type = + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT; + in_actions[i].conf = actions->conf; + break; + case RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP: + in_actions[i].type = + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT; + in_actions[i].conf = actions->conf; + break; default: plt_npc_dbg("Action is not supported = %d", actions->type); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v3 00/62] Marvell CNXK Ethdev Driver 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram ` (61 preceding siblings ...) 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 62/62] net/cnxk: add marking and VLAN tagging support Nithin Dabilpuram @ 2021-06-21 13:41 ` Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-06-21 13:41 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra On Fri, Jun 18, 2021 at 4:09 PM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > driver when code is ready and 'net/octeontx2' will be deprecated. Looks good to me in general, a couple of final comments 1) Build issue with RHEL 7 (gcc 4.8) http://mails.dpdk.org/archives/test-report/2021-June/199620.html 2) Shorten git subject messages( for explicit "add" etc) Example: From: net/cnxk: add DMAC filter support to: net/cnxk: support DMAC filter 3) Add Cc: stable@dpdk.org for following patch common/cnxk: fix batch alloc completion poll logic 4) Fix following camel case CHECK:CAMELCASE: Avoid CamelCase: <roc_model_is_cn96_Ax> #92: FILE: drivers/net/cnxk/cnxk_ethdev.c:122: + if (roc_model_is_cn96_Ax() && 5) Replace INTERNAL in drivers/net/cnxk/version.map 6) Add [rte_flow items] and [rte_flow actions] in doc/guides/nics/features/cnxk_vec.ini and doc/guides/nics/features/cnxk_vf.ini ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram ` (45 preceding siblings ...) 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram ` (62 more replies) 46 siblings, 63 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' driver. In future, CN9K a.k.a octeontx2 will also be supported by same driver when code is ready and 'net/octeontx2' will be deprecated. Harman Kalra (1): common/cnxk: allocate lmt region in userspace Jerin Jacob (7): common/cnxk: fix batch alloc completion poll logic net/cnxk: add Rx burst for cn9k net/cnxk: add Rx vector version for cn9k net/cnxk: add Tx burst for cn9k net/cnxk: add Rx burst for cn10k net/cnxk: add Rx vector version for cn10k net/cnxk: add Tx burst for cn10k Kiran Kumar K (2): net/cnxk: add support to configure npc net/cnxk: support initial version of rte flow Nithin Dabilpuram (18): common/cnxk: change model API to not use camel case net/cnxk: add build infra and common probe net/cnxk: add platform specific probe and remove net/cnxk: add common devargs parsing function net/cnxk: support common dev infos get net/cnxk: add device configuration operation net/cnxk: support link status update net/cnxk: add Rx queue setup and release net/cnxk: add Tx queue setup and release net/cnxk: support packet type net/cnxk: support queue start and stop net/cnxk: add Rx multi-segmented version for cn9k net/cnxk: add Tx multi-segment version for cn9k net/cnxk: add Tx vector version for cn9k net/cnxk: add Rx multi-segment version for cn10k net/cnxk: add Tx multi-segment version for cn10k net/cnxk: add Tx vector version for cn10k net/cnxk: add device start and stop operations Satha Rao (8): common/cnxk: add support to lock NIX RQ contexts common/cnxk: add provision to enable RED on RQ net/cnxk: add port/queue stats net/cnxk: add xstats apis net/cnxk: add rxq/txq info get operations net/cnxk: add ethdev firmware version get net/cnxk: add get register operation net/cnxk: added RETA and RSS hash operations Satheesh Paul (6): common/cnxk: add support to dump flow entries common/cnxk: support for mark and flag flow actions common/cnxk: support for VLAN push and pop flow actions net/cnxk: add flow ops get operation net/cnxk: support for RSS in rte flow net/cnxk: support marking and VLAN tagging Sunil Kumar Kori (20): net/cnxk: add MAC address set ops net/cnxk: add MTU set device operation net/cnxk: add promiscuous mode enable and disable net/cnxk: support DMAC filter net/cnxk: add all multicast enable/disable ethops net/cnxk: add Rx/Tx burst mode get ops net/cnxk: add flow ctrl set/get ops net/cnxk: add link up/down operations net/cnxk: add EEPROM module info get operations net/cnxk: add Rx queue interrupt enable/disable ops net/cnxk: add validation API for mempool ops net/cnxk: add device close and reset operations net/cnxk: add pending Tx mbuf cleanup operation net/cnxk: register callback to get PTP status net/cnxk: support base PTP timesync net/cnxk: add timesync enable/disable operations net/cnxk: add Rx/Tx timestamp read operations net/cnxk: add time read/write/adjust operations net/cnxk: add read clock operation net/cnxk: support multicast filter -- v4: - Fixed build issue with gcc 4.8 - Shortened subject lines of few commits - Removed camel case for model API - Updated rte_flow features in cnxk_vec.ini and cnxk_vf.ini - Added CC stable to "fix batch alloc.." patch - Squashed cn98xx flow create related common patch to VLAN push and pop flow actions patch. - Changed INTERNAL to DPDK_21 in version.map v3: - Updated release notes - Removed RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag and add support for queue stats in xstats - Fixed issue with LSO format indices - Removed mbox sync changes patch from this series - Fixed documentation issues - Removed repetitive code in fast path SIMD - Optimize cn10k LMTST logic - Make rte_flow_create implementation specific to handle VLAN Stripping and MARK actions/offloads - Use rte_atomic_thread_fence() instead of rte_rmb() - Handle other comments from Jerin. - Merged rte flow dump API patch to flow ops get patch - Added marking and vlan tagging support. - Fixed some checkpatch and git check log issues. v2: - Fixed issue with flow validate and flow create for 98xx - Fixed issue batch alloc logic - Fix lmtline allocation to be cached - Sync Inline IPSec Rx mbox with kernel - Add support for mark and flag flow actions - Add reta key and hash update ops - Added PTP and multicast filter support MAINTAINERS | 5 +- doc/guides/nics/cnxk.rst | 232 +++++ doc/guides/nics/features/cnxk.ini | 90 ++ doc/guides/nics/features/cnxk_vec.ini | 86 ++ doc/guides/nics/features/cnxk_vf.ini | 82 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + doc/guides/rel_notes/release_21_08.rst | 5 + drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 +- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 3 + drivers/common/cnxk/roc_model.h | 12 +- drivers/common/cnxk/roc_nix.h | 39 +- drivers/common/cnxk/roc_nix_queue.c | 52 + drivers/common/cnxk/roc_nix_rss.c | 51 +- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +- drivers/common/cnxk/roc_npa.c | 10 +- drivers/common/cnxk/roc_npa.h | 35 +- drivers/common/cnxk/roc_npc.c | 296 +++++- drivers/common/cnxk/roc_npc.h | 39 +- drivers/common/cnxk/roc_npc_mcam.c | 2 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++ drivers/common/cnxk/roc_npc_priv.h | 3 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/roc_platform.h | 13 + drivers/common/cnxk/version.map | 7 + drivers/net/cnxk/cn10k_ethdev.c | 551 +++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 41 + drivers/net/cnxk/cn10k_rte_flow.c | 72 ++ drivers/net/cnxk/cn10k_rte_flow.h | 17 + drivers/net/cnxk/cn10k_rx.c | 79 ++ drivers/net/cnxk/cn10k_rx.h | 653 +++++++++++++ drivers/net/cnxk/cn10k_rx_mseg.c | 17 + drivers/net/cnxk/cn10k_rx_vec.c | 22 + drivers/net/cnxk/cn10k_tx.c | 82 ++ drivers/net/cnxk/cn10k_tx.h | 1605 +++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 + drivers/net/cnxk/cn10k_tx_vec.c | 26 + drivers/net/cnxk/cn9k_ethdev.c | 574 +++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 39 + drivers/net/cnxk/cn9k_rte_flow.c | 72 ++ drivers/net/cnxk/cn9k_rte_flow.h | 17 + drivers/net/cnxk/cn9k_rx.c | 79 ++ drivers/net/cnxk/cn9k_rx.h | 655 +++++++++++++ drivers/net/cnxk/cn9k_rx_mseg.c | 17 + drivers/net/cnxk/cn9k_rx_vec.c | 20 + drivers/net/cnxk/cn9k_tx.c | 81 ++ drivers/net/cnxk/cn9k_tx.h | 1436 +++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 + drivers/net/cnxk/cn9k_tx_vec.c | 26 + drivers/net/cnxk/cnxk_ethdev.c | 1540 +++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 495 ++++++++++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 173 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 912 ++++++++++++++++++ drivers/net/cnxk/cnxk_link.c | 113 +++ drivers/net/cnxk/cnxk_lookup.c | 326 +++++++ drivers/net/cnxk/cnxk_ptp.c | 287 ++++++ drivers/net/cnxk/cnxk_rte_flow.c | 433 +++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 27 + drivers/net/cnxk/cnxk_stats.c | 320 ++++++ drivers/net/cnxk/meson.build | 48 + drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 66 files changed, 12678 insertions(+), 102 deletions(-) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn10k_rte_flow.c create mode 100644 drivers/net/cnxk/cn10k_rte_flow.h create mode 100644 drivers/net/cnxk/cn10k_rx.c create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c create mode 100644 drivers/net/cnxk/cn10k_tx.c create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_rte_flow.c create mode 100644 drivers/net/cnxk/cn9k_rte_flow.h create mode 100644 drivers/net/cnxk/cn9k_rx.c create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c create mode 100644 drivers/net/cnxk/cn9k_tx.c create mode 100644 drivers/net/cnxk/cn9k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c create mode 100644 drivers/net/cnxk/cnxk_link.c create mode 100644 drivers/net/cnxk/cnxk_lookup.c create mode 100644 drivers/net/cnxk/cnxk_ptp.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h create mode 100644 drivers/net/cnxk/cnxk_stats.c create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 01/62] common/cnxk: add support to lock NIX RQ contexts 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram ` (61 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch will consider device argument to lock rss table in NIX. This patch also adds few misc fixes such as disabling NIX Tx vlan insertion conf in SMQ, enabling SSO in NIX Tx SQ for Tx completions and TM related stats API. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 31 ++++++++++-- drivers/common/cnxk/roc_nix_queue.c | 2 + drivers/common/cnxk/roc_nix_rss.c | 51 ++++++++++++++++++-- drivers/common/cnxk/roc_nix_tm_utils.c | 86 +++++++++++++++++++++++++++++++++- drivers/common/cnxk/roc_platform.h | 2 + drivers/common/cnxk/version.map | 1 + 6 files changed, 163 insertions(+), 10 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index b39f461..6d9ac10 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -85,10 +85,11 @@ struct roc_nix_eeprom_info { #define ROC_NIX_LF_RX_CFG_LEN_OL3 BIT_ULL(41) /* Group 0 will be used for RSS, 1 -7 will be used for npc_flow RSS action*/ -#define ROC_NIX_RSS_GROUP_DEFAULT 0 -#define ROC_NIX_RSS_GRPS 8 -#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 -#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_GROUP_DEFAULT 0 +#define ROC_NIX_RSS_GRPS 8 +#define ROC_NIX_RSS_RETA_MAX ROC_NIX_RSS_RETA_SZ_256 +#define ROC_NIX_RSS_KEY_LEN 48 /* 352 Bits */ +#define ROC_NIX_RSS_MCAM_IDX_DEFAULT (-1) #define ROC_NIX_DEFAULT_HW_FRS 1514 @@ -184,6 +185,7 @@ struct roc_nix_sq { enum roc_nix_sq_max_sqe_sz max_sqe_sz; uint32_t nb_desc; uint16_t qid; + bool sso_ena; /* End of Input parameters */ uint16_t sqes_per_sqb_log2; struct roc_nix *roc_nix; @@ -241,6 +243,8 @@ struct roc_nix { uint16_t max_sqb_count; enum roc_nix_rss_reta_sz reta_sz; bool enable_loop; + bool hw_vlan_ins; + uint8_t lock_rx_ctx; /* End of input parameters */ /* LMT line base for "Per Core Tx LMT line" mode*/ uintptr_t lmt_base; @@ -371,6 +375,22 @@ struct roc_nix_tm_shaper_profile { void (*free_fn)(void *profile); }; +enum roc_nix_tm_node_stats_type { + ROC_NIX_TM_NODE_PKTS_DROPPED, + ROC_NIX_TM_NODE_BYTES_DROPPED, + ROC_NIX_TM_NODE_GREEN_PKTS, + ROC_NIX_TM_NODE_GREEN_BYTES, + ROC_NIX_TM_NODE_YELLOW_PKTS, + ROC_NIX_TM_NODE_YELLOW_BYTES, + ROC_NIX_TM_NODE_RED_PKTS, + ROC_NIX_TM_NODE_RED_BYTES, + ROC_NIX_TM_NODE_STATS_MAX, +}; + +struct roc_nix_tm_node_stats { + uint64_t stats[ROC_NIX_TM_NODE_STATS_MAX]; +}; + int __roc_api roc_nix_tm_node_add(struct roc_nix *roc_nix, struct roc_nix_tm_node *roc_node); int __roc_api roc_nix_tm_node_delete(struct roc_nix *roc_nix, uint32_t node_id, @@ -408,6 +428,9 @@ roc_nix_tm_shaper_profile_get(struct roc_nix *roc_nix, uint32_t profile_id); struct roc_nix_tm_shaper_profile *__roc_api roc_nix_tm_shaper_profile_next( struct roc_nix *roc_nix, struct roc_nix_tm_shaper_profile *__prev); +int __roc_api roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, + uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *stats); /* * TM ratelimit tree API. */ diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index fbf7efa..1c62aa2 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -582,6 +582,7 @@ sq_cn9k_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); @@ -679,6 +680,7 @@ sq_init(struct nix *nix, struct roc_nix_sq *sq, uint32_t rr_quantum, aq->sq.default_chan = nix->tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; aq->sq.ena = 1; + aq->sq.sso_ena = !!sq->sso_ena; if (aq->sq.max_sqe_size == NIX_MAXSQESZ_W8) aq->sq.sqe_stype = NIX_STYPE_STP; aq->sq.sqb_aura = roc_npa_aura_handle_to_aura(sq->aura_handle); diff --git a/drivers/common/cnxk/roc_nix_rss.c b/drivers/common/cnxk/roc_nix_rss.c index 2d7b84a..7de69aa 100644 --- a/drivers/common/cnxk/roc_nix_rss.c +++ b/drivers/common/cnxk/roc_nix_rss.c @@ -52,7 +52,7 @@ roc_nix_rss_key_get(struct roc_nix *roc_nix, uint8_t key[ROC_NIX_RSS_KEY_LEN]) static int nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_aq_enq_req *req; @@ -77,6 +77,27 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -88,7 +109,7 @@ nix_cn9k_rss_reta_set(struct nix *nix, uint8_t group, static int nix_rss_reta_set(struct nix *nix, uint8_t group, - uint16_t reta[ROC_NIX_RSS_RETA_MAX]) + uint16_t reta[ROC_NIX_RSS_RETA_MAX], uint8_t lock_rx_ctx) { struct mbox *mbox = (&nix->dev)->mbox; struct nix_cn10k_aq_enq_req *req; @@ -113,6 +134,27 @@ nix_rss_reta_set(struct nix *nix, uint8_t group, req->qidx = (group * nix->reta_sz) + idx; req->ctype = NIX_AQ_CTYPE_RSS; req->op = NIX_AQ_INSTOP_INIT; + + if (!lock_rx_ctx) + continue; + + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) { + /* The shared memory buffer can be full. + * Flush it and retry + */ + rc = mbox_process(mbox); + if (rc < 0) + return rc; + req = mbox_alloc_msg_nix_cn10k_aq_enq(mbox); + if (!req) + return NIX_ERR_NO_MEM; + } + req->rss.rq = reta[idx]; + /* Fill AQ info */ + req->qidx = (group * nix->reta_sz) + idx; + req->ctype = NIX_AQ_CTYPE_RSS; + req->op = NIX_AQ_INSTOP_LOCK; } rc = mbox_process(mbox); @@ -133,9 +175,10 @@ roc_nix_rss_reta_set(struct roc_nix *roc_nix, uint8_t group, return NIX_ERR_PARAM; if (roc_model_is_cn9k()) - rc = nix_cn9k_rss_reta_set(nix, group, reta); + rc = nix_cn9k_rss_reta_set(nix, group, reta, + roc_nix->lock_rx_ctx); else - rc = nix_rss_reta_set(nix, group, reta); + rc = nix_rss_reta_set(nix, group, reta, roc_nix->lock_rx_ctx); if (rc) return rc; diff --git a/drivers/common/cnxk/roc_nix_tm_utils.c b/drivers/common/cnxk/roc_nix_tm_utils.c index 1d7dd68..6b9543e 100644 --- a/drivers/common/cnxk/roc_nix_tm_utils.c +++ b/drivers/common/cnxk/roc_nix_tm_utils.c @@ -409,6 +409,7 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, volatile uint64_t *reg, volatile uint64_t *regval, volatile uint64_t *regval_mask) { + struct roc_nix *roc_nix = nix_priv_to_roc_nix(nix); uint8_t k = 0, hw_lvl, parent_lvl; uint64_t parent = 0, child = 0; enum roc_nix_tm_tree tree; @@ -454,8 +455,11 @@ nix_tm_topology_reg_prep(struct nix *nix, struct nix_tm_node *node, reg[k] = NIX_AF_SMQX_CFG(schq); regval[k] = (BIT_ULL(50) | NIX_MIN_HW_FRS | ((nix->mtu & 0xFFFF) << 8)); - regval_mask[k] = - ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | GENMASK_ULL(23, 8)); + /* Maximum Vtag insertion size as a multiple of four bytes */ + if (roc_nix->hw_vlan_ins) + regval[k] |= (0x2ULL << 36); + regval_mask[k] = ~(BIT_ULL(50) | GENMASK_ULL(6, 0) | + GENMASK_ULL(23, 8) | GENMASK_ULL(38, 36)); k++; /* Parent and schedule conf */ @@ -1000,3 +1004,81 @@ nix_tm_shaper_profile_free(struct nix_tm_shaper_profile *profile) (profile->free_fn)(profile); } + +int +roc_nix_tm_node_stats_get(struct roc_nix *roc_nix, uint32_t node_id, bool clear, + struct roc_nix_tm_node_stats *n_stats) +{ + struct nix *nix = roc_nix_to_nix_priv(roc_nix); + struct mbox *mbox = (&nix->dev)->mbox; + struct nix_txschq_config *req, *rsp; + struct nix_tm_node *node; + uint32_t schq; + int rc, i; + + node = nix_tm_node_search(nix, node_id, ROC_NIX_TM_USER); + if (!node) + return NIX_ERR_TM_INVALID_NODE; + + if (node->hw_lvl != NIX_TXSCH_LVL_TL1) + return NIX_ERR_OP_NOTSUP; + + schq = node->hw_id; + /* Skip fetch if not requested */ + if (!n_stats) + goto clear_stats; + + memset(n_stats, 0, sizeof(struct roc_nix_tm_node_stats)); + /* Check if node has HW resource */ + if (!(node->flags & NIX_TM_NODE_HWRES)) + return 0; + + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->read = 1; + req->lvl = NIX_TXSCH_LVL_TL1; + + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + rc = mbox_process_msg(mbox, (void **)&rsp); + if (rc) + return rc; + + /* Return stats */ + n_stats->stats[ROC_NIX_TM_NODE_PKTS_DROPPED] = rsp->regval[0]; + n_stats->stats[ROC_NIX_TM_NODE_BYTES_DROPPED] = rsp->regval[1]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_PKTS] = rsp->regval[2]; + n_stats->stats[ROC_NIX_TM_NODE_GREEN_BYTES] = rsp->regval[3]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_PKTS] = rsp->regval[4]; + n_stats->stats[ROC_NIX_TM_NODE_YELLOW_BYTES] = rsp->regval[5]; + n_stats->stats[ROC_NIX_TM_NODE_RED_PKTS] = rsp->regval[6]; + n_stats->stats[ROC_NIX_TM_NODE_RED_BYTES] = rsp->regval[7]; + +clear_stats: + if (!clear) + return 0; + + /* Clear all the stats */ + req = mbox_alloc_msg_nix_txschq_cfg(mbox); + req->lvl = NIX_TXSCH_LVL_TL1; + i = 0; + req->reg[i++] = NIX_AF_TL1X_DROPPED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_DROPPED_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_GREEN_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_YELLOW_BYTES(schq); + req->reg[i++] = NIX_AF_TL1X_RED_PACKETS(schq); + req->reg[i++] = NIX_AF_TL1X_RED_BYTES(schq); + req->num_regs = i; + + return mbox_process_msg(mbox, (void **)&rsp); +} diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 7864fa4..911ae15 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -127,6 +127,8 @@ #define plt_memzone_reserve_cache_align(name, sz) \ rte_memzone_reserve_aligned(name, sz, 0, 0, RTE_CACHE_LINE_SIZE) #define plt_memzone_free rte_memzone_free +#define plt_memzone_reserve_aligned(name, len, flags, align) \ + rte_memzone_reserve_aligned((name), (len), 0, (flags), (align)) #define plt_tsc_hz rte_get_tsc_hz #define plt_delay_ms rte_delay_ms diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 8e67c83..c39d76f 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -123,6 +123,7 @@ INTERNAL { roc_nix_tm_node_parent_update; roc_nix_tm_node_pkt_mode_update; roc_nix_tm_node_shaper_update; + roc_nix_tm_node_stats_get; roc_nix_tm_node_suspend_resume; roc_nix_tm_prealloc_res; roc_nix_tm_rlimit_sq; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 02/62] common/cnxk: fix batch alloc completion poll logic 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram ` (60 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, stable From: Jerin Jacob <jerinj@marvell.com> The instruction generation was not correct due to fact that volatile suppose to use with ccode variable as well. Change the logic to use gcc atomic builtin to simplify and avoid explicit volatile from the code. Fixes: 81af26789316 ("common/cnxk: support NPA batch alloc/free") Cc: stable@dpdk.org Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com> --- drivers/common/cnxk/roc_npa.c | 2 +- drivers/common/cnxk/roc_npa.h | 30 +++++++++++++++--------------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index f1e03b7..5ba6e81 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -236,7 +236,7 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Block size should be cache line aligned and in range of 128B-128KB */ if (block_size % ROC_ALIGN || block_size < 128 || - block_size > 128 * 1024) + block_size > ROC_NPA_MAX_BLOCK_SZ) return NPA_ERR_INVALID_BLOCK_SZ; pos = 0; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 89f5c6f..59d6223 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -8,6 +8,7 @@ #define ROC_AURA_ID_MASK (BIT_ULL(16) - 1) #define ROC_AURA_OP_LIMIT_MASK (BIT_ULL(36) - 1) +#define ROC_NPA_MAX_BLOCK_SZ (128 * 1024) #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 @@ -219,6 +220,17 @@ roc_npa_aura_batch_alloc_issue(uint64_t aura_handle, uint64_t *buf, return 0; } +static inline void +roc_npa_batch_alloc_wait(uint64_t *cache_line) +{ + /* Batch alloc status code is updated in bits [5:6] of the first word + * of the 128 byte cache line. + */ + while (((__atomic_load_n(cache_line, __ATOMIC_RELAXED) >> 5) & 0x3) == + ALLOC_CCODE_INVAL) + ; +} + static inline unsigned int roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) { @@ -231,17 +243,10 @@ roc_npa_aura_batch_alloc_count(uint64_t *aligned_buf, unsigned int num) /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int ccode; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); - + roc_npa_batch_alloc_wait(&aligned_buf[i]); count += status->count; } @@ -261,16 +266,11 @@ roc_npa_aura_batch_alloc_extract(uint64_t *buf, uint64_t *aligned_buf, /* Check each ROC cache line one by one */ for (i = 0; i < num; i += (ROC_ALIGN >> 3)) { struct npa_batch_alloc_status_s *status; - int line_count, ccode; + int line_count; status = (struct npa_batch_alloc_status_s *)&aligned_buf[i]; - /* Status is updated in first 7 bits of each 128 byte cache - * line. Wait until the status gets updated. - */ - do { - ccode = (volatile int)status->ccode; - } while (ccode == ALLOC_CCODE_INVAL); + roc_npa_batch_alloc_wait(&aligned_buf[i]); line_count = status->count; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 03/62] common/cnxk: add support to dump flow entries 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram ` (59 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add NPC support API to dump created flow entries. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/hw/npc.h | 2 + drivers/common/cnxk/meson.build | 1 + drivers/common/cnxk/roc_npc.c | 20 ++ drivers/common/cnxk/roc_npc.h | 12 +- drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npc_priv.h | 2 +- drivers/common/cnxk/roc_npc_utils.c | 4 + drivers/common/cnxk/version.map | 2 + 8 files changed, 652 insertions(+), 2 deletions(-) create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c diff --git a/drivers/common/cnxk/hw/npc.h b/drivers/common/cnxk/hw/npc.h index e0f06bf..68c5037 100644 --- a/drivers/common/cnxk/hw/npc.h +++ b/drivers/common/cnxk/hw/npc.h @@ -193,6 +193,7 @@ enum npc_kpu_lb_ltype { NPC_LT_LB_EXDSA, NPC_LT_LB_EXDSA_VLAN, NPC_LT_LB_FDSA, + NPC_LT_LB_VLAN_EXDSA, NPC_LT_LB_CUSTOM0 = 0xE, NPC_LT_LB_CUSTOM1 = 0xF, }; @@ -208,6 +209,7 @@ enum npc_kpu_lc_ltype { NPC_LT_LC_MPLS, NPC_LT_LC_NSH, NPC_LT_LC_FCOE, + NPC_LT_LC_NGIO, NPC_LT_LC_CUSTOM0 = 0xE, NPC_LT_LC_CUSTOM1 = 0xF, }; diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index 178bce7..e7ab79f 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -37,6 +37,7 @@ sources = files( 'roc_npa_irq.c', 'roc_npc.c', 'roc_npc_mcam.c', + 'roc_npc_mcam_dump.c', 'roc_npc_parse.c', 'roc_npc_utils.c', 'roc_platform.c', diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index abaef77..81c7fd9 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -870,3 +870,23 @@ roc_npc_flow_destroy(struct roc_npc *roc_npc, struct roc_npc_flow *flow) plt_free(flow); return 0; } + +void +roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + struct roc_npc_flow *flow_iter; + struct npc_flow_list *list; + uint32_t max_prio, i; + + max_prio = npc->flow_max_priority; + + for (i = 0; i < max_prio; i++) { + list = &npc->flow_list[i]; + + /* List in ascending order of mcam entries */ + TAILQ_FOREACH(flow_iter, list, next) { + roc_npc_flow_mcam_dump(file, roc_npc, flow_iter); + } + } +} diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 223c4ba..115bcd5 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -90,6 +90,11 @@ struct roc_npc_attr { uint32_t reserved : 30; /**< Reserved, must be zero. */ }; +struct roc_npc_flow_dump_data { + uint8_t lid; + uint16_t ltype; +}; + struct roc_npc_flow { uint8_t nix_intf; uint8_t enable; @@ -102,6 +107,9 @@ struct roc_npc_flow { uint64_t mcam_mask[ROC_NPC_MAX_MCAM_WIDTH_DWORDS]; uint64_t npc_action; uint64_t vtag_action; +#define ROC_NPC_MAX_FLOW_PATTERNS 32 + struct roc_npc_flow_dump_data dump_data[ROC_NPC_MAX_FLOW_PATTERNS]; + uint16_t num_patterns; TAILQ_ENTRY(roc_npc_flow) next; }; @@ -185,5 +193,7 @@ int __roc_api roc_npc_mcam_clear_counter(struct roc_npc *roc_npc, uint32_t ctr_id); int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); - +void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); +void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *mcam); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/roc_npc_mcam_dump.c b/drivers/common/cnxk/roc_npc_mcam_dump.c new file mode 100644 index 0000000..19b4901 --- /dev/null +++ b/drivers/common/cnxk/roc_npc_mcam_dump.c @@ -0,0 +1,611 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#define NPC_MAX_FIELD_NAME_SIZE 80 +#define NPC_RX_ACTIONOP_MASK GENMASK(3, 0) +#define NPC_RX_ACTION_PFFUNC_MASK GENMASK(19, 4) +#define NPC_RX_ACTION_INDEX_MASK GENMASK(39, 20) +#define NPC_RX_ACTION_MATCH_MASK GENMASK(55, 40) +#define NPC_RX_ACTION_FLOWKEY_MASK GENMASK(60, 56) + +#define NPC_TX_ACTION_INDEX_MASK GENMASK(31, 12) +#define NPC_TX_ACTION_MATCH_MASK GENMASK(47, 32) + +#define NIX_RX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_RX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_RX_VTAGACT_VTAG0_TYPE_MASK GENMASK(14, 12) +#define NIX_RX_VTAGACT_VTAG0_VALID_MASK BIT_ULL(15) + +#define NIX_RX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_RX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_RX_VTAGACT_VTAG1_TYPE_MASK GENMASK(46, 44) +#define NIX_RX_VTAGACT_VTAG1_VALID_MASK BIT_ULL(47) + +#define NIX_TX_VTAGACT_VTAG0_RELPTR_MASK GENMASK(7, 0) +#define NIX_TX_VTAGACT_VTAG0_LID_MASK GENMASK(10, 8) +#define NIX_TX_VTAGACT_VTAG0_OP_MASK GENMASK(13, 12) +#define NIX_TX_VTAGACT_VTAG0_DEF_MASK GENMASK(25, 16) + +#define NIX_TX_VTAGACT_VTAG1_RELPTR_MASK GENMASK(39, 32) +#define NIX_TX_VTAGACT_VTAG1_LID_MASK GENMASK(42, 40) +#define NIX_TX_VTAGACT_VTAG1_OP_MASK GENMASK(45, 44) +#define NIX_TX_VTAGACT_VTAG1_DEF_MASK GENMASK(57, 48) + +struct npc_rx_parse_nibble_s { + uint16_t chan : 3; + uint16_t errlev : 1; + uint16_t errcode : 2; + uint16_t l2l3bm : 1; + uint16_t laflags : 2; + uint16_t latype : 1; + uint16_t lbflags : 2; + uint16_t lbtype : 1; + uint16_t lcflags : 2; + uint16_t lctype : 1; + uint16_t ldflags : 2; + uint16_t ldtype : 1; + uint16_t leflags : 2; + uint16_t letype : 1; + uint16_t lfflags : 2; + uint16_t lftype : 1; + uint16_t lgflags : 2; + uint16_t lgtype : 1; + uint16_t lhflags : 2; + uint16_t lhtype : 1; +} __plt_packed; + +static const char *const intf_str[] = { + "NIX-RX", + "NIX-TX", +}; + +static const char *const ltype_str[NPC_MAX_LID][NPC_MAX_LT] = { + [NPC_LID_LA][0] = "NONE", + [NPC_LID_LA][NPC_LT_LA_ETHER] = "LA_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_ETHER] = "LA_IH_NIX_ETHER", + [NPC_LID_LA][NPC_LT_LA_HIGIG2_ETHER] = "LA_HIGIG2_ETHER", + [NPC_LID_LA][NPC_LT_LA_IH_NIX_HIGIG2_ETHER] = "LA_IH_NIX_HIGIG2_ETHER", + [NPC_LID_LB][0] = "NONE", + [NPC_LID_LB][NPC_LT_LB_CTAG] = "LB_CTAG", + [NPC_LID_LB][NPC_LT_LB_STAG_QINQ] = "LB_STAG_QINQ", + [NPC_LID_LB][NPC_LT_LB_ETAG] = "LB_ETAG", + [NPC_LID_LB][NPC_LT_LB_EXDSA] = "LB_EXDSA", + [NPC_LID_LB][NPC_LT_LB_VLAN_EXDSA] = "LB_VLAN_EXDSA", + [NPC_LID_LC][0] = "NONE", + [NPC_LID_LC][NPC_LT_LC_IP] = "LC_IP", + [NPC_LID_LC][NPC_LT_LC_IP6] = "LC_IP6", + [NPC_LID_LC][NPC_LT_LC_ARP] = "LC_ARP", + [NPC_LID_LC][NPC_LT_LC_IP6_EXT] = "LC_IP6_EXT", + [NPC_LID_LC][NPC_LT_LC_NGIO] = "LC_NGIO", + [NPC_LID_LD][0] = "NONE", + [NPC_LID_LD][NPC_LT_LD_ICMP] = "LD_ICMP", + [NPC_LID_LD][NPC_LT_LD_ICMP6] = "LD_ICMP6", + [NPC_LID_LD][NPC_LT_LD_UDP] = "LD_UDP", + [NPC_LID_LD][NPC_LT_LD_TCP] = "LD_TCP", + [NPC_LID_LD][NPC_LT_LD_SCTP] = "LD_SCTP", + [NPC_LID_LD][NPC_LT_LD_GRE] = "LD_GRE", + [NPC_LID_LD][NPC_LT_LD_NVGRE] = "LD_NVGRE", + [NPC_LID_LE][0] = "NONE", + [NPC_LID_LE][NPC_LT_LE_VXLAN] = "LE_VXLAN", + [NPC_LID_LE][NPC_LT_LE_ESP] = "LE_ESP", + [NPC_LID_LE][NPC_LT_LE_GTPC] = "LE_GTPC", + [NPC_LID_LE][NPC_LT_LE_GTPU] = "LE_GTPU", + [NPC_LID_LE][NPC_LT_LE_GENEVE] = "LE_GENEVE", + [NPC_LID_LE][NPC_LT_LE_VXLANGPE] = "LE_VXLANGPE", + [NPC_LID_LF][0] = "NONE", + [NPC_LID_LF][NPC_LT_LF_TU_ETHER] = "LF_TU_ETHER", + [NPC_LID_LG][0] = "NONE", + [NPC_LID_LG][NPC_LT_LG_TU_IP] = "LG_TU_IP", + [NPC_LID_LG][NPC_LT_LG_TU_IP6] = "LG_TU_IP6", + [NPC_LID_LH][0] = "NONE", + [NPC_LID_LH][NPC_LT_LH_TU_UDP] = "LH_TU_UDP", + [NPC_LID_LH][NPC_LT_LH_TU_TCP] = "LH_TU_TCP", + [NPC_LID_LH][NPC_LT_LH_TU_SCTP] = "LH_TU_SCTP", + [NPC_LID_LH][NPC_LT_LH_TU_ESP] = "LH_TU_ESP", +}; + +static uint16_t +npc_get_nibbles(struct roc_npc_flow *flow, uint16_t size, uint32_t bit_offset) +{ + uint32_t byte_index, noffset; + uint16_t data, mask; + uint8_t *bytes; + + bytes = (uint8_t *)flow->mcam_data; + mask = (1ULL << (size * 4)) - 1; + byte_index = bit_offset / 8; + noffset = bit_offset % 8; + data = *(unaligned_uint16_t *)&bytes[byte_index]; + data >>= noffset; + data &= mask; + + return data; +} + +static void +npc_flow_print_parse_nibbles(FILE *file, struct roc_npc_flow *flow, + uint64_t parse_nibbles) +{ + struct npc_rx_parse_nibble_s *rx_parse; + uint32_t data, offset = 0; + + rx_parse = (struct npc_rx_parse_nibble_s *)&parse_nibbles; + + if (rx_parse->chan) { + data = npc_get_nibbles(flow, 3, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_CHAN:%#03X\n", data); + offset += 12; + } + + if (rx_parse->errlev) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRLEV:%#X\n", data); + offset += 4; + } + + if (rx_parse->errcode) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_ERRCODE:%#02X\n", data); + offset += 8; + } + + if (rx_parse->l2l3bm) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_L2L3_BCAST:%#X\n", data); + offset += 4; + } + + if (rx_parse->latype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_LTYPE:%s\n", + ltype_str[NPC_LID_LA][data]); + offset += 4; + } + + if (rx_parse->laflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LA_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lbtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_LTYPE:%s\n", + ltype_str[NPC_LID_LB][data]); + offset += 4; + } + + if (rx_parse->lbflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LB_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lctype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_LTYPE:%s\n", + ltype_str[NPC_LID_LC][data]); + offset += 4; + } + + if (rx_parse->lcflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LC_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->ldtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_LTYPE:%s\n", + ltype_str[NPC_LID_LD][data]); + offset += 4; + } + + if (rx_parse->ldflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LD_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->letype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_LTYPE:%s\n", + ltype_str[NPC_LID_LE][data]); + offset += 4; + } + + if (rx_parse->leflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LE_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lftype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_LTYPE:%s\n", + ltype_str[NPC_LID_LF][data]); + offset += 4; + } + + if (rx_parse->lfflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LF_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lgtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_LTYPE:%s\n", + ltype_str[NPC_LID_LG][data]); + offset += 4; + } + + if (rx_parse->lgflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LG_FLAGS:%#02X\n", data); + offset += 8; + } + + if (rx_parse->lhtype) { + data = npc_get_nibbles(flow, 1, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_LTYPE:%s\n", + ltype_str[NPC_LID_LH][data]); + offset += 4; + } + + if (rx_parse->lhflags) { + data = npc_get_nibbles(flow, 2, offset); + fprintf(file, "\tNPC_PARSE_NIBBLE_LH_FLAGS:%#02X\n", data); + } +} + +static void +npc_flow_print_xtractinfo(FILE *file, struct npc_xtract_info *lfinfo, + struct roc_npc_flow *flow, int lid, int lt) +{ + uint8_t *datastart, *maskstart; + int i; + + datastart = (uint8_t *)&flow->mcam_data + lfinfo->key_off; + maskstart = (uint8_t *)&flow->mcam_mask + lfinfo->key_off; + + fprintf(file, "\t%s, hdr offset:%#X, len:%#X, key offset:%#X, ", + ltype_str[lid][lt], lfinfo->hdr_off, lfinfo->len, + lfinfo->key_off); + + fprintf(file, "Data:0X"); + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", datastart[i]); + + fprintf(file, ", Mask:0X"); + + for (i = lfinfo->len - 1; i >= 0; i--) + fprintf(file, "%02X", maskstart[i]); + + fprintf(file, "\n"); +} + +static void +npc_flow_print_item(FILE *file, struct npc *npc, struct npc_xtract_info *xinfo, + struct roc_npc_flow *flow, int intf, int lid, int lt, + int ld) +{ + struct npc_xtract_info *lflags_info; + int i, lf_cfg = 0; + + npc_flow_print_xtractinfo(file, xinfo, flow, lid, lt); + + if (xinfo->flags_enable) { + lf_cfg = npc->prx_lfcfg[ld].i; + + if (lf_cfg != lid) + return; + + for (i = 0; i < NPC_MAX_LFL; i++) { + lflags_info = npc->prx_fxcfg[intf][ld][i].xtract; + + npc_flow_print_xtractinfo(file, lflags_info, flow, lid, + lt); + } + } +} + +static void +npc_flow_dump_patterns(FILE *file, struct npc *npc, struct roc_npc_flow *flow) +{ + struct npc_lid_lt_xtract_info *lt_xinfo; + struct npc_xtract_info *xinfo; + uint32_t intf, lid, ld, i; + uint64_t parse_nibbles; + uint16_t ltype; + + intf = flow->nix_intf; + parse_nibbles = npc->keyx_supp_nmask[intf]; + npc_flow_print_parse_nibbles(file, flow, parse_nibbles); + + for (i = 0; i < flow->num_patterns; i++) { + lid = flow->dump_data[i].lid; + ltype = flow->dump_data[i].ltype; + lt_xinfo = &npc->prx_dxcfg[intf][lid][ltype]; + + for (ld = 0; ld < NPC_MAX_LD; ld++) { + xinfo = <_xinfo->xtract[ld]; + if (!xinfo->enable) + continue; + npc_flow_print_item(file, npc, xinfo, flow, intf, lid, + ltype, ld); + } + } +} + +static void +npc_flow_dump_tx_action(FILE *file, uint64_t npc_action) +{ + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + uint32_t tx_op, index, match_id; + + tx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (tx_op) { + case NIX_TX_ACTIONOP_DROP: + fprintf(file, "NIX_TX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_TX_ACTIONOP_UCAST_DEFAULT: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_DEFAULT); + break; + case NIX_TX_ACTIONOP_UCAST_CHAN: + fprintf(file, "NIX_TX_ACTIONOP_UCAST_DEFAULT (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_UCAST_CHAN); + plt_strlcpy(index_name, + "Transmit Channel:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_MCAST: + fprintf(file, "NIX_TX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_MCAST); + plt_strlcpy(index_name, + "Multicast Table Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_TX_ACTIONOP_DROP_VIOL: + fprintf(file, "NIX_TX_ACTIONOP_DROP_VIOL (%" PRIu64 ")\n", + (uint64_t)NIX_TX_ACTIONOP_DROP_VIOL); + break; + default: + plt_err("Unknown NIX_TX_ACTIONOP found"); + return; + } + + index = ((npc_action & NPC_TX_ACTION_INDEX_MASK) >> 12) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_TX_ACTION_MATCH_MASK) >> 32) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); +} + +static void +npc_flow_dump_rx_action(FILE *file, uint64_t npc_action) +{ + uint32_t rx_op, pf_func, index, match_id, flowkey_alg; + char index_name[NPC_MAX_FIELD_NAME_SIZE] = "Index:"; + + rx_op = npc_action & NPC_RX_ACTIONOP_MASK; + + fprintf(file, "\tActionOp:"); + + switch (rx_op) { + case NIX_RX_ACTIONOP_DROP: + fprintf(file, "NIX_RX_ACTIONOP_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_DROP); + break; + case NIX_RX_ACTIONOP_UCAST: + fprintf(file, "NIX_RX_ACTIONOP_UCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST); + plt_strlcpy(index_name, "RQ Index", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_UCAST_IPSEC: + fprintf(file, "NIX_RX_ACTIONOP_UCAST_IPSEC (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_UCAST_IPSEC); + plt_strlcpy(index_name, "RQ Index:", NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_MCAST: + fprintf(file, "NIX_RX_ACTIONOP_MCAST (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MCAST); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_RSS: + fprintf(file, "NIX_RX_ACTIONOP_RSS (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_RSS); + plt_strlcpy(index_name, "RSS Group Index", + NPC_MAX_FIELD_NAME_SIZE); + break; + case NIX_RX_ACTIONOP_PF_FUNC_DROP: + fprintf(file, "NIX_RX_ACTIONOP_PF_FUNC_DROP (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_PF_FUNC_DROP); + break; + case NIX_RX_ACTIONOP_MIRROR: + fprintf(file, "NIX_RX_ACTIONOP_MIRROR (%" PRIu64 ")\n", + (uint64_t)NIX_RX_ACTIONOP_MIRROR); + plt_strlcpy(index_name, "Multicast/mirror table index", + NPC_MAX_FIELD_NAME_SIZE); + break; + default: + plt_err("Unknown NIX_RX_ACTIONOP found"); + return; + } + + pf_func = ((npc_action & NPC_RX_ACTION_PFFUNC_MASK) >> 4) & + GENMASK(15, 0); + + fprintf(file, "\tPF_FUNC: %#04X\n", pf_func); + + index = ((npc_action & NPC_RX_ACTION_INDEX_MASK) >> 20) & + GENMASK(19, 0); + + fprintf(file, "\t%s:%#05X\n", index_name, index); + + match_id = ((npc_action & NPC_RX_ACTION_MATCH_MASK) >> 40) & + GENMASK(15, 0); + + fprintf(file, "\tMatch Id:%#04X\n", match_id); + + flowkey_alg = ((npc_action & NPC_RX_ACTION_FLOWKEY_MASK) >> 56) & + GENMASK(4, 0); + + fprintf(file, "\tFlow Key Alg:%#X\n", flowkey_alg); +} + +static void +npc_flow_dump_parsed_action(FILE *file, uint64_t npc_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX Action:%#016lX\n", npc_action); + npc_flow_dump_rx_action(file, npc_action); + } else { + fprintf(file, "NPC TX Action:%#016lX\n", npc_action); + npc_flow_dump_tx_action(file, npc_action); + } +} + +static void +npc_flow_dump_rx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t type, lid, relptr; + + if (vtag_action & NIX_RX_VTAGACT_VTAG0_VALID_MASK) { + relptr = vtag_action & NIX_RX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG0_TYPE_MASK) >> 12) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG0:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } + + if (vtag_action & NIX_RX_VTAGACT_VTAG1_VALID_MASK) { + relptr = ((vtag_action & NIX_RX_VTAGACT_VTAG1_RELPTR_MASK) >> + 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_RX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + type = ((vtag_action & NIX_RX_VTAGACT_VTAG1_TYPE_MASK) >> 44) & + GENMASK(2, 0); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\ttype:%#X\n", type); + } +} + +static void +npc_get_vtag_opname(uint32_t op, char *opname, int len) +{ + switch (op) { + case 0x0: + plt_strlcpy(opname, "NOP", len - 1); + break; + case 0x1: + plt_strlcpy(opname, "INSERT", len - 1); + break; + case 0x2: + plt_strlcpy(opname, "REPLACE", len - 1); + break; + default: + plt_err("Unknown vtag op found"); + break; + } +} + +static void +npc_flow_dump_tx_vtag_action(FILE *file, uint64_t vtag_action) +{ + uint32_t relptr, lid, op, vtag_def; + char opname[10]; + + relptr = vtag_action & NIX_TX_VTAGACT_VTAG0_RELPTR_MASK; + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG0_LID_MASK) >> 8) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG0_OP_MASK) >> 12) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG0_DEF_MASK) >> 16) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG0 relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); + + relptr = ((vtag_action & NIX_TX_VTAGACT_VTAG1_RELPTR_MASK) >> 32) & + GENMASK(7, 0); + lid = ((vtag_action & NIX_TX_VTAGACT_VTAG1_LID_MASK) >> 40) & + GENMASK(2, 0); + op = ((vtag_action & NIX_TX_VTAGACT_VTAG1_OP_MASK) >> 44) & + GENMASK(1, 0); + vtag_def = ((vtag_action & NIX_TX_VTAGACT_VTAG1_DEF_MASK) >> 48) & + GENMASK(9, 0); + + npc_get_vtag_opname(op, opname, sizeof(opname)); + + fprintf(file, "\tVTAG1:relptr:%#X\n", relptr); + fprintf(file, "\tlid:%#X\n", lid); + fprintf(file, "\top:%s\n", opname); + fprintf(file, "\tvtag_def:%#X\n", vtag_def); +} + +static void +npc_flow_dump_vtag_action(FILE *file, uint64_t vtag_action, bool is_rx) +{ + if (is_rx) { + fprintf(file, "NPC RX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_rx_vtag_action(file, vtag_action); + } else { + fprintf(file, "NPC TX VTAG Action:%#016lX\n", vtag_action); + npc_flow_dump_tx_vtag_action(file, vtag_action); + } +} + +void +roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, + struct roc_npc_flow *flow) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + bool is_rx = 0; + int i; + + fprintf(file, "MCAM Index:%d\n", flow->mcam_id); + fprintf(file, "Interface :%s (%d)\n", intf_str[flow->nix_intf], + flow->nix_intf); + fprintf(file, "Priority :%d\n", flow->priority); + + if (flow->nix_intf == NIX_INTF_RX) + is_rx = 1; + + npc_flow_dump_parsed_action(file, flow->npc_action, is_rx); + npc_flow_dump_vtag_action(file, flow->vtag_action, is_rx); + fprintf(file, "Patterns:\n"); + npc_flow_dump_patterns(file, npc, flow); + + fprintf(file, "MCAM Raw Data :\n"); + + for (i = 0; i < ROC_NPC_MAX_MCAM_WIDTH_DWORDS; i++) { + fprintf(file, "\tDW%d :%016lX\n", i, flow->mcam_data[i]); + fprintf(file, "\tDW%d_Mask:%016lX\n", i, flow->mcam_mask[i]); + } + + fprintf(file, "\n"); +} diff --git a/drivers/common/cnxk/roc_npc_priv.h b/drivers/common/cnxk/roc_npc_priv.h index 8bc5bac..961583b 100644 --- a/drivers/common/cnxk/roc_npc_priv.h +++ b/drivers/common/cnxk/roc_npc_priv.h @@ -47,7 +47,7 @@ #define NPC_RVUPF_MAX_9XXX 0x10 /* HRM: RVU_PRIV_CONST */ #define NPC_RVUPF_MAX_10XX 0x20 /* HRM: RVU_PRIV_CONST */ #define NPC_NIXLF_MAX 0x80 /* HRM: NIX_AF_CONST2 */ -#define NPC_MCAME_PER_PF 2 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ +#define NPC_MCAME_PER_PF 3 /* DRV: RSVD_MCAM_ENTRIES_PER_PF */ #define NPC_MCAME_PER_LF 1 /* DRV: RSVD_MCAM_ENTRIES_PER_NIXLF */ #define NPC_MCAME_RESVD_9XXX \ (NPC_NIXLF_MAX * NPC_MCAME_PER_LF + \ diff --git a/drivers/common/cnxk/roc_npc_utils.c b/drivers/common/cnxk/roc_npc_utils.c index 1fb8973..5c97588 100644 --- a/drivers/common/cnxk/roc_npc_utils.c +++ b/drivers/common/cnxk/roc_npc_utils.c @@ -206,6 +206,7 @@ npc_update_parse_state(struct npc_parse_state *pst, uint8_t flags) { struct npc_lid_lt_xtract_info *xinfo; + struct roc_npc_flow_dump_data *dump; struct npc_xtract_info *lfinfo; int intf, lf_cfg; int i, j, rc = 0; @@ -248,6 +249,9 @@ npc_update_parse_state(struct npc_parse_state *pst, } done: + dump = &pst->flow->dump_data[pst->flow->num_patterns++]; + dump->lid = lid; + dump->ltype = lt; pst->pattern++; return 0; } diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index c39d76f..a11ba4d 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -160,6 +160,8 @@ INTERNAL { roc_npc_fini; roc_npc_flow_create; roc_npc_flow_destroy; + roc_npc_flow_dump; + roc_npc_flow_mcam_dump; roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 04/62] common/cnxk: support for mark and flag flow actions 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (2 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram ` (58 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add roc API to get mark action. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 17 +++++++++++++++++ drivers/common/cnxk/roc_npc.h | 3 +++ drivers/common/cnxk/version.map | 2 ++ 3 files changed, 22 insertions(+) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 81c7fd9..e6a5036 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -757,6 +757,23 @@ npc_rss_action_program(struct roc_npc *roc_npc, return 0; } +int +roc_npc_mark_actions_get(struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + return npc->mark_actions; +} + +int +roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + npc->mark_actions -= count; + return npc->mark_actions; +} + struct roc_npc_flow * roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, const struct roc_npc_item_info pattern[], diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index 115bcd5..cf6f732 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -196,4 +196,7 @@ int __roc_api roc_npc_mcam_free_all_resources(struct roc_npc *roc_npc); void __roc_api roc_npc_flow_dump(FILE *file, struct roc_npc *roc_npc); void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, struct roc_npc_flow *mcam); +int __roc_api roc_npc_mark_actions_get(struct roc_npc *roc_npc); +int __roc_api roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, + uint32_t count); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index a11ba4d..554459b 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -165,6 +165,8 @@ INTERNAL { roc_npc_flow_parse; roc_npc_get_low_priority_mcam; roc_npc_init; + roc_npc_mark_actions_get; + roc_npc_mark_actions_sub_return; roc_npc_mcam_alloc_entries; roc_npc_mcam_alloc_entry; roc_npc_mcam_clear_counter; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 05/62] common/cnxk: allocate lmt region in userspace 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (3 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram ` (57 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Harman Kalra <hkalra@marvell.com> As per the new LMTST design, userspace shall allocate lmt region, setup the DMA translation and share the IOVA with kernel via MBOX. Kernel will convert this IOVA to physical memory and update the LMT table entry with the same. With this new design also shared mode (i.e. all pci funcs sharing the LMT region allocated by primary/base pci func) is intact. Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/common/cnxk/roc_api.h | 2 + drivers/common/cnxk/roc_dev.c | 98 ++++++++++++++++++-------------------- drivers/common/cnxk/roc_dev_priv.h | 1 + drivers/common/cnxk/roc_mbox.h | 3 ++ drivers/common/cnxk/roc_platform.h | 11 +++++ 5 files changed, 63 insertions(+), 52 deletions(-) diff --git a/drivers/common/cnxk/roc_api.h b/drivers/common/cnxk/roc_api.h index 67f5d13..32e383c 100644 --- a/drivers/common/cnxk/roc_api.h +++ b/drivers/common/cnxk/roc_api.h @@ -24,6 +24,8 @@ /* Platform definition */ #include "roc_platform.h" +#define ROC_LMT_LINE_SZ 128 +#define ROC_NUM_LMT_LINES 2048 #define ROC_LMT_LINES_PER_CORE_LOG2 5 #define ROC_LMT_LINE_SIZE_LOG2 7 #define ROC_LMT_BASE_PER_CORE_LOG2 \ diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index a39acc9..adff779 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -915,43 +915,30 @@ dev_vf_mbase_put(struct plt_pci_device *pci_dev, uintptr_t vf_mbase) mbox_mem_unmap((void *)vf_mbase, MBOX_SIZE * pci_dev->max_vfs); } -static uint16_t -dev_pf_total_vfs(struct plt_pci_device *pci_dev) -{ - uint16_t total_vfs = 0; - int sriov_pos, rc; - - sriov_pos = - plt_pci_find_ext_capability(pci_dev, ROC_PCI_EXT_CAP_ID_SRIOV); - if (sriov_pos <= 0) { - plt_warn("Unable to find SRIOV cap, rc=%d", sriov_pos); - return 0; - } - - rc = plt_pci_read_config(pci_dev, &total_vfs, 2, - sriov_pos + ROC_PCI_SRIOV_TOTAL_VF); - if (rc < 0) { - plt_warn("Unable to read SRIOV cap, rc=%d", rc); - return 0; - } - - return total_vfs; -} - static int -dev_setup_shared_lmt_region(struct mbox *mbox) +dev_setup_shared_lmt_region(struct mbox *mbox, bool valid_iova, uint64_t iova) { struct lmtst_tbl_setup_req *req; req = mbox_alloc_msg_lmtst_tbl_setup(mbox); - req->pcifunc = idev_lmt_pffunc_get(); + /* This pcifunc is defined with primary pcifunc whose LMT address + * will be shared. If call contains valid IOVA, following pcifunc + * field is of no use. + */ + req->pcifunc = valid_iova ? 0 : idev_lmt_pffunc_get(); + req->use_local_lmt_region = valid_iova; + req->lmt_iova = iova; return mbox_process(mbox); } +/* Total no of lines * size of each lmtline */ +#define LMT_REGION_SIZE (ROC_NUM_LMT_LINES * ROC_LMT_LINE_SZ) static int -dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) +dev_lmt_setup(struct dev *dev) { + char name[PLT_MEMZONE_NAMESIZE]; + const struct plt_memzone *mz; struct idev_cfg *idev; int rc; @@ -965,8 +952,11 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) /* Set common lmt region from second pf_func onwards. */ if (!dev->disable_shared_lmt && idev_lmt_pffunc_get() && dev->pf_func != idev_lmt_pffunc_get()) { - rc = dev_setup_shared_lmt_region(dev->mbox); + rc = dev_setup_shared_lmt_region(dev->mbox, false, 0); if (!rc) { + /* On success, updating lmt base of secondary pf_funcs + * with primary pf_func's lmt base. + */ dev->lmt_base = roc_idev_lmt_base_addr_get(); return rc; } @@ -975,34 +965,30 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) dev->pf_func, rc); } - if (dev_is_vf(dev)) { - /* VF BAR4 should always be sufficient enough to - * hold LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (RVU_LMT_LINE_MAX * RVU_LMT_SZ)) { - plt_err("Not enough bar4 space for lmt lines"); - return -EFAULT; - } + /* Allocating memory for LMT region */ + sprintf(name, "LMT_MAP%x", dev->pf_func); - dev->lmt_base = dev->bar4; - } else { - uint64_t bar4_mbox_sz = MBOX_SIZE; - - /* PF BAR4 should always be sufficient enough to - * hold PF-AF MBOX + PF-VF MBOX + LMT lines. - */ - if (pci_dev->mem_resource[4].len < - (bar4_mbox_sz + (RVU_LMT_LINE_MAX * RVU_LMT_SZ))) { - plt_err("Not enough bar4 space for lmt lines and mbox"); - return -EFAULT; - } + /* Setting alignment to ensure correct masking for resetting to lmt base + * of a core after all lmt lines under that core are used. + * Alignment value LMT_REGION_SIZE to handle the case where all lines + * are used by 1 core. + */ + mz = plt_lmt_region_reserve_aligned(name, LMT_REGION_SIZE, + LMT_REGION_SIZE); + if (!mz) { + plt_err("Memory alloc failed: %s", strerror(errno)); + goto fail; + } - /* LMT base is just after total VF MBOX area */ - bar4_mbox_sz += (MBOX_SIZE * dev_pf_total_vfs(pci_dev)); - dev->lmt_base = dev->bar4 + bar4_mbox_sz; + /* Share the IOVA address with Kernel */ + rc = dev_setup_shared_lmt_region(dev->mbox, true, mz->iova); + if (rc) { + errno = rc; + goto free; } + dev->lmt_base = mz->iova; + dev->lmt_mz = mz; /* Base LMT address should be chosen from only those pci funcs which * participate in LMT shared mode. */ @@ -1016,6 +1002,10 @@ dev_lmt_setup(struct plt_pci_device *pci_dev, struct dev *dev) } return 0; +free: + plt_memzone_free(mz); +fail: + return -errno; } int @@ -1130,7 +1120,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) goto iounmap; /* Setup LMT line base */ - rc = dev_lmt_setup(pci_dev, dev); + rc = dev_lmt_setup(dev); if (rc) goto iounmap; @@ -1161,6 +1151,10 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) /* Clear references to this pci dev */ npa_lf_fini(); + /* Releasing memory allocated for lmt region */ + if (dev->lmt_mz) + plt_memzone_free(dev->lmt_mz); + mbox_unregister_irq(pci_dev, dev); if (!dev_is_vf(dev)) diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h index 910cfb6..7ee604e 100644 --- a/drivers/common/cnxk/roc_dev_priv.h +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -84,6 +84,7 @@ struct dev { struct dev_ops *ops; void *roc_nix; bool disable_shared_lmt; /* false(default): shared lmt mode enabled */ + const struct plt_memzone *lmt_mz; } __plt_cache_aligned; struct npa { diff --git a/drivers/common/cnxk/roc_mbox.h b/drivers/common/cnxk/roc_mbox.h index f6b11b6..9c529d7 100644 --- a/drivers/common/cnxk/roc_mbox.h +++ b/drivers/common/cnxk/roc_mbox.h @@ -403,6 +403,9 @@ struct lmtst_tbl_setup_req { uint64_t __io dis_line_pref : 1; uint64_t __io ssow_pf_func : 13; uint16_t __io pcifunc; + uint8_t __io use_local_lmt_region; + uint64_t __io lmt_iova; + uint64_t __io rsvd[2]; /* Future use */ }; /* CGX mbox message formats */ diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 911ae15..be58625 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -194,4 +194,15 @@ int roc_plt_init(void); typedef int (*roc_plt_init_cb_t)(void); int __roc_api roc_plt_init_cb_register(roc_plt_init_cb_t cb); +static inline const void * +plt_lmt_region_reserve_aligned(const char *name, size_t len, uint32_t align) +{ + /* To ensure returned memory is physically contiguous, bounding + * the start and end address in 2M range. + */ + return rte_memzone_reserve_bounded(name, len, SOCKET_ID_ANY, + RTE_MEMZONE_IOVA_CONTIG, + align, RTE_PGSIZE_2M); +} + #endif /* _ROC_PLATFORM_H_ */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 06/62] common/cnxk: add provision to enable RED on RQ 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (4 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 07/62] common/cnxk: change model API to not use camel case Nithin Dabilpuram ` (56 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Send RED pass/drop levels based on rq configurations to kernel. Fixed the aura and pool shift value calculation. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/common/cnxk/roc_nix.h | 8 ++++++ drivers/common/cnxk/roc_nix_queue.c | 50 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_npa.c | 8 ++++-- drivers/common/cnxk/roc_npa.h | 5 ++++ 4 files changed, 69 insertions(+), 2 deletions(-) diff --git a/drivers/common/cnxk/roc_nix.h b/drivers/common/cnxk/roc_nix.h index 6d9ac10..bb69027 100644 --- a/drivers/common/cnxk/roc_nix.h +++ b/drivers/common/cnxk/roc_nix.h @@ -161,6 +161,14 @@ struct roc_nix_rq { uint32_t vwqe_max_sz_exp; uint64_t vwqe_wait_tmo; uint64_t vwqe_aura_handle; + /* Average LPB aura level drop threshold for RED */ + uint8_t red_drop; + /* Average LPB aura level pass threshold for RED */ + uint8_t red_pass; + /* Average SPB aura level drop threshold for RED */ + uint8_t spb_red_drop; + /* Average SPB aura level pass threshold for RED */ + uint8_t spb_red_pass; /* End of Input parameters */ struct roc_nix *roc_nix; }; diff --git a/drivers/common/cnxk/roc_nix_queue.c b/drivers/common/cnxk/roc_nix_queue.c index 1c62aa2..0604e7a 100644 --- a/drivers/common/cnxk/roc_nix_queue.c +++ b/drivers/common/cnxk/roc_nix_queue.c @@ -119,6 +119,15 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_aura_pass = rq->spb_red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + + aq->rq.spb_aura_drop = rq->spb_red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -155,6 +164,14 @@ rq_cn9k_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + } } return 0; @@ -244,6 +261,23 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq.qint_idx = rq->qid % nix->qints; aq->rq.xqe_drop_ena = 1; + /* If RED enabled, then fill enable for all cases */ + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq.spb_pool_pass = rq->red_pass; + aq->rq.spb_aura_pass = rq->red_pass; + aq->rq.lpb_pool_pass = rq->red_pass; + aq->rq.lpb_aura_pass = rq->red_pass; + aq->rq.wqe_pool_pass = rq->red_pass; + aq->rq.xqe_pass = rq->red_pass; + + aq->rq.spb_pool_drop = rq->red_drop; + aq->rq.spb_aura_drop = rq->red_drop; + aq->rq.lpb_pool_drop = rq->red_drop; + aq->rq.lpb_aura_drop = rq->red_drop; + aq->rq.wqe_pool_drop = rq->red_drop; + aq->rq.xqe_drop = rq->red_drop; + } + if (cfg) { if (rq->sso_ena) { /* SSO mode */ @@ -296,6 +330,22 @@ rq_cfg(struct nix *nix, struct roc_nix_rq *rq, bool cfg, bool ena) aq->rq_mask.rq_int_ena = ~aq->rq_mask.rq_int_ena; aq->rq_mask.qint_idx = ~aq->rq_mask.qint_idx; aq->rq_mask.xqe_drop_ena = ~aq->rq_mask.xqe_drop_ena; + + if (rq->red_pass && (rq->red_pass >= rq->red_drop)) { + aq->rq_mask.spb_pool_pass = ~aq->rq_mask.spb_pool_pass; + aq->rq_mask.spb_aura_pass = ~aq->rq_mask.spb_aura_pass; + aq->rq_mask.lpb_pool_pass = ~aq->rq_mask.lpb_pool_pass; + aq->rq_mask.lpb_aura_pass = ~aq->rq_mask.lpb_aura_pass; + aq->rq_mask.wqe_pool_pass = ~aq->rq_mask.wqe_pool_pass; + aq->rq_mask.xqe_pass = ~aq->rq_mask.xqe_pass; + + aq->rq_mask.spb_pool_drop = ~aq->rq_mask.spb_pool_drop; + aq->rq_mask.spb_aura_drop = ~aq->rq_mask.spb_aura_drop; + aq->rq_mask.lpb_pool_drop = ~aq->rq_mask.lpb_pool_drop; + aq->rq_mask.lpb_aura_drop = ~aq->rq_mask.lpb_aura_drop; + aq->rq_mask.wqe_pool_drop = ~aq->rq_mask.wqe_pool_drop; + aq->rq_mask.xqe_drop = ~aq->rq_mask.xqe_drop; + } } return 0; diff --git a/drivers/common/cnxk/roc_npa.c b/drivers/common/cnxk/roc_npa.c index 5ba6e81..d064d12 100644 --- a/drivers/common/cnxk/roc_npa.c +++ b/drivers/common/cnxk/roc_npa.c @@ -278,13 +278,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, /* Update aura fields */ aura->pool_addr = pool_id; /* AF will translate to associated poolctx */ aura->ena = 1; - aura->shift = __builtin_clz(block_count) - 8; + aura->shift = plt_log2_u32(block_count); + aura->shift = aura->shift < 8 ? 0 : aura->shift - 8; aura->limit = block_count; aura->pool_caching = 1; aura->err_int_ena = BIT(NPA_AURA_ERR_INT_AURA_ADD_OVER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_ADD_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_AURA_FREE_UNDER); aura->err_int_ena |= BIT(NPA_AURA_ERR_INT_POOL_DIS); + aura->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ aura->err_qint_idx = aura_id % lf->qints; @@ -293,13 +295,15 @@ npa_aura_pool_pair_alloc(struct npa_lf *lf, const uint32_t block_size, pool->ena = 1; pool->buf_size = block_size / ROC_ALIGN; pool->stack_max_pages = stack_size; - pool->shift = __builtin_clz(block_count) - 8; + pool->shift = plt_log2_u32(block_count); + pool->shift = pool->shift < 8 ? 0 : pool->shift - 8; pool->ptr_start = 0; pool->ptr_end = ~0; pool->stack_caching = 1; pool->err_int_ena = BIT(NPA_POOL_ERR_INT_OVFLS); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_RANGE); pool->err_int_ena |= BIT(NPA_POOL_ERR_INT_PERR); + pool->avg_con = ROC_NPA_AVG_CONT; /* Many to one reduction */ pool->err_qint_idx = pool_id % lf->qints; diff --git a/drivers/common/cnxk/roc_npa.h b/drivers/common/cnxk/roc_npa.h index 59d6223..3fc6192 100644 --- a/drivers/common/cnxk/roc_npa.h +++ b/drivers/common/cnxk/roc_npa.h @@ -12,6 +12,11 @@ #define ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS 512 #define ROC_CN10K_NPA_BATCH_FREE_MAX_PTRS 15 +/* This value controls how much of the present average resource level is used to + * calculate the new resource level. + */ +#define ROC_NPA_AVG_CONT 0xE0 + /* 16 CASP instructions can be outstanding in CN9k, but we use only 15 * outstanding CASPs as we run out of registers. */ -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 07/62] common/cnxk: change model API to not use camel case 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (5 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 08/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram ` (55 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Change model check API's to not use Camel case in function names. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/common/cnxk/roc_model.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/common/cnxk/roc_model.h b/drivers/common/cnxk/roc_model.h index fb774ac..5aaad53 100644 --- a/drivers/common/cnxk/roc_model.h +++ b/drivers/common/cnxk/roc_model.h @@ -88,19 +88,19 @@ roc_model_is_cn10k(void) } static inline uint64_t -roc_model_is_cn96_A0(void) +roc_model_is_cn96_a0(void) { return roc_model->flag & ROC_MODEL_CN96xx_A0; } static inline uint64_t -roc_model_is_cn96_Ax(void) +roc_model_is_cn96_ax(void) { return (roc_model->flag & ROC_MODEL_CN96xx_Ax); } static inline uint64_t -roc_model_is_cn95_A0(void) +roc_model_is_cn95_a0(void) { return roc_model->flag & ROC_MODEL_CNF95xx_A0; } -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 08/62] common/cnxk: support for VLAN push and pop flow actions 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (6 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 07/62] common/cnxk: change model API to not use camel case Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram ` (54 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Add roc API to configure VLAN tag addition and removal. This patch also adds 98xx support for increased MCAM entries for rte flow. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_model.h | 6 + drivers/common/cnxk/roc_npc.c | 257 ++++++++++++++++++++++++++++++++++--- drivers/common/cnxk/roc_npc.h | 24 ++++ drivers/common/cnxk/roc_npc_mcam.c | 2 +- drivers/common/cnxk/roc_npc_priv.h | 1 + drivers/common/cnxk/version.map | 2 + 6 files changed, 276 insertions(+), 16 deletions(-) diff --git a/drivers/common/cnxk/roc_model.h b/drivers/common/cnxk/roc_model.h index 5aaad53..c1d11b7 100644 --- a/drivers/common/cnxk/roc_model.h +++ b/drivers/common/cnxk/roc_model.h @@ -88,6 +88,12 @@ roc_model_is_cn10k(void) } static inline uint64_t +roc_model_is_cn98xx(void) +{ + return (roc_model->flag & ROC_MODEL_CN98xx_A0); +} + +static inline uint64_t roc_model_is_cn96_a0(void) { return roc_model->flag & ROC_MODEL_CN96xx_A0; diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index e6a5036..8a76823 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -6,6 +6,23 @@ #include "roc_priv.h" int +roc_npc_vtag_actions_get(struct roc_npc *roc_npc) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + return npc->vtag_actions; +} + +int +roc_npc_vtag_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) +{ + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + + npc->vtag_actions -= count; + return npc->vtag_actions; +} + +int roc_npc_mcam_free_counter(struct roc_npc *roc_npc, uint16_t ctr_id) { struct npc *npc = roc_npc_to_npc_priv(roc_npc); @@ -101,7 +118,7 @@ npc_mcam_tot_entries(void) /* FIXME: change to reading in AF from NPC_AF_CONST1/2 * MCAM_BANK_DEPTH(_EXT) * MCAM_BANKS */ - if (roc_model_is_cn10k()) + if (roc_model_is_cn10k() || roc_model_is_cn98xx()) return 16 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 4096 */ else return 4 * 1024; /* MCAM_BANKS = 4, BANK_DEPTH_EXT = 1024 */ @@ -330,6 +347,7 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, const struct roc_npc_action_mark *act_mark; const struct roc_npc_action_queue *act_q; const struct roc_npc_action_vf *vf_act; + bool vlan_insert_action = false; int sel_act, req_act = 0; uint16_t pf_func, vf_id; int errcode = 0; @@ -417,25 +435,69 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, req_act |= ROC_NPC_ACTION_TYPE_SEC; rq = 0; break; + case ROC_NPC_ACTION_TYPE_VLAN_STRIP: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_STRIP; + break; + case ROC_NPC_ACTION_TYPE_VLAN_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_INSERT; + break; + case ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT; + break; + case ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT: + req_act |= ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT; + break; default: errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; } } + if (req_act & (ROC_NPC_ACTION_TYPE_VLAN_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT)) + vlan_insert_action = true; + + if ((req_act & (ROC_NPC_ACTION_TYPE_VLAN_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT | + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT)) == + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT) { + plt_err("PCP insert action can't be supported alone"); + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } + + /* Both STRIP and INSERT actions are not supported */ + if (vlan_insert_action && (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP)) { + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } + /* Check if actions specified are compatible */ if (attr->egress) { - /* Only DROP/COUNT is supported */ - if (!(req_act & ROC_NPC_ACTION_TYPE_DROP)) { + if (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP) { + plt_err("VLAN pop action is not supported on Egress"); errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; - } else if (req_act & ~(ROC_NPC_ACTION_TYPE_DROP | - ROC_NPC_ACTION_TYPE_COUNT)) { + } + + if (req_act & ROC_NPC_ACTION_TYPE_DROP) { + flow->npc_action = NIX_TX_ACTIONOP_DROP; + } else if ((req_act & ROC_NPC_ACTION_TYPE_COUNT) || + vlan_insert_action) { + flow->npc_action = NIX_TX_ACTIONOP_UCAST_DEFAULT; + } else { + plt_err("Unsupported action for egress"); errcode = NPC_ERR_ACTION_NOTSUP; goto err_exit; } - flow->npc_action = NIX_TX_ACTIONOP_DROP; + goto set_pf_func; + } else { + if (vlan_insert_action) { + errcode = NPC_ERR_ACTION_NOTSUP; + goto err_exit; + } } /* We have already verified the attr, this is ingress. @@ -463,6 +525,13 @@ npc_parse_actions(struct npc *npc, const struct roc_npc_attr *attr, goto err_exit; } + if (req_act & ROC_NPC_ACTION_TYPE_VLAN_STRIP) + npc->vtag_actions++; + + /* Only VLAN action is provided */ + if (req_act == ROC_NPC_ACTION_TYPE_VLAN_STRIP) + flow->npc_action = NIX_RX_ACTIONOP_UCAST; + /* Set NIX_RX_ACTIONOP */ if (req_act & (ROC_NPC_ACTION_TYPE_PF | ROC_NPC_ACTION_TYPE_VF)) { flow->npc_action = NIX_RX_ACTIONOP_UCAST; @@ -774,6 +843,161 @@ roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count) return npc->mark_actions; } +static int +npc_vtag_cfg_delete(struct roc_npc *roc_npc, struct roc_npc_flow *flow) +{ + struct roc_nix *roc_nix = roc_npc->roc_nix; + struct nix_vtag_config *vtag_cfg; + struct nix_vtag_config_rsp *rsp; + struct mbox *mbox; + struct nix *nix; + int rc = 0; + + union { + uint64_t reg; + struct nix_tx_vtag_action_s act; + } tx_vtag_action; + + nix = roc_nix_to_nix_priv(roc_nix); + mbox = (&nix->dev)->mbox; + + tx_vtag_action.reg = flow->vtag_action; + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_TX; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->tx.vtag0_idx = tx_vtag_action.act.vtag0_def; + vtag_cfg->tx.free_vtag0 = true; + + rc = mbox_process_msg(mbox, (void *)&rsp); + if (rc) + return rc; + + return 0; +} + +static int +npc_vtag_action_program(struct roc_npc *roc_npc, + const struct roc_npc_action actions[], + struct roc_npc_flow *flow) +{ + uint16_t vlan_id = 0, vlan_ethtype = ROC_ETHER_TYPE_VLAN; + struct npc *npc = roc_npc_to_npc_priv(roc_npc); + struct roc_nix *roc_nix = roc_npc->roc_nix; + struct nix_vtag_config *vtag_cfg; + struct nix_vtag_config_rsp *rsp; + uint64_t rx_vtag_action = 0; + uint8_t vlan_pcp = 0; + struct mbox *mbox; + struct nix *nix; + int rc; + + union { + uint64_t reg; + struct nix_tx_vtag_action_s act; + } tx_vtag_action; + + nix = roc_nix_to_nix_priv(roc_nix); + mbox = (&nix->dev)->mbox; + + flow->vtag_insert_enabled = false; + + for (; actions->type != ROC_NPC_ACTION_TYPE_END; actions++) { + if (actions->type == ROC_NPC_ACTION_TYPE_VLAN_STRIP) { + if (npc->vtag_actions == 1) { + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_RX; + vtag_cfg->rx.strip_vtag = 1; + /* Always capture */ + vtag_cfg->rx.capture_vtag = 1; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->rx.vtag_type = 0; + + rc = mbox_process(mbox); + if (rc) + return rc; + } + + rx_vtag_action |= (NIX_RX_VTAGACTION_VTAG_VALID << 15); + rx_vtag_action |= ((uint64_t)NPC_LID_LB << 8); + rx_vtag_action |= NIX_RX_VTAGACTION_VTAG0_RELPTR; + flow->vtag_action = rx_vtag_action; + } else if (actions->type == ROC_NPC_ACTION_TYPE_VLAN_INSERT) { + const struct roc_npc_action_of_set_vlan_vid *vtag = + (const struct roc_npc_action_of_set_vlan_vid *) + actions->conf; + vlan_id = plt_be_to_cpu_16(vtag->vlan_vid); + if (vlan_id > 0xfff) { + plt_err("Invalid vlan_id for set vlan action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } else if (actions->type == + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT) { + const struct roc_npc_action_of_push_vlan *ethtype = + (const struct roc_npc_action_of_push_vlan *) + actions->conf; + vlan_ethtype = plt_be_to_cpu_16(ethtype->ethertype); + if (vlan_ethtype != ROC_ETHER_TYPE_VLAN && + vlan_ethtype != ROC_ETHER_TYPE_QINQ) { + plt_err("Invalid ethtype specified for push" + " vlan action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } else if (actions->type == + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT) { + const struct roc_npc_action_of_set_vlan_pcp *pcp = + (const struct roc_npc_action_of_set_vlan_pcp *) + actions->conf; + vlan_pcp = pcp->vlan_pcp; + if (vlan_pcp > 0x7) { + plt_err("Invalid PCP value for pcp action"); + return -EINVAL; + } + flow->vtag_insert_enabled = true; + } + } + + if (flow->vtag_insert_enabled) { + vtag_cfg = mbox_alloc_msg_nix_vtag_cfg(mbox); + + if (vtag_cfg == NULL) + return -ENOSPC; + + vtag_cfg->cfg_type = VTAG_TX; + vtag_cfg->vtag_size = NIX_VTAGSIZE_T4; + vtag_cfg->tx.vtag0 = + ((vlan_ethtype << 16) | (vlan_pcp << 13) | vlan_id); + + vtag_cfg->tx.cfg_vtag0 = 1; + rc = mbox_process_msg(mbox, (void *)&rsp); + if (rc) + return rc; + + if (rsp->vtag0_idx < 0) { + plt_err("Failed to config TX VTAG action"); + return -EINVAL; + } + + tx_vtag_action.reg = 0; + tx_vtag_action.act.vtag0_def = rsp->vtag0_idx; + tx_vtag_action.act.vtag0_lid = NPC_LID_LA; + tx_vtag_action.act.vtag0_op = NIX_TX_VTAGOP_INSERT; + tx_vtag_action.act.vtag0_relptr = + NIX_TX_VTAGACTION_VTAG0_RELPTR; + flow->vtag_action = tx_vtag_action.reg; + } + return 0; +} + struct roc_npc_flow * roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, const struct roc_npc_item_info pattern[], @@ -798,6 +1022,12 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, goto err_exit; } + rc = npc_vtag_action_program(roc_npc, actions, flow); + if (rc != 0) { + *errcode = rc; + goto err_exit; + } + parse_state.is_vf = !roc_nix_is_pf(roc_npc->roc_nix); rc = npc_program_mcam(npc, &parse_state, 1); @@ -858,23 +1088,20 @@ roc_npc_flow_destroy(struct roc_npc *roc_npc, struct roc_npc_flow *flow) { struct npc *npc = roc_npc_to_npc_priv(roc_npc); struct plt_bitmap *bmap; - uint16_t match_id; int rc; - match_id = (flow->npc_action >> NPC_RX_ACT_MATCH_OFFSET) & - NPC_RX_ACT_MATCH_MASK; - - if (match_id && match_id < NPC_ACTION_FLAG_DEFAULT) { - if (npc->mark_actions == 0) - return NPC_ERR_PARAM; - } - rc = npc_rss_group_free(npc, flow); if (rc != 0) { plt_err("Failed to free rss action rc = %d", rc); return rc; } + if (flow->vtag_insert_enabled) { + rc = npc_vtag_cfg_delete(roc_npc, flow); + if (rc != 0) + return rc; + } + rc = npc_mcam_free_entry(npc, flow->mcam_id); if (rc != 0) return rc; diff --git a/drivers/common/cnxk/roc_npc.h b/drivers/common/cnxk/roc_npc.h index cf6f732..2c0a536 100644 --- a/drivers/common/cnxk/roc_npc.h +++ b/drivers/common/cnxk/roc_npc.h @@ -62,6 +62,10 @@ enum roc_npc_action_type { ROC_NPC_ACTION_TYPE_COUNT = (1 << 9), ROC_NPC_ACTION_TYPE_PF = (1 << 10), ROC_NPC_ACTION_TYPE_VF = (1 << 11), + ROC_NPC_ACTION_TYPE_VLAN_STRIP = (1 << 12), + ROC_NPC_ACTION_TYPE_VLAN_INSERT = (1 << 13), + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT = (1 << 14), + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT = (1 << 15), }; struct roc_npc_action { @@ -83,6 +87,18 @@ struct roc_npc_action_queue { uint16_t index; /**< Queue index to use. */ }; +struct roc_npc_action_of_push_vlan { + uint16_t ethertype; /**< EtherType. */ +}; + +struct roc_npc_action_of_set_vlan_vid { + uint16_t vlan_vid; /**< VLAN id. */ +}; + +struct roc_npc_action_of_set_vlan_pcp { + uint8_t vlan_pcp; /**< VLAN priority. */ +}; + struct roc_npc_attr { uint32_t priority; /**< Rule priority level within group. */ uint32_t ingress : 1; /**< Rule applies to ingress traffic. */ @@ -107,6 +123,7 @@ struct roc_npc_flow { uint64_t mcam_mask[ROC_NPC_MAX_MCAM_WIDTH_DWORDS]; uint64_t npc_action; uint64_t vtag_action; + bool vtag_insert_enabled; #define ROC_NPC_MAX_FLOW_PATTERNS 32 struct roc_npc_flow_dump_data dump_data[ROC_NPC_MAX_FLOW_PATTERNS]; uint16_t num_patterns; @@ -138,6 +155,10 @@ enum roc_npc_intf { ROC_NPC_INTF_MAX = 2, }; +enum flow_vtag_cfg_dir { VTAG_TX, VTAG_RX }; +#define ROC_ETHER_TYPE_VLAN 0x8100 /**< IEEE 802.1Q VLAN tagging. */ +#define ROC_ETHER_TYPE_QINQ 0x88A8 /**< IEEE 802.1ad QinQ tagging. */ + struct roc_npc { struct roc_nix *roc_nix; uint8_t switch_header_type; @@ -199,4 +220,7 @@ void __roc_api roc_npc_flow_mcam_dump(FILE *file, struct roc_npc *roc_npc, int __roc_api roc_npc_mark_actions_get(struct roc_npc *roc_npc); int __roc_api roc_npc_mark_actions_sub_return(struct roc_npc *roc_npc, uint32_t count); +int __roc_api roc_npc_vtag_actions_get(struct roc_npc *roc_npc); +int __roc_api roc_npc_vtag_actions_sub_return(struct roc_npc *roc_npc, + uint32_t count); #endif /* _ROC_NPC_H_ */ diff --git a/drivers/common/cnxk/roc_npc_mcam.c b/drivers/common/cnxk/roc_npc_mcam.c index ff0676d..8ccaaad 100644 --- a/drivers/common/cnxk/roc_npc_mcam.c +++ b/drivers/common/cnxk/roc_npc_mcam.c @@ -546,7 +546,7 @@ npc_mcam_alloc_and_write(struct npc *npc, struct roc_npc_flow *flow, * * Second approach is used now. */ - req->entry_data.vtag_action = 0ULL; + req->entry_data.vtag_action = flow->vtag_action; for (idx = 0; idx < ROC_NPC_MAX_MCAM_WIDTH_DWORDS; idx++) { req->entry_data.kw[idx] = flow->mcam_data[idx]; diff --git a/drivers/common/cnxk/roc_npc_priv.h b/drivers/common/cnxk/roc_npc_priv.h index 961583b..484c3ae 100644 --- a/drivers/common/cnxk/roc_npc_priv.h +++ b/drivers/common/cnxk/roc_npc_priv.h @@ -350,6 +350,7 @@ struct npc { uint16_t flow_max_priority; /* Max priority for flow */ uint16_t switch_header_type; /* Suppprted switch header type */ uint32_t mark_actions; /* Number of mark actions */ + uint32_t vtag_actions; /* vtag insert/strip actions */ uint16_t pf_func; /* pf_func of device */ npc_dxcfg_t prx_dxcfg; /* intf, lid, lt, extract */ npc_fxcfg_t prx_fxcfg; /* Flag extract */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 554459b..8a5c839 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -167,6 +167,8 @@ INTERNAL { roc_npc_init; roc_npc_mark_actions_get; roc_npc_mark_actions_sub_return; + roc_npc_vtag_actions_get; + roc_npc_vtag_actions_sub_return; roc_npc_mcam_alloc_entries; roc_npc_mcam_alloc_entry; roc_npc_mcam_clear_counter; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (7 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 08/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-07-05 21:55 ` Thomas Monjalon 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram ` (53 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add build infrastructure and common probe and remove for cnxk driver which is used by both CN10K and CN9K SoC. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- MAINTAINERS | 5 +- doc/guides/nics/cnxk.rst | 29 +++++ doc/guides/nics/features/cnxk.ini | 9 ++ doc/guides/nics/features/cnxk_vec.ini | 9 ++ doc/guides/nics/features/cnxk_vf.ini | 9 ++ doc/guides/nics/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + doc/guides/rel_notes/release_21_08.rst | 5 + drivers/net/cnxk/cnxk_ethdev.c | 218 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ drivers/net/cnxk/meson.build | 14 +++ drivers/net/cnxk/version.map | 3 + drivers/net/meson.build | 1 + 13 files changed, 362 insertions(+), 1 deletion(-) create mode 100644 doc/guides/nics/cnxk.rst create mode 100644 doc/guides/nics/features/cnxk.ini create mode 100644 doc/guides/nics/features/cnxk_vec.ini create mode 100644 doc/guides/nics/features/cnxk_vf.ini create mode 100644 drivers/net/cnxk/cnxk_ethdev.c create mode 100644 drivers/net/cnxk/cnxk_ethdev.h create mode 100644 drivers/net/cnxk/meson.build create mode 100644 drivers/net/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 5877a16..b39a1c2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -745,8 +745,11 @@ M: Kiran Kumar K <kirankumark@marvell.com> M: Sunil Kumar Kori <skori@marvell.com> M: Satha Rao <skoteshwar@marvell.com> T: git://dpdk.org/next/dpdk-next-net-mrvl -F: drivers/common/cnxk/ +F: doc/guides/nics/cnxk.rst +F: doc/guides/nics/features/cnxk*.ini F: doc/guides/platform/cnxk.rst +F: drivers/common/cnxk/ +F: drivers/net/cnxk/ Marvell mvpp2 M: Liron Himi <lironh@marvell.com> diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst new file mode 100644 index 0000000..ca21842 --- /dev/null +++ b/doc/guides/nics/cnxk.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK Poll Mode driver +===================== + +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** +SoC family as well as for their virtual functions (VF) in SR-IOV context. + +More information can be found at `Marvell Official Website +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_. + +Features +-------- + +Features of the CNXK Ethdev PMD are: + +Prerequisites +------------- + +See :doc:`../platform/cnxk` for setup information. + + +Driver compilation and testing +------------------------------ + +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` +for details. diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini new file mode 100644 index 0000000..2c23464 --- /dev/null +++ b/doc/guides/nics/features/cnxk.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini new file mode 100644 index 0000000..de78516 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vec' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini new file mode 100644 index 0000000..9c96351 --- /dev/null +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -0,0 +1,9 @@ +; +; Supported features of the 'cnxk_vf' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Linux = Y +ARMv8 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 799697c..c1a04d9 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -19,6 +19,7 @@ Network Interface Controller Drivers axgbe bnx2x bnxt + cnxk cxgbe dpaa dpaa2 diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index cebb3d0..b506c11 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -142,6 +142,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in cnxk SoC. +#. **Ethdev Driver** + See :doc:`../nics/cnxk` for NIX Ethdev driver information. + #. **Mempool Driver** See :doc:`../mempool/cnxk` for NPA mempool driver information. diff --git a/doc/guides/rel_notes/release_21_08.rst b/doc/guides/rel_notes/release_21_08.rst index a6ecfdf..31e49e1 100644 --- a/doc/guides/rel_notes/release_21_08.rst +++ b/doc/guides/rel_notes/release_21_08.rst @@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **Added support for Marvell CN10K SoC ethernet device.** + + * Added net/cnxk driver which provides the support for the integrated ethernet + device. + Removed Items ------------- diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c new file mode 100644 index 0000000..589b0da --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -0,0 +1,218 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_ethdev.h> + +/* CNXK platform independent eth dev ops */ +struct eth_dev_ops cnxk_eth_dev_ops; + +static int +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct rte_pci_device *pci_dev; + int rc, max_entries; + + eth_dev->dev_ops = &cnxk_eth_dev_ops; + + /* For secondary processes, the primary has done all the work */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + rte_eth_copy_pci_info(eth_dev, pci_dev); + + /* Initialize base roc nix */ + nix->pci_dev = pci_dev; + rc = roc_nix_dev_init(nix); + if (rc) { + plt_err("Failed to initialize roc nix rc=%d", rc); + goto error; + } + + dev->eth_dev = eth_dev; + + /* For vfs, returned max_entries will be 0. but to keep default mac + * address, one entry must be allocated. so setting up to 1. + */ + if (roc_nix_is_vf_or_sdp(nix)) + max_entries = 1; + else + max_entries = roc_nix_mac_max_entries_get(nix); + + if (max_entries <= 0) { + plt_err("Failed to get max entries for mac addr"); + rc = -ENOTSUP; + goto dev_fini; + } + + eth_dev->data->mac_addrs = + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); + if (eth_dev->data->mac_addrs == NULL) { + plt_err("Failed to allocate memory for mac addr"); + rc = -ENOMEM; + goto dev_fini; + } + + dev->max_mac_entries = max_entries; + + /* Get mac address */ + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to get mac addr, rc=%d", rc); + goto free_mac_addrs; + } + + /* Update the mac address */ + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); + + if (!roc_nix_is_vf_or_sdp(nix)) { + /* Sync same MAC address to CGX/RPM table */ + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); + if (rc) { + plt_err("Failed to set mac addr, rc=%d", rc); + goto free_mac_addrs; + } + } + + /* Initialize roc npc */ + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, + eth_dev->data->port_id, roc_nix_get_pf(nix), + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, + dev->rx_offload_capa, dev->tx_offload_capa); + return 0; + +free_mac_addrs: + rte_free(eth_dev->data->mac_addrs); +dev_fini: + roc_nix_dev_fini(nix); +error: + plt_err("Failed to init nix eth_dev rc=%d", rc); + return rc; +} + +static int +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct roc_nix *nix = &dev->nix; + int rc, i; + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + roc_nix_npc_rx_ena_dis(nix, false); + + /* Free up SQs */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); + eth_dev->data->tx_queues[i] = NULL; + } + eth_dev->data->nb_tx_queues = 0; + + /* Free up RQ's and CQ's */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); + eth_dev->data->rx_queues[i] = NULL; + } + eth_dev->data->nb_rx_queues = 0; + + /* Free tm resources */ + roc_nix_tm_fini(nix); + + /* Unregister queue irqs */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister cq irqs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Free nix lf resources */ + rc = roc_nix_lf_free(nix); + if (rc) + plt_err("Failed to free nix lf, rc=%d", rc); + + rte_free(eth_dev->data->mac_addrs); + eth_dev->data->mac_addrs = NULL; + + /* Check if mbox close is needed */ + if (!mbox_close) + return 0; + + rc = roc_nix_dev_fini(nix); + /* Can be freed later by PMD if NPA LF is in use */ + if (rc == -EAGAIN) { + eth_dev->data->dev_private = NULL; + return 0; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + } + + return rc; +} + +int +cnxk_nix_remove(struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct roc_nix *nix; + int rc = -EINVAL; + + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (eth_dev) { + /* Cleanup eth dev */ + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + rte_eth_dev_release_port(eth_dev); + } + + /* Nothing to be done for secondary processes */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + /* Check if this device is hosting common resource */ + nix = roc_idev_npa_nix_get(); + if (nix->pci_dev != pci_dev) + return 0; + + /* Try nix fini now */ + rc = roc_nix_dev_fini(nix); + if (rc == -EAGAIN) { + plt_info("%s: common resource in use by other devices", + pci_dev->name); + goto exit; + } else if (rc) { + plt_err("Failed in nix dev fini, rc=%d", rc); + goto exit; + } + + /* Free device pointer as rte_ethdev does not have it anymore */ + rte_free(nix); +exit: + return rc; +} + +int +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + int rc; + + RTE_SET_USED(pci_drv); + + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), + cnxk_eth_dev_init); + + /* On error on secondary, recheck if port exists in primary or + * in mid of detach state. + */ + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) + if (!rte_eth_dev_allocated(pci_dev->device.name)) + return 0; + return rc; +} diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h new file mode 100644 index 0000000..0460d1e --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_ETHDEV_H__ +#define __CNXK_ETHDEV_H__ + +#include <math.h> +#include <stdint.h> + +#include <ethdev_driver.h> +#include <ethdev_pci.h> + +#include "roc_api.h" + +#define CNXK_ETH_DEV_PMD_VERSION "1.0" + +struct cnxk_eth_dev { + /* ROC NIX */ + struct roc_nix nix; + + /* Max macfilter entries */ + uint8_t max_mac_entries; + + uint16_t flags; + + /* Pointer back to rte */ + struct rte_eth_dev *eth_dev; + + /* HW capabilities / Limitations */ + union { + uint64_t hwcap; + }; + + /* Rx and Tx offload capabilities */ + uint64_t rx_offload_capa; + uint64_t tx_offload_capa; + uint32_t speed_capa; + + /* Default mac address */ + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; +}; + +static inline struct cnxk_eth_dev * +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) +{ + return eth_dev->data->dev_private; +} + +/* Common ethdev ops */ +extern struct eth_dev_ops cnxk_eth_dev_ops; + +/* Ops */ +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); +int cnxk_nix_remove(struct rte_pci_device *pci_dev); + +#endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build new file mode 100644 index 0000000..7dd4bca --- /dev/null +++ b/drivers/net/cnxk/meson.build @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit' + subdir_done() +endif + +sources = files('cnxk_ethdev.c') + +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] +deps += ['common_cnxk', 'mempool_cnxk'] diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map new file mode 100644 index 0000000..4a76d1d --- /dev/null +++ b/drivers/net/cnxk/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/drivers/net/meson.build b/drivers/net/meson.build index c8b5ce2..5b066fd 100644 --- a/drivers/net/meson.build +++ b/drivers/net/meson.build @@ -12,6 +12,7 @@ drivers = [ 'bnx2x', 'bnxt', 'bonding', + 'cnxk', 'cxgbe', 'dpaa', 'dpaa2', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-07-05 21:55 ` Thomas Monjalon 0 siblings, 0 replies; 262+ messages in thread From: Thomas Monjalon @ 2021-07-05 21:55 UTC (permalink / raw) To: jerinj Cc: dev, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram 23/06/2021 06:46, Nithin Dabilpuram: > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -12,6 +12,7 @@ drivers = [ > 'bnx2x', > 'bnxt', > 'bonding', > + 'cnxk', > 'cxgbe', > 'dpaa', > 'dpaa2', Wrong indent spotted with devtools/check-meson.py Will fix while applying if no serious issue. ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 10/62] net/cnxk: add platform specific probe and remove 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (8 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram ` (52 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add platform specific probe and remove callbacks for CN9K and CN10K which use common probe and remove functions. Register ethdev driver for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 64 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 9 +++++ drivers/net/cnxk/cn9k_ethdev.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 9 +++++ drivers/net/cnxk/cnxk_ethdev.c | 42 +++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 19 ++++++++++ drivers/net/cnxk/meson.build | 5 +++ 7 files changed, 230 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_ethdev.c create mode 100644 drivers/net/cnxk/cn10k_ethdev.h create mode 100644 drivers/net/cnxk/cn9k_ethdev.c create mode 100644 drivers/net/cnxk/cn9k_ethdev.h diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c new file mode 100644 index 0000000..ff8ce31 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn10k_ethdev.h" + +static int +cn10k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 64) { + plt_err("Driver not compiled for CN10K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + } + return 0; +} + +static const struct rte_pci_id cn10k_pci_nix_map[] = { + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_PF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KA, PCI_DEVID_CNXK_RVU_AF_VF), + CNXK_PCI_ID(PCI_SUBSYSTEM_DEVID_CN10KAS, PCI_DEVID_CNXK_RVU_AF_VF), + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn10k_pci_nix = { + .id_table = cn10k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn10k_nix_probe, + .remove = cn10k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn10k, cn10k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn10k, cn10k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn10k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h new file mode 100644 index 0000000..1bf4a65 --- /dev/null +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_ETHDEV_H__ +#define __CN10K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c new file mode 100644 index 0000000..98d2d3a --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include "cn9k_ethdev.h" + +static int +cn9k_nix_remove(struct rte_pci_device *pci_dev) +{ + return cnxk_nix_remove(pci_dev); +} + +static int +cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + int rc; + + if (RTE_CACHE_LINE_SIZE != 128) { + plt_err("Driver not compiled for CN9K"); + return -EFAULT; + } + + rc = roc_plt_init(); + if (rc) { + plt_err("Failed to initialize platform model, rc=%d", rc); + return rc; + } + + /* Common probe */ + rc = cnxk_nix_probe(pci_drv, pci_dev); + if (rc) + return rc; + + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + + dev = cnxk_eth_pmd_priv(eth_dev); + /* Update capabilities already set for TSO. + * TSO not supported for earlier chip revisions + */ + if (roc_model_is_cn96_a0() || roc_model_is_cn95_a0()) + dev->tx_offload_capa &= ~(DEV_TX_OFFLOAD_TCP_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_GRE_TNL_TSO); + + /* 50G and 100G to be supported for board version C0 + * and above of CN9K. + */ + if (roc_model_is_cn96_a0() || roc_model_is_cn95_a0()) { + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_50G; + dev->speed_capa &= ~(uint64_t)ETH_LINK_SPEED_100G; + } + + dev->hwcap = 0; + + /* Update HW erratas */ + if (roc_model_is_cn96_a0() || roc_model_is_cn95_a0()) + dev->cq_min_4k = 1; + return 0; +} + +static const struct rte_pci_id cn9k_pci_nix_map[] = { + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver cn9k_pci_nix = { + .id_table = cn9k_pci_nix_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA | + RTE_PCI_DRV_INTR_LSC, + .probe = cn9k_nix_probe, + .remove = cn9k_nix_remove, +}; + +RTE_PMD_REGISTER_PCI(net_cn9k, cn9k_pci_nix); +RTE_PMD_REGISTER_PCI_TABLE(net_cn9k, cn9k_pci_nix_map); +RTE_PMD_REGISTER_KMOD_DEP(net_cn9k, "vfio-pci"); diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h new file mode 100644 index 0000000..15d9397 --- /dev/null +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_ETHDEV_H__ +#define __CN9K_ETHDEV_H__ + +#include <cnxk_ethdev.h> + +#endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 589b0da..526c19b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -3,6 +3,40 @@ */ #include <cnxk_ethdev.h> +static inline uint64_t +nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) +{ + uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; + + return capa; +} + +static inline uint64_t +nix_get_tx_offload_capa(struct cnxk_eth_dev *dev) +{ + RTE_SET_USED(dev); + return CNXK_NIX_TX_OFFLOAD_CAPA; +} + +static inline uint32_t +nix_get_speed_capa(struct cnxk_eth_dev *dev) +{ + uint32_t speed_capa; + + /* Auto negotiation disabled */ + speed_capa = ETH_LINK_SPEED_FIXED; + if (!roc_nix_is_vf_or_sdp(&dev->nix) && !roc_nix_is_lbk(&dev->nix)) { + speed_capa |= ETH_LINK_SPEED_1G | ETH_LINK_SPEED_10G | + ETH_LINK_SPEED_25G | ETH_LINK_SPEED_40G | + ETH_LINK_SPEED_50G | ETH_LINK_SPEED_100G; + } + + return speed_capa; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops; @@ -76,6 +110,14 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } } + /* Union of all capabilities supported by CNXK. + * Platform specific capabilities will be + * updated later. + */ + dev->rx_offload_capa = nix_get_rx_offload_capa(dev); + dev->tx_offload_capa = nix_get_tx_offload_capa(dev); + dev->speed_capa = nix_get_speed_capa(dev); + /* Initialize roc npc */ plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 0460d1e..ba2bfcd 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -14,6 +14,22 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +#define CNXK_NIX_TX_OFFLOAD_CAPA \ + (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | DEV_TX_OFFLOAD_TCP_TSO | \ + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | DEV_TX_OFFLOAD_GENEVE_TNL_TSO | \ + DEV_TX_OFFLOAD_GRE_TNL_TSO | DEV_TX_OFFLOAD_MULTI_SEGS | \ + DEV_TX_OFFLOAD_IPV4_CKSUM) + +#define CNXK_NIX_RX_OFFLOAD_CAPA \ + (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_RX_OFFLOAD_RSS_HASH) + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -28,6 +44,9 @@ struct cnxk_eth_dev { /* HW capabilities / Limitations */ union { + struct { + uint64_t cq_min_4k : 1; + }; uint64_t hwcap; }; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7dd4bca..089e4fc 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,5 +10,10 @@ endif sources = files('cnxk_ethdev.c') +# CN9K +sources += files('cn9k_ethdev.c') +# CN10K +sources += files('cn10k_ethdev.c') + deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 11/62] net/cnxk: add common devargs parsing function 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (9 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 12/62] net/cnxk: support common dev infos get Nithin Dabilpuram ` (51 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add various devargs parsing command line arguments parsing functions supported by CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 94 +++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 7 ++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 166 +++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index ca21842..6652e17 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -27,3 +27,97 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. + +Runtime Config Options +---------------------- + +- ``Rx&Tx scalar mode enable`` (default ``0``) + + PMD supports both scalar and vector mode, it may be selected at runtime + using ``scalar_enable`` ``devargs`` parameter. + +- ``RSS reta size`` (default ``64``) + + RSS redirection table size may be configured during runtime using ``reta_size`` + ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,reta_size=256 + + With the above configuration, reta table of size 256 is populated. + +- ``Flow priority levels`` (default ``3``) + + RTE Flow priority levels can be configured during runtime using + ``flow_max_priority`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_max_priority=10 + + With the above configuration, priority level was set to 10 (0-9). Max + priority level supported is 32. + +- ``Reserve Flow entries`` (default ``8``) + + RTE flow entries can be pre allocated and the size of pre allocation can be + selected runtime using ``flow_prealloc_size`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,flow_prealloc_size=4 + + With the above configuration, pre alloc size was set to 4. Max pre alloc + size supported is 32. + +- ``Max SQB buffer count`` (default ``512``) + + Send queue descriptor buffer count may be limited during runtime using + ``max_sqb_count`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,max_sqb_count=64 + + With the above configuration, each send queue's descriptor buffer count is + limited to a maximum of 64 buffers. + +- ``Switch header enable`` (default ``none``) + + A port can be configured to a specific switch header type by using + ``switch_header`` ``devargs`` parameter. + + For example:: + + -a 0002:02:00.0,switch_header="higig2" + + With the above configuration, higig2 will be enabled on that port and the + traffic on this port should be higig2 traffic only. Supported switch header + types are "higig2", "dsa", "chlen90b" and "chlen24b". + +- ``RSS tag as XOR`` (default ``0``) + + The HW gives two options to configure the RSS adder i.e + + * ``rss_adder<7:0> = flow_tag<7:0> ^ flow_tag<15:8> ^ flow_tag<23:16> ^ flow_tag<31:24>`` + + * ``rss_adder<7:0> = flow_tag<7:0>`` + + Latter one aligns with standard NIC behavior vs former one is a legacy + RSS adder scheme used in OCTEON TX2 products. + + By default, the driver runs in the latter mode. + Setting this flag to 1 to select the legacy mode. + + For example to select the legacy mode(RSS tag adder as XOR):: + + -a 0002:02:00.0,tag_as_xor=1 + + + +.. note:: + + Above devarg parameters are configurable per device, user needs to pass the + parameters to all the PCIe devices if application requires to configure on + all the ethdev ports. diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 526c19b..109fd35 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -57,6 +57,13 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); rte_eth_copy_pci_info(eth_dev, pci_dev); + /* Parse devargs string */ + rc = cnxk_ethdev_parse_devargs(eth_dev->device->devargs, dev); + if (rc) { + plt_err("Failed to parse devargs rc=%d", rc); + goto error; + } + /* Initialize base roc nix */ nix->pci_dev = pci_dev; rc = roc_nix_dev_init(nix); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ba2bfcd..97e3a15 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -9,11 +9,15 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> +#include <rte_kvargs.h> #include "roc_api.h" #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Max supported SQB count */ +#define CNXK_NIX_TX_MAX_SQB 512 + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -38,6 +42,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + bool scalar_ena; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -73,4 +78,8 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +/* Devargs */ +int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, + struct cnxk_eth_dev *dev); + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c new file mode 100644 index 0000000..4af2803 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <inttypes.h> +#include <math.h> + +#include "cnxk_ethdev.h" + +static int +parse_flow_max_priority(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the max priority to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flow_prealloc_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint16_t val; + + val = atoi(value); + + /* Limit the prealloc size to 32 */ + if (val < 1 || val > 32) + return -EINVAL; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_reta_size(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + if (val <= ETH_RSS_RETA_SIZE_64) + val = ROC_NIX_RSS_RETA_SZ_64; + else if (val > ETH_RSS_RETA_SIZE_64 && val <= ETH_RSS_RETA_SIZE_128) + val = ROC_NIX_RSS_RETA_SZ_128; + else if (val > ETH_RSS_RETA_SIZE_128 && val <= ETH_RSS_RETA_SIZE_256) + val = ROC_NIX_RSS_RETA_SZ_256; + else + val = ROC_NIX_RSS_RETA_SZ_64; + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_flag(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + *(uint16_t *)extra_args = atoi(value); + + return 0; +} + +static int +parse_sqb_count(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + + *(uint16_t *)extra_args = val; + + return 0; +} + +static int +parse_switch_header_type(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + + if (strcmp(value, "higig2") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_HIGIG; + + if (strcmp(value, "dsa") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_EDSA; + + if (strcmp(value, "chlen90b") == 0) + *(uint16_t *)extra_args = ROC_PRIV_FLAGS_LEN_90B; + return 0; +} + +#define CNXK_RSS_RETA_SIZE "reta_size" +#define CNXK_SCL_ENABLE "scalar_enable" +#define CNXK_MAX_SQB_COUNT "max_sqb_count" +#define CNXK_FLOW_PREALLOC_SIZE "flow_prealloc_size" +#define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" +#define CNXK_SWITCH_HEADER_TYPE "switch_header" +#define CNXK_RSS_TAG_AS_XOR "tag_as_xor" + +int +cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) +{ + uint16_t reta_sz = ROC_NIX_RSS_RETA_SZ_64; + uint16_t sqb_count = CNXK_NIX_TX_MAX_SQB; + uint16_t flow_prealloc_size = 8; + uint16_t switch_header_type = 0; + uint16_t flow_max_priority = 3; + uint16_t rss_tag_as_xor = 0; + uint16_t scalar_enable = 0; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto null_devargs; + + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_RSS_RETA_SIZE, &parse_reta_size, + &reta_sz); + rte_kvargs_process(kvlist, CNXK_SCL_ENABLE, &parse_flag, + &scalar_enable); + rte_kvargs_process(kvlist, CNXK_MAX_SQB_COUNT, &parse_sqb_count, + &sqb_count); + rte_kvargs_process(kvlist, CNXK_FLOW_PREALLOC_SIZE, + &parse_flow_prealloc_size, &flow_prealloc_size); + rte_kvargs_process(kvlist, CNXK_FLOW_MAX_PRIORITY, + &parse_flow_max_priority, &flow_max_priority); + rte_kvargs_process(kvlist, CNXK_SWITCH_HEADER_TYPE, + &parse_switch_header_type, &switch_header_type); + rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, + &rss_tag_as_xor); + rte_kvargs_free(kvlist); + +null_devargs: + dev->scalar_ena = !!scalar_enable; + dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; + dev->nix.max_sqb_count = sqb_count; + dev->nix.reta_sz = reta_sz; + return 0; + +exit: + return -EINVAL; +} + +RTE_PMD_REGISTER_PARAM_STRING(net_cnxk, + CNXK_RSS_RETA_SIZE "=<64|128|256>" + CNXK_SCL_ENABLE "=1" + CNXK_MAX_SQB_COUNT "=<8-512>" + CNXK_FLOW_PREALLOC_SIZE "=<1-32>" + CNXK_FLOW_MAX_PRIORITY "=<1-32>" + CNXK_SWITCH_HEADER_TYPE "=<higig2|dsa|chlen90b>" + CNXK_RSS_TAG_AS_XOR "=1"); diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 089e4fc..e7e43f0 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -8,7 +8,8 @@ if not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_ethdev.c') +sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_devargs.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 12/62] net/cnxk: support common dev infos get 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (10 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram ` (50 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add support to retrieve dev infos get for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 3 ++ doc/guides/nics/features/cnxk.ini | 4 ++ doc/guides/nics/features/cnxk_vec.ini | 4 ++ doc/guides/nics/features/cnxk_vf.ini | 3 ++ drivers/net/cnxk/cnxk_ethdev.c | 4 +- drivers/net/cnxk/cnxk_ethdev.h | 33 ++++++++++++++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 71 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 8 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6652e17..6bd410b 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,9 @@ Features Features of the CNXK Ethdev PMD are: +- SR-IOV VF +- Lock-free Tx queue + Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 2c23464..b426340 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index de78516..292ac1e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -4,6 +4,10 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +SR-IOV = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 9c96351..bc2eb8a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -4,6 +4,9 @@ ; Refer to default.ini for the full list of available PMD features. ; [Features] +Speed capabilities = Y +Lock-free Tx queue = Y +Multiprocess aware = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 109fd35..066e01c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -38,7 +38,9 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) } /* CNXK platform independent eth dev ops */ -struct eth_dev_ops cnxk_eth_dev_ops; +struct eth_dev_ops cnxk_eth_dev_ops = { + .dev_infos_get = cnxk_nix_info_get, +}; static int cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 97e3a15..8d9a7e0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,9 +15,40 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* VLAN tag inserted by NIX_TX_VTAG_ACTION. + * In Tx space is always reserved for this in FRS. + */ +#define CNXK_NIX_MAX_VTAG_INS 2 +#define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) + +/* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) + +#define CNXK_NIX_RX_MIN_DESC 16 +#define CNXK_NIX_RX_MIN_DESC_ALIGN 16 +#define CNXK_NIX_RX_NB_SEG_MAX 6 +#define CNXK_NIX_RX_DEFAULT_RING_SZ 4096 /* Max supported SQB count */ #define CNXK_NIX_TX_MAX_SQB 512 +/* If PTP is enabled additional SEND MEM DESC is required which + * takes 2 words, hence max 7 iova address are possible + */ +#if defined(RTE_LIBRTE_IEEE1588) +#define CNXK_NIX_TX_NB_SEG_MAX 7 +#else +#define CNXK_NIX_TX_NB_SEG_MAX 9 +#endif + +#define CNXK_NIX_RSS_L3_L4_SRC_DST \ + (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ + ETH_RSS_L4_DST_ONLY) + +#define CNXK_NIX_RSS_OFFLOAD \ + (ETH_RSS_PORT | ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP | \ + ETH_RSS_SCTP | ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \ + CNXK_NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK | ETH_RSS_C_VLAN) + #define CNXK_NIX_TX_OFFLOAD_CAPA \ (DEV_TX_OFFLOAD_MBUF_FAST_FREE | DEV_TX_OFFLOAD_MT_LOCKFREE | \ DEV_TX_OFFLOAD_VLAN_INSERT | DEV_TX_OFFLOAD_QINQ_INSERT | \ @@ -77,6 +108,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_info *dev_info); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c new file mode 100644 index 0000000..4a45956 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <cnxk_ethdev.h> + +int +cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) +{ + struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int max_rx_pktlen; + + max_rx_pktlen = (roc_nix_max_pkt_len(&dev->nix) + RTE_ETHER_CRC_LEN - + CNXK_NIX_MAX_VTAG_ACT_SIZE); + + devinfo->min_rx_bufsize = NIX_MIN_HW_FRS + RTE_ETHER_CRC_LEN; + devinfo->max_rx_pktlen = max_rx_pktlen; + devinfo->max_rx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; + devinfo->max_mac_addrs = dev->max_mac_entries; + devinfo->max_vfs = pci_dev->max_vfs; + devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; + + devinfo->rx_offload_capa = dev->rx_offload_capa; + devinfo->tx_offload_capa = dev->tx_offload_capa; + devinfo->rx_queue_offload_capa = 0; + devinfo->tx_queue_offload_capa = 0; + + devinfo->reta_size = dev->nix.reta_sz; + devinfo->hash_key_size = ROC_NIX_RSS_KEY_LEN; + devinfo->flow_type_rss_offloads = CNXK_NIX_RSS_OFFLOAD; + + devinfo->default_rxconf = (struct rte_eth_rxconf){ + .rx_drop_en = 0, + .offloads = 0, + }; + + devinfo->default_txconf = (struct rte_eth_txconf){ + .offloads = 0, + }; + + devinfo->default_rxportconf = (struct rte_eth_dev_portconf){ + .ring_size = CNXK_NIX_RX_DEFAULT_RING_SZ, + }; + + devinfo->rx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = CNXK_NIX_RX_MIN_DESC, + .nb_align = CNXK_NIX_RX_MIN_DESC_ALIGN, + .nb_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_RX_NB_SEG_MAX, + }; + devinfo->rx_desc_lim.nb_max = + RTE_ALIGN_MUL_FLOOR(devinfo->rx_desc_lim.nb_max, + CNXK_NIX_RX_MIN_DESC_ALIGN); + + devinfo->tx_desc_lim = (struct rte_eth_desc_lim){ + .nb_max = UINT16_MAX, + .nb_min = 1, + .nb_align = 1, + .nb_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + .nb_mtu_seg_max = CNXK_NIX_TX_NB_SEG_MAX, + }; + + devinfo->speed_capa = dev->speed_capa; + devinfo->dev_capa = RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP | + RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index e7e43f0..8495732 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -9,6 +9,7 @@ if not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_ethdev.c', + 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c') # CN9K -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 13/62] net/cnxk: add device configuration operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (11 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 12/62] net/cnxk: support common dev infos get Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 14/62] net/cnxk: support link status update Nithin Dabilpuram ` (49 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add device configuration op for CN9K and CN10K. Most of the device configuration is common between two platforms except for some supported offloads. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 2 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cn10k_ethdev.c | 34 ++ drivers/net/cnxk/cn9k_ethdev.c | 45 +++ drivers/net/cnxk/cnxk_ethdev.c | 568 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 85 +++++ 8 files changed, 740 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6bd410b..0c2ea89 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -18,6 +18,8 @@ Features of the CNXK Ethdev PMD are: - SR-IOV VF - Lock-free Tx queue +- Multiple queues for TX and RX +- Receiver Side Scaling (RSS) Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b426340..96dba2a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 292ac1e..616991c 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index bc2eb8a..a0bd2f1 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +RSS hash = Y +Inner RSS = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index ff8ce31..d971bbd 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,38 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; +} + +static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -26,6 +58,8 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 98d2d3a..2fb7c14 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_conf *conf = ð_dev->data->dev_conf; + struct rte_eth_txmode *txmode = &conf->txmode; + int rc; + + /* Platform specific checks */ + if ((roc_model_is_cn96_a0() || roc_model_is_cn95_a0()) && + (txmode->offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) && + ((txmode->offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) || + (txmode->offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM))) { + plt_err("Outer IP and SCTP checksum unsupported"); + return -EINVAL; + } + + /* Common nix configure */ + rc = cnxk_nix_configure(eth_dev); + if (rc) + return rc; + + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" + " tx_offload_flags=0x%x", + eth_dev->data->port_id, dev->rx_offload_flags, + dev->tx_offload_flags); + return 0; +} + +/* Update platform specific eth dev ops */ +static void +nix_eth_dev_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; +} + +static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { return cnxk_nix_remove(pci_dev); @@ -27,6 +70,8 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) return rc; } + nix_eth_dev_ops_override(); + /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 066e01c..251d6eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,567 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint32_t +cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level) +{ + uint32_t flow_key_type[RSS_MAX_LEVELS][6] = { + {FLOW_KEY_TYPE_IPV4, FLOW_KEY_TYPE_IPV6, FLOW_KEY_TYPE_TCP, + FLOW_KEY_TYPE_UDP, FLOW_KEY_TYPE_SCTP, FLOW_KEY_TYPE_ETH_DMAC}, + {FLOW_KEY_TYPE_INNR_IPV4, FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_INNR_TCP, FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_INNR_SCTP, FLOW_KEY_TYPE_INNR_ETH_DMAC}, + {FLOW_KEY_TYPE_IPV4 | FLOW_KEY_TYPE_INNR_IPV4, + FLOW_KEY_TYPE_IPV6 | FLOW_KEY_TYPE_INNR_IPV6, + FLOW_KEY_TYPE_TCP | FLOW_KEY_TYPE_INNR_TCP, + FLOW_KEY_TYPE_UDP | FLOW_KEY_TYPE_INNR_UDP, + FLOW_KEY_TYPE_SCTP | FLOW_KEY_TYPE_INNR_SCTP, + FLOW_KEY_TYPE_ETH_DMAC | FLOW_KEY_TYPE_INNR_ETH_DMAC} + }; + uint32_t flowkey_cfg = 0; + + dev->ethdev_rss_hf = ethdev_rss; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + + if (ethdev_rss & ETH_RSS_C_VLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VLAN; + + if (ethdev_rss & ETH_RSS_L3_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_SRC; + + if (ethdev_rss & ETH_RSS_L3_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L3_DST; + + if (ethdev_rss & ETH_RSS_L4_SRC_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_SRC; + + if (ethdev_rss & ETH_RSS_L4_DST_ONLY) + flowkey_cfg |= FLOW_KEY_TYPE_L4_DST; + + if (ethdev_rss & RSS_IPV4_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV4_INDEX]; + + if (ethdev_rss & RSS_IPV6_ENABLE) + flowkey_cfg |= flow_key_type[rss_level][RSS_IPV6_INDEX]; + + if (ethdev_rss & ETH_RSS_TCP) + flowkey_cfg |= flow_key_type[rss_level][RSS_TCP_INDEX]; + + if (ethdev_rss & ETH_RSS_UDP) + flowkey_cfg |= flow_key_type[rss_level][RSS_UDP_INDEX]; + + if (ethdev_rss & ETH_RSS_SCTP) + flowkey_cfg |= flow_key_type[rss_level][RSS_SCTP_INDEX]; + + if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + flowkey_cfg |= flow_key_type[rss_level][RSS_DMAC_INDEX]; + + if (ethdev_rss & RSS_IPV6_EX_ENABLE) + flowkey_cfg |= FLOW_KEY_TYPE_IPV6_EXT; + + if (ethdev_rss & ETH_RSS_PORT) + flowkey_cfg |= FLOW_KEY_TYPE_PORT; + + if (ethdev_rss & ETH_RSS_NVGRE) + flowkey_cfg |= FLOW_KEY_TYPE_NVGRE; + + if (ethdev_rss & ETH_RSS_VXLAN) + flowkey_cfg |= FLOW_KEY_TYPE_VXLAN; + + if (ethdev_rss & ETH_RSS_GENEVE) + flowkey_cfg |= FLOW_KEY_TYPE_GENEVE; + + if (ethdev_rss & ETH_RSS_GTPU) + flowkey_cfg |= FLOW_KEY_TYPE_GTPU; + + return flowkey_cfg; +} + +static void +nix_free_queue_mem(struct cnxk_eth_dev *dev) +{ + plt_free(dev->rqs); + plt_free(dev->cqs); + plt_free(dev->sqs); + dev->rqs = NULL; + dev->cqs = NULL; + dev->sqs = NULL; +} + +static int +nix_rss_default_setup(struct cnxk_eth_dev *dev) +{ + struct rte_eth_dev *eth_dev = dev->eth_dev; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + uint64_t rss_hf; + + rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf; + rss_hash_level = ETH_RSS_LEVEL(rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + + flowkey_cfg = cnxk_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level); + return roc_nix_rss_default_setup(&dev->nix, flowkey_cfg); +} + +static int +nix_store_queue_cfg_and_then_release(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = NULL; + struct cnxk_eth_qconf *rx_qconf = NULL; + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_txq_sp *txq_sp; + int i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + tx_qconf = malloc(nb_txq * sizeof(*tx_qconf)); + if (tx_qconf == NULL) { + plt_err("Failed to allocate memory for tx_qconf"); + goto fail; + } + + rx_qconf = malloc(nb_rxq * sizeof(*rx_qconf)); + if (rx_qconf == NULL) { + plt_err("Failed to allocate memory for rx_qconf"); + goto fail; + } + + txq = eth_dev->data->tx_queues; + for (i = 0; i < nb_txq; i++) { + if (txq[i] == NULL) { + tx_qconf[i].valid = false; + plt_info("txq[%d] is already released", i); + continue; + } + txq_sp = cnxk_eth_txq_to_sp(txq[i]); + memcpy(&tx_qconf[i], &txq_sp->qconf, sizeof(*tx_qconf)); + tx_qconf[i].valid = true; + dev_ops->tx_queue_release(txq[i]); + eth_dev->data->tx_queues[i] = NULL; + } + + rxq = eth_dev->data->rx_queues; + for (i = 0; i < nb_rxq; i++) { + if (rxq[i] == NULL) { + rx_qconf[i].valid = false; + plt_info("rxq[%d] is already released", i); + continue; + } + rxq_sp = cnxk_eth_rxq_to_sp(rxq[i]); + memcpy(&rx_qconf[i], &rxq_sp->qconf, sizeof(*rx_qconf)); + rx_qconf[i].valid = true; + dev_ops->rx_queue_release(rxq[i]); + eth_dev->data->rx_queues[i] = NULL; + } + + dev->tx_qconf = tx_qconf; + dev->rx_qconf = rx_qconf; + return 0; + +fail: + free(tx_qconf); + free(rx_qconf); + return -ENOMEM; +} + +static int +nix_restore_queue_cfg(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_qconf *tx_qconf = dev->tx_qconf; + struct cnxk_eth_qconf *rx_qconf = dev->rx_qconf; + int rc, i, nb_rxq, nb_txq; + void **txq, **rxq; + + nb_rxq = RTE_MIN(dev->nb_rxq, eth_dev->data->nb_rx_queues); + nb_txq = RTE_MIN(dev->nb_txq, eth_dev->data->nb_tx_queues); + + rc = -ENOMEM; + /* Setup tx & rx queues with previous configuration so + * that the queues can be functional in cases like ports + * are started without re configuring queues. + * + * Usual re config sequence is like below: + * port_configure() { + * if(reconfigure) { + * queue_release() + * queue_setup() + * } + * queue_configure() { + * queue_release() + * queue_setup() + * } + * } + * port_start() + * + * In some application's control path, queue_configure() would + * NOT be invoked for TXQs/RXQs in port_configure(). + * In such cases, queues can be functional after start as the + * queues are already setup in port_configure(). + */ + for (i = 0; i < nb_txq; i++) { + if (!tx_qconf[i].valid) + continue; + rc = dev_ops->tx_queue_setup(eth_dev, i, tx_qconf[i].nb_desc, 0, + &tx_qconf[i].conf.tx); + if (rc) { + plt_err("Failed to setup tx queue rc=%d", rc); + txq = eth_dev->data->tx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->tx_queue_release(txq[i]); + goto fail; + } + } + + free(tx_qconf); + tx_qconf = NULL; + + for (i = 0; i < nb_rxq; i++) { + if (!rx_qconf[i].valid) + continue; + rc = dev_ops->rx_queue_setup(eth_dev, i, rx_qconf[i].nb_desc, 0, + &rx_qconf[i].conf.rx, + rx_qconf[i].mp); + if (rc) { + plt_err("Failed to setup rx queue rc=%d", rc); + rxq = eth_dev->data->rx_queues; + for (i -= 1; i >= 0; i--) + dev_ops->rx_queue_release(rxq[i]); + goto tx_queue_release; + } + } + + free(rx_qconf); + rx_qconf = NULL; + + return 0; + +tx_queue_release: + txq = eth_dev->data->tx_queues; + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_release(txq[i]); +fail: + if (tx_qconf) + free(tx_qconf); + if (rx_qconf) + free(rx_qconf); + + return rc; +} + +static uint16_t +nix_eth_nop_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + RTE_SET_USED(queue); + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + return 0; +} + +static void +nix_set_nop_rxtx_function(struct rte_eth_dev *eth_dev) +{ + /* These dummy functions are required for supporting + * some applications which reconfigure queues without + * stopping tx burst and rx burst threads(eg kni app) + * When the queues context is saved, txq/rxqs are released + * which caused app crash since rx/tx burst is still + * on different lcores + */ + eth_dev->tx_pkt_burst = nix_eth_nop_burst; + eth_dev->rx_pkt_burst = nix_eth_nop_burst; + rte_mb(); +} + +static int +nix_lso_tun_fmt_update(struct cnxk_eth_dev *dev) +{ + uint8_t udp_tun[ROC_NIX_LSO_TUN_MAX]; + uint8_t tun[ROC_NIX_LSO_TUN_MAX]; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_lso_fmt_get(nix, udp_tun, tun); + if (rc) + return rc; + + dev->lso_tun_fmt = ((uint64_t)tun[ROC_NIX_LSO_TUN_V4V4] | + (uint64_t)tun[ROC_NIX_LSO_TUN_V4V6] << 8 | + (uint64_t)tun[ROC_NIX_LSO_TUN_V6V4] << 16 | + (uint64_t)tun[ROC_NIX_LSO_TUN_V6V6] << 24); + + dev->lso_tun_fmt |= ((uint64_t)udp_tun[ROC_NIX_LSO_TUN_V4V4] << 32 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V4V6] << 40 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V6V4] << 48 | + (uint64_t)udp_tun[ROC_NIX_LSO_TUN_V6V6] << 56); + return 0; +} + +static int +nix_lso_fmt_setup(struct cnxk_eth_dev *dev) +{ + struct roc_nix *nix = &dev->nix; + int rc; + + /* Nothing much to do if offload is not enabled */ + if (!(dev->tx_offloads & + (DEV_TX_OFFLOAD_TCP_TSO | DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO))) + return 0; + + /* Setup LSO formats in AF. Its a no-op if other ethdev has + * already set it up + */ + rc = roc_nix_lso_fmt_setup(nix); + if (rc) + return rc; + + return nix_lso_tun_fmt_update(dev); +} + +int +cnxk_nix_configure(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + struct rte_eth_txmode *txmode = &conf->txmode; + char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix *nix = &dev->nix; + struct rte_ether_addr *ea; + uint8_t nb_rxq, nb_txq; + uint64_t rx_cfg; + void *qs; + int rc; + + rc = -EINVAL; + + /* Sanity checks */ + if (rte_eal_has_hugepages() == 0) { + plt_err("Huge page is not configured"); + goto fail_configure; + } + + if (conf->dcb_capability_en == 1) { + plt_err("dcb enable is not supported"); + goto fail_configure; + } + + if (conf->fdir_conf.mode != RTE_FDIR_MODE_NONE) { + plt_err("Flow director is not supported"); + goto fail_configure; + } + + if (rxmode->mq_mode != ETH_MQ_RX_NONE && + rxmode->mq_mode != ETH_MQ_RX_RSS) { + plt_err("Unsupported mq rx mode %d", rxmode->mq_mode); + goto fail_configure; + } + + if (txmode->mq_mode != ETH_MQ_TX_NONE) { + plt_err("Unsupported mq tx mode %d", txmode->mq_mode); + goto fail_configure; + } + + /* Free the resources allocated from the previous configure */ + if (dev->configured == 1) { + /* Unregister queue irq's */ + roc_nix_unregister_queue_irqs(nix); + + /* Unregister CQ irqs if present */ + if (eth_dev->data->dev_conf.intr_conf.rxq) + roc_nix_unregister_cq_irqs(nix); + + /* Set no-op functions */ + nix_set_nop_rxtx_function(eth_dev); + /* Store queue config for later */ + rc = nix_store_queue_cfg_and_then_release(eth_dev); + if (rc) + goto fail_configure; + roc_nix_tm_fini(nix); + roc_nix_lf_free(nix); + } + + dev->rx_offloads = rxmode->offloads; + dev->tx_offloads = txmode->offloads; + + /* Prepare rx cfg */ + rx_cfg = ROC_NIX_LF_RX_CFG_DIS_APAD; + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) { + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_OL4; + rx_cfg |= ROC_NIX_LF_RX_CFG_CSUM_IL4; + } + rx_cfg |= (ROC_NIX_LF_RX_CFG_DROP_RE | ROC_NIX_LF_RX_CFG_L2_LEN_ERR | + ROC_NIX_LF_RX_CFG_LEN_IL4 | ROC_NIX_LF_RX_CFG_LEN_IL3 | + ROC_NIX_LF_RX_CFG_LEN_OL4 | ROC_NIX_LF_RX_CFG_LEN_OL3); + + nb_rxq = RTE_MAX(data->nb_rx_queues, 1); + nb_txq = RTE_MAX(data->nb_tx_queues, 1); + + /* Alloc a nix lf */ + rc = roc_nix_lf_alloc(nix, nb_rxq, nb_txq, rx_cfg); + if (rc) { + plt_err("Failed to init nix_lf rc=%d", rc); + goto fail_configure; + } + + nb_rxq = data->nb_rx_queues; + nb_txq = data->nb_tx_queues; + rc = -ENOMEM; + if (nb_rxq) { + /* Allocate memory for roc rq's and cq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_rq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc rqs"); + goto free_nix_lf; + } + dev->rqs = qs; + + qs = plt_zmalloc(sizeof(struct roc_nix_cq) * nb_rxq, 0); + if (!qs) { + plt_err("Failed to alloc cqs"); + goto free_nix_lf; + } + dev->cqs = qs; + } + + if (nb_txq) { + /* Allocate memory for roc sq's */ + qs = plt_zmalloc(sizeof(struct roc_nix_sq) * nb_txq, 0); + if (!qs) { + plt_err("Failed to alloc sqs"); + goto free_nix_lf; + } + dev->sqs = qs; + } + + /* Re-enable NIX LF error interrupts */ + roc_nix_err_intr_ena_dis(nix, true); + roc_nix_ras_intr_ena_dis(nix, true); + + if (nix->rx_ptp_ena) { + plt_err("Both PTP and switch header enabled"); + goto free_nix_lf; + } + + /* Setup LSO if needed */ + rc = nix_lso_fmt_setup(dev); + if (rc) { + plt_err("Failed to setup nix lso format fields, rc=%d", rc); + goto free_nix_lf; + } + + /* Configure RSS */ + rc = nix_rss_default_setup(dev); + if (rc) { + plt_err("Failed to configure rss rc=%d", rc); + goto free_nix_lf; + } + + /* Init the default TM scheduler hierarchy */ + rc = roc_nix_tm_init(nix); + if (rc) { + plt_err("Failed to init traffic manager, rc=%d", rc); + goto free_nix_lf; + } + + rc = roc_nix_tm_hierarchy_enable(nix, ROC_NIX_TM_DEFAULT, false); + if (rc) { + plt_err("Failed to enable default tm hierarchy, rc=%d", rc); + goto tm_fini; + } + + /* Register queue IRQs */ + rc = roc_nix_register_queue_irqs(nix); + if (rc) { + plt_err("Failed to register queue interrupts rc=%d", rc); + goto tm_fini; + } + + /* Register cq IRQs */ + if (eth_dev->data->dev_conf.intr_conf.rxq) { + if (eth_dev->data->nb_rx_queues > dev->nix.cints) { + plt_err("Rx interrupt cannot be enabled, rxq > %d", + dev->nix.cints); + goto q_irq_fini; + } + /* Rx interrupt feature cannot work with vector mode because, + * vector mode does not process packets unless min 4 pkts are + * received, while cq interrupts are generated even for 1 pkt + * in the CQ. + */ + dev->scalar_ena = true; + + rc = roc_nix_register_cq_irqs(nix); + if (rc) { + plt_err("Failed to register CQ interrupts rc=%d", rc); + goto q_irq_fini; + } + } + + /* Configure loop back mode */ + rc = roc_nix_mac_loopback_enable(nix, + eth_dev->data->dev_conf.lpbk_mode); + if (rc) { + plt_err("Failed to configure cgx loop back mode rc=%d", rc); + goto cq_fini; + } + + /* + * Restore queue config when reconfigure followed by + * reconfigure and no queue configure invoked from application case. + */ + if (dev->configured == 1) { + rc = nix_restore_queue_cfg(eth_dev); + if (rc) + goto cq_fini; + } + + /* Update the mac address */ + ea = eth_dev->data->mac_addrs; + memcpy(ea, dev->mac_addr, RTE_ETHER_ADDR_LEN); + if (rte_is_zero_ether_addr(ea)) + rte_eth_random_addr((uint8_t *)ea); + + rte_ether_format_addr(ea_fmt, RTE_ETHER_ADDR_FMT_SIZE, ea); + + plt_nix_dbg("Configured port%d mac=%s nb_rxq=%d nb_txq=%d" + " rx_offloads=0x%" PRIx64 " tx_offloads=0x%" PRIx64 "", + eth_dev->data->port_id, ea_fmt, nb_rxq, nb_txq, + dev->rx_offloads, dev->tx_offloads); + + /* All good */ + dev->configured = 1; + dev->nb_rxq = data->nb_rx_queues; + dev->nb_txq = data->nb_tx_queues; + return 0; + +cq_fini: + roc_nix_unregister_cq_irqs(nix); +q_irq_fini: + roc_nix_unregister_queue_irqs(nix); +tm_fini: + roc_nix_tm_fini(nix); +free_nix_lf: + nix_free_queue_mem(dev); + rc |= roc_nix_lf_free(nix); +fail_configure: + dev->configured = 0; + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, @@ -75,6 +636,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->eth_dev = eth_dev; + dev->configured = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. @@ -156,6 +718,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; + /* Clear the flag since we are closing down */ + dev->configured = 0; + roc_nix_npc_rx_ena_dis(nix, false); /* Free up SQs */ @@ -182,6 +747,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) if (eth_dev->data->dev_conf.intr_conf.rxq) roc_nix_unregister_cq_irqs(nix); + /* Free ROC RQ's, SQ's and CQ's memory */ + nix_free_queue_mem(dev); + /* Free nix lf resources */ rc = roc_nix_lf_free(nix); if (rc) diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 8d9a7e0..291f5f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -65,10 +65,50 @@ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ DEV_RX_OFFLOAD_RSS_HASH) +#define RSS_IPV4_ENABLE \ + (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ + ETH_RSS_NONFRAG_IPV4_TCP | ETH_RSS_NONFRAG_IPV4_SCTP) + +#define RSS_IPV6_ENABLE \ + (ETH_RSS_IPV6 | ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_UDP | \ + ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_NONFRAG_IPV6_SCTP) + +#define RSS_IPV6_EX_ENABLE \ + (ETH_RSS_IPV6_EX | ETH_RSS_IPV6_TCP_EX | ETH_RSS_IPV6_UDP_EX) + +#define RSS_MAX_LEVELS 3 + +#define RSS_IPV4_INDEX 0 +#define RSS_IPV6_INDEX 1 +#define RSS_TCP_INDEX 2 +#define RSS_UDP_INDEX 3 +#define RSS_SCTP_INDEX 4 +#define RSS_DMAC_INDEX 5 + +struct cnxk_eth_qconf { + union { + struct rte_eth_txconf tx; + struct rte_eth_rxconf rx; + } conf; + struct rte_mempool *mp; + uint16_t nb_desc; + uint8_t valid; +}; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC RQs, SQs and CQs */ + struct roc_nix_rq *rqs; + struct roc_nix_sq *sqs; + struct roc_nix_cq *cqs; + + /* Configured queue count */ + uint16_t nb_rxq; + uint16_t nb_txq; + uint8_t configured; + /* Max macfilter entries */ uint8_t max_mac_entries; @@ -90,17 +130,57 @@ struct cnxk_eth_dev { uint64_t rx_offload_capa; uint64_t tx_offload_capa; uint32_t speed_capa; + /* Configured Rx and Tx offloads */ + uint64_t rx_offloads; + uint64_t tx_offloads; + /* Platform specific offload flags */ + uint16_t rx_offload_flags; + uint16_t tx_offload_flags; + + /* ETHDEV RSS HF bitmask */ + uint64_t ethdev_rss_hf; + + /* Saved qconf before lf realloc */ + struct cnxk_eth_qconf *tx_qconf; + struct cnxk_eth_qconf *rx_qconf; /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; + + /* LSO Tunnel format indices */ + uint64_t lso_tun_fmt; }; +struct cnxk_eth_rxq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + +struct cnxk_eth_txq_sp { + struct cnxk_eth_dev *dev; + struct cnxk_eth_qconf qconf; + uint16_t qid; +} __plt_cache_aligned; + static inline struct cnxk_eth_dev * cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) { return eth_dev->data->dev_private; } +static inline struct cnxk_eth_rxq_sp * +cnxk_eth_rxq_to_sp(void *__rxq) +{ + return ((struct cnxk_eth_rxq_sp *)__rxq) - 1; +} + +static inline struct cnxk_eth_txq_sp * +cnxk_eth_txq_to_sp(void *__txq) +{ + return ((struct cnxk_eth_txq_sp *)__txq) - 1; +} + /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; @@ -110,6 +190,11 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_configure(struct rte_eth_dev *eth_dev); + +/* RSS */ +uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, + uint8_t rss_level); /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 14/62] net/cnxk: support link status update 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (12 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram ` (48 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add link status update callback to get current link status. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 7 +++ drivers/net/cnxk/cnxk_ethdev.h | 8 +++ drivers/net/cnxk/cnxk_link.c | 102 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_link.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 0c2ea89..7bf6cf5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -20,6 +20,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Link state information Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 96dba2a..affbbd9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 616991c..836cc9f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -8,6 +8,8 @@ Speed capabilities = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a0bd2f1..29bb24f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -7,6 +7,8 @@ Speed capabilities = Y Lock-free Tx queue = Y Multiprocess aware = Y +Link status = Y +Link status event = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 251d6eb..ea49809 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -601,6 +601,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, + .link_update = cnxk_nix_link_update, }; static int @@ -635,6 +636,9 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) goto error; } + /* Register up msg callbacks */ + roc_nix_mac_link_cb_register(nix, cnxk_eth_dev_link_status_cb); + dev->eth_dev = eth_dev; dev->configured = 0; @@ -723,6 +727,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable link status events */ + roc_nix_mac_link_event_start_stop(nix, false); + /* Free up SQs */ for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 291f5f9..daa87af 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -15,6 +15,9 @@ #define CNXK_ETH_DEV_PMD_VERSION "1.0" +/* Used for struct cnxk_eth_dev::flags */ +#define CNXK_LINK_CFG_IN_PROGRESS_F BIT_ULL(0) + /* VLAN tag inserted by NIX_TX_VTAG_ACTION. * In Tx space is always reserved for this in FRS. */ @@ -196,6 +199,11 @@ int cnxk_nix_configure(struct rte_eth_dev *eth_dev); uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +/* Link */ +void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, + struct roc_nix_link_info *link); +int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c new file mode 100644 index 0000000..b0273e7 --- /dev/null +++ b/drivers/net/cnxk/cnxk_link.c @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +static inline int +nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) +{ + uint16_t wait = 1000; + + do { + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); + if (!(dev->flags & CNXK_LINK_CFG_IN_PROGRESS_F)) + break; + wait--; + rte_delay_ms(1); + } while (wait); + + return wait ? 0 : -1; +} + +static void +nix_link_status_print(struct rte_eth_dev *eth_dev, struct rte_eth_link *link) +{ + if (link && link->link_status) + plt_info("Port %d: Link Up - speed %u Mbps - %s", + (int)(eth_dev->data->port_id), + (uint32_t)link->link_speed, + link->link_duplex == ETH_LINK_FULL_DUPLEX + ? "full-duplex" + : "half-duplex"); + else + plt_info("Port %d: Link Down", (int)(eth_dev->data->port_id)); +} + +void +cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_link eth_link; + struct rte_eth_dev *eth_dev; + + if (!link || !nix) + return; + + eth_dev = dev->eth_dev; + if (!eth_dev || !eth_dev->data->dev_conf.intr_conf.lsc) + return; + + if (nix_wait_for_link_cfg(dev)) { + plt_err("Timeout waiting for link_cfg to complete"); + return; + } + + eth_link.link_status = link->status; + eth_link.link_speed = link->speed; + eth_link.link_autoneg = ETH_LINK_AUTONEG; + eth_link.link_duplex = link->full_duplex; + + /* Print link info */ + nix_link_status_print(eth_dev, ð_link); + + /* Update link info */ + rte_eth_linkstatus_set(eth_dev, ð_link); + + /* Set the flag and execute application callbacks */ + rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); +} + +int +cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_link_info info; + struct rte_eth_link link; + int rc; + + RTE_SET_USED(wait_to_complete); + memset(&link, 0, sizeof(struct rte_eth_link)); + + if (roc_nix_is_sdp(&dev->nix)) + return 0; + + if (roc_nix_is_lbk(&dev->nix)) { + link.link_status = ETH_LINK_UP; + link.link_speed = ETH_SPEED_NUM_100G; + link.link_autoneg = ETH_LINK_FIXED; + link.link_duplex = ETH_LINK_FULL_DUPLEX; + } else { + rc = roc_nix_mac_link_info_get(&dev->nix, &info); + if (rc) + return rc; + link.link_status = info.status; + link.link_speed = info.speed; + link.link_autoneg = ETH_LINK_AUTONEG; + if (info.full_duplex) + link.link_duplex = info.full_duplex; + } + + return rte_eth_linkstatus_set(eth_dev, &link); +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 8495732..1ac3d08 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -10,7 +10,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', - 'cnxk_ethdev_devargs.c') + 'cnxk_ethdev_devargs.c', + 'cnxk_link.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 15/62] net/cnxk: add Rx queue setup and release 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (13 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 14/62] net/cnxk: support link status update Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 16/62] net/cnxk: add Tx " Nithin Dabilpuram ` (47 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx queue setup and release op for CN9K and CN10K SoC. Release is completely common while setup is platform dependent due to fast path Rx queue structure variation. Fastpath is platform dependent partly due to core cacheline size difference. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn10k_ethdev.h | 14 +++ drivers/net/cnxk/cn9k_ethdev.c | 44 +++++++++ drivers/net/cnxk/cn9k_ethdev.h | 14 +++ drivers/net/cnxk/cnxk_ethdev.c | 172 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 9 ++ 9 files changed, 300 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index affbbd9..a9d2b03 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 836cc9f..6a8ca1f 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -10,6 +10,7 @@ SR-IOV = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 29bb24f..f761638 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -9,6 +9,7 @@ Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y Link status event = Y +Runtime Rx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d971bbd..b87c4e5 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -4,6 +4,49 @@ #include "cn10k_ethdev.h" static int +cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -33,6 +76,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 1bf4a65..08e11bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_rxq { + uint64_t mbuf_initializer; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t data_off; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 2fb7c14..2ab035e 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -4,6 +4,49 @@ #include "cn9k_ethdev.h" static int +cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_rxq *rxq; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + int rc; + + RTE_SET_USED(socket); + + /* CQ Errata needs min 4K ring */ + if (dev->cq_min_4k && nb_desc < 4096) + nb_desc = 4096; + + /* Common Rx queue setup */ + rc = cnxk_nix_rx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_rxq), rx_conf, mp); + if (rc) + return rc; + + rq = &dev->rqs[qid]; + cq = &dev->cqs[qid]; + + /* Update fast path queue */ + rxq = eth_dev->data->rx_queues[qid]; + rxq->rq = qid; + rxq->desc = (uintptr_t)cq->desc_base; + rxq->cq_door = cq->door; + rxq->cq_status = cq->status; + rxq->wdata = cq->wdata; + rxq->head = cq->head; + rxq->qmask = cq->qmask; + + /* Data offset from data to start of mbuf is first_skip */ + rxq->data_off = rq->first_skip; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -44,6 +87,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } static int diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 15d9397..6384609 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,4 +6,18 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_rxq { + uint64_t mbuf_initializer; + uint64_t data_off; + uintptr_t desc; + void *lookup_mem; + uintptr_t cq_door; + uint64_t wdata; + int64_t *cq_status; + uint32_t head; + uint32_t qmask; + uint32_t available; + uint16_t rq; +} __plt_cache_aligned; + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index ea49809..2775fe4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,177 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +uint64_t +cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) +{ + uint16_t port_id = dev->eth_dev->data->port_id; + struct rte_mbuf mb_def; + uint64_t *tmp; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) % 8 != 0); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, refcnt) - + offsetof(struct rte_mbuf, data_off) != + 2); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, nb_segs) - + offsetof(struct rte_mbuf, data_off) != + 4); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, port) - + offsetof(struct rte_mbuf, data_off) != + 6); + mb_def.nb_segs = 1; + mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.port = port_id; + rte_mbuf_refcnt_set(&mb_def, 1); + + /* Prevent compiler reordering: rearm_data covers previous fields */ + rte_compiler_barrier(); + tmp = (uint64_t *)&mb_def.rearm_data; + + return *tmp; +} + +int +cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_mempool_ops *ops; + const char *platform_ops; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t first_skip; + int rc = -EINVAL; + size_t rxq_sz; + + /* Sanity checks */ + if (rx_conf->rx_deferred_start == 1) { + plt_err("Deferred Rx start is not supported"); + goto fail; + } + + platform_ops = rte_mbuf_platform_mempool_ops(); + /* This driver needs cnxk_npa mempool ops to work */ + ops = rte_mempool_get_ops(mp->ops_index); + if (strncmp(ops->name, platform_ops, RTE_MEMPOOL_OPS_NAMESIZE)) { + plt_err("mempool ops should be of cnxk_npa type"); + goto fail; + } + + if (mp->pool_id == 0) { + plt_err("Invalid pool_id"); + goto fail; + } + + /* Free memory prior to re-allocation if needed */ + if (eth_dev->data->rx_queues[qid] != NULL) { + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->rx_queue_release(eth_dev->data->rx_queues[qid]); + eth_dev->data->rx_queues[qid] = NULL; + } + + /* Setup ROC CQ */ + cq = &dev->cqs[qid]; + cq->qid = qid; + cq->nb_desc = nb_desc; + rc = roc_nix_cq_init(&dev->nix, cq); + if (rc) { + plt_err("Failed to init roc cq for rq=%d, rc=%d", qid, rc); + goto fail; + } + + /* Setup ROC RQ */ + rq = &dev->rqs[qid]; + rq->qid = qid; + rq->aura_handle = mp->pool_id; + rq->flow_tag_width = 32; + rq->sso_ena = false; + + /* Calculate first mbuf skip */ + first_skip = (sizeof(struct rte_mbuf)); + first_skip += RTE_PKTMBUF_HEADROOM; + first_skip += rte_pktmbuf_priv_size(mp); + rq->first_skip = first_skip; + rq->later_skip = sizeof(struct rte_mbuf); + rq->lpb_size = mp->elt_size; + + rc = roc_nix_rq_init(&dev->nix, rq, !!eth_dev->data->dev_started); + if (rc) { + plt_err("Failed to init roc rq for rq=%d, rc=%d", qid, rc); + goto cq_fini; + } + + /* Allocate and setup fast path rx queue */ + rc = -ENOMEM; + rxq_sz = sizeof(struct cnxk_eth_rxq_sp) + fp_rx_q_sz; + rxq_sp = plt_zmalloc(rxq_sz, PLT_CACHE_LINE_SIZE); + if (!rxq_sp) { + plt_err("Failed to alloc rx queue for rq=%d", qid); + goto rq_fini; + } + + /* Setup slow path fields */ + rxq_sp->dev = dev; + rxq_sp->qid = qid; + rxq_sp->qconf.conf.rx = *rx_conf; + rxq_sp->qconf.nb_desc = nb_desc; + rxq_sp->qconf.mp = mp; + + plt_nix_dbg("rq=%d pool=%s nb_desc=%d->%d", qid, mp->name, nb_desc, + cq->nb_desc); + + /* Store start of fast path area */ + eth_dev->data->rx_queues[qid] = rxq_sp + 1; + eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + + return 0; +rq_fini: + rc |= roc_nix_rq_fini(rq); +cq_fini: + rc |= roc_nix_cq_fini(cq); +fail: + return rc; +} + +static void +cnxk_nix_rx_queue_release(void *rxq) +{ + struct cnxk_eth_rxq_sp *rxq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_rq *rq; + struct roc_nix_cq *cq; + uint16_t qid; + int rc; + + if (!rxq) + return; + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + dev = rxq_sp->dev; + qid = rxq_sp->qid; + + plt_nix_dbg("Releasing rxq %u", qid); + + /* Cleanup ROC RQ */ + rq = &dev->rqs[qid]; + rc = roc_nix_rq_fini(rq); + if (rc) + plt_err("Failed to cleanup rq, rc=%d", rc); + + /* Cleanup ROC CQ */ + cq = &dev->cqs[qid]; + rc = roc_nix_cq_fini(cq); + if (rc) + plt_err("Failed to cleanup cq, rc=%d", rc); + + /* Finally free fast path area */ + plt_free(rxq_sp); +} + uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level) @@ -602,6 +773,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .rx_queue_release = cnxk_nix_rx_queue_release, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index daa87af..4a7c2ca 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -10,6 +10,9 @@ #include <ethdev_driver.h> #include <ethdev_pci.h> #include <rte_kvargs.h> +#include <rte_mbuf.h> +#include <rte_mbuf_pool_ops.h> +#include <rte_mempool.h> #include "roc_api.h" @@ -194,6 +197,12 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_rx_q_sz, + const struct rte_eth_rxconf *rx_conf, + struct rte_mempool *mp); + +uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 16/62] net/cnxk: add Tx queue setup and release 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (14 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 17/62] net/cnxk: support packet type Nithin Dabilpuram ` (46 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram aDD tx queue setup and release for CN9K and CN10K. Release is common while setup is platform dependent due to differences in fast path Tx queue structures. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 72 +++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.h | 13 +++++ drivers/net/cnxk/cn10k_tx.h | 13 +++++ drivers/net/cnxk/cn9k_ethdev.c | 70 +++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.h | 11 ++++ drivers/net/cnxk/cn9k_tx.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev.c | 98 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++ 11 files changed, 296 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_tx.h create mode 100644 drivers/net/cnxk/cn9k_tx.h diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index a9d2b03..462d7c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6a8ca1f..09e0d3a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -11,6 +11,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index f761638..4a93a35 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -10,6 +10,7 @@ Multiprocess aware = Y Link status = Y Link status event = Y Runtime Rx queue setup = Y +Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y Linux = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index b87c4e5..454c8ca 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,77 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + union nix_send_hdr_w0_u send_hdr_w0; + union nix_send_sg_s sg_w0; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + send_hdr_w0.u = 0; + sg_w0.u = 0; + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr_w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + } else { + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr_w0.sizem1 = 1; + } + + send_hdr_w0.sq = qid; + sg_w0.subdc = NIX_SUBDC_SG; + sg_w0.segs = 1; + sg_w0.ld_type = NIX_SENDLDTYPE_LDD; + + txq->send_hdr_w0 = send_hdr_w0.u; + txq->sg_w0 = sg_w0.u; + + rte_wmb(); +} + +static int +cn10k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn10k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn10k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + /* Store lmt base in tx queue for easy access */ + txq->lmt_base = dev->nix.lmt_base; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + txq->lso_tun_fmt = dev->lso_tun_fmt; + return 0; +} static int cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -76,6 +147,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 08e11bb..18deb95 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -6,6 +6,19 @@ #include <cnxk_ethdev.h> +struct cn10k_eth_txq { + uint64_t send_hdr_w0; + uint64_t sg_w0; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + uintptr_t lmt_base; + rte_iova_t io_addr; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; + uint64_t cmd[4]; + uint64_t lso_tun_fmt; +} __plt_cache_aligned; + struct cn10k_eth_rxq { uint64_t mbuf_initializer; uintptr_t desc; diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h new file mode 100644 index 0000000..39d4755 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_TX_H__ +#define __CN10K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 2ab035e..5c696c8 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,75 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +static void +nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, + uint16_t qid) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + + RTE_SET_USED(dev); + + /* Initialize the fields based on basic single segment packet */ + memset(&txq->cmd, 0, sizeof(txq->cmd)); + + if (dev->tx_offload_flags & NIX_TX_NEED_EXT_HDR) { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 2(EXT_HDR) + 1(SG) + 1(IOVA) = 6/2 - 1 = 2 */ + send_hdr->w0.sizem1 = 2; + + send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + sg = (union nix_send_sg_s *)&txq->cmd[4]; + } else { + send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; + /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ + send_hdr->w0.sizem1 = 1; + sg = (union nix_send_sg_s *)&txq->cmd[2]; + } + + send_hdr->w0.sq = qid; + sg->subdc = NIX_SUBDC_SG; + sg->segs = 1; + sg->ld_type = NIX_SENDLDTYPE_LDD; + + rte_wmb(); +} + +static int +cn9k_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, unsigned int socket, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cn9k_eth_txq *txq; + struct roc_nix_sq *sq; + int rc; + + RTE_SET_USED(socket); + + /* Common Tx queue setup */ + rc = cnxk_nix_tx_queue_setup(eth_dev, qid, nb_desc, + sizeof(struct cn9k_eth_txq), tx_conf); + if (rc) + return rc; + + sq = &dev->sqs[qid]; + /* Update fast path queue */ + txq = eth_dev->data->tx_queues[qid]; + txq->fc_mem = sq->fc; + txq->lmt_addr = sq->lmt_addr; + txq->io_addr = sq->io_addr; + txq->nb_sqb_bufs_adj = sq->nb_sqb_bufs_adj; + txq->sqes_per_sqb_log2 = sq->sqes_per_sqb_log2; + + nix_form_default_desc(dev, txq, qid); + txq->lso_tun_fmt = dev->lso_tun_fmt; + return 0; +} static int cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, @@ -87,6 +156,7 @@ nix_eth_dev_ops_override(void) /* Update platform specific ops */ cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; + cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; } diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index 6384609..bd7bf50 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -6,6 +6,17 @@ #include <cnxk_ethdev.h> +struct cn9k_eth_txq { + uint64_t cmd[8]; + int64_t fc_cache_pkts; + uint64_t *fc_mem; + void *lmt_addr; + rte_iova_t io_addr; + uint64_t lso_tun_fmt; + uint16_t sqes_per_sqb_log2; + int16_t nb_sqb_bufs_adj; +} __plt_cache_aligned; + struct cn9k_eth_rxq { uint64_t mbuf_initializer; uint64_t data_off; diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h new file mode 100644 index 0000000..bb6379b --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN9K_TX_H__ +#define __CN9K_TX_H__ + +#define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_TSO_F BIT(4) + +#define NIX_TX_NEED_EXT_HDR \ + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + +#endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 2775fe4..424512c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -66,6 +66,103 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) return *tmp; } +static inline uint8_t +nix_sq_max_sqe_sz(struct cnxk_eth_dev *dev) +{ + /* + * Maximum three segments can be supported with W8, Choose + * NIX_MAXSQESZ_W16 for multi segment offload. + */ + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + return NIX_MAXSQESZ_W16; + else + return NIX_MAXSQESZ_W8; +} + +int +cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct cnxk_eth_txq_sp *txq_sp; + struct roc_nix_sq *sq; + size_t txq_sz; + int rc; + + /* Free memory prior to re-allocation if needed. */ + if (eth_dev->data->tx_queues[qid] != NULL) { + plt_nix_dbg("Freeing memory prior to re-allocation %d", qid); + dev_ops->tx_queue_release(eth_dev->data->tx_queues[qid]); + eth_dev->data->tx_queues[qid] = NULL; + } + + /* Setup ROC SQ */ + sq = &dev->sqs[qid]; + sq->qid = qid; + sq->nb_desc = nb_desc; + sq->max_sqe_sz = nix_sq_max_sqe_sz(dev); + + rc = roc_nix_sq_init(&dev->nix, sq); + if (rc) { + plt_err("Failed to init sq=%d, rc=%d", qid, rc); + return rc; + } + + rc = -ENOMEM; + txq_sz = sizeof(struct cnxk_eth_txq_sp) + fp_tx_q_sz; + txq_sp = plt_zmalloc(txq_sz, PLT_CACHE_LINE_SIZE); + if (!txq_sp) { + plt_err("Failed to alloc tx queue mem"); + rc |= roc_nix_sq_fini(sq); + return rc; + } + + txq_sp->dev = dev; + txq_sp->qid = qid; + txq_sp->qconf.conf.tx = *tx_conf; + txq_sp->qconf.nb_desc = nb_desc; + + plt_nix_dbg("sq=%d fc=%p offload=0x%" PRIx64 " lmt_addr=%p" + " nb_sqb_bufs=%d sqes_per_sqb_log2=%d", + qid, sq->fc, dev->tx_offloads, sq->lmt_addr, + sq->nb_sqb_bufs, sq->sqes_per_sqb_log2); + + /* Store start of fast path area */ + eth_dev->data->tx_queues[qid] = txq_sp + 1; + eth_dev->data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + return 0; +} + +static void +cnxk_nix_tx_queue_release(void *txq) +{ + struct cnxk_eth_txq_sp *txq_sp; + struct cnxk_eth_dev *dev; + struct roc_nix_sq *sq; + uint16_t qid; + int rc; + + if (!txq) + return; + + txq_sp = cnxk_eth_txq_to_sp(txq); + dev = txq_sp->dev; + qid = txq_sp->qid; + + plt_nix_dbg("Releasing txq %u", qid); + + /* Cleanup ROC SQ */ + sq = &dev->sqs[qid]; + rc = roc_nix_sq_fini(sq); + if (rc) + plt_err("Failed to cleanup sq, rc=%d", rc); + + /* Finally free */ + plt_free(txq_sp); +} + int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, @@ -773,6 +870,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, + .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4a7c2ca..ef8e408 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -197,6 +197,9 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); +int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, + uint16_t nb_desc, uint16_t fp_tx_q_sz, + const struct rte_eth_txconf *tx_conf); int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 17/62] net/cnxk: support packet type 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (15 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 16/62] net/cnxk: add Tx " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 18/62] net/cnxk: support queue start and stop Nithin Dabilpuram ` (45 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add support for packet type lookup on Rx to translate HW specific types to RTE_PTYPE_* defines Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 21 +++ drivers/net/cnxk/cn10k_rx.h | 11 ++ drivers/net/cnxk/cn9k_ethdev.c | 21 +++ drivers/net/cnxk/cn9k_rx.h | 12 ++ drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 14 ++ drivers/net/cnxk/cnxk_lookup.c | 326 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 12 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.h create mode 100644 drivers/net/cnxk/cn9k_rx.h create mode 100644 drivers/net/cnxk/cnxk_lookup.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 7bf6cf5..8bc85c0 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -16,6 +16,7 @@ Features Features of the CNXK Ethdev PMD are: +- Packet type information - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 462d7c4..503582c 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 09e0d3a..9ad225a 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -14,6 +14,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 4a93a35..8c93ba7 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -13,6 +13,7 @@ Runtime Rx queue setup = Y Runtime Tx queue setup = Y RSS hash = Y Inner RSS = Y +Packet type parsing = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 454c8ca..f79d03c 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rx.h" #include "cn10k_tx.h" +static int +cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, uint16_t qid) @@ -114,6 +131,9 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -149,6 +169,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h new file mode 100644 index 0000000..d3d1661 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CN10K_RX_H__ +#define __CN10K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 5c696c8..19b3727 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,8 +2,25 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rx.h" #include "cn9k_tx.h" +static int +cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (ptype_mask) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 0; + } else { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_PTYPE_F; + dev->ptype_disable = 1; + } + + return 0; +} + static void nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, uint16_t qid) @@ -112,6 +129,9 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + + /* Lookup mem */ + rxq->lookup_mem = cnxk_nix_fastpath_lookup_mem_get(); return 0; } @@ -158,6 +178,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } static int diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h new file mode 100644 index 0000000..95a1e69 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef __CN9K_RX_H__ +#define __CN9K_RX_H__ + +#include <rte_ether.h> + +#define NIX_RX_OFFLOAD_PTYPE_F BIT(1) + +#endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 424512c..b1ed046 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -872,6 +872,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; static int @@ -911,6 +912,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->eth_dev = eth_dev; dev->configured = 0; + dev->ptype_disable = 0; /* For vfs, returned max_entries will be 0. but to keep default mac * address, one entry must be allocated. so setting up to 1. diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ef8e408..b23df4a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,15 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +#define PTYPE_NON_TUNNEL_WIDTH 16 +#define PTYPE_TUNNEL_WIDTH 12 +#define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) +#define PTYPE_TUNNEL_ARRAY_SZ BIT(PTYPE_TUNNEL_WIDTH) +#define PTYPE_ARRAY_SZ \ + ((PTYPE_NON_TUNNEL_ARRAY_SZ + PTYPE_TUNNEL_ARRAY_SZ) * sizeof(uint16_t)) +/* Fastpath lookup */ +#define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -119,6 +128,7 @@ struct cnxk_eth_dev { uint8_t max_mac_entries; uint16_t flags; + uint8_t ptype_disable; bool scalar_ena; /* Pointer back to rte */ @@ -216,6 +226,10 @@ void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +/* Lookup configuration */ +const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); +void *cnxk_nix_fastpath_lookup_mem_get(void); + /* Devargs */ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_lookup.c b/drivers/net/cnxk/cnxk_lookup.c new file mode 100644 index 0000000..0152ad9 --- /dev/null +++ b/drivers/net/cnxk/cnxk_lookup.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include <rte_common.h> +#include <rte_memzone.h> + +#include "cnxk_ethdev.h" + +/* NIX_RX_PARSE_S's ERRCODE + ERRLEV (12 bits) */ +#define ERRCODE_ERRLEN_WIDTH 12 +#define ERR_ARRAY_SZ ((BIT(ERRCODE_ERRLEN_WIDTH)) * sizeof(uint32_t)) + +#define SA_TBL_SZ (RTE_MAX_ETHPORTS * sizeof(uint64_t)) +#define LOOKUP_ARRAY_SZ (PTYPE_ARRAY_SZ + ERR_ARRAY_SZ + SA_TBL_SZ) +const uint32_t * +cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev) +{ + RTE_SET_USED(eth_dev); + + static const uint32_t ptypes[] = { + RTE_PTYPE_L2_ETHER_QINQ, /* LB */ + RTE_PTYPE_L2_ETHER_VLAN, /* LB */ + RTE_PTYPE_L2_ETHER_TIMESYNC, /* LB */ + RTE_PTYPE_L2_ETHER_ARP, /* LC */ + RTE_PTYPE_L2_ETHER_NSH, /* LC */ + RTE_PTYPE_L2_ETHER_FCOE, /* LC */ + RTE_PTYPE_L2_ETHER_MPLS, /* LC */ + RTE_PTYPE_L3_IPV4, /* LC */ + RTE_PTYPE_L3_IPV4_EXT, /* LC */ + RTE_PTYPE_L3_IPV6, /* LC */ + RTE_PTYPE_L3_IPV6_EXT, /* LC */ + RTE_PTYPE_L4_TCP, /* LD */ + RTE_PTYPE_L4_UDP, /* LD */ + RTE_PTYPE_L4_SCTP, /* LD */ + RTE_PTYPE_L4_ICMP, /* LD */ + RTE_PTYPE_L4_IGMP, /* LD */ + RTE_PTYPE_TUNNEL_GRE, /* LD */ + RTE_PTYPE_TUNNEL_ESP, /* LD */ + RTE_PTYPE_TUNNEL_NVGRE, /* LD */ + RTE_PTYPE_TUNNEL_VXLAN, /* LE */ + RTE_PTYPE_TUNNEL_GENEVE, /* LE */ + RTE_PTYPE_TUNNEL_GTPC, /* LE */ + RTE_PTYPE_TUNNEL_GTPU, /* LE */ + RTE_PTYPE_TUNNEL_VXLAN_GPE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_GRE, /* LE */ + RTE_PTYPE_TUNNEL_MPLS_IN_UDP, /* LE */ + RTE_PTYPE_INNER_L2_ETHER, /* LF */ + RTE_PTYPE_INNER_L3_IPV4, /* LG */ + RTE_PTYPE_INNER_L3_IPV6, /* LG */ + RTE_PTYPE_INNER_L4_TCP, /* LH */ + RTE_PTYPE_INNER_L4_UDP, /* LH */ + RTE_PTYPE_INNER_L4_SCTP, /* LH */ + RTE_PTYPE_INNER_L4_ICMP, /* LH */ + RTE_PTYPE_UNKNOWN, + }; + + return ptypes; +} + +/* + * +------------------ +------------------ + + * | | IL4 | IL3| IL2 | TU | L4 | L3 | L2 | + * +-------------------+-------------------+ + * + * +-------------------+------------------ + + * | | LH | LG | LF | LE | LD | LC | LB | + * +-------------------+-------------------+ + * + * ptype [LE - LD - LC - LB] = TU - L4 - L3 - T2 + * ptype_tunnel[LH - LG - LF] = IL4 - IL3 - IL2 - TU + * + */ +static void +nix_create_non_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lb, lc, ld, le; + uint16_t val; + uint32_t idx; + + for (idx = 0; idx < PTYPE_NON_TUNNEL_ARRAY_SZ; idx++) { + lb = idx & 0xF; + lc = (idx & 0xF0) >> 4; + ld = (idx & 0xF00) >> 8; + le = (idx & 0xF000) >> 12; + val = RTE_PTYPE_UNKNOWN; + + switch (lb) { + case NPC_LT_LB_STAG_QINQ: + val |= RTE_PTYPE_L2_ETHER_QINQ; + break; + case NPC_LT_LB_CTAG: + val |= RTE_PTYPE_L2_ETHER_VLAN; + break; + } + + switch (lc) { + case NPC_LT_LC_ARP: + val |= RTE_PTYPE_L2_ETHER_ARP; + break; + case NPC_LT_LC_NSH: + val |= RTE_PTYPE_L2_ETHER_NSH; + break; + case NPC_LT_LC_FCOE: + val |= RTE_PTYPE_L2_ETHER_FCOE; + break; + case NPC_LT_LC_MPLS: + val |= RTE_PTYPE_L2_ETHER_MPLS; + break; + case NPC_LT_LC_IP: + val |= RTE_PTYPE_L3_IPV4; + break; + case NPC_LT_LC_IP_OPT: + val |= RTE_PTYPE_L3_IPV4_EXT; + break; + case NPC_LT_LC_IP6: + val |= RTE_PTYPE_L3_IPV6; + break; + case NPC_LT_LC_IP6_EXT: + val |= RTE_PTYPE_L3_IPV6_EXT; + break; + case NPC_LT_LC_PTP: + val |= RTE_PTYPE_L2_ETHER_TIMESYNC; + break; + } + + switch (ld) { + case NPC_LT_LD_TCP: + val |= RTE_PTYPE_L4_TCP; + break; + case NPC_LT_LD_UDP: + val |= RTE_PTYPE_L4_UDP; + break; + case NPC_LT_LD_SCTP: + val |= RTE_PTYPE_L4_SCTP; + break; + case NPC_LT_LD_ICMP: + case NPC_LT_LD_ICMP6: + val |= RTE_PTYPE_L4_ICMP; + break; + case NPC_LT_LD_IGMP: + val |= RTE_PTYPE_L4_IGMP; + break; + case NPC_LT_LD_GRE: + val |= RTE_PTYPE_TUNNEL_GRE; + break; + case NPC_LT_LD_NVGRE: + val |= RTE_PTYPE_TUNNEL_NVGRE; + break; + } + + switch (le) { + case NPC_LT_LE_VXLAN: + val |= RTE_PTYPE_TUNNEL_VXLAN; + break; + case NPC_LT_LE_ESP: + val |= RTE_PTYPE_TUNNEL_ESP; + break; + case NPC_LT_LE_VXLANGPE: + val |= RTE_PTYPE_TUNNEL_VXLAN_GPE; + break; + case NPC_LT_LE_GENEVE: + val |= RTE_PTYPE_TUNNEL_GENEVE; + break; + case NPC_LT_LE_GTPC: + val |= RTE_PTYPE_TUNNEL_GTPC; + break; + case NPC_LT_LE_GTPU: + val |= RTE_PTYPE_TUNNEL_GTPU; + break; + case NPC_LT_LE_TU_MPLS_IN_GRE: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_GRE; + break; + case NPC_LT_LE_TU_MPLS_IN_UDP: + val |= RTE_PTYPE_TUNNEL_MPLS_IN_UDP; + break; + } + ptype[idx] = val; + } +} + +#define TU_SHIFT(x) ((x) >> PTYPE_NON_TUNNEL_WIDTH) +static void +nix_create_tunnel_ptype_array(uint16_t *ptype) +{ + uint8_t lf, lg, lh; + uint16_t val; + uint32_t idx; + + /* Skip non tunnel ptype array memory */ + ptype = ptype + PTYPE_NON_TUNNEL_ARRAY_SZ; + + for (idx = 0; idx < PTYPE_TUNNEL_ARRAY_SZ; idx++) { + lf = idx & 0xF; + lg = (idx & 0xF0) >> 4; + lh = (idx & 0xF00) >> 8; + val = RTE_PTYPE_UNKNOWN; + + switch (lf) { + case NPC_LT_LF_TU_ETHER: + val |= TU_SHIFT(RTE_PTYPE_INNER_L2_ETHER); + break; + } + switch (lg) { + case NPC_LT_LG_TU_IP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV4); + break; + case NPC_LT_LG_TU_IP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L3_IPV6); + break; + } + switch (lh) { + case NPC_LT_LH_TU_TCP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_TCP); + break; + case NPC_LT_LH_TU_UDP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_UDP); + break; + case NPC_LT_LH_TU_SCTP: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_SCTP); + break; + case NPC_LT_LH_TU_ICMP: + case NPC_LT_LH_TU_ICMP6: + val |= TU_SHIFT(RTE_PTYPE_INNER_L4_ICMP); + break; + } + + ptype[idx] = val; + } +} + +static void +nix_create_rx_ol_flags_array(void *mem) +{ + uint16_t idx, errcode, errlev; + uint32_t val, *ol_flags; + + /* Skip ptype array memory */ + ol_flags = (uint32_t *)((uint8_t *)mem + PTYPE_ARRAY_SZ); + + for (idx = 0; idx < BIT(ERRCODE_ERRLEN_WIDTH); idx++) { + errlev = idx & 0xf; + errcode = (idx & 0xff0) >> 4; + + val = PKT_RX_IP_CKSUM_UNKNOWN; + val |= PKT_RX_L4_CKSUM_UNKNOWN; + val |= PKT_RX_OUTER_L4_CKSUM_UNKNOWN; + + switch (errlev) { + case NPC_ERRLEV_RE: + /* Mark all errors as BAD checksum errors + * including Outer L2 length mismatch error + */ + if (errcode) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_L4_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LC: + if (errcode == NPC_EC_OIP4_CSUM || + errcode == NPC_EC_IP_FRAG_OFFSET_1) { + val |= PKT_RX_IP_CKSUM_BAD; + val |= PKT_RX_OUTER_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + } + break; + case NPC_ERRLEV_LG: + if (errcode == NPC_EC_IIP4_CSUM) + val |= PKT_RX_IP_CKSUM_BAD; + else + val |= PKT_RX_IP_CKSUM_GOOD; + break; + case NPC_ERRLEV_NIX: + if (errcode == NIX_RX_PERRCODE_OL4_CHK || + errcode == NIX_RX_PERRCODE_OL4_LEN || + errcode == NIX_RX_PERRCODE_OL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + val |= PKT_RX_OUTER_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL4_CHK || + errcode == NIX_RX_PERRCODE_IL4_LEN || + errcode == NIX_RX_PERRCODE_IL4_PORT) { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_BAD; + } else if (errcode == NIX_RX_PERRCODE_IL3_LEN || + errcode == NIX_RX_PERRCODE_OL3_LEN) { + val |= PKT_RX_IP_CKSUM_BAD; + } else { + val |= PKT_RX_IP_CKSUM_GOOD; + val |= PKT_RX_L4_CKSUM_GOOD; + } + break; + } + ol_flags[idx] = val; + } +} + +void * +cnxk_nix_fastpath_lookup_mem_get(void) +{ + const char name[] = CNXK_NIX_FASTPATH_LOOKUP_MEM; + const struct rte_memzone *mz; + void *mem; + + mz = rte_memzone_lookup(name); + if (mz != NULL) + return mz->addr; + + /* Request for the first time */ + mz = rte_memzone_reserve_aligned(name, LOOKUP_ARRAY_SZ, SOCKET_ID_ANY, + 0, ROC_ALIGN); + if (mz != NULL) { + mem = mz->addr; + /* Form the ptype array lookup memory */ + nix_create_non_tunnel_ptype_array(mem); + nix_create_tunnel_ptype_array(mem); + /* Form the rx ol_flags based on errcode */ + nix_create_rx_ol_flags_array(mem); + return mem; + } + return NULL; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1ac3d08..5bc0bb5 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -11,7 +11,8 @@ endif sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', - 'cnxk_link.c') + 'cnxk_link.c', + 'cnxk_lookup.c') # CN9K sources += files('cn9k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 18/62] net/cnxk: support queue start and stop 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (16 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 17/62] net/cnxk: support packet type Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 19/62] net/cnxk: add Rx burst for cn9k Nithin Dabilpuram ` (44 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx/Tx queue start and stop callbacks for CN9K and CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cn10k_ethdev.c | 16 ++++++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++++++ drivers/net/cnxk/cnxk_ethdev.c | 92 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 1 + 7 files changed, 128 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 503582c..712f8d5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 9ad225a..82f2af0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8c93ba7..61fed11 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Queue start/stop = Y RSS hash = Y Inner RSS = Y Packet type parsing = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index f79d03c..d70ab00 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -138,6 +138,21 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn10k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn10k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn10k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -169,6 +184,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn10k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 19b3727..806e95f 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -136,6 +136,21 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, } static int +cn9k_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qidx) +{ + struct cn9k_eth_txq *txq = eth_dev->data->tx_queues[qidx]; + int rc; + + rc = cnxk_nix_tx_queue_stop(eth_dev, qidx); + if (rc) + return rc; + + /* Clear fc cache pkts to trigger worker stop */ + txq->fc_cache_pkts = 0; + return 0; +} + +static int cn9k_nix_configure(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -178,6 +193,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_configure = cn9k_nix_configure; cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; + cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b1ed046..6c20098 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -866,12 +866,104 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } +static int +cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc = -EINVAL; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, true); + if (rc) { + plt_err("Failed to enable sq aura fc, txq=%u, rc=%d", qid, rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +int +cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_sq *sq = &dev->sqs[qid]; + int rc; + + if (data->tx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_tm_sq_aura_fc(sq, false); + if (rc) { + plt_err("Failed to disable sqb aura fc, txq=%u, rc=%d", qid, + rc); + goto done; + } + + data->tx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STARTED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, true); + if (rc) { + plt_err("Failed to enable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STARTED; +done: + return rc; +} + +static int +cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix_rq *rq = &dev->rqs[qid]; + int rc; + + if (data->rx_queue_state[qid] == RTE_ETH_QUEUE_STATE_STOPPED) + return 0; + + rc = roc_nix_rq_ena_dis(rq, false); + if (rc) { + plt_err("Failed to disable rxq=%u, rc=%d", qid, rc); + goto done; + } + + data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; +done: + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .tx_queue_start = cnxk_nix_tx_queue_start, + .rx_queue_start = cnxk_nix_rx_queue_start, + .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, }; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index b23df4a..5a52489 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -214,6 +214,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 19/62] net/cnxk: add Rx burst for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (17 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 18/62] net/cnxk: support queue start and stop Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram ` (43 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Jerin Jacob <jerinj@marvell.com> Add Rx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 3 + drivers/net/cnxk/cn9k_rx.c | 46 ++++++++ drivers/net/cnxk/cn9k_rx.h | 237 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/meson.build | 3 +- 5 files changed, 291 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_rx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index bd7bf50..bab5540 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -31,4 +31,7 @@ struct cn9k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c new file mode 100644 index 0000000..a4297f9 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 95a1e69..92f3c7c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -7,6 +7,243 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->cn9k.pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = + nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn9k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN9K_RX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5a52489..a6f5d36 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -91,6 +91,9 @@ #define RSS_SCTP_INDEX 4 #define RSS_DMAC_INDEX 5 +/* Default mark value used when none is provided. */ +#define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 5bc0bb5..7a44001 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -15,7 +15,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_lookup.c') # CN9K -sources += files('cn9k_ethdev.c') +sources += files('cn9k_ethdev.c', + 'cn9k_rx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 20/62] net/cnxk: add Rx multi-segmented version for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (18 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 19/62] net/cnxk: add Rx burst for cn9k Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (42 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 17 ++++++++++++ drivers/net/cnxk/cn9k_rx.h | 60 ++++++++++++++++++++++++++++++++++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 17 ++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/meson.build | 3 ++- 5 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index a4297f9..87a62c9 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -40,7 +42,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 92f3c7c..49f80ce 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -104,6 +104,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -138,8 +185,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,8 +290,11 @@ R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c new file mode 100644 index 0000000..6ad8c1d --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a6f5d36..333a54c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7a44001..5bfab21 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -16,7 +16,8 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', - 'cn9k_rx.c') + 'cn9k_rx.c', + 'cn9k_rx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 21/62] net/cnxk: add Rx vector version for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (19 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 22/62] net/cnxk: add Tx burst " Nithin Dabilpuram ` (41 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- drivers/net/cnxk/cn9k_rx.c | 13 ++- drivers/net/cnxk/cn9k_rx.h | 221 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx_vec.c | 17 ++++ drivers/net/cnxk/meson.build | 11 +- 4 files changed, 260 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 87a62c9..01eb21f 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -50,7 +50,18 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 49f80ce..bc04f5c 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -6,6 +6,7 @@ #define __CN9K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -266,6 +267,223 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn9k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn9k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -295,6 +513,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c new file mode 100644 index 0000000..997177f --- /dev/null +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 5bfab21..9aba7d4 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -17,9 +17,18 @@ sources = files('cnxk_ethdev.c', # CN9K sources += files('cn9k_ethdev.c', 'cn9k_rx.c', - 'cn9k_rx_mseg.c') + 'cn9k_rx_mseg.c', + 'cn9k_rx_vec.c') # CN10K sources += files('cn10k_ethdev.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] + +# Allow implicit vector conversions +extra_flags = ['-flax-vector-conversions'] +foreach flag: extra_flags + if cc.has_argument(flag) + cflags += flag + endif +endforeach -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 22/62] net/cnxk: add Tx burst for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (20 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (40 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN9K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_tx.c | 53 ++++++ drivers/net/cnxk/cn9k_tx.h | 419 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 71 +++++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 546 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx.c diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index bab5540..f8344e3 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -33,5 +33,6 @@ struct cn9k_eth_rxq { /* Rx and Tx routines */ void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN9K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c new file mode 100644 index 0000000..a0b022a --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx.c @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, flags);\ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index bb6379b..7acecc6 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,10 +4,429 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn9k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags, + const uint64_t lso_tun_fmt) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + uint8_t shift = is_udp_tun ? 32 : 0; + + shift += (!!(ol_flags & PKT_TX_OUTER_IPV6) << 4); + shift += (!!(ol_flags & PKT_TX_IPV6) << 3); + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format = (lso_tun_fmt >> shift); + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(++sg) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } +} + +static __rte_always_inline void +cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, + const uint32_t flags) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_prep_lmt(uint64_t *cmd, void *lmt_addr, const uint32_t flags) +{ + roc_lmt_mov(lmt_addr, cmd, cn9k_nix_tx_ext_subs(flags)); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeor(io_addr); +} + +static __rte_always_inline uint64_t +cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return roc_lmt_submit_ldeorl(io_addr); +} + +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint64_t lso_tun_fmt; + uint16_t i; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + lso_tun_fmt = txq->lso_tun_fmt; + + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN9K_TX_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 333a54c..58cc6b7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -103,6 +103,10 @@ /* Fastpath lookup */ #define CNXK_NIX_FASTPATH_LOOKUP_MEM "cnxk_nix_fastpath_lookup_mem" +#define CNXK_NIX_UDP_TUN_BITMASK \ + ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ + (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -241,4 +245,71 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Inlines */ +static __rte_always_inline uint64_t +cnxk_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +cnxk_nix_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return cnxk_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 9aba7d4..6c2cd13 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -18,7 +18,8 @@ sources = files('cnxk_ethdev.c', sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', - 'cn9k_rx_vec.c') + 'cn9k_rx_vec.c', + 'cn9k_tx.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 23/62] net/cnxk: add Tx multi-segment version for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (21 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 22/62] net/cnxk: add Tx burst " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (39 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst multi-segment version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 14 ++++ drivers/net/cnxk/cn9k_tx.h | 150 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_mseg.c | 25 +++++++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 195 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index a0b022a..8f1d5f5 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,6 +21,7 @@ NIX_TX_FASTPATH_MODES #undef T + static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -39,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, @@ -47,7 +50,18 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + pick_tx_func(eth_dev, nix_eth_tx_burst); + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); + rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 7acecc6..d9aa406 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -311,6 +311,111 @@ cn9k_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) } static __rte_always_inline uint16_t +cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Commit changes to mbuf */ + rte_io_wmb(); + } + /* Mark mempool object as "put" since it is freed by NIX */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + rte_io_wmb(); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_prep_lmt(uint64_t *cmd, void *lmt_addr, uint16_t segdw) +{ + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, rte_iova_t io_addr, + uint16_t segdw) +{ + uint64_t lmt_status; + + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline void +cn9k_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_io_wmb(); + do { + roc_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); +} + +static __rte_always_inline uint16_t cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -349,6 +454,48 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn9k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + void *lmt_addr = txq->lmt_addr; + uint64_t lso_tun_fmt; + uint16_t segdw; + uint64_t i; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + roc_lmt_mov(cmd, &txq->cmd[0], cn9k_nix_tx_ext_subs(flags)); + + /* Perform header writes before barrier for TSO */ + if (flags & NIX_TX_OFFLOAD_TSO_F) { + lso_tun_fmt = txq->lso_tun_fmt; + + for (i = 0; i < pkts; i++) + cn9k_nix_xmit_prepare_tso(tx_pkts[i], flags); + } + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + for (i = 0; i < pkts; i++) { + cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); + } + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + return pkts; +} + + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -424,6 +571,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c new file mode 100644 index 0000000..65c5f36 --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz) + CNXK_NIX_TX_MSEG_SG_DWORDS - 2]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn9k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58cc6b7..276e569 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -46,6 +46,10 @@ #define CNXK_NIX_TX_NB_SEG_MAX 9 #endif +#define CNXK_NIX_TX_MSEG_SG_DWORDS \ + ((RTE_ALIGN_MUL_CEIL(CNXK_NIX_TX_NB_SEG_MAX, 3) / 3) + \ + CNXK_NIX_TX_NB_SEG_MAX) + #define CNXK_NIX_RSS_L3_L4_SRC_DST \ (ETH_RSS_L3_SRC_ONLY | ETH_RSS_L3_DST_ONLY | ETH_RSS_L4_SRC_ONLY | \ ETH_RSS_L4_DST_ONLY) diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 6c2cd13..a3cd200 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -19,7 +19,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', - 'cn9k_tx.c') + 'cn9k_tx.c', + 'cn9k_tx_mseg.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 24/62] net/cnxk: add Tx vector version for cn9k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (22 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 25/62] net/cnxk: add Rx burst for cn10k Nithin Dabilpuram ` (38 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst vector version for CN9K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn9k_tx.c | 16 +- drivers/net/cnxk/cn9k_tx.h | 743 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 784 insertions(+), 3 deletions(-) create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 8f1d5f5..2ff9720 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -21,7 +21,6 @@ NIX_TX_FASTPATH_MODES #undef T - static inline void pick_tx_func(struct rte_eth_dev *eth_dev, const eth_tx_burst_t tx_burst[2][2][2][2][2]) @@ -58,7 +57,20 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index d9aa406..7b0d536 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN9K_TX_H__ #define __CN9K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -495,6 +497,744 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64x2_t cmd0[NIX_DESCS_PER_LOOP], cmd1[NIX_DESCS_PER_LOOP]; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn9k_eth_txq *txq = tx_queue; + uint64_t *lmt_addr = txq->lmt_addr; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint64_t lmt_status, i; + uint16_t pkts_left; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_io_wmb(); + + senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->cmd[2]); + sgdesc23_w0 = sgdesc01_w0; + + for (i = 0; i < pkts; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + /* Move mbuf to point to pool_id. */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, pool_id)); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf0)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf1)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf2)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf3)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * cnxk_nix_prefree_seg are written before LMTST. + */ + rte_io_wmb(); + } else { + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + /* Mark mempool object as "put" since + * it is freed by NIX + */ + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + rte_io_wmb(); +#endif + } + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd0[0] = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd0[1] = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd0[2] = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd0[3] = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + cmd1[0] = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[1] = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[2] = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd1[3] = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + do { + vst1q_u64(lmt_addr, cmd0[0]); + vst1q_u64(lmt_addr + 2, cmd1[0]); + vst1q_u64(lmt_addr + 4, cmd0[1]); + vst1q_u64(lmt_addr + 6, cmd1[1]); + vst1q_u64(lmt_addr + 8, cmd0[2]); + vst1q_u64(lmt_addr + 10, cmd1[2]); + vst1q_u64(lmt_addr + 12, cmd0[3]); + vst1q_u64(lmt_addr + 14, cmd1[3]); + lmt_status = roc_lmt_submit_ldeor(io_addr); + } while (lmt_status == 0); + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + if (unlikely(pkts_left)) + pkts += cn9k_nix_xmit_pkts(tx_queue, tx_pkts, pkts_left, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F @@ -574,6 +1314,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c new file mode 100644 index 0000000..21ffc2c --- /dev/null +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn9k_ethdev.h" +#include "cn9k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index a3cd200..534c4bd 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -20,7 +20,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', 'cn9k_tx.c', - 'cn9k_tx_mseg.c') + 'cn9k_tx_mseg.c', + 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 25/62] net/cnxk: add Rx burst for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (23 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram ` (37 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst support for CN10K SoC. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.h | 3 + drivers/net/cnxk/cn10k_rx.c | 45 ++++++++ drivers/net/cnxk/cn10k_rx.h | 236 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 286 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_rx.c diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 18deb95..596985f 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -33,4 +33,7 @@ struct cn10k_eth_rxq { uint16_t rq; } __plt_cache_aligned; +/* Rx and Tx routines */ +void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); + #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c new file mode 100644 index 0000000..8b422d0 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx.c @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R + +static inline void +pick_rx_func(struct rte_eth_dev *eth_dev, + const eth_rx_burst_t rx_burst[2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [MARK] [CKSUM] [PTYPE] [RSS] */ + eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_RSS_F)]; +} + +void +cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) +{ + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + pick_rx_func(eth_dev, nix_eth_rx_burst); + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index d3d1661..01c9d29 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -6,6 +6,242 @@ #include <rte_ether.h> +#define NIX_RX_OFFLOAD_NONE (0) +#define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) +#define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) +#define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) + +/* Flags to control cqe_to_mbuf conversion function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_RX_MULTI_SEG_F BIT(15) + +#define CNXK_NIX_CQ_ENTRY_SZ 128 +#define NIX_DESCS_PER_LOOP 4 +#define CQE_CAST(x) ((struct nix_cqe_hdr_s *)(x)) +#define CQE_SZ(x) ((x) * CNXK_NIX_CQ_ENTRY_SZ) + +union mbuf_initializer { + struct { + uint16_t data_off; + uint16_t refcnt; + uint16_t nb_segs; + uint16_t port; + } fields; + uint64_t value; +}; + +static __rte_always_inline uint64_t +nix_clear_data_off(uint64_t oldval) +{ + union mbuf_initializer mbuf_init = {.value = oldval}; + + mbuf_init.fields.data_off = 0; + return mbuf_init.value; +} + +static __rte_always_inline struct rte_mbuf * +nix_get_mbuf_from_cqe(void *cq, const uint64_t data_off) +{ + rte_iova_t buff; + + /* Skip CQE, NIX_RX_PARSE_S and SG HDR(9 DWORDs) and peek buff addr */ + buff = *((rte_iova_t *)((uint64_t *)cq + 9)); + return (struct rte_mbuf *)(buff - data_off); +} + +static __rte_always_inline uint32_t +nix_ptype_get(const void *const lookup_mem, const uint64_t in) +{ + const uint16_t *const ptype = lookup_mem; + const uint16_t lh_lg_lf = (in & 0xFFF0000000000000) >> 52; + const uint16_t tu_l2 = ptype[(in & 0x000FFFF000000000) >> 36]; + const uint16_t il4_tu = ptype[PTYPE_NON_TUNNEL_ARRAY_SZ + lh_lg_lf]; + + return (il4_tu << PTYPE_NON_TUNNEL_WIDTH) | tu_l2; +} + +static __rte_always_inline uint32_t +nix_rx_olflags_get(const void *const lookup_mem, const uint64_t in) +{ + const uint32_t *const ol_flags = + (const uint32_t *)((const uint8_t *)lookup_mem + + PTYPE_ARRAY_SZ); + + return ol_flags[(in & 0xfff00000) >> 20]; +} + +static inline uint64_t +nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, + struct rte_mbuf *mbuf) +{ + /* There is no separate bit to check match_id + * is valid or not? and no flag to identify it is an + * RTE_FLOW_ACTION_TYPE_FLAG vs RTE_FLOW_ACTION_TYPE_MARK + * action. The former case addressed through 0 being invalid + * value and inc/dec match_id pair when MARK is activated. + * The later case addressed through defining + * CNXK_FLOW_MARK_DEFAULT as value for + * RTE_FLOW_ACTION_TYPE_MARK. + * This would translate to not use + * CNXK_FLOW_ACTION_FLAG_DEFAULT - 1 and + * CNXK_FLOW_ACTION_FLAG_DEFAULT for match_id. + * i.e valid mark_id's are from + * 0 to CNXK_FLOW_ACTION_FLAG_DEFAULT - 2 + */ + if (likely(match_id)) { + ol_flags |= PKT_RX_FDIR; + if (match_id != CNXK_FLOW_ACTION_FLAG_DEFAULT) { + ol_flags |= PKT_RX_FDIR_ID; + mbuf->hash.fdir.hi = match_id - 1; + } + } + + return ol_flags; +} + +static __rte_always_inline void +cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, + struct rte_mbuf *mbuf, const void *lookup_mem, + const uint64_t val, const uint16_t flag) +{ + const union nix_rx_parse_u *rx = + (const union nix_rx_parse_u *)((const uint64_t *)cq + 1); + const uint16_t len = rx->pkt_lenm1 + 1; + const uint64_t w1 = *(const uint64_t *)rx; + uint64_t ol_flags = 0; + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + if (flag & NIX_RX_OFFLOAD_PTYPE_F) + mbuf->packet_type = nix_ptype_get(lookup_mem, w1); + else + mbuf->packet_type = 0; + + if (flag & NIX_RX_OFFLOAD_RSS_F) { + mbuf->hash.rss = tag; + ol_flags |= PKT_RX_RSS_HASH; + } + + if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) + ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) + ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); + + mbuf->ol_flags = ol_flags; + *(uint64_t *)(&mbuf->rearm_data) = val; + mbuf->pkt_len = len; + + mbuf->data_len = len; + mbuf->next = NULL; +} + +static inline uint16_t +nix_rx_nb_pkts(struct cn10k_eth_rxq *rxq, const uint64_t wdata, + const uint16_t pkts, const uint32_t qmask) +{ + uint32_t available = rxq->available; + + /* Update the available count if cached value is not enough */ + if (unlikely(available < pkts)) { + uint64_t reg, head, tail; + + /* Use LDADDA version to avoid reorder */ + reg = roc_atomic64_add_sync(wdata, rxq->cq_status); + /* CQ_OP_STATUS operation error */ + if (reg & BIT_ULL(NIX_CQ_OP_STAT_OP_ERR) || + reg & BIT_ULL(NIX_CQ_OP_STAT_CQ_ERR)) + return 0; + + tail = reg & 0xFFFFF; + head = (reg >> 20) & 0xFFFFF; + if (tail < head) + available = tail - head + qmask + 1; + else + available = tail - head; + + rxq->available = available; + } + + return RTE_MIN(pkts, available); +} + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, + const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + const uint64_t mbuf_init = rxq->mbuf_initializer; + const void *lookup_mem = rxq->lookup_mem; + const uint64_t data_off = rxq->data_off; + const uintptr_t desc = rxq->desc; + const uint64_t wdata = rxq->wdata; + const uint32_t qmask = rxq->qmask; + uint16_t packets = 0, nb_pkts; + uint32_t head = rxq->head; + struct nix_cqe_hdr_s *cq; + struct rte_mbuf *mbuf; + + nb_pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + + while (packets < nb_pkts) { + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal( + (void *)(desc + (CQE_SZ((head + 2) & qmask)))); + cq = (struct nix_cqe_hdr_s *)(desc + CQE_SZ(head)); + + mbuf = nix_get_mbuf_from_cqe(cq, data_off); + + cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, + flags); + rx_pkts[packets++] = mbuf; + roc_prefetch_store_keep(mbuf); + head++; + head &= qmask; + } + + rxq->head = head; + rxq->available -= nb_pkts; + + /* Free all the CQs that we've processed */ + plt_write64((wdata | nb_pkts), rxq->cq_door); + + return nb_pkts; +} + +#define RSS_F NIX_RX_OFFLOAD_RSS_F +#define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F +#define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F +#define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F + +/* [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ +R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); + +NIX_RX_FASTPATH_MODES +#undef R #endif /* __CN10K_RX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 534c4bd..1fcc211 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -23,7 +23,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_mseg.c', 'cn9k_tx_vec.c') # CN10K -sources += files('cn10k_ethdev.c') +sources += files('cn10k_ethdev.c', + 'cn10k_rx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 26/62] net/cnxk: add Rx multi-segment version for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (24 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 25/62] net/cnxk: add Rx burst for cn10k Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram ` (36 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Rx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- doc/guides/nics/cnxk.rst | 2 ++ doc/guides/nics/features/cnxk.ini | 2 ++ doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 2 ++ drivers/net/cnxk/cn10k_rx.c | 20 +++++++++++- drivers/net/cnxk/cn10k_rx.h | 57 +++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx_mseg.c | 17 +++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 100 insertions(+), 4 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 8bc85c0..fd7f2dd 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,11 +17,13 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Jumbo frames - SR-IOV VF - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - Link state information +- Scatter-Gather IO support Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 712f8d5..23564b7 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 82f2af0..421048d 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 61fed11..e901fa2 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,8 @@ Runtime Tx queue setup = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y +Jumbo frame = Y +Scattered Rx = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 8b422d0..ce2cfee 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -10,7 +10,7 @@ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ - } + } \ NIX_RX_FASTPATH_MODES #undef R @@ -32,6 +32,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, @@ -40,6 +42,22 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 01c9d29..c667c9a 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -103,6 +103,52 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + ((rx->desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -136,8 +182,12 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } static inline uint16_t @@ -239,6 +289,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) #define R(name, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c new file mode 100644 index 0000000..9d283f7 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 1fcc211..20bb00b 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -24,7 +24,8 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c', - 'cn10k_rx.c') + 'cn10k_rx.c', + 'cn10k_rx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 27/62] net/cnxk: add Rx vector version for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (25 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 28/62] net/cnxk: add Tx burst " Nithin Dabilpuram ` (35 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Rx burst vector version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + drivers/net/cnxk/cn10k_rx.c | 13 ++- drivers/net/cnxk/cn10k_rx.h | 222 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_rx_vec.c | 19 ++++ drivers/net/cnxk/meson.build | 3 +- 5 files changed, 256 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index fd7f2dd..481bc7e 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Link state information - Scatter-Gather IO support +- Vector Poll mode driver Prerequisites ------------- diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index ce2cfee..0598111 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -50,7 +50,18 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; - pick_rx_func(eth_dev, nix_eth_rx_burst); + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + + if (dev->scalar_ena) + pick_rx_func(eth_dev, nix_eth_rx_burst); + else + pick_rx_func(eth_dev, nix_eth_rx_vec_burst); if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index c667c9a..7bb9dd8 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -5,6 +5,7 @@ #define __CN10K_RX_H__ #include <rte_ether.h> +#include <rte_vect.h> #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) @@ -263,6 +264,224 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, return nb_pkts; } +#if defined(RTE_ARCH_ARM64) + +static __rte_always_inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + struct cn10k_eth_rxq *rxq = rx_queue; + uint16_t packets = 0; + uint64x2_t cq0_w8, cq1_w8, cq2_w8, cq3_w8, mbuf01, mbuf23; + const uint64_t mbuf_initializer = rxq->mbuf_initializer; + const uint64x2_t data_off = vdupq_n_u64(rxq->data_off); + uint64_t ol_flags0, ol_flags1, ol_flags2, ol_flags3; + uint64x2_t rearm0 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm1 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm2 = vdupq_n_u64(mbuf_initializer); + uint64x2_t rearm3 = vdupq_n_u64(mbuf_initializer); + struct rte_mbuf *mbuf0, *mbuf1, *mbuf2, *mbuf3; + const uint16_t *lookup_mem = rxq->lookup_mem; + const uint32_t qmask = rxq->qmask; + const uint64_t wdata = rxq->wdata; + const uintptr_t desc = rxq->desc; + uint8x16_t f0, f1, f2, f3; + uint32_t head = rxq->head; + uint16_t pkts_left; + + pkts = nix_rx_nb_pkts(rxq, wdata, pkts, qmask); + pkts_left = pkts & (NIX_DESCS_PER_LOOP - 1); + + /* Packets has to be floor-aligned to NIX_DESCS_PER_LOOP */ + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + while (packets < pkts) { + /* Exit loop if head is about to wrap and become unaligned */ + if (((head + NIX_DESCS_PER_LOOP - 1) & qmask) < + NIX_DESCS_PER_LOOP) { + pkts_left += (pkts - packets); + break; + } + + const uintptr_t cq0 = desc + CQE_SZ(head); + + /* Prefetch N desc ahead */ + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(8))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(9))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(10))); + rte_prefetch_non_temporal((void *)(cq0 + CQE_SZ(11))); + + /* Get NIX_RX_SG_S for size and buffer pointer */ + cq0_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(0) + 64)); + cq1_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(1) + 64)); + cq2_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(2) + 64)); + cq3_w8 = vld1q_u64((uint64_t *)(cq0 + CQE_SZ(3) + 64)); + + /* Extract mbuf from NIX_RX_SG_S */ + mbuf01 = vzip2q_u64(cq0_w8, cq1_w8); + mbuf23 = vzip2q_u64(cq2_w8, cq3_w8); + mbuf01 = vqsubq_u64(mbuf01, data_off); + mbuf23 = vqsubq_u64(mbuf23, data_off); + + /* Move mbufs to scalar registers for future use */ + mbuf0 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 0); + mbuf1 = (struct rte_mbuf *)vgetq_lane_u64(mbuf01, 1); + mbuf2 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 0); + mbuf3 = (struct rte_mbuf *)vgetq_lane_u64(mbuf23, 1); + + /* Mask to get packet len from NIX_RX_SG_S */ + const uint8x16_t shuf_msk = { + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0xFF, 0xFF, /* pkt_type set as unknown */ + 0, 1, /* octet 1~0, low 16 bits pkt_len */ + 0xFF, 0xFF, /* skip high 16 bits pkt_len, zero out */ + 0, 1, /* octet 1~0, 16 bits data_len */ + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + /* Form the rx_descriptor_fields1 with pkt_len and data_len */ + f0 = vqtbl1q_u8(cq0_w8, shuf_msk); + f1 = vqtbl1q_u8(cq1_w8, shuf_msk); + f2 = vqtbl1q_u8(cq2_w8, shuf_msk); + f3 = vqtbl1q_u8(cq3_w8, shuf_msk); + + /* Load CQE word0 and word 1 */ + uint64_t cq0_w0 = ((uint64_t *)(cq0 + CQE_SZ(0)))[0]; + uint64_t cq0_w1 = ((uint64_t *)(cq0 + CQE_SZ(0)))[1]; + uint64_t cq1_w0 = ((uint64_t *)(cq0 + CQE_SZ(1)))[0]; + uint64_t cq1_w1 = ((uint64_t *)(cq0 + CQE_SZ(1)))[1]; + uint64_t cq2_w0 = ((uint64_t *)(cq0 + CQE_SZ(2)))[0]; + uint64_t cq2_w1 = ((uint64_t *)(cq0 + CQE_SZ(2)))[1]; + uint64_t cq3_w0 = ((uint64_t *)(cq0 + CQE_SZ(3)))[0]; + uint64_t cq3_w1 = ((uint64_t *)(cq0 + CQE_SZ(3)))[1]; + + if (flags & NIX_RX_OFFLOAD_RSS_F) { + /* Fill rss in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(cq0_w0, f0, 3); + f1 = vsetq_lane_u32(cq1_w0, f1, 3); + f2 = vsetq_lane_u32(cq2_w0, f2, 3); + f3 = vsetq_lane_u32(cq3_w0, f3, 3); + ol_flags0 = PKT_RX_RSS_HASH; + ol_flags1 = PKT_RX_RSS_HASH; + ol_flags2 = PKT_RX_RSS_HASH; + ol_flags3 = PKT_RX_RSS_HASH; + } else { + ol_flags0 = 0; + ol_flags1 = 0; + ol_flags2 = 0; + ol_flags3 = 0; + } + + if (flags & NIX_RX_OFFLOAD_PTYPE_F) { + /* Fill packet_type in the rx_descriptor_fields1 */ + f0 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq0_w1), + f0, 0); + f1 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq1_w1), + f1, 0); + f2 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq2_w1), + f2, 0); + f3 = vsetq_lane_u32(nix_ptype_get(lookup_mem, cq3_w1), + f3, 0); + } + + if (flags & NIX_RX_OFFLOAD_CHECKSUM_F) { + ol_flags0 |= nix_rx_olflags_get(lookup_mem, cq0_w1); + ol_flags1 |= nix_rx_olflags_get(lookup_mem, cq1_w1); + ol_flags2 |= nix_rx_olflags_get(lookup_mem, cq2_w1); + ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); + } + + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { + ol_flags0 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, + mbuf0); + ol_flags1 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(1) + 38), ol_flags1, + mbuf1); + ol_flags2 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(2) + 38), ol_flags2, + mbuf2); + ol_flags3 = nix_update_match_id( + *(uint16_t *)(cq0 + CQE_SZ(3) + 38), ol_flags3, + mbuf3); + } + + /* Form rearm_data with ol_flags */ + rearm0 = vsetq_lane_u64(ol_flags0, rearm0, 1); + rearm1 = vsetq_lane_u64(ol_flags1, rearm1, 1); + rearm2 = vsetq_lane_u64(ol_flags2, rearm2, 1); + rearm3 = vsetq_lane_u64(ol_flags3, rearm3, 1); + + /* Update rx_descriptor_fields1 */ + vst1q_u64((uint64_t *)mbuf0->rx_descriptor_fields1, f0); + vst1q_u64((uint64_t *)mbuf1->rx_descriptor_fields1, f1); + vst1q_u64((uint64_t *)mbuf2->rx_descriptor_fields1, f2); + vst1q_u64((uint64_t *)mbuf3->rx_descriptor_fields1, f3); + + /* Update rearm_data */ + vst1q_u64((uint64_t *)mbuf0->rearm_data, rearm0); + vst1q_u64((uint64_t *)mbuf1->rearm_data, rearm1); + vst1q_u64((uint64_t *)mbuf2->rearm_data, rearm2); + vst1q_u64((uint64_t *)mbuf3->rearm_data, rearm3); + + /* Update that no more segments */ + mbuf0->next = NULL; + mbuf1->next = NULL; + mbuf2->next = NULL; + mbuf3->next = NULL; + + /* Store the mbufs to rx_pkts */ + vst1q_u64((uint64_t *)&rx_pkts[packets], mbuf01); + vst1q_u64((uint64_t *)&rx_pkts[packets + 2], mbuf23); + + /* Prefetch mbufs */ + roc_prefetch_store_keep(mbuf0); + roc_prefetch_store_keep(mbuf1); + roc_prefetch_store_keep(mbuf2); + roc_prefetch_store_keep(mbuf3); + + /* Mark mempool obj as "get" as it is alloc'ed by NIX */ + __mempool_check_cookies(mbuf0->pool, (void **)&mbuf0, 1, 1); + __mempool_check_cookies(mbuf1->pool, (void **)&mbuf1, 1, 1); + __mempool_check_cookies(mbuf2->pool, (void **)&mbuf2, 1, 1); + __mempool_check_cookies(mbuf3->pool, (void **)&mbuf3, 1, 1); + + /* Advance head pointer and packets */ + head += NIX_DESCS_PER_LOOP; + head &= qmask; + packets += NIX_DESCS_PER_LOOP; + } + + rxq->head = head; + rxq->available -= packets; + + rte_io_wmb(); + /* Free all the CQs that we've processed */ + plt_write64((rxq->wdata | packets), rxq->cq_door); + + if (unlikely(pkts_left)) + packets += cn10k_nix_recv_pkts(rx_queue, &rx_pkts[packets], + pkts_left, flags); + + return packets; +} + +#else + +static inline uint16_t +cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, + uint16_t pkts, const uint16_t flags) +{ + RTE_SET_USED(rx_queue); + RTE_SET_USED(rx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(flags); + + return 0; +} + +#endif + + #define RSS_F NIX_RX_OFFLOAD_RSS_F #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F @@ -292,6 +511,9 @@ R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ + void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); NIX_RX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c new file mode 100644 index 0000000..0fa079c --- /dev/null +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_rx.h" + +#define R(name, f3, f2, f1, f0, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ + (flags)); \ + } + +NIX_RX_FASTPATH_MODES +#undef R diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 20bb00b..6748e9c 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -25,7 +25,8 @@ sources += files('cn9k_ethdev.c', # CN10K sources += files('cn10k_ethdev.c', 'cn10k_rx.c', - 'cn10k_rx_mseg.c') + 'cn10k_rx_mseg.c', + 'cn10k_rx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 28/62] net/cnxk: add Tx burst for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (26 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram ` (34 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram From: Jerin Jacob <jerinj@marvell.com> Add Tx burst scalar version for CN10K. Signed-off-by: Jerin Jacob <jerinj@marvell.com> Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> Signed-off-by: Harman Kalra <hkalra@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 7 + doc/guides/nics/features/cnxk_vec.ini | 6 + doc/guides/nics/features/cnxk_vf.ini | 7 + drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_tx.c | 54 ++++ drivers/net/cnxk/cn10k_tx.h | 491 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 7 +- 8 files changed, 571 insertions(+), 3 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_tx.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 481bc7e..17da141 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -22,6 +22,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- Inner and Outer Checksum offload - Link state information - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 23564b7..02be26b 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,11 +12,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 421048d..8c63853 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,10 +12,16 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index e901fa2..a1bd49b 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,11 +11,18 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Fast mbuf free = Y +Free Tx mbuf on demand = Y Queue start/stop = Y +TSO = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index 596985f..d39ca31 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -35,5 +35,6 @@ struct cn10k_eth_rxq { /* Rx and Tx routines */ void cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev); +void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev); #endif /* __CN10K_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c new file mode 100644 index 0000000..13c605f --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts(tx_queue, tx_pkts, pkts, cmd, \ + flags); \ + } + +NIX_TX_FASTPATH_MODES +#undef T + +static inline void +pick_tx_func(struct rte_eth_dev *eth_dev, + const eth_tx_burst_t tx_burst[2][2][2][2][2]) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)]; +} + +void +cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) +{ + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + pick_tx_func(eth_dev, nix_eth_tx_burst); + + rte_mb(); +} diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 39d4755..c54fbfe 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,10 +4,501 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#define NIX_TX_OFFLOAD_NONE (0) +#define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) +#define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) +#define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +/* Flags to control xmit_prepare function. + * Defining it from backwards to denote its been + * not used as offload flags to pick function + */ +#define NIX_TX_MULTI_SEG_F BIT(15) + +#define NIX_TX_NEED_SEND_HDR_W1 \ + (NIX_TX_OFFLOAD_L3_L4_CSUM_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | \ + NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + #define NIX_TX_NEED_EXT_HDR \ (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) +#define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ + do { \ + /* Cached value is low, Update the fc_cache_pkts */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) { \ + /* Multiply with sqe_per_sqb to express in pkts */ \ + (txq)->fc_cache_pkts = \ + ((txq)->nb_sqb_bufs_adj - *(txq)->fc_mem) \ + << (txq)->sqes_per_sqb_log2; \ + /* Check it again for the room */ \ + if (unlikely((txq)->fc_cache_pkts < (pkts))) \ + return 0; \ + } \ + } while (0) + +/* Function to determine no of tx subdesc required in case ext + * sub desc is enabled. + */ +static __rte_always_inline int +cn10k_nix_tx_ext_subs(const uint16_t flags) +{ + return (flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; +} + +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_data(const uint16_t flags) +{ + const uint64_t dw_m1 = cn10k_nix_tx_ext_subs(flags) + 1; + uint64_t data; + + /* This will be moved to addr area */ + data = dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + +static __rte_always_inline void +cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, + const uint16_t flags) +{ + /* Send hdr */ + cmd[0] = txq->send_hdr_w0; + cmd[1] = 0; + cmd += 2; + + /* Send ext if present */ + if (flags & NIX_TX_NEED_EXT_HDR) { + *(__uint128_t *)cmd = *(const __uint128_t *)txq->cmd; + cmd += 2; + } + + /* Send sg */ + cmd[0] = txq->sg_w0; + cmd[1] = 0; +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare_tso(struct rte_mbuf *m, const uint64_t flags) +{ + uint64_t mask, ol_flags = m->ol_flags; + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uintptr_t mdata = rte_pktmbuf_mtod(m, uintptr_t); + uint16_t *iplen, *oiplen, *oudplen; + uint16_t lso_sb, paylen; + + mask = -!!(ol_flags & (PKT_TX_OUTER_IPV4 | PKT_TX_OUTER_IPV6)); + lso_sb = (mask & (m->outer_l2_len + m->outer_l3_len)) + + m->l2_len + m->l3_len + m->l4_len; + + /* Reduce payload len from base headers */ + paylen = m->pkt_len - lso_sb; + + /* Get iplen position assuming no tunnel hdr */ + iplen = (uint16_t *)(mdata + m->l2_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + + oiplen = (uint16_t *)(mdata + m->outer_l2_len + + (2 << !!(ol_flags & + PKT_TX_OUTER_IPV6))); + *oiplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*oiplen) - + paylen); + + /* Update format for UDP tunneled packet */ + if (is_udp_tun) { + oudplen = (uint16_t *)(mdata + m->outer_l2_len + + m->outer_l3_len + 4); + *oudplen = rte_cpu_to_be_16( + rte_be_to_cpu_16(*oudplen) - paylen); + } + + /* Update iplen position to inner ip hdr */ + iplen = (uint16_t *)(mdata + lso_sb - m->l3_len - + m->l4_len + + (2 << !!(ol_flags & PKT_TX_IPV6))); + } + + *iplen = rte_cpu_to_be_16(rte_be_to_cpu_16(*iplen) - paylen); + } +} + +static __rte_always_inline void +cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, + const uint16_t flags, const uint64_t lso_tun_fmt) +{ + struct nix_send_ext_s *send_hdr_ext; + struct nix_send_hdr_s *send_hdr; + uint64_t ol_flags = 0, mask; + union nix_send_hdr_w1_u w1; + union nix_send_sg_s *sg; + + send_hdr = (struct nix_send_hdr_s *)cmd; + if (flags & NIX_TX_NEED_EXT_HDR) { + send_hdr_ext = (struct nix_send_ext_s *)(cmd + 2); + sg = (union nix_send_sg_s *)(cmd + 4); + /* Clear previous markings */ + send_hdr_ext->w0.lso = 0; + send_hdr_ext->w1.u = 0; + } else { + sg = (union nix_send_sg_s *)(cmd + 2); + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) { + ol_flags = m->ol_flags; + w1.u = 0; + } + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + send_hdr->w0.total = m->data_len; + send_hdr->w0.aura = + roc_npa_aura_handle_to_aura(m->pool->pool_id); + } + + /* + * L3type: 2 => IPV4 + * 3 => IPV4 with csum + * 4 => IPV6 + * L3type and L3ptr needs to be set for either + * L3 csum or L4 csum or LSO + * + */ + + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t ol3type = + ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L3 */ + w1.ol3type = ol3type; + mask = 0xffffull << ((!!ol3type) << 4); + w1.ol3ptr = ~mask & m->outer_l2_len; + w1.ol4ptr = ~mask & (w1.ol3ptr + m->outer_l3_len); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + /* Inner L3 */ + w1.il3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2); + w1.il3ptr = w1.ol4ptr + m->l2_len; + w1.il4ptr = w1.il3ptr + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.il3type = w1.il3type + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.il4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + + /* In case of no tunnel header use only + * shift IL3/IL4 fields a bit to use + * OL3/OL4 for header checksum + */ + mask = !ol3type; + w1.u = ((w1.u & 0xFFFFFFFF00000000) >> (mask << 3)) | + ((w1.u & 0X00000000FFFFFFFF) >> (mask << 4)); + + } else if (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + const uint8_t csum = !!(ol_flags & PKT_TX_OUTER_UDP_CKSUM); + const uint8_t outer_l2_len = m->outer_l2_len; + + /* Outer L3 */ + w1.ol3ptr = outer_l2_len; + w1.ol4ptr = outer_l2_len + m->outer_l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_OUTER_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_OUTER_IPV6)) << 2) + + !!(ol_flags & PKT_TX_OUTER_IP_CKSUM); + + /* Outer L4 */ + w1.ol4type = csum + (csum << 1); + + } else if (flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) { + const uint8_t l2_len = m->l2_len; + + /* Always use OLXPTR and OLXTYPE when only + * when one header is present + */ + + /* Inner L3 */ + w1.ol3ptr = l2_len; + w1.ol4ptr = l2_len + m->l3_len; + /* Increment it by 1 if it is IPV4 as 3 is with csum */ + w1.ol3type = ((!!(ol_flags & PKT_TX_IPV4)) << 1) + + ((!!(ol_flags & PKT_TX_IPV6)) << 2) + + !!(ol_flags & PKT_TX_IP_CKSUM); + + /* Inner L4 */ + w1.ol4type = (ol_flags & PKT_TX_L4_MASK) >> 52; + } + + if (flags & NIX_TX_NEED_EXT_HDR && flags & NIX_TX_OFFLOAD_VLAN_QINQ_F) { + send_hdr_ext->w1.vlan1_ins_ena = !!(ol_flags & PKT_TX_VLAN); + /* HW will update ptr after vlan0 update */ + send_hdr_ext->w1.vlan1_ins_ptr = 12; + send_hdr_ext->w1.vlan1_ins_tci = m->vlan_tci; + + send_hdr_ext->w1.vlan0_ins_ena = !!(ol_flags & PKT_TX_QINQ); + /* 2B before end of l2 header */ + send_hdr_ext->w1.vlan0_ins_ptr = 12; + send_hdr_ext->w1.vlan0_ins_tci = m->vlan_tci_outer; + } + + if (flags & NIX_TX_OFFLOAD_TSO_F && (ol_flags & PKT_TX_TCP_SEG)) { + uint16_t lso_sb; + uint64_t mask; + + mask = -(!w1.il3type); + lso_sb = (mask & w1.ol4ptr) + (~mask & w1.il4ptr) + m->l4_len; + + send_hdr_ext->w0.lso_sb = lso_sb; + send_hdr_ext->w0.lso = 1; + send_hdr_ext->w0.lso_mps = m->tso_segsz; + send_hdr_ext->w0.lso_format = + NIX_LSO_FORMAT_IDX_TSOV4 + !!(ol_flags & PKT_TX_IPV6); + w1.ol4type = NIX_SENDL4TYPE_TCP_CKSUM; + + /* Handle tunnel tso */ + if ((flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (ol_flags & PKT_TX_TUNNEL_MASK)) { + const uint8_t is_udp_tun = + (CNXK_NIX_UDP_TUN_BITMASK >> + ((ol_flags & PKT_TX_TUNNEL_MASK) >> 45)) & + 0x1; + uint8_t shift = is_udp_tun ? 32 : 0; + + shift += (!!(ol_flags & PKT_TX_OUTER_IPV6) << 4); + shift += (!!(ol_flags & PKT_TX_IPV6) << 3); + + w1.il4type = NIX_SENDL4TYPE_TCP_CKSUM; + w1.ol4type = is_udp_tun ? NIX_SENDL4TYPE_UDP_CKSUM : 0; + /* Update format for UDP tunneled packet */ + send_hdr_ext->w0.lso_format = (lso_tun_fmt >> shift); + } + } + + if (flags & NIX_TX_NEED_SEND_HDR_W1) + send_hdr->w1.u = w1.u; + + if (!(flags & NIX_TX_MULTI_SEG_F)) { + sg->seg1_size = m->data_len; + *(rte_iova_t *)(sg + 1) = rte_mbuf_data_iova(m); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* DF bit = 1 if refcount of current mbuf or parent mbuf + * is greater than 1 + * DF bit = 0 otherwise + */ + send_hdr->w0.df = cnxk_nix_prefree_seg(m); + } + /* Mark mempool object as "put" since it is freed by NIX */ + if (!send_hdr->w0.df) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); + } + + /* With minimal offloads, 'cmd' being local could be optimized out to + * registers. In other cases, 'cmd' will be in stack. Intent is + * 'cmd' stores content from txq->cmd which is copied only once. + */ + *((struct nix_send_hdr_s *)lmt_addr) = *send_hdr; + lmt_addr += 16; + if (flags & NIX_TX_NEED_EXT_HDR) { + *((struct nix_send_ext_s *)lmt_addr) = *send_hdr_ext; + lmt_addr += 16; + } + /* In case of multi-seg, sg template is stored here */ + *((union nix_send_sg_s *)lmt_addr) = *sg; + *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); +} + +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, + uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + const rte_iova_t io_addr = txq->io_addr; + uintptr_t pa, lmt_addr = txq->lmt_base; + uint16_t lmt_id, burst, left, i; + uint64_t lso_tun_fmt; + uint64_t data; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + /* Get cmd skeleton */ + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + if (flags & NIX_TX_OFFLOAD_TSO_F) + lso_tun_fmt = txq->lso_tun_fmt; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, + lso_tun_fmt); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + } + + /* Trigger LMTST */ + if (burst > 16) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (burst) { + data = cn10k_nix_tx_steor_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(burst - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + +#define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F +#define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F +#define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F +#define TSO_F NIX_TX_OFFLOAD_TSO_F + +/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); + +NIX_TX_FASTPATH_MODES +#undef T + #endif /* __CN10K_TX_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 6748e9c..b1ba824 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -26,13 +26,14 @@ sources += files('cn9k_ethdev.c', sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', - 'cn10k_rx_vec.c') + 'cn10k_rx_vec.c', + 'cn10k_tx.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -# Allow implicit vector conversions -extra_flags = ['-flax-vector-conversions'] +# Allow implicit vector conversions and strict aliasing warning +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] foreach flag: extra_flags if cc.has_argument(flag) cflags += flag -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 29/62] net/cnxk: add Tx multi-segment version for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (27 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 28/62] net/cnxk: add Tx burst " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram ` (33 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst multi-segment version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 18 ++++- drivers/net/cnxk/cn10k_tx.h | 171 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_mseg.c | 25 ++++++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 215 insertions(+), 2 deletions(-) create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 13c605f..9803002 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -40,6 +40,8 @@ pick_tx_func(struct rte_eth_dev *eth_dev, void cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { #define T(name, f4, f3, f2, f1, f0, sz, flags) \ [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, @@ -48,7 +50,21 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; - pick_tx_func(eth_dev, nix_eth_tx_burst); + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + + if (dev->scalar_ena || + (dev->tx_offload_flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + pick_tx_func(eth_dev, nix_eth_tx_burst); + + if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index c54fbfe..63e9848 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -339,6 +339,77 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, } static __rte_always_inline uint16_t +cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) +{ + struct nix_send_hdr_s *send_hdr; + union nix_send_sg_s *sg; + struct rte_mbuf *m_next; + uint64_t *slist, sg_u; + uint64_t nb_segs; + uint64_t segdw; + uint8_t off, i; + + send_hdr = (struct nix_send_hdr_s *)cmd; + send_hdr->w0.total = m->pkt_len; + send_hdr->w0.aura = roc_npa_aura_handle_to_aura(m->pool->pool_id); + + if (flags & NIX_TX_NEED_EXT_HDR) + off = 2; + else + off = 0; + + sg = (union nix_send_sg_s *)&cmd[2 + off]; + /* Clear sg->u header before use */ + sg->u &= 0xFC00000000000000; + sg_u = sg->u; + slist = &cmd[3 + off]; + + i = 0; + nb_segs = m->nb_segs; + + /* Fill mbuf segments */ + do { + m_next = m->next; + sg_u = sg_u | ((uint64_t)m->data_len << (i << 4)); + *slist = rte_mbuf_data_iova(m); + /* Set invert df if buffer is not to be freed by H/W */ + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) + sg_u |= (cnxk_nix_prefree_seg(m) << (i + 55)); + /* Mark mempool object as "put" since it is freed by NIX + */ +#ifdef RTE_LIBRTE_MEMPOOL_DEBUG + if (!(sg_u & (1ULL << (i + 55)))) + __mempool_check_cookies(m->pool, (void **)&m, 1, 0); +#endif + slist++; + i++; + nb_segs--; + if (i > 2 && nb_segs) { + i = 0; + /* Next SG subdesc */ + *(uint64_t *)slist = sg_u & 0xFC00000000000000; + sg->u = sg_u; + sg->segs = 3; + sg = (union nix_send_sg_s *)slist; + sg_u = sg->u; + slist++; + } + m = m_next; + } while (nb_segs); + + sg->u = sg_u; + sg->segs = i; + segdw = (uint64_t *)slist - (uint64_t *)&cmd[2 + off]; + /* Roundup extra dwords to multiple of 2 */ + segdw = (segdw >> 1) + (segdw & 0x1); + /* Default dwords */ + segdw += (off >> 1) + 1; + send_hdr->w0.sizem1 = segdw - 1; + + return segdw; +} + +static __rte_always_inline uint16_t cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, uint64_t *cmd, const uint16_t flags) { @@ -421,6 +492,103 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, return pkts; } +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t pa0, pa1, lmt_addr = txq->lmt_base; + const rte_iova_t io_addr = txq->io_addr; + uint16_t segdw, lmt_id, burst, left, i; + uint64_t data0, data1; + uint64_t lso_tun_fmt; + __uint128_t data128; + uint16_t shft; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + cn10k_nix_tx_skeleton(txq, cmd, flags); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + if (flags & NIX_TX_OFFLOAD_TSO_F) + lso_tun_fmt = txq->lso_tun_fmt; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + left = pkts; +again: + burst = left > 32 ? 32 : left; + shft = 16; + data128 = 0; + for (i = 0; i < burst; i++) { + /* Perform header writes for TSO, barrier at + * lmt steorl will suffice. + */ + if (flags & NIX_TX_OFFLOAD_TSO_F) + cn10k_nix_xmit_prepare_tso(tx_pkts[i], flags); + + cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, + lso_tun_fmt); + /* Store sg list directly on lmt line */ + segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, + flags); + lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); + data128 |= (((__uint128_t)(segdw - 1)) << shft); + shft += 3; + } + + data0 = (uint64_t)data128; + data1 = (uint64_t)(data128 >> 64); + /* Make data0 similar to data1 */ + data0 >>= 16; + /* Trigger LMTST */ + if (burst > 16) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= (15ULL << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + + pa1 = io_addr | (data1 & 0x7) << 4; + data1 &= ~0x7ULL; + data1 <<= 16; + data1 |= ((uint64_t)(burst - 17)) << 12; + data1 |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data1, pa1); + } else if (burst) { + pa0 = io_addr | (data0 & 0x7) << 4; + data0 &= ~0x7ULL; + /* Move lmtst1..15 sz to bits 63:19 */ + data0 <<= 16; + data0 |= ((burst - 1) << 12); + data0 |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data0, pa0); + } + + left -= burst; + rte_io_wmb(); + if (left) { + /* Start processing another burst */ + tx_pkts += burst; + /* Reset lmt base addr */ + lmt_addr -= (1ULL << ROC_LMT_LINE_SIZE_LOG2); + lmt_addr &= (~(BIT_ULL(ROC_LMT_BASE_PER_CORE_LOG2) - 1)); + goto again; + } + + return pkts; +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -496,6 +664,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ #define T(name, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c new file mode 100644 index 0000000..6ae6907 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[(sz)]; \ + \ + /* For TSO inner checksum is a must */ \ + if (((flags) & NIX_TX_OFFLOAD_TSO_F) && \ + !((flags) & NIX_TX_OFFLOAD_L3_L4_CSUM_F)) \ + return 0; \ + return cn10k_nix_xmit_pkts_mseg(tx_queue, tx_pkts, pkts, cmd, \ + (flags) | NIX_TX_MULTI_SEG_F); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index b1ba824..21e5676 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -27,7 +27,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', - 'cn10k_tx.c') + 'cn10k_tx.c', + 'cn10k_tx_mseg.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 30/62] net/cnxk: add Tx vector version for cn10k 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (28 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram ` (32 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add Tx burst vector version for CN10K. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- drivers/net/cnxk/cn10k_tx.c | 10 + drivers/net/cnxk/cn10k_tx.h | 815 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_tx_vec.c | 25 ++ drivers/net/cnxk/meson.build | 3 +- 4 files changed, 852 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index 9803002..e6eb101 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -58,10 +58,20 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) #undef T }; + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + + NIX_TX_FASTPATH_MODES +#undef T + }; + if (dev->scalar_ena || (dev->tx_offload_flags & (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); + else + pick_tx_func(eth_dev, nix_eth_tx_vec_burst); if (dev->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) pick_tx_func(eth_dev, nix_eth_tx_burst_mseg); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index 63e9848..b74df10 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -4,6 +4,8 @@ #ifndef __CN10K_TX_H__ #define __CN10K_TX_H__ +#include <rte_vect.h> + #define NIX_TX_OFFLOAD_NONE (0) #define NIX_TX_OFFLOAD_L3_L4_CSUM_F BIT(0) #define NIX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(1) @@ -38,6 +40,9 @@ } \ } while (0) +#define LMT_OFF(lmt_addr, lmt_num, offset) \ + (void *)((lmt_addr) + ((lmt_num) << ROC_LMT_LINE_SIZE_LOG2) + (offset)) + /* Function to determine no of tx subdesc required in case ext * sub desc is enabled. */ @@ -48,6 +53,14 @@ cn10k_nix_tx_ext_subs(const uint16_t flags) (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; } +static __rte_always_inline uint8_t +cn10k_nix_pkts_per_vec_brst(const uint16_t flags) +{ + RTE_SET_USED(flags); + /* We can pack up to 4 packets per LMTLINE if there are no offloads. */ + return 4 << ROC_LMT_LINES_PER_CORE_LOG2; +} + static __rte_always_inline uint64_t cn10k_nix_tx_steor_data(const uint16_t flags) { @@ -76,6 +89,35 @@ cn10k_nix_tx_steor_data(const uint16_t flags) return data; } +static __rte_always_inline uint64_t +cn10k_nix_tx_steor_vec_data(const uint16_t flags) +{ + const uint64_t dw_m1 = 0x7; + uint64_t data; + + RTE_SET_USED(flags); + /* This will be moved to addr area */ + data = dw_m1; + /* 15 vector sizes for single seg */ + data |= dw_m1 << 19; + data |= dw_m1 << 22; + data |= dw_m1 << 25; + data |= dw_m1 << 28; + data |= dw_m1 << 31; + data |= dw_m1 << 34; + data |= dw_m1 << 37; + data |= dw_m1 << 40; + data |= dw_m1 << 43; + data |= dw_m1 << 46; + data |= dw_m1 << 49; + data |= dw_m1 << 52; + data |= dw_m1 << 55; + data |= dw_m1 << 58; + data |= dw_m1 << 61; + + return data; +} + static __rte_always_inline void cn10k_nix_tx_skeleton(const struct cn10k_eth_txq *txq, uint64_t *cmd, const uint16_t flags) @@ -589,6 +631,776 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, return pkts; } +#if defined(RTE_ARCH_ARM64) + +#define NIX_DESCS_PER_LOOP 4 +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + uint64x2_t dataoff_iova0, dataoff_iova1, dataoff_iova2, dataoff_iova3; + uint64x2_t len_olflags0, len_olflags1, len_olflags2, len_olflags3; + uint64x2_t cmd0[NIX_DESCS_PER_LOOP], cmd1[NIX_DESCS_PER_LOOP]; + uint64_t *mbuf0, *mbuf1, *mbuf2, *mbuf3, data, pa; + uint64x2_t senddesc01_w0, senddesc23_w0; + uint64x2_t senddesc01_w1, senddesc23_w1; + uint16_t left, scalar, burst, i, lmt_id; + uint64x2_t sgdesc01_w0, sgdesc23_w0; + uint64x2_t sgdesc01_w1, sgdesc23_w1; + struct cn10k_eth_txq *txq = tx_queue; + uintptr_t laddr = txq->lmt_base; + rte_iova_t io_addr = txq->io_addr; + uint64x2_t ltypes01, ltypes23; + uint64x2_t xtmp128, ytmp128; + uint64x2_t xmask01, xmask23; + uint8_t lnum; + + NIX_XMIT_FC_OR_RETURN(txq, pkts); + + scalar = pkts & (NIX_DESCS_PER_LOOP - 1); + pkts = RTE_ALIGN_FLOOR(pkts, NIX_DESCS_PER_LOOP); + + /* Reduce the cached count */ + txq->fc_cache_pkts -= pkts; + + senddesc01_w0 = vld1q_dup_u64(&txq->send_hdr_w0); + senddesc23_w0 = senddesc01_w0; + senddesc01_w1 = vdupq_n_u64(0); + senddesc23_w1 = senddesc01_w1; + sgdesc01_w0 = vld1q_dup_u64(&txq->sg_w0); + sgdesc23_w0 = sgdesc01_w0; + + /* Get LMT base address and LMT ID as lcore id */ + ROC_LMT_BASE_ID_GET(laddr, lmt_id); + left = pkts; +again: + /* Number of packets to prepare depends on offloads enabled. */ + burst = left > cn10k_nix_pkts_per_vec_brst(flags) ? + cn10k_nix_pkts_per_vec_brst(flags) : + left; + lnum = 0; + for (i = 0; i < burst; i += NIX_DESCS_PER_LOOP) { + /* Clear lower 32bit of SEND_HDR_W0 and SEND_SG_W0 */ + senddesc01_w0 = + vbicq_u64(senddesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + sgdesc01_w0 = vbicq_u64(sgdesc01_w0, vdupq_n_u64(0xFFFFFFFF)); + + senddesc23_w0 = senddesc01_w0; + sgdesc23_w0 = sgdesc01_w0; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, buf_iova)); + /* + * Get mbuf's, olflags, iova, pktlen, dataoff + * dataoff_iovaX.D[0] = iova, + * dataoff_iovaX.D[1](15:0) = mbuf->dataoff + * len_olflagsX.D[0] = ol_flags, + * len_olflagsX.D[1](63:32) = mbuf->pkt_len + */ + dataoff_iova0 = vld1q_u64(mbuf0); + len_olflags0 = vld1q_u64(mbuf0 + 2); + dataoff_iova1 = vld1q_u64(mbuf1); + len_olflags1 = vld1q_u64(mbuf1 + 2); + dataoff_iova2 = vld1q_u64(mbuf2); + len_olflags2 = vld1q_u64(mbuf2 + 2); + dataoff_iova3 = vld1q_u64(mbuf3); + len_olflags3 = vld1q_u64(mbuf3 + 2); + + /* Move mbufs to point pool */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mbuf, pool) - + offsetof(struct rte_mbuf, buf_iova)); + + if (flags & (NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F)) { + /* Get tx_offload for ol2, ol3, l2, l3 lengths */ + /* + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + * E(8):OL2_LEN(7):OL3_LEN(9):E(24):L3_LEN(9):L2_LEN(7) + */ + + asm volatile("LD1 {%[a].D}[0],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf0 + 2) + : "memory"); + + asm volatile("LD1 {%[a].D}[1],[%[in]]\n\t" + : [a] "+w"(senddesc01_w1) + : [in] "r"(mbuf1 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[0],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf2 + 2) + : "memory"); + + asm volatile("LD1 {%[b].D}[1],[%[in]]\n\t" + : [b] "+w"(senddesc23_w1) + : [in] "r"(mbuf3 + 2) + : "memory"); + + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } else { + /* Get pool pointer alone */ + mbuf0 = (uint64_t *)*mbuf0; + mbuf1 = (uint64_t *)*mbuf1; + mbuf2 = (uint64_t *)*mbuf2; + mbuf3 = (uint64_t *)*mbuf3; + } + + const uint8x16_t shuf_mask2 = { + 0x4, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xc, 0xd, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + xtmp128 = vzip2q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip2q_u64(len_olflags2, len_olflags3); + + /* Clear dataoff_iovaX.D[1] bits other than dataoff(15:0) */ + const uint64x2_t and_mask0 = { + 0xFFFFFFFFFFFFFFFF, + 0x000000000000FFFF, + }; + + dataoff_iova0 = vandq_u64(dataoff_iova0, and_mask0); + dataoff_iova1 = vandq_u64(dataoff_iova1, and_mask0); + dataoff_iova2 = vandq_u64(dataoff_iova2, and_mask0); + dataoff_iova3 = vandq_u64(dataoff_iova3, and_mask0); + + /* + * Pick only 16 bits of pktlen preset at bits 63:32 + * and place them at bits 15:0. + */ + xtmp128 = vqtbl1q_u8(xtmp128, shuf_mask2); + ytmp128 = vqtbl1q_u8(ytmp128, shuf_mask2); + + /* Add pairwise to get dataoff + iova in sgdesc_w1 */ + sgdesc01_w1 = vpaddq_u64(dataoff_iova0, dataoff_iova1); + sgdesc23_w1 = vpaddq_u64(dataoff_iova2, dataoff_iova3); + + /* Orr both sgdesc_w0 and senddesc_w0 with 16 bits of + * pktlen at 15:0 position. + */ + sgdesc01_w0 = vorrq_u64(sgdesc01_w0, xtmp128); + sgdesc23_w0 = vorrq_u64(sgdesc23_w0, ytmp128); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xtmp128); + senddesc23_w0 = vorrq_u64(senddesc23_w0, ytmp128); + + /* Move mbuf to point to pool_id. */ + mbuf0 = (uint64_t *)((uintptr_t)mbuf0 + + offsetof(struct rte_mempool, pool_id)); + mbuf1 = (uint64_t *)((uintptr_t)mbuf1 + + offsetof(struct rte_mempool, pool_id)); + mbuf2 = (uint64_t *)((uintptr_t)mbuf2 + + offsetof(struct rte_mempool, pool_id)); + mbuf3 = (uint64_t *)((uintptr_t)mbuf3 + + offsetof(struct rte_mempool, pool_id)); + + if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + !(flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * il3/il4 types. But we still use ol3/ol4 types in + * senddesc_w1 as only one header processing is enabled. + */ + const uint8x16_t tbl = { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6 assumed) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6 assumed) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6 assumed) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6 assumed) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | PKT_TX_TCP_CKSUM */ + 0x23, /* PKT_TX_IP_CKSUM | PKT_TX_SCTP_CKSUM */ + 0x33, /* PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | PKT_TX_TCP_CKSUM */ + 0x22, /* PKT_TX_IPV4 | PKT_TX_SCTP_CKSUM */ + 0x32, /* PKT_TX_IPV4 | PKT_TX_UDP_CKSUM */ + 0x03, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):L3_LEN(9):L2_LEN(7+z) + * E(47):L3_LEN(9):L2_LEN(7+z) + */ + senddesc01_w1 = vshlq_n_u64(senddesc01_w1, 1); + senddesc23_w1 = vshlq_n_u64(senddesc23_w1, 1); + + /* Move OLFLAGS bits 55:52 to 51:48 + * with zeros preprended on the byte and rest + * don't care + */ + xtmp128 = vshrq_n_u8(xtmp128, 4); + ytmp128 = vshrq_n_u8(ytmp128, 4); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x6, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xE, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if (!(flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* + * Lookup table to translate ol_flags to + * ol3/ol4 types. + */ + + const uint8x16_t tbl = { + /* [0-15] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IP_CKSUM */ + 0x32, /* OUTER_UDP_CKSUM | OUTER_IPV4 */ + 0x33, /* OUTER_UDP_CKSUM | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | OUTER_IPV6 */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }; + + /* Extract olflags to translate to iltypes */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(47):OL3_LEN(9):OL2_LEN(7+z) + * E(47):OL3_LEN(9):OL2_LEN(7+z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer ol flags only */ + const uint64x2_t o_cksum_mask = { + 0x1C00020000000000, + 0x1C00020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, o_cksum_mask); + ytmp128 = vandq_u64(ytmp128, o_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift oltype by 2 to start nibble from BIT(56) + * instead of BIT(58) + */ + xtmp128 = vshrq_n_u8(xtmp128, 2); + ytmp128 = vshrq_n_u8(ytmp128, 2); + /* + * E(48):L3_LEN(8):L2_LEN(z+7) + * E(48):L3_LEN(8):L2_LEN(z+7) + */ + const int8x16_t tshft3 = { + -1, 0, 8, 8, 8, 8, 8, 8, + -1, 0, 8, 8, 8, 8, 8, 8, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Do the lookup */ + ltypes01 = vqtbl1q_u8(tbl, xtmp128); + ltypes23 = vqtbl1q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 56:63 of oltype + * and place it in ol3/ol4type of senddesc_w1 + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xFF, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare ol4ptr, ol3ptr from ol3len, ol2len. + * a [E(32):E(16):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E(32):E(16):(OL3+OL2):OL2] + * => E(32):E(16)::OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u16(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u16(senddesc23_w1, 8)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } else if ((flags & NIX_TX_OFFLOAD_L3_L4_CSUM_F) && + (flags & NIX_TX_OFFLOAD_OL3_OL4_CSUM_F)) { + /* Lookup table to translate ol_flags to + * ol4type, ol3type, il4type, il3type of senddesc_w1 + */ + const uint8x16x2_t tbl = {{ + { + /* [0-15] = il4type:il3type */ + 0x04, /* none (IPv6) */ + 0x14, /* PKT_TX_TCP_CKSUM (IPv6) */ + 0x24, /* PKT_TX_SCTP_CKSUM (IPv6) */ + 0x34, /* PKT_TX_UDP_CKSUM (IPv6) */ + 0x03, /* PKT_TX_IP_CKSUM */ + 0x13, /* PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + 0x02, /* PKT_TX_IPV4 */ + 0x12, /* PKT_TX_IPV4 | + * PKT_TX_TCP_CKSUM + */ + 0x22, /* PKT_TX_IPV4 | + * PKT_TX_SCTP_CKSUM + */ + 0x32, /* PKT_TX_IPV4 | + * PKT_TX_UDP_CKSUM + */ + 0x03, /* PKT_TX_IPV4 | + * PKT_TX_IP_CKSUM + */ + 0x13, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_TCP_CKSUM + */ + 0x23, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_SCTP_CKSUM + */ + 0x33, /* PKT_TX_IPV4 | PKT_TX_IP_CKSUM | + * PKT_TX_UDP_CKSUM + */ + }, + + { + /* [16-31] = ol4type:ol3type */ + 0x00, /* none */ + 0x03, /* OUTER_IP_CKSUM */ + 0x02, /* OUTER_IPV4 */ + 0x03, /* OUTER_IPV4 | OUTER_IP_CKSUM */ + 0x04, /* OUTER_IPV6 */ + 0x00, /* OUTER_IPV6 | OUTER_IP_CKSUM */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 */ + 0x00, /* OUTER_IPV6 | OUTER_IPV4 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IP_CKSUM + */ + 0x32, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 + */ + 0x33, /* OUTER_UDP_CKSUM | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + 0x34, /* OUTER_UDP_CKSUM | + * OUTER_IPV6 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IP_CKSUM + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 + */ + 0x00, /* OUTER_UDP_CKSUM | OUTER_IPV6 | + * OUTER_IPV4 | OUTER_IP_CKSUM + */ + }, + }}; + + /* Extract olflags to translate to oltype & iltype */ + xtmp128 = vzip1q_u64(len_olflags0, len_olflags1); + ytmp128 = vzip1q_u64(len_olflags2, len_olflags3); + + /* + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + * E(8):OL2_LN(7):OL3_LN(9):E(23):L3_LN(9):L2_LN(7+z) + */ + const uint32x4_t tshft_4 = { + 1, + 0, + 1, + 0, + }; + senddesc01_w1 = vshlq_u32(senddesc01_w1, tshft_4); + senddesc23_w1 = vshlq_u32(senddesc23_w1, tshft_4); + + /* + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + * E(32):L3_LEN(8):L2_LEN(7+Z):OL3_LEN(8):OL2_LEN(7+Z) + */ + const uint8x16_t shuf_mask5 = { + 0x6, 0x5, 0x0, 0x1, 0xFF, 0xFF, 0xFF, 0xFF, + 0xE, 0xD, 0x8, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, + }; + senddesc01_w1 = vqtbl1q_u8(senddesc01_w1, shuf_mask5); + senddesc23_w1 = vqtbl1q_u8(senddesc23_w1, shuf_mask5); + + /* Extract outer and inner header ol_flags */ + const uint64x2_t oi_cksum_mask = { + 0x1CF0020000000000, + 0x1CF0020000000000, + }; + + xtmp128 = vandq_u64(xtmp128, oi_cksum_mask); + ytmp128 = vandq_u64(ytmp128, oi_cksum_mask); + + /* Extract OUTER_UDP_CKSUM bit 41 and + * move it to bit 61 + */ + + xtmp128 = xtmp128 | vshlq_n_u64(xtmp128, 20); + ytmp128 = ytmp128 | vshlq_n_u64(ytmp128, 20); + + /* Shift right oltype by 2 and iltype by 4 + * to start oltype nibble from BIT(58) + * instead of BIT(56) and iltype nibble from BIT(48) + * instead of BIT(52). + */ + const int8x16_t tshft5 = { + 8, 8, 8, 8, 8, 8, -4, -2, + 8, 8, 8, 8, 8, 8, -4, -2, + }; + + xtmp128 = vshlq_u8(xtmp128, tshft5); + ytmp128 = vshlq_u8(ytmp128, tshft5); + /* + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + * E(32):L3_LEN(8):L2_LEN(8):OL3_LEN(8):OL2_LEN(8) + */ + const int8x16_t tshft3 = { + -1, 0, -1, 0, 0, 0, 0, 0, + -1, 0, -1, 0, 0, 0, 0, 0, + }; + + senddesc01_w1 = vshlq_u8(senddesc01_w1, tshft3); + senddesc23_w1 = vshlq_u8(senddesc23_w1, tshft3); + + /* Mark Bit(4) of oltype */ + const uint64x2_t oi_cksum_mask2 = { + 0x1000000000000000, + 0x1000000000000000, + }; + + xtmp128 = vorrq_u64(xtmp128, oi_cksum_mask2); + ytmp128 = vorrq_u64(ytmp128, oi_cksum_mask2); + + /* Do the lookup */ + ltypes01 = vqtbl2q_u8(tbl, xtmp128); + ltypes23 = vqtbl2q_u8(tbl, ytmp128); + + /* Pick only relevant fields i.e Bit 48:55 of iltype and + * Bit 56:63 of oltype and place it in corresponding + * place in senddesc_w1. + */ + const uint8x16_t shuf_mask0 = { + 0xFF, 0xFF, 0xFF, 0xFF, 0x7, 0x6, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xF, 0xE, 0xFF, 0xFF, + }; + + ltypes01 = vqtbl1q_u8(ltypes01, shuf_mask0); + ltypes23 = vqtbl1q_u8(ltypes23, shuf_mask0); + + /* Prepare l4ptr, l3ptr, ol4ptr, ol3ptr from + * l3len, l2len, ol3len, ol2len. + * a [E(32):L3(8):L2(8):OL3(8):OL2(8)] + * a = a + (a << 8) + * a [E:(L3+L2):(L2+OL3):(OL3+OL2):OL2] + * a = a + (a << 16) + * a [E:(L3+L2+OL3+OL2):(L2+OL3+OL2):(OL3+OL2):OL2] + * => E(32):IL4PTR(8):IL3PTR(8):OL4PTR(8):OL3PTR(8) + */ + senddesc01_w1 = vaddq_u8(senddesc01_w1, + vshlq_n_u32(senddesc01_w1, 8)); + senddesc23_w1 = vaddq_u8(senddesc23_w1, + vshlq_n_u32(senddesc23_w1, 8)); + + /* Continue preparing l4ptr, l3ptr, ol4ptr, ol3ptr */ + senddesc01_w1 = vaddq_u8( + senddesc01_w1, vshlq_n_u32(senddesc01_w1, 16)); + senddesc23_w1 = vaddq_u8( + senddesc23_w1, vshlq_n_u32(senddesc23_w1, 16)); + + /* Move ltypes to senddesc*_w1 */ + senddesc01_w1 = vorrq_u64(senddesc01_w1, ltypes01); + senddesc23_w1 = vorrq_u64(senddesc23_w1, ltypes23); + } + + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + asm volatile("LD1 {%[a].H}[0],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf0) + : "memory"); + + asm volatile("LD1 {%[a].H}[4],[%[in]]\n\t" + : [a] "+w"(xmask01) + : [in] "r"(mbuf1) + : "memory"); + + asm volatile("LD1 {%[b].H}[0],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf2) + : "memory"); + + asm volatile("LD1 {%[b].H}[4],[%[in]]\n\t" + : [b] "+w"(xmask23) + : [in] "r"(mbuf3) + : "memory"); + xmask01 = vshlq_n_u64(xmask01, 20); + xmask23 = vshlq_n_u64(xmask23, 20); + + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + + if (flags & NIX_TX_OFFLOAD_MBUF_NOFF_F) { + /* Set don't free bit if reference count > 1 */ + xmask01 = vdupq_n_u64(0); + xmask23 = xmask01; + + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf0)) + vsetq_lane_u64(0x80000, xmask01, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf1)) + vsetq_lane_u64(0x80000, xmask01, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf2)) + vsetq_lane_u64(0x80000, xmask23, 0); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + if (cnxk_nix_prefree_seg((struct rte_mbuf *)mbuf3)) + vsetq_lane_u64(0x80000, xmask23, 1); + else + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); + senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + } else { + /* Move mbufs to iova */ + mbuf0 = (uint64_t *)tx_pkts[0]; + mbuf1 = (uint64_t *)tx_pkts[1]; + mbuf2 = (uint64_t *)tx_pkts[2]; + mbuf3 = (uint64_t *)tx_pkts[3]; + + /* Mark mempool object as "put" since + * it is freed by NIX + */ + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf0)->pool, + (void **)&mbuf0, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf1)->pool, + (void **)&mbuf1, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf2)->pool, + (void **)&mbuf2, 1, 0); + + __mempool_check_cookies( + ((struct rte_mbuf *)mbuf3)->pool, + (void **)&mbuf3, 1, 0); + } + + /* Create 4W cmd for 4 mbufs (sendhdr, sgdesc) */ + cmd0[0] = vzip1q_u64(senddesc01_w0, senddesc01_w1); + cmd0[1] = vzip2q_u64(senddesc01_w0, senddesc01_w1); + cmd0[2] = vzip1q_u64(senddesc23_w0, senddesc23_w1); + cmd0[3] = vzip2q_u64(senddesc23_w0, senddesc23_w1); + + cmd1[0] = vzip1q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[1] = vzip2q_u64(sgdesc01_w0, sgdesc01_w1); + cmd1[2] = vzip1q_u64(sgdesc23_w0, sgdesc23_w1); + cmd1[3] = vzip2q_u64(sgdesc23_w0, sgdesc23_w1); + + /* Store the prepared send desc to LMT lines */ + vst1q_u64(LMT_OFF(laddr, lnum, 0), cmd0[0]); + vst1q_u64(LMT_OFF(laddr, lnum, 16), cmd1[0]); + vst1q_u64(LMT_OFF(laddr, lnum, 32), cmd0[1]); + vst1q_u64(LMT_OFF(laddr, lnum, 48), cmd1[1]); + vst1q_u64(LMT_OFF(laddr, lnum, 64), cmd0[2]); + vst1q_u64(LMT_OFF(laddr, lnum, 80), cmd1[2]); + vst1q_u64(LMT_OFF(laddr, lnum, 96), cmd0[3]); + vst1q_u64(LMT_OFF(laddr, lnum, 112), cmd1[3]); + lnum += 1; + + tx_pkts = tx_pkts + NIX_DESCS_PER_LOOP; + } + + /* Trigger LMTST */ + if (lnum > 16) { + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= (15ULL << 12); + data |= (uint64_t)lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(lnum - 17)) << 12; + data |= (uint64_t)(lmt_id + 16); + + /* STEOR1 */ + roc_lmt_submit_steorl(data, pa); + } else if (lnum) { + data = cn10k_nix_tx_steor_vec_data(flags); + pa = io_addr | (data & 0x7) << 4; + data &= ~0x7ULL; + data |= ((uint64_t)(lnum - 1)) << 12; + data |= lmt_id; + + /* STEOR0 */ + roc_lmt_submit_steorl(data, pa); + } + + left -= burst; + rte_io_wmb(); + if (left) + goto again; + + if (unlikely(scalar)) + pkts += cn10k_nix_xmit_pkts(tx_queue, tx_pkts, scalar, cmd, + flags); + + return pkts; +} + +#else +static __rte_always_inline uint16_t +cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t pkts, uint64_t *cmd, const uint16_t flags) +{ + RTE_SET_USED(tx_queue); + RTE_SET_USED(tx_pkts); + RTE_SET_USED(pkts); + RTE_SET_USED(cmd); + RTE_SET_USED(flags); + return 0; +} +#endif + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F @@ -667,6 +1479,9 @@ T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_mseg_##name( \ + void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ + \ + uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_vec_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); NIX_TX_FASTPATH_MODES diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c new file mode 100644 index 0000000..42baeb5 --- /dev/null +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cn10k_ethdev.h" +#include "cn10k_tx.h" + +#define T(name, f4, f3, f2, f1, f0, sz, flags) \ + uint16_t __rte_noinline __rte_hot \ + cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ + struct rte_mbuf **tx_pkts, \ + uint16_t pkts) \ + { \ + uint64_t cmd[sz]; \ + \ + /* VLAN, TSTMP, TSO is not supported by vec */ \ + if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSO_F) \ + return 0; \ + return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ + (flags)); \ + } + +NIX_TX_FASTPATH_MODES +#undef T diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 21e5676..8f32dc7 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -28,7 +28,8 @@ sources += files('cn10k_ethdev.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', 'cn10k_tx.c', - 'cn10k_tx_mseg.c') + 'cn10k_tx_mseg.c', + 'cn10k_tx_vec.c') deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] deps += ['common_cnxk', 'mempool_cnxk'] -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 31/62] net/cnxk: add device start and stop operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (29 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram ` (31 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra, Nithin Dabilpuram Add device start and stop operation callbacks for CN9K and CN10K. Device stop is common for both platforms while device start as some platform dependent portion where the platform specific offload flags are recomputed and the right Rx/Tx burst function is chosen. Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> --- doc/guides/nics/cnxk.rst | 84 ++++++++++++++++++++++++++ drivers/net/cnxk/cn10k_ethdev.c | 124 +++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 127 ++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.c | 90 ++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 2 + drivers/net/cnxk/cnxk_link.c | 11 ++++ 6 files changed, 438 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 17da141..15911ee 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -39,6 +39,58 @@ Driver compilation and testing Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for details. +#. Running testpmd: + + Follow instructions available in the document + :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` + to run testpmd. + + Example output: + + .. code-block:: console + + ./<build_dir>/app/dpdk-testpmd -c 0xc -a 0002:02:00.0 -- --portmask=0x1 --nb-cores=1 --port-topology=loop --rxq=1 --txq=1 + EAL: Detected 4 lcore(s) + EAL: Detected 1 NUMA nodes + EAL: Multi-process socket /var/run/dpdk/rte/mp_socket + EAL: Selected IOVA mode 'VA' + EAL: No available hugepages reported in hugepages-16777216kB + EAL: No available hugepages reported in hugepages-2048kB + EAL: Probing VFIO support... + EAL: VFIO support initialized + EAL: using IOMMU type 1 (Type 1) + [ 2003.202721] vfio-pci 0002:02:00.0: vfio_cap_init: hiding cap 0x14@0x98 + EAL: Probe PCI driver: net_cn10k (177d:a063) device: 0002:02:00.0 (socket 0) + PMD: RoC Model: cn10k + EAL: No legacy callbacks, legacy socket not created + testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0 + testpmd: preferred mempool ops selected: cn10k_mempool_ops + Configuring Port 0 (socket 0) + PMD: Port 0: Link Up - speed 25000 Mbps - full-duplex + + Port 0: link state change event + Port 0: 96:D4:99:72:A5:BF + Checking link statuses... + Done + No commandline core given, start packet forwarding + io packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP allocation mode: native + Logical Core 3 (socket 0) forwards packets on 1 streams: + RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 + + io packet forwarding packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=1 + port 0: RX queue number: 1 Tx queue number: 1 + Rx offloads=0x0 Tx offloads=0x10000 + RX queue: 0 + RX desc=4096 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + RX Offloads=0x0 + TX queue: 0 + TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX offloads=0x0 - TX RS bit threshold=0 + Press enter to exit + Runtime Config Options ---------------------- @@ -132,3 +184,35 @@ Runtime Config Options Above devarg parameters are configurable per device, user needs to pass the parameters to all the PCIe devices if application requires to configure on all the ethdev ports. + +Limitations +----------- + +``mempool_cnxk`` external mempool handler dependency +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NIC has inbuilt HW assisted external mempool manager. +``net_cnxk`` pmd only works with ``mempool_cnxk`` mempool handler +as it is performance wise most effective way for packet allocation and Tx buffer +recycling on OCTEON TX2 SoC platform. + +CRC stripping +~~~~~~~~~~~~~ + +The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by +the host interface irrespective of the offload configuration. + +Debugging Options +----------------- + +.. _table_cnxk_ethdev_debug_options: + +.. table:: cnxk ethdev debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NIX | --log-level='pmd\.net.cnxk,8' | + +---+------------+-------------------------------------------------------+ + | 2 | NPC | --log-level='pmd\.net.cnxk\.flow,8' | + +---+------------+-------------------------------------------------------+ diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index d70ab00..5ff36bb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -5,6 +5,98 @@ #include "cn10k_rx.h" #include "cn10k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn10k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn10k_eth_set_rx_function(eth_dev); return 0; } @@ -163,6 +256,10 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -170,6 +267,28 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -185,6 +304,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn10k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn10k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } @@ -222,6 +342,10 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (!eth_dev) return -ENOENT; + + /* Setup callbacks for secondary process */ + cn10k_eth_set_tx_function(eth_dev); + cn10k_eth_set_rx_function(eth_dev); } return 0; } diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 806e95f..2157dca 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -5,6 +5,98 @@ #include "cn9k_rx.h" #include "cn9k_tx.h" +static uint16_t +nix_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS && + (dev->rx_offloads & DEV_RX_OFFLOAD_RSS_HASH)) + flags |= NIX_RX_OFFLOAD_RSS_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_TCP_CKSUM | DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & + (DEV_RX_OFFLOAD_IPV4_CKSUM | DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= NIX_RX_OFFLOAD_CHECKSUM_F; + + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + flags |= NIX_RX_MULTI_SEG_F; + + if (!dev->ptype_disable) + flags |= NIX_RX_OFFLOAD_PTYPE_F; + + return flags; +} + +static uint16_t +nix_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t conf = dev->tx_offloads; + uint16_t flags = 0; + + /* Fastpath is dependent on these enums */ + RTE_BUILD_BUG_ON(PKT_TX_TCP_CKSUM != (1ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_SCTP_CKSUM != (2ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_UDP_CKSUM != (3ULL << 52)); + RTE_BUILD_BUG_ON(PKT_TX_IP_CKSUM != (1ULL << 54)); + RTE_BUILD_BUG_ON(PKT_TX_IPV4 != (1ULL << 55)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IP_CKSUM != (1ULL << 58)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV4 != (1ULL << 59)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_IPV6 != (1ULL << 60)); + RTE_BUILD_BUG_ON(PKT_TX_OUTER_UDP_CKSUM != (1ULL << 41)); + RTE_BUILD_BUG_ON(RTE_MBUF_L2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_L3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL2_LEN_BITS != 7); + RTE_BUILD_BUG_ON(RTE_MBUF_OUTL3_LEN_BITS != 9); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_mbuf, buf_iova) + 8); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_mbuf, buf_iova) + 16); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_mbuf, ol_flags) + 12); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, tx_offload) != + offsetof(struct rte_mbuf, pool) + 2 * sizeof(void *)); + + if (conf & DEV_TX_OFFLOAD_VLAN_INSERT || + conf & DEV_TX_OFFLOAD_QINQ_INSERT) + flags |= NIX_TX_OFFLOAD_VLAN_QINQ_F; + + if (conf & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= NIX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (conf & DEV_TX_OFFLOAD_IPV4_CKSUM || + conf & DEV_TX_OFFLOAD_TCP_CKSUM || + conf & DEV_TX_OFFLOAD_UDP_CKSUM || conf & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= NIX_TX_OFFLOAD_L3_L4_CSUM_F; + + if (!(conf & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= NIX_TX_OFFLOAD_MBUF_NOFF_F; + + if (conf & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= NIX_TX_MULTI_SEG_F; + + /* Enable Inner checksum for TSO */ + if (conf & DEV_TX_OFFLOAD_TCP_TSO) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + /* Enable Inner and Outer checksum for Tunnel TSO */ + if (conf & (DEV_TX_OFFLOAD_VXLAN_TNL_TSO | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | DEV_TX_OFFLOAD_GRE_TNL_TSO)) + flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | + NIX_TX_OFFLOAD_L3_L4_CSUM_F); + + return flags; +} + static int cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) { @@ -18,6 +110,7 @@ cn9k_nix_ptypes_set(struct rte_eth_dev *eth_dev, uint32_t ptype_mask) dev->ptype_disable = 1; } + cn9k_eth_set_rx_function(eth_dev); return 0; } @@ -172,6 +265,10 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update offload flags */ + dev->rx_offload_flags = nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags = nix_tx_offload_flags(eth_dev); + plt_nix_dbg("Configured port%d platform specific rx_offload_flags=%x" " tx_offload_flags=0x%x", eth_dev->data->port_id, dev->rx_offload_flags, @@ -179,6 +276,28 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +static int +cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc; + + /* Common eth dev start */ + rc = cnxk_nix_dev_start(eth_dev); + if (rc) + return rc; + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + dev->rx_offload_flags |= nix_rx_offload_flags(eth_dev); + dev->tx_offload_flags |= nix_tx_offload_flags(eth_dev); + + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; +} + /* Update platform specific eth dev ops */ static void nix_eth_dev_ops_override(void) @@ -194,6 +313,7 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_setup = cn9k_nix_tx_queue_setup; cnxk_eth_dev_ops.rx_queue_setup = cn9k_nix_rx_queue_setup; cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; + cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } @@ -233,6 +353,13 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (!eth_dev) return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + /* Setup callbacks for secondary process */ + cn9k_eth_set_tx_function(eth_dev); + cn9k_eth_set_rx_function(eth_dev); + return 0; + } + dev = cnxk_eth_pmd_priv(eth_dev); /* Update capabilities already set for TSO. * TSO not supported for earlier chip revisions diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 6c20098..d4587f0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -955,12 +955,102 @@ cnxk_nix_rx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid) return rc; } +static int +cnxk_nix_dev_stop(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; + struct rte_mbuf *rx_pkts[32]; + int count, i, j, rc; + void *rxq; + + /* Disable switch hdr pkind */ + roc_nix_switch_hdr_set(&dev->nix, 0); + + /* Stop link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) + roc_nix_mac_link_event_start_stop(&dev->nix, false); + + /* Disable Rx via NPC */ + roc_nix_npc_rx_ena_dis(&dev->nix, false); + + /* Stop rx queues and free up pkts pending */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = dev_ops->rx_queue_stop(eth_dev, i); + if (rc) + continue; + + rxq = eth_dev->data->rx_queues[i]; + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + while (count) { + for (j = 0; j < count; j++) + rte_pktmbuf_free(rx_pkts[j]); + count = dev->rx_pkt_burst_no_offload(rxq, rx_pkts, 32); + } + } + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + dev_ops->tx_queue_stop(eth_dev, i); + + return 0; +} + +int +cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int rc, i; + + /* Start rx queues */ + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rc = cnxk_nix_rx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + return rc; + } + + /* Enable Rx in NPC */ + rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); + if (rc) { + plt_err("Failed to enable NPC rx %d", rc); + return rc; + } + + cnxk_nix_toggle_flag_link_cfg(dev, true); + + /* Start link change events */ + if (!roc_nix_is_vf_or_sdp(&dev->nix)) { + rc = roc_nix_mac_link_event_start_stop(&dev->nix, true); + if (rc) { + plt_err("Failed to start cgx link event %d", rc); + goto rx_disable; + } + } + + cnxk_nix_toggle_flag_link_cfg(dev, false); + + return 0; + +rx_disable: + roc_nix_npc_rx_ena_dis(&dev->nix, false); + cnxk_nix_toggle_flag_link_cfg(dev, false); + return rc; +} + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, + .dev_stop = cnxk_nix_dev_stop, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 276e569..50c75e1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -229,6 +229,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); +int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -237,6 +238,7 @@ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); /* Link */ +void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); diff --git a/drivers/net/cnxk/cnxk_link.c b/drivers/net/cnxk/cnxk_link.c index b0273e7..caf35ee 100644 --- a/drivers/net/cnxk/cnxk_link.c +++ b/drivers/net/cnxk/cnxk_link.c @@ -4,6 +4,17 @@ #include "cnxk_ethdev.h" +void +cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set) +{ + if (set) + dev->flags |= CNXK_LINK_CFG_IN_PROGRESS_F; + else + dev->flags &= ~CNXK_LINK_CFG_IN_PROGRESS_F; + + rte_wmb(); +} + static inline int nix_wait_for_link_cfg(struct cnxk_eth_dev *dev) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 32/62] net/cnxk: add MAC address set ops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (30 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram ` (30 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Default mac address set operation is implemented for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index d4587f0..58d2f7d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1046,6 +1046,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, .tx_queue_release = cnxk_nix_tx_queue_release, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 50c75e1..a5380a5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -218,6 +218,8 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 4a45956..87cf4ee 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -69,3 +69,32 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP; return 0; } + +int +cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Update mac address at NPC */ + rc = roc_nix_npc_mac_addr_set(nix, addr->addr_bytes); + if (rc) + goto exit; + + /* Update mac address at CGX for PFs only */ + if (!roc_nix_is_vf_or_sdp(nix)) { + rc = roc_nix_mac_addr_set(nix, addr->addr_bytes); + if (rc) { + /* Rollback to previous mac address */ + roc_nix_npc_mac_addr_set(nix, dev->mac_addr); + goto exit; + } + } + + /* Update mac address to cnxk ethernet device */ + rte_memcpy(dev->mac_addr, addr->addr_bytes, RTE_ETHER_ADDR_LEN); + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 33/62] net/cnxk: add MTU set device operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (31 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram ` (29 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> This Patch implements mtu set dev op for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 51 +++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 77 ++++++++++++++++++++++++++++++++++- 7 files changed, 135 insertions(+), 2 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 15911ee..d34d3fa 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - Inner and Outer Checksum offload - Link state information +- MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 02be26b..6fef725 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8c63853..79cb1e2 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -15,6 +15,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index a1bd49b..5cc9f3f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -14,6 +14,7 @@ Runtime Tx queue setup = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y +MTU update = Y TSO = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 58d2f7d..3f92780 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -37,6 +37,50 @@ nix_get_speed_capa(struct cnxk_eth_dev *dev) return speed_capa; } +static void +nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; + uint32_t buffsz; + + dev = rxq->dev; + eth_dev = dev->eth_dev; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->qconf.mp); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + if (eth_dev->data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + dev->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + dev->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } +} + +static int +nix_recalc_mtu(struct rte_eth_dev *eth_dev) +{ + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_eth_rxq_sp *rxq; + uint16_t mtu; + int rc; + + rxq = ((struct cnxk_eth_rxq_sp *)data->rx_queues[0]) - 1; + /* Setup scatter mode if needed by jumbo */ + nix_enable_mseg_on_jumbo(rxq); + + /* Setup MTU based on max_rx_pkt_len */ + mtu = data->dev_conf.rxmode.max_rx_pkt_len - CNXK_NIX_L2_OVERHEAD + + CNXK_NIX_MAX_VTAG_ACT_SIZE; + + rc = cnxk_nix_mtu_set(eth_dev, mtu); + if (rc) + plt_err("Failed to set default MTU size, rc=%d", rc); + + return rc; +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -1002,6 +1046,12 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; + if (eth_dev->data->nb_rx_queues != 0) { + rc = nix_recalc_mtu(eth_dev); + if (rc) + return rc; + } + /* Start rx queues */ for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { rc = cnxk_nix_rx_queue_start(eth_dev, i); @@ -1046,6 +1096,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { + .mtu_set = cnxk_nix_mtu_set, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a5380a5..c216dd5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -28,7 +28,9 @@ #define CNXK_NIX_MAX_VTAG_ACT_SIZE (4 * CNXK_NIX_MAX_VTAG_INS) /* ETH_HLEN+ETH_FCS+2*VLAN_HLEN */ -#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN + 8) +#define CNXK_NIX_L2_OVERHEAD (RTE_ETHER_HDR_LEN + \ + RTE_ETHER_CRC_LEN + \ + CNXK_NIX_MAX_VTAG_ACT_SIZE) #define CNXK_NIX_RX_MIN_DESC 16 #define CNXK_NIX_RX_MIN_DESC_ALIGN 16 @@ -218,6 +220,7 @@ extern struct eth_dev_ops cnxk_eth_dev_ops; int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 87cf4ee..21b55c4 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -20,7 +20,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->max_tx_queues = RTE_MAX_QUEUES_PER_PORT; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; - devinfo->max_mtu = devinfo->max_rx_pktlen - CNXK_NIX_L2_OVERHEAD; + devinfo->max_mtu = devinfo->max_rx_pktlen - + (RTE_ETHER_HDR_LEN + RTE_ETHER_CRC_LEN); devinfo->min_mtu = devinfo->min_rx_bufsize - CNXK_NIX_L2_OVERHEAD; devinfo->rx_offload_capa = dev->rx_offload_capa; @@ -98,3 +99,77 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) exit: return rc; } + +int +cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) +{ + uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + uint32_t buffsz; + + /* Check if MTU is within the allowed range */ + if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { + plt_err("MTU is lesser than minimum"); + goto exit; + } + + if ((frame_size - RTE_ETHER_CRC_LEN) > + ((uint32_t)roc_nix_max_pkt_len(nix))) { + plt_err("MTU is greater than maximum"); + goto exit; + } + + buffsz = data->min_rx_buf_size - RTE_PKTMBUF_HEADROOM; + old_frame_size = data->mtu + CNXK_NIX_L2_OVERHEAD; + + /* Refuse MTU that requires the support of scattered packets + * when this feature has not been enabled before. + */ + if (data->dev_started && frame_size > buffsz && + !(dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER)) { + plt_err("Scatter offload is not enabled for mtu"); + goto exit; + } + + /* Check <seg size> * <max_seg> >= max_frame */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) && + frame_size > (buffsz * CNXK_NIX_RX_NB_SEG_MAX)) { + plt_err("Greater than maximum supported packet length"); + goto exit; + } + + frame_size -= RTE_ETHER_CRC_LEN; + + /* Update mtu on Tx */ + rc = roc_nix_mac_mtu_set(nix, frame_size); + if (rc) { + plt_err("Failed to set MTU, rc=%d", rc); + goto exit; + } + + /* Sync same frame size on Rx */ + rc = roc_nix_mac_max_rx_len_set(nix, frame_size); + if (rc) { + /* Rollback to older mtu */ + roc_nix_mac_mtu_set(nix, + old_frame_size - RTE_ETHER_CRC_LEN); + plt_err("Failed to max Rx frame length, rc=%d", rc); + goto exit; + } + + frame_size += RTE_ETHER_CRC_LEN; + + if (frame_size > RTE_ETHER_MAX_LEN) + dev->rx_offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; + else + dev->rx_offloads &= ~DEV_RX_OFFLOAD_JUMBO_FRAME; + + /* Update max_rx_pkt_len */ + data->dev_conf.rxmode.max_rx_pkt_len = frame_size; + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 34/62] net/cnxk: add promiscuous mode enable and disable 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (32 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 35/62] net/cnxk: support DMAC filter Nithin Dabilpuram ` (28 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Add device operations to enable and disable promisc mode for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 56 +++++++++++++++++++++++++++++++++++ 6 files changed, 63 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index d34d3fa..87ceda5 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -17,6 +17,7 @@ Features Features of the CNXK Ethdev PMD are: - Packet type information +- Promiscuous mode - Jumbo frames - SR-IOV VF - Lock-free Tx queue diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 6fef725..9b2e163 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y TSO = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 79cb1e2..31471e0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -16,6 +16,7 @@ Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y +Promiscuous mode = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 3f92780..e5590a0 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1107,6 +1107,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, + .promiscuous_enable = cnxk_nix_promisc_enable, + .promiscuous_disable = cnxk_nix_promisc_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c216dd5..2fce20e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -223,6 +223,8 @@ int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); +int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 21b55c4..6feb3a9 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -173,3 +173,59 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) exit: return rc; } + +int +cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, true); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, false); + return rc; + } + + return 0; +} + +int +cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0; + + if (roc_nix_is_vf_or_sdp(nix)) + return rc; + + rc = roc_nix_npc_promisc_ena_dis(nix, false); + if (rc < 0) { + plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + return rc; + } + + rc = roc_nix_mac_promisc_mode_enable(nix, false); + if (rc) { + plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, + roc_error_msg_get(rc)); + roc_nix_npc_promisc_ena_dis(nix, true); + return rc; + } + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 35/62] net/cnxk: support DMAC filter 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (33 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram ` (27 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> DMAC filter support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 44 ++++++++++++++++++++++++++++++++--- 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 87ceda5..af7141f 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -23,6 +23,7 @@ Features of the CNXK Ethdev PMD are: - Lock-free Tx queue - Multiple queues for TX and RX - Receiver Side Scaling (RSS) +- MAC filtering - Inner and Outer Checksum offload - Link state information - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9b2e163..20d4d12 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 31471e0..e1de8ab 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Unicast MAC filter = Y RSS hash = Y Inner RSS = Y Jumbo frame = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index e5590a0..7b9e37b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1097,6 +1097,8 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, + .mac_addr_add = cnxk_nix_mac_addr_add, + .mac_addr_remove = cnxk_nix_mac_addr_del, .mac_addr_set = cnxk_nix_mac_addr_set, .dev_infos_get = cnxk_nix_info_get, .link_update = cnxk_nix_link_update, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 2fce20e..ab94b99 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -139,6 +139,7 @@ struct cnxk_eth_dev { /* Max macfilter entries */ uint8_t max_mac_entries; + bool dmac_filter_enable; uint16_t flags; uint8_t ptype_disable; @@ -221,6 +222,10 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *addr, uint32_t index, + uint32_t pool); +void cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index); int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 6feb3a9..fc60576 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -101,6 +101,43 @@ cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) } int +cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, + uint32_t index, uint32_t pool) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + PLT_SET_USED(index); + PLT_SET_USED(pool); + + rc = roc_nix_mac_addr_add(nix, addr->addr_bytes); + if (rc < 0) { + plt_err("Failed to add mac address, rc=%d", rc); + return rc; + } + + /* Enable promiscuous mode at NIX level */ + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} + +void +cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_mac_addr_del(nix, index); + if (rc) + plt_err("Failed to delete mac address, rc=%d", rc); +} + +int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) { uint32_t old_frame_size, frame_size = mtu + CNXK_NIX_L2_OVERHEAD; @@ -212,8 +249,8 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (roc_nix_is_vf_or_sdp(nix)) return rc; - rc = roc_nix_npc_promisc_ena_dis(nix, false); - if (rc < 0) { + rc = roc_nix_npc_promisc_ena_dis(nix, dev->dmac_filter_enable); + if (rc) { plt_err("Failed to setup promisc mode in npc, rc=%d(%s)", rc, roc_error_msg_get(rc)); return rc; @@ -223,9 +260,10 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) if (rc) { plt_err("Failed to setup promisc mode in mac, rc=%d(%s)", rc, roc_error_msg_get(rc)); - roc_nix_npc_promisc_ena_dis(nix, true); + roc_nix_npc_promisc_ena_dis(nix, !dev->dmac_filter_enable); return rc; } + dev->dmac_filter_enable = false; return 0; } -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 36/62] net/cnxk: add all multicast enable/disable ethops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (34 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 35/62] net/cnxk: support DMAC filter Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram ` (26 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> L2 multicast packets can be allowed or blocked. Patch implements corresponding ethops. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 17 +++++++++++++++++ 5 files changed, 23 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 20d4d12..b41af2d 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -18,6 +18,7 @@ Queue start/stop = Y MTU update = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e1de8ab..7fe8018 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -17,6 +17,7 @@ Free Tx mbuf on demand = Y Queue start/stop = Y MTU update = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 7b9e37b..516788c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1111,6 +1111,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_supported_ptypes_get = cnxk_nix_supported_ptypes_get, .promiscuous_enable = cnxk_nix_promisc_enable, .promiscuous_disable = cnxk_nix_promisc_disable, + .allmulticast_enable = cnxk_nix_allmulticast_enable, + .allmulticast_disable = cnxk_nix_allmulticast_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index ab94b99..70bc374 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -230,6 +230,8 @@ int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr); int cnxk_nix_promisc_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index fc60576..61ecbab 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -267,3 +267,20 @@ cnxk_nix_promisc_disable(struct rte_eth_dev *eth_dev) dev->dmac_filter_enable = false; return 0; } + +int +cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, true, false); +} + +int +cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_npc_mcast_config(&dev->nix, false, + eth_dev->data->promiscuous); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 37/62] net/cnxk: add Rx/Tx burst mode get ops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (35 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram ` (25 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements ethdev operations to get Rx and Tx burst mode. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 127 ++++++++++++++++++++++++++++++++++ 6 files changed, 136 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b41af2d..298f167 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 7fe8018..a673cc1 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -12,6 +12,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 5cc9f3f..335d082 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -11,6 +11,7 @@ Link status = Y Link status event = Y Runtime Rx queue setup = Y Runtime Tx queue setup = Y +Burst mode info = Y Fast mbuf free = Y Free Tx mbuf on demand = Y Queue start/stop = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 516788c..0311df3 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1113,6 +1113,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .promiscuous_disable = cnxk_nix_promisc_disable, .allmulticast_enable = cnxk_nix_allmulticast_enable, .allmulticast_disable = cnxk_nix_allmulticast_disable, + .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, + .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 70bc374..aea0005 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -234,6 +234,10 @@ int cnxk_nix_allmulticast_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev); int cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *dev_info); +int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); +int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 61ecbab..7ae961a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -72,6 +72,133 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) } int +cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } rx_offload_map[] = { + {DEV_RX_OFFLOAD_VLAN_STRIP, " VLAN Strip,"}, + {DEV_RX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_RX_OFFLOAD_TCP_LRO, " TCP LRO,"}, + {DEV_RX_OFFLOAD_QINQ_STRIP, " QinQ VLAN Strip,"}, + {DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_RX_OFFLOAD_MACSEC_STRIP, " MACsec Strip,"}, + {DEV_RX_OFFLOAD_HEADER_SPLIT, " Header Split,"}, + {DEV_RX_OFFLOAD_VLAN_FILTER, " VLAN Filter,"}, + {DEV_RX_OFFLOAD_VLAN_EXTEND, " VLAN Extend,"}, + {DEV_RX_OFFLOAD_JUMBO_FRAME, " Jumbo Frame,"}, + {DEV_RX_OFFLOAD_SCATTER, " Scattered,"}, + {DEV_RX_OFFLOAD_TIMESTAMP, " Timestamp,"}, + {DEV_RX_OFFLOAD_SECURITY, " Security,"}, + {DEV_RX_OFFLOAD_KEEP_CRC, " Keep CRC,"}, + {DEV_RX_OFFLOAD_SCTP_CKSUM, " SCTP,"}, + {DEV_RX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_RX_OFFLOAD_RSS_HASH, " RSS,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Rx Offloads:", + "Scalar, Rx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Rx offload info */ + for (i = 0; i < RTE_DIM(rx_offload_map); i++) { + if (dev->rx_offloads & rx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + rx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int +cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, + struct rte_eth_burst_mode *mode) +{ + ssize_t bytes = 0, str_size = RTE_ETH_BURST_MODE_INFO_SIZE, rc; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const struct burst_info { + uint64_t flags; + const char *output; + } tx_offload_map[] = { + {DEV_TX_OFFLOAD_VLAN_INSERT, " VLAN Insert,"}, + {DEV_TX_OFFLOAD_IPV4_CKSUM, " Inner IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_UDP_CKSUM, " UDP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_CKSUM, " TCP Checksum,"}, + {DEV_TX_OFFLOAD_SCTP_CKSUM, " SCTP Checksum,"}, + {DEV_TX_OFFLOAD_TCP_TSO, " TCP TSO,"}, + {DEV_TX_OFFLOAD_UDP_TSO, " UDP TSO,"}, + {DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM, " Outer IPv4 Checksum,"}, + {DEV_TX_OFFLOAD_QINQ_INSERT, " QinQ VLAN Insert,"}, + {DEV_TX_OFFLOAD_VXLAN_TNL_TSO, " VXLAN Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GRE_TNL_TSO, " GRE Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IPIP_TNL_TSO, " IP-in-IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_GENEVE_TNL_TSO, " Geneve Tunnel TSO,"}, + {DEV_TX_OFFLOAD_MACSEC_INSERT, " MACsec Insert,"}, + {DEV_TX_OFFLOAD_MT_LOCKFREE, " Multi Thread Lockless Tx,"}, + {DEV_TX_OFFLOAD_MULTI_SEGS, " Scattered,"}, + {DEV_TX_OFFLOAD_MBUF_FAST_FREE, " H/W MBUF Free,"}, + {DEV_TX_OFFLOAD_SECURITY, " Security,"}, + {DEV_TX_OFFLOAD_UDP_TNL_TSO, " UDP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_IP_TNL_TSO, " IP Tunnel TSO,"}, + {DEV_TX_OFFLOAD_OUTER_UDP_CKSUM, " Outer UDP Checksum,"}, + {DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP, " Timestamp,"} + }; + static const char *const burst_mode[] = {"Vector Neon, Tx Offloads:", + "Scalar, Tx Offloads:" + }; + uint32_t i; + + PLT_SET_USED(queue_id); + + /* Update burst mode info */ + rc = rte_strscpy(mode->info + bytes, burst_mode[dev->scalar_ena], + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + + /* Update Tx offload info */ + for (i = 0; i < RTE_DIM(tx_offload_map); i++) { + if (dev->tx_offloads & tx_offload_map[i].flags) { + rc = rte_strscpy(mode->info + bytes, + tx_offload_map[i].output, + str_size - bytes); + if (rc < 0) + goto done; + + bytes += rc; + } + } + +done: + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 38/62] net/cnxk: add flow ctrl set/get ops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (36 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram ` (24 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements set and get operations for flow control. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 74 +++++++++++++++++++++++++ drivers/net/cnxk/cnxk_ethdev.h | 13 +++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 101 ++++++++++++++++++++++++++++++++++ 6 files changed, 191 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index af7141f..88ca313 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -26,6 +26,7 @@ Features of the CNXK Ethdev PMD are: - MAC filtering - Inner and Outer Checksum offload - Link state information +- Link flow control - MTU update - Scatter-Gather IO support - Vector Poll mode driver diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 298f167..afd0f01 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y Scattered Rx = Y L3 checksum offload = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index a673cc1..4bd11ce 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,7 @@ Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y Inner RSS = Y +Flow control = Y Jumbo frame = Y L3 checksum offload = Y L4 checksum offload = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 0311df3..abca49b 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -81,6 +81,55 @@ nix_recalc_mtu(struct rte_eth_dev *eth_dev) return rc; } +static int +nix_init_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_conf = {0}; + int rc; + + /* Both Rx & Tx flow ctrl get enabled(RTE_FC_FULL) in HW + * by AF driver, update those info in PMD structure. + */ + rc = cnxk_nix_flow_ctrl_get(eth_dev, &fc_conf); + if (rc) + goto exit; + + fc->mode = fc_conf.mode; + fc->rx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_RX_PAUSE); + fc->tx_pause = (fc_conf.mode == RTE_FC_FULL) || + (fc_conf.mode == RTE_FC_TX_PAUSE); + +exit: + return rc; +} + +static int +nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct rte_eth_fc_conf fc_cfg = {0}; + + if (roc_nix_is_vf_or_sdp(&dev->nix)) + return 0; + + fc_cfg.mode = fc->mode; + + /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ + if (roc_model_is_cn96_ax() && + (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { + fc_cfg.mode = + (fc_cfg.mode == RTE_FC_FULL || + fc_cfg.mode == RTE_FC_TX_PAUSE) ? + RTE_FC_TX_PAUSE : RTE_FC_NONE; + } + + return cnxk_nix_flow_ctrl_set(eth_dev, &fc_cfg); +} + uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) { @@ -686,6 +735,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) struct rte_eth_rxmode *rxmode = &conf->rxmode; struct rte_eth_txmode *txmode = &conf->txmode; char ea_fmt[RTE_ETHER_ADDR_FMT_SIZE]; + struct roc_nix_fc_cfg fc_cfg = {0}; struct roc_nix *nix = &dev->nix; struct rte_ether_addr *ea; uint8_t nb_rxq, nb_txq; @@ -867,6 +917,21 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto cq_fini; } + /* Init flow control configuration */ + fc_cfg.cq_cfg_valid = false; + fc_cfg.rxchan_cfg.enable = true; + rc = roc_nix_fc_config_set(nix, &fc_cfg); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } + + /* Update flow control configuration to PMD */ + rc = nix_init_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to initialize flow control rc=%d", rc); + goto cq_fini; + } /* * Restore queue config when reconfigure followed by * reconfigure and no queue configure invoked from application case. @@ -1066,6 +1131,13 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } + /* Update Flow control configuration */ + rc = nix_update_flow_ctrl_config(eth_dev); + if (rc) { + plt_err("Failed to enable flow control. error code(%d)", rc); + return rc; + } + /* Enable Rx in NPC */ rc = roc_nix_npc_rx_ena_dis(&dev->nix, true); if (rc) { @@ -1115,6 +1187,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .allmulticast_disable = cnxk_nix_allmulticast_disable, .rx_burst_mode_get = cnxk_nix_rx_burst_mode_get, .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, + .flow_ctrl_get = cnxk_nix_flow_ctrl_get, + .flow_ctrl_set = cnxk_nix_flow_ctrl_set, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index aea0005..e788a42 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -113,6 +113,12 @@ ((1ull << (PKT_TX_TUNNEL_VXLAN >> 45)) | \ (1ull << (PKT_TX_TUNNEL_GENEVE >> 45))) +struct cnxk_fc_cfg { + enum rte_eth_fc_mode mode; + uint8_t rx_pause; + uint8_t tx_pause; +}; + struct cnxk_eth_qconf { union { struct rte_eth_txconf tx; @@ -174,6 +180,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Flow control configuration */ + struct cnxk_fc_cfg fc_cfg; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -238,6 +247,10 @@ int cnxk_nix_rx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); int cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, struct rte_eth_burst_mode *mode); +int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7ae961a..97c5428 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -199,6 +199,107 @@ cnxk_nix_tx_burst_mode_get(struct rte_eth_dev *eth_dev, uint16_t queue_id, } int +cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum rte_eth_fc_mode mode_map[] = { + RTE_FC_NONE, RTE_FC_RX_PAUSE, + RTE_FC_TX_PAUSE, RTE_FC_FULL + }; + struct roc_nix *nix = &dev->nix; + int mode; + + mode = roc_nix_fc_mode_get(nix); + if (mode < 0) + return mode; + + memset(fc_conf, 0, sizeof(struct rte_eth_fc_conf)); + fc_conf->mode = mode_map[mode]; + return 0; +} + +static int +nix_fc_cq_config_set(struct cnxk_eth_dev *dev, uint16_t qid, bool enable) +{ + struct roc_nix *nix = &dev->nix; + struct roc_nix_fc_cfg fc_cfg; + struct roc_nix_cq *cq; + + memset(&fc_cfg, 0, sizeof(struct roc_nix_fc_cfg)); + cq = &dev->cqs[qid]; + fc_cfg.cq_cfg_valid = true; + fc_cfg.cq_cfg.enable = enable; + fc_cfg.cq_cfg.rq = qid; + fc_cfg.cq_cfg.cq_drop = cq->drop_thresh; + + return roc_nix_fc_config_set(nix, &fc_cfg); +} + +int +cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, + struct rte_eth_fc_conf *fc_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + enum roc_nix_fc_mode mode_map[] = { + ROC_NIX_FC_NONE, ROC_NIX_FC_RX, + ROC_NIX_FC_TX, ROC_NIX_FC_FULL + }; + struct rte_eth_dev_data *data = eth_dev->data; + struct cnxk_fc_cfg *fc = &dev->fc_cfg; + struct roc_nix *nix = &dev->nix; + uint8_t rx_pause, tx_pause; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) { + plt_err("Flow control configuration is not allowed on VFs"); + return -ENOTSUP; + } + + if (fc_conf->high_water || fc_conf->low_water || fc_conf->pause_time || + fc_conf->mac_ctrl_frame_fwd || fc_conf->autoneg) { + plt_info("Only MODE configuration is supported"); + return -EINVAL; + } + + if (fc_conf->mode == fc->mode) + return 0; + + rx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_RX_PAUSE); + tx_pause = (fc_conf->mode == RTE_FC_FULL) || + (fc_conf->mode == RTE_FC_TX_PAUSE); + + /* Check if TX pause frame is already enabled or not */ + if (fc->tx_pause ^ tx_pause) { + if (roc_model_is_cn96_ax() && data->dev_started) { + /* On Ax, CQ should be in disabled state + * while setting flow control configuration. + */ + plt_info("Stop the port=%d for setting flow control", + data->port_id); + return 0; + } + + for (i = 0; i < data->nb_rx_queues; i++) { + rc = nix_fc_cq_config_set(dev, i, tx_pause); + if (rc) + return rc; + } + } + + rc = roc_nix_fc_mode_set(nix, mode_map[fc_conf->mode]); + if (rc) + return rc; + + fc->rx_pause = rx_pause; + fc->tx_pause = tx_pause; + fc->mode = fc_conf->mode; + + return rc; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 39/62] net/cnxk: add link up/down operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (37 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram ` (23 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements link up/down ethdev operations for cn9k and cn10k platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 4 +++- drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 47 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index abca49b..a18d1d5 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -975,7 +975,7 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) return rc; } -static int +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -1189,6 +1189,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_burst_mode_get = cnxk_nix_tx_burst_mode_get, .flow_ctrl_get = cnxk_nix_flow_ctrl_get, .flow_ctrl_set = cnxk_nix_flow_ctrl_set, + .dev_set_link_up = cnxk_nix_set_link_up, + .dev_set_link_down = cnxk_nix_set_link_down, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e788a42..5e982f9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -251,6 +251,9 @@ int cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); +int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); +int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); + int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, @@ -259,6 +262,7 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_rx_q_sz, const struct rte_eth_rxconf *rx_conf, struct rte_mempool *mp); +int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 97c5428..c8da129 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -512,3 +512,50 @@ cnxk_nix_allmulticast_disable(struct rte_eth_dev *eth_dev) return roc_nix_npc_mcast_config(&dev->nix, false, eth_dev->data->promiscuous); } + +int +cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + rc = roc_nix_mac_link_state_set(nix, true); + if (rc) + goto exit; + + /* Start tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_start(eth_dev, i); + if (rc) + goto exit; + } + +exit: + return rc; +} + +int +cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc, i; + + if (roc_nix_is_vf_or_sdp(nix)) + return -ENOTSUP; + + /* Stop tx queues */ + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { + rc = cnxk_nix_tx_queue_stop(eth_dev, i); + if (rc) + goto exit; + } + + rc = roc_nix_mac_link_state_set(nix, false); +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 40/62] net/cnxk: add EEPROM module info get operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (38 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram ` (22 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements eeprom module info get ethops for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 39 +++++++++++++++++++++++++++++++++++ 6 files changed, 48 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index afd0f01..b1e8641 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -31,6 +31,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4bd11ce..0f99634 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -29,6 +29,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 335d082..cecced9 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -26,6 +26,7 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index a18d1d5..165c354 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1191,6 +1191,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .flow_ctrl_set = cnxk_nix_flow_ctrl_set, .dev_set_link_up = cnxk_nix_set_link_up, .dev_set_link_down = cnxk_nix_set_link_down, + .get_module_info = cnxk_nix_get_module_info, + .get_module_eeprom = cnxk_nix_get_module_eeprom, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5e982f9..083af29 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -253,6 +253,10 @@ int cnxk_nix_flow_ctrl_get(struct rte_eth_dev *eth_dev, struct rte_eth_fc_conf *fc_conf); int cnxk_nix_set_link_up(struct rte_eth_dev *eth_dev); int cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev); +int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo); +int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index c8da129..55b0411 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -559,3 +559,42 @@ cnxk_nix_set_link_down(struct rte_eth_dev *eth_dev) exit: return rc; } + +int +cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, + struct rte_eth_dev_module_info *modinfo) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + modinfo->type = eeprom_info.sff_id; + modinfo->eeprom_len = ROC_NIX_EEPROM_SIZE; + return 0; +} + +int +cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, + struct rte_dev_eeprom_info *info) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_eeprom_info eeprom_info = {0}; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL; + + if (!info->data || !info->length || + (info->offset + info->length > ROC_NIX_EEPROM_SIZE)) + return rc; + + rc = roc_nix_eeprom_info_get(nix, &eeprom_info); + if (rc) + return rc; + + rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 41/62] net/cnxk: add Rx queue interrupt enable/disable ops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (39 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram ` (21 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Application may choose to enable/disable interrupts on Rx queues so that application can select its processing if no packets are available on queues for a longer period. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 7 files changed, 29 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 88ca313..6678486 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -30,6 +30,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Support Rx interrupt Prerequisites ------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index b1e8641..e5669f5 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 0f99634..dff0c9b 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y SR-IOV = Y Multiprocess aware = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index cecced9..b950d2f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -5,6 +5,7 @@ ; [Features] Speed capabilities = Y +Rx interrupt = Y Lock-free Tx queue = Y Multiprocess aware = Y Link status = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 165c354..1ff3afa 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1193,6 +1193,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .dev_set_link_down = cnxk_nix_set_link_down, .get_module_info = cnxk_nix_get_module_info, .get_module_eeprom = cnxk_nix_get_module_eeprom, + .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, + .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 083af29..a01b72a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -257,6 +257,10 @@ int cnxk_nix_get_module_info(struct rte_eth_dev *eth_dev, struct rte_eth_dev_module_info *modinfo); int cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, struct rte_dev_eeprom_info *info); +int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); +int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 55b0411..45683dd 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -598,3 +598,22 @@ cnxk_nix_get_module_eeprom(struct rte_eth_dev *eth_dev, rte_memcpy(info->data, eeprom_info.buf + info->offset, info->length); return 0; } + +int +cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_enable(&dev->nix, rx_queue_id); + return 0; +} + +int +cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, + uint16_t rx_queue_id) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 42/62] net/cnxk: add validation API for mempool ops 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (40 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram ` (20 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> cn9k and cn10k supports platform specific mempool ops. This patch implements API to validate whether given mempool ops is supported or not. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 11 +++++++++++ 3 files changed, 13 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 1ff3afa..77bbab6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1195,6 +1195,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_module_eeprom = cnxk_nix_get_module_eeprom, .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, + .pool_ops_supported = cnxk_nix_pool_ops_supported, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a01b72a..e6dac95 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -261,6 +261,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); +int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 45683dd..7f46551 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -617,3 +617,14 @@ cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, roc_nix_rx_queue_intr_disable(&dev->nix, rx_queue_id); return 0; } + +int +cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) +{ + RTE_SET_USED(eth_dev); + + if (!strcmp(pool, rte_mbuf_platform_mempool_ops())) + return 0; + + return -ENOTSUP; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 43/62] net/cnxk: add port/queue stats 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (41 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 44/62] net/cnxk: add xstats apis Nithin Dabilpuram ` (19 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch implements regular port statistics and queue mapping set api to get queue statistics Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 8 ++++ drivers/net/cnxk/cnxk_stats.c | 85 +++++++++++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 3 +- 8 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 drivers/net/cnxk/cnxk_stats.c diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index 6678486..e875e55 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -25,6 +25,7 @@ Features of the CNXK Ethdev PMD are: - Receiver Side Scaling (RSS) - MAC filtering - Inner and Outer Checksum offload +- Port hardware statistics - Link state information - Link flow control - MTU update diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e5669f5..40952a9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index dff0c9b..32035bb 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -30,6 +30,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b950d2f..8060a68 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -27,6 +27,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Basic stats = Y +Stats per queue = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 77bbab6..32b6f6c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1196,6 +1196,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rx_queue_intr_enable = cnxk_nix_rx_queue_intr_enable, .rx_queue_intr_disable = cnxk_nix_rx_queue_intr_disable, .pool_ops_supported = cnxk_nix_pool_ops_supported, + .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, + .stats_get = cnxk_nix_stats_get, + .stats_reset = cnxk_nix_stats_reset, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index e6dac95..860cfe1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -191,6 +191,10 @@ struct cnxk_eth_dev { /* LSO Tunnel format indices */ uint64_t lso_tun_fmt; + + /* Per queue statistics counters */ + uint32_t txq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; + uint32_t rxq_stat_map[RTE_ETHDEV_QUEUE_STAT_CNTRS]; }; struct cnxk_eth_rxq_sp { @@ -286,6 +290,10 @@ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); void cnxk_eth_dev_link_status_cb(struct roc_nix *nix, struct roc_nix_link_info *link); int cnxk_nix_link_update(struct rte_eth_dev *eth_dev, int wait_to_complete); +int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx); +int cnxk_nix_stats_reset(struct rte_eth_dev *dev); +int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c new file mode 100644 index 0000000..24bff0b --- /dev/null +++ b/drivers/net/cnxk/cnxk_stats.c @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +int +cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + struct roc_nix_stats nix_stats; + int rc = 0, i; + + rc = roc_nix_stats_get(nix, &nix_stats); + if (rc) + goto exit; + + stats->opackets = nix_stats.tx_ucast; + stats->opackets += nix_stats.tx_mcast; + stats->opackets += nix_stats.tx_bcast; + stats->oerrors = nix_stats.tx_drop; + stats->obytes = nix_stats.tx_octs; + + stats->ipackets = nix_stats.rx_ucast; + stats->ipackets += nix_stats.rx_mcast; + stats->ipackets += nix_stats.rx_bcast; + stats->imissed = nix_stats.rx_drop; + stats->ibytes = nix_stats.rx_octs; + stats->ierrors = nix_stats.rx_err; + + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS; i++) { + struct roc_nix_stats_queue qstats; + uint16_t qidx; + + if (dev->txq_stat_map[i] & (1U << 31)) { + qidx = dev->txq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 0, &qstats); + if (rc) + goto exit; + stats->q_opackets[i] = qstats.tx_pkts; + stats->q_obytes[i] = qstats.tx_octs; + stats->q_errors[i] = qstats.tx_drop_pkts; + } + + if (dev->rxq_stat_map[i] & (1U << 31)) { + qidx = dev->rxq_stat_map[i] & 0xFFFF; + rc = roc_nix_stats_queue_get(nix, qidx, 1, &qstats); + if (rc) + goto exit; + stats->q_ipackets[i] = qstats.rx_pkts; + stats->q_ibytes[i] = qstats.rx_octs; + stats->q_errors[i] += qstats.rx_drop_pkts; + } + } +exit: + return rc; +} + +int +cnxk_nix_stats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + return roc_nix_stats_reset(&dev->nix); +} + +int +cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, + uint8_t stat_idx, uint8_t is_rx) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (is_rx) { + if (queue_id >= dev->nb_rxq) + return -EINVAL; + dev->rxq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } else { + if (queue_id >= dev->nb_txq) + return -EINVAL; + dev->txq_stat_map[stat_idx] = ((1U << 31) | queue_id); + } + + return 0; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 8f32dc7..7d711a2 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -12,7 +12,8 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_ops.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', - 'cnxk_lookup.c') + 'cnxk_lookup.c', + 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 44/62] net/cnxk: add xstats apis 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (42 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram ` (18 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Initial implementation of xstats operations. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 5 + drivers/net/cnxk/cnxk_ethdev.h | 11 ++ drivers/net/cnxk/cnxk_stats.c | 235 ++++++++++++++++++++++++++++++++++ 6 files changed, 254 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 40952a9..192c15a 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -34,6 +34,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 32035bb..e990480 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -32,6 +32,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8060a68..3a4417c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -29,6 +29,7 @@ Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Stats per queue = Y +Extended stats = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 32b6f6c..071c293 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1199,6 +1199,11 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .queue_stats_mapping_set = cnxk_nix_queue_stats_mapping, .stats_get = cnxk_nix_stats_get, .stats_reset = cnxk_nix_stats_reset, + .xstats_get = cnxk_nix_xstats_get, + .xstats_get_names = cnxk_nix_xstats_get_names, + .xstats_reset = cnxk_nix_xstats_reset, + .xstats_get_by_id = cnxk_nix_xstats_get_by_id, + .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 860cfe1..5a0dc13 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -294,6 +294,17 @@ int cnxk_nix_queue_stats_mapping(struct rte_eth_dev *dev, uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx); int cnxk_nix_stats_reset(struct rte_eth_dev *dev); int cnxk_nix_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats); +int cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat *xstats, unsigned int n); +int cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit); +int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit); +int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n); +int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_stats.c b/drivers/net/cnxk/cnxk_stats.c index 24bff0b..19bab21 100644 --- a/drivers/net/cnxk/cnxk_stats.c +++ b/drivers/net/cnxk/cnxk_stats.c @@ -4,6 +4,9 @@ #include "cnxk_ethdev.h" +#define CNXK_NB_RXQ_STATS 5 +#define CNXK_NB_TXQ_STATS 4 + int cnxk_nix_stats_get(struct rte_eth_dev *eth_dev, struct rte_eth_stats *stats) { @@ -83,3 +86,235 @@ cnxk_nix_queue_stats_mapping(struct rte_eth_dev *eth_dev, uint16_t queue_id, return 0; } + +int +cnxk_nix_xstats_get(struct rte_eth_dev *eth_dev, struct rte_eth_xstat *xstats, + unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat roc_xstats[n]; + struct roc_nix *nix = &dev->nix; + int roc_size, q, idx = 0, size; + + roc_size = roc_nix_xstats_get(nix, roc_xstats, n); + + if (roc_size < 0) + return roc_size; + + /* Per Queue statistics also returned as part of xstats */ + size = roc_size + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + + /* If requested array do not have space then return with count */ + if (size > (int)n || xstats == NULL) + return size; + + for (idx = 0; idx < roc_size; idx++) { + xstats[idx].id = roc_xstats[idx].id; + xstats[idx].value = roc_xstats[idx].value; + } + for (q = 0; q < dev->nb_rxq; q++) { + struct roc_nix_stats_queue qstats; + int rc; + + rc = roc_nix_stats_queue_get(nix, q, 1, &qstats); + if (rc) + return rc; + + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_drop_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_drop_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.rx_error_pkts; + idx++; + } + for (q = 0; q < dev->nb_txq; q++) { + struct roc_nix_stats_queue qstats; + int rc; + + rc = roc_nix_stats_queue_get(nix, q, 0, &qstats); + if (rc) + return rc; + + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_octs; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_drop_pkts; + idx++; + xstats[idx].id = idx; + xstats[idx].value = qstats.tx_drop_octs; + idx++; + } + + return size; +} + +int +cnxk_nix_xstats_get_names(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix_xstat_name roc_xstats_name[limit]; + struct roc_nix *nix = &dev->nix; + int roc_size, size, i, q; + + roc_size = roc_nix_num_xstats_get(nix); + /* Per Queue statistics also returned as part of xstats */ + size = roc_size + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + + if (xstats_names == NULL) + return size; + + if ((int)limit < size && xstats_names != NULL) + return size; + + roc_size = roc_nix_xstats_names_get(nix, roc_xstats_name, limit); + + for (i = 0; i < roc_size; i++) + rte_strscpy(xstats_names[i].name, roc_xstats_name[i].name, + sizeof(xstats_names[i].name)); + + for (q = 0; q < dev->nb_rxq; q++) { + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_drop_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_drop_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "rxq_%d_err_pkts", q); + i++; + } + + for (q = 0; q < dev->nb_txq; q++) { + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_octs", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_drop_pkts", q); + i++; + snprintf(xstats_names[i].name, sizeof(xstats_names[i].name), + "txq_%d_drop_octs", q); + i++; + } + + return size; +} + +int +cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, + struct rte_eth_xstat_name *xstats_names, + const uint64_t *ids, unsigned int limit) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t nix_cnt = roc_nix_num_xstats_get(&dev->nix); + uint32_t stat_cnt = nix_cnt + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + struct rte_eth_xstat_name xnames[stat_cnt]; + uint32_t i; + + if (limit < stat_cnt && ids == NULL) + return stat_cnt; + + if (limit > stat_cnt) + return -EINVAL; + + if (xstats_names == NULL) + return -ENOMEM; + + cnxk_nix_xstats_get_names(eth_dev, xnames, stat_cnt); + + for (i = 0; i < limit; i++) { + if (ids[i] >= stat_cnt) + return -EINVAL; + + rte_strscpy(xstats_names[i].name, xnames[ids[i]].name, + sizeof(xstats_names[i].name)); + } + + return limit; +} + +int +cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, + uint64_t *values, unsigned int n) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint32_t nix_cnt = roc_nix_num_xstats_get(&dev->nix); + uint32_t stat_cnt = nix_cnt + (dev->nb_rxq * CNXK_NB_RXQ_STATS) + + (dev->nb_txq * CNXK_NB_TXQ_STATS); + struct rte_eth_xstat xstats[stat_cnt]; + uint32_t i; + + if (n < stat_cnt && ids == NULL) + return stat_cnt; + + if (n > stat_cnt) + return -EINVAL; + + if (values == NULL) + return -ENOMEM; + + cnxk_nix_xstats_get(eth_dev, xstats, stat_cnt); + + for (i = 0; i < n; i++) { + if (ids[i] >= stat_cnt) + return -EINVAL; + values[i] = xstats[ids[i]].value; + } + + return n; +} + +int +cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc = 0, i; + + rc = roc_nix_stats_reset(nix); + if (rc) + goto exit; + + /* Reset Rx Queues */ + for (i = 0; i < dev->nb_rxq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 1); + if (rc) + goto exit; + } + + /* Reset Tx Queues */ + for (i = 0; i < dev->nb_txq; i++) { + rc = roc_nix_stats_queue_reset(nix, i, 0); + if (rc) + goto exit; + } + +exit: + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 45/62] net/cnxk: add rxq/txq info get operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (43 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 44/62] net/cnxk: add xstats apis Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram ` (17 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Initial apis to get default queue information. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 071c293..81cd04e 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1204,6 +1204,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .rxq_info_get = cnxk_nix_rxq_info_get, + .txq_info_get = cnxk_nix_txq_info_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 5a0dc13..32977eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -305,6 +305,10 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo); +void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo); /* Lookup configuration */ const uint32_t *cnxk_nix_supported_ptypes_get(struct rte_eth_dev *eth_dev); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 7f46551..782f163 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -628,3 +628,33 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } + +void +cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_rxq_info *qinfo) +{ + void *rxq = eth_dev->data->rx_queues[qid]; + struct cnxk_eth_rxq_sp *rxq_sp = cnxk_eth_rxq_to_sp(rxq); + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->mp = rxq_sp->qconf.mp; + qinfo->scattered_rx = eth_dev->data->scattered_rx; + qinfo->nb_desc = rxq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &rxq_sp->qconf.conf.rx, sizeof(qinfo->conf)); +} + +void +cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, + struct rte_eth_txq_info *qinfo) +{ + void *txq = eth_dev->data->tx_queues[qid]; + struct cnxk_eth_txq_sp *txq_sp = cnxk_eth_txq_to_sp(txq); + + memset(qinfo, 0, sizeof(*qinfo)); + + qinfo->nb_desc = txq_sp->qconf.nb_desc; + + memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 46/62] net/cnxk: add device close and reset operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (44 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram ` (16 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements device close and reset operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 81cd04e..24f02e6 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1166,6 +1166,9 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) return rc; } +static int cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev); +static int cnxk_nix_dev_close(struct rte_eth_dev *eth_dev); + /* CNXK platform independent eth dev ops */ struct eth_dev_ops cnxk_eth_dev_ops = { .mtu_set = cnxk_nix_mtu_set, @@ -1177,6 +1180,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_queue_release = cnxk_nix_tx_queue_release, .rx_queue_release = cnxk_nix_rx_queue_release, .dev_stop = cnxk_nix_dev_stop, + .dev_close = cnxk_nix_dev_close, + .dev_reset = cnxk_nix_dev_reset, .tx_queue_start = cnxk_nix_tx_queue_start, .rx_queue_start = cnxk_nix_rx_queue_start, .rx_queue_stop = cnxk_nix_rx_queue_stop, @@ -1316,7 +1321,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } static int -cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; @@ -1370,14 +1375,11 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* Check if mbox close is needed */ - if (!mbox_close) - return 0; - rc = roc_nix_dev_fini(nix); /* Can be freed later by PMD if NPA LF is in use */ if (rc == -EAGAIN) { - eth_dev->data->dev_private = NULL; + if (!reset) + eth_dev->data->dev_private = NULL; return 0; } else if (rc) { plt_err("Failed in nix dev fini, rc=%d", rc); @@ -1386,6 +1388,25 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) return rc; } +static int +cnxk_nix_dev_close(struct rte_eth_dev *eth_dev) +{ + cnxk_eth_dev_uninit(eth_dev, false); + return 0; +} + +static int +cnxk_nix_dev_reset(struct rte_eth_dev *eth_dev) +{ + int rc; + + rc = cnxk_eth_dev_uninit(eth_dev, true); + if (rc) + return rc; + + return cnxk_eth_dev_init(eth_dev); +} + int cnxk_nix_remove(struct rte_pci_device *pci_dev) { @@ -1396,7 +1417,7 @@ cnxk_nix_remove(struct rte_pci_device *pci_dev) eth_dev = rte_eth_dev_allocated(pci_dev->device.name); if (eth_dev) { /* Cleanup eth dev */ - rc = cnxk_eth_dev_uninit(eth_dev, true); + rc = cnxk_eth_dev_uninit(eth_dev, false); if (rc) return rc; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 47/62] net/cnxk: add pending Tx mbuf cleanup operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (45 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram ` (15 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Once mbufs are transmitted, mbufs are freed by H/W. No mbufs are accumalated as a pending mbuf. Hence operation is NOP for cnxk platform. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ 3 files changed, 12 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 24f02e6..529adf7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1211,6 +1211,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, + .tx_done_cleanup = cnxk_nix_tx_done_cleanup, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 32977eb..1c4403c 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -266,6 +266,7 @@ int cnxk_nix_rx_queue_intr_enable(struct rte_eth_dev *eth_dev, int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); +int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 782f163..305220a 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -658,3 +658,13 @@ cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, memcpy(&qinfo->conf, &txq_sp->qconf.conf.tx, sizeof(qinfo->conf)); } + +/* It is a NOP for cnxk as HW frees the buffer on xmit */ +int +cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) +{ + RTE_SET_USED(txq); + RTE_SET_USED(free_cnt); + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 48/62] net/cnxk: add support to configure npc 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (46 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow Nithin Dabilpuram ` (14 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Kiran Kumar K <kirankumark@marvell.com> Adding support to configure NPC on device initialization. This involves reading the MKEX and initializing the necessary data. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 25 ++++++++++++++++++++++--- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 3 +++ 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 529adf7..7088e67 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -8,7 +8,8 @@ nix_get_rx_offload_capa(struct cnxk_eth_dev *dev) { uint64_t capa = CNXK_NIX_RX_OFFLOAD_CAPA; - if (roc_nix_is_vf_or_sdp(&dev->nix)) + if (roc_nix_is_vf_or_sdp(&dev->nix) || + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) capa &= ~DEV_RX_OFFLOAD_TIMESTAMP; return capa; @@ -120,6 +121,7 @@ nix_update_flow_ctrl_config(struct rte_eth_dev *eth_dev) /* To avoid Link credit deadlock on Ax, disable Tx FC if it's enabled */ if (roc_model_is_cn96_ax() && + dev->npc.switch_header_type != ROC_PRIV_FLAGS_HIGIG && (fc_cfg.mode == RTE_FC_FULL || fc_cfg.mode == RTE_FC_RX_PAUSE)) { fc_cfg.mode = (fc_cfg.mode == RTE_FC_FULL || @@ -419,8 +421,10 @@ cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, dev->ethdev_rss_hf = ethdev_rss; - if (ethdev_rss & ETH_RSS_L2_PAYLOAD) + if (ethdev_rss & ETH_RSS_L2_PAYLOAD && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_LEN_90B) { flowkey_cfg |= FLOW_KEY_TYPE_CH_LEN_90B; + } if (ethdev_rss & ETH_RSS_C_VLAN) flowkey_cfg |= FLOW_KEY_TYPE_VLAN; @@ -849,11 +853,18 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) roc_nix_err_intr_ena_dis(nix, true); roc_nix_ras_intr_ena_dis(nix, true); - if (nix->rx_ptp_ena) { + if (nix->rx_ptp_ena && + dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { plt_err("Both PTP and switch header enabled"); goto free_nix_lf; } + rc = roc_nix_switch_hdr_set(nix, dev->npc.switch_header_type); + if (rc) { + plt_err("Failed to enable switch type nix_lf rc=%d", rc); + goto free_nix_lf; + } + /* Setup LSO if needed */ rc = nix_lso_fmt_setup(dev); if (rc) { @@ -1305,6 +1316,11 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) dev->speed_capa = nix_get_speed_capa(dev); /* Initialize roc npc */ + dev->npc.roc_nix = nix; + rc = roc_npc_init(&dev->npc); + if (rc) + goto free_mac_addrs; + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, eth_dev->data->port_id, roc_nix_get_pf(nix), @@ -1338,6 +1354,9 @@ cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool reset) roc_nix_npc_rx_ena_dis(nix, false); + /* Disable and free rte_flow entries */ + roc_npc_fini(&dev->npc); + /* Disable link status events */ roc_nix_mac_link_event_start_stop(nix, false); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1c4403c..4879ef9 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -133,6 +133,9 @@ struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; + /* ROC NPC */ + struct roc_npc npc; + /* ROC RQs, SQs and CQs */ struct roc_nix_rq *rqs; struct roc_nix_sq *sqs; diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 4af2803..7fd06eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -150,6 +150,9 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->npc.flow_prealloc_size = flow_prealloc_size; + dev->npc.flow_max_priority = flow_max_priority; + dev->npc.switch_header_type = switch_header_type; return 0; exit: -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (47 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-07-05 22:01 ` Thomas Monjalon 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram ` (13 subsequent siblings) 62 siblings, 1 reply; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Kiran Kumar K <kirankumark@marvell.com> Adding initial version of rte_flow support for cnxk family device. Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_flush, flow_query, flow_isolate. Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> --- doc/guides/nics/cnxk.rst | 7 + doc/guides/nics/features/cnxk.ini | 36 ++++ doc/guides/nics/features/cnxk_vec.ini | 36 ++++ doc/guides/nics/features/cnxk_vf.ini | 36 ++++ drivers/net/cnxk/cn10k_ethdev.c | 16 ++ drivers/net/cnxk/cn10k_rte_flow.c | 30 ++++ drivers/net/cnxk/cn10k_rte_flow.h | 17 ++ drivers/net/cnxk/cn9k_ethdev.c | 16 ++ drivers/net/cnxk/cn9k_rte_flow.c | 30 ++++ drivers/net/cnxk/cn9k_rte_flow.h | 17 ++ drivers/net/cnxk/cnxk_ethdev.h | 3 + drivers/net/cnxk/cnxk_rte_flow.c | 330 ++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cnxk_rte_flow.h | 27 +++ drivers/net/cnxk/meson.build | 3 + 14 files changed, 604 insertions(+) create mode 100644 drivers/net/cnxk/cn10k_rte_flow.c create mode 100644 drivers/net/cnxk/cn10k_rte_flow.h create mode 100644 drivers/net/cnxk/cn9k_rte_flow.c create mode 100644 drivers/net/cnxk/cn9k_rte_flow.h create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index e875e55..cc71b22 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -24,6 +24,7 @@ Features of the CNXK Ethdev PMD are: - Multiple queues for TX and RX - Receiver Side Scaling (RSS) - MAC filtering +- Generic flow API - Inner and Outer Checksum offload - Port hardware statistics - Link state information @@ -208,6 +209,12 @@ CRC stripping The OCTEON CN9K/CN10K SoC family NICs strip the CRC for every packet being received by the host interface irrespective of the offload configuration. +RTE Flow GRE support +~~~~~~~~~~~~~~~~~~~~ + +- ``RTE_FLOW_ITEM_TYPE_GRE_KEY`` works only when checksum and routing + bits in the GRE header are equal to 0. + Debugging Options ----------------- diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 192c15a..e2da5dd 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -39,3 +39,39 @@ Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y + +[rte_flow items] +any = Y +arp_eth_ipv4 = Y +esp = Y +eth = Y +e_tag = Y +geneve = Y +gre = Y +gre_key = Y +gtpc = Y +gtpu = Y +higig2 = Y +icmp = Y +ipv4 = Y +ipv6 = Y +ipv6_ext = Y +mpls = Y +nvgre = Y +raw = Y +sctp = Y +tcp = Y +udp = Y +vlan = Y +vxlan = Y +vxlan_gpe = Y + +[rte_flow actions] +count = Y +drop = Y +flag = Y +pf = Y +port_id = Y +queue = Y +security = Y +vf = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index e990480..89a3c37 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -37,3 +37,39 @@ Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y + +[rte_flow items] +any = Y +arp_eth_ipv4 = Y +esp = Y +eth = Y +e_tag = Y +geneve = Y +gre = Y +gre_key = Y +gtpc = Y +gtpu = Y +higig2 = Y +icmp = Y +ipv4 = Y +ipv6 = Y +ipv6_ext = Y +mpls = Y +nvgre = Y +raw = Y +sctp = Y +tcp = Y +udp = Y +vlan = Y +vxlan = Y +vxlan_gpe = Y + +[rte_flow actions] +count = Y +drop = Y +flag = Y +pf = Y +port_id = Y +queue = Y +security = Y +vf = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 3a4417c..b0e469f 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -34,3 +34,39 @@ Module EEPROM dump = Y Linux = Y ARMv8 = Y Usage doc = Y + +[rte_flow items] +any = Y +arp_eth_ipv4 = Y +esp = Y +eth = Y +e_tag = Y +geneve = Y +gre = Y +gre_key = Y +gtpc = Y +gtpu = Y +higig2 = Y +icmp = Y +ipv4 = Y +ipv6 = Y +ipv6_ext = Y +mpls = Y +nvgre = Y +raw = Y +sctp = Y +tcp = Y +udp = Y +vlan = Y +vxlan = Y +vxlan_gpe = Y + +[rte_flow actions] +count = Y +drop = Y +flag = Y +pf = Y +port_id = Y +queue = Y +security = Y +vf = Y diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 5ff36bb..0396ff6 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ #include "cn10k_ethdev.h" +#include "cn10k_rte_flow.h" #include "cn10k_rx.h" #include "cn10k_tx.h" @@ -308,6 +309,20 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; } +static void +npc_flow_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_flow_ops.create = cn10k_flow_create; + cnxk_flow_ops.destroy = cn10k_flow_destroy; +} + static int cn10k_nix_remove(struct rte_pci_device *pci_dev) { @@ -332,6 +347,7 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } nix_eth_dev_ops_override(); + npc_flow_ops_override(); /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); diff --git a/drivers/net/cnxk/cn10k_rte_flow.c b/drivers/net/cnxk/cn10k_rte_flow.c new file mode 100644 index 0000000..65893cc --- /dev/null +++ b/drivers/net/cnxk/cn10k_rte_flow.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#include <cnxk_rte_flow.h> +#include "cn10k_rte_flow.h" +#include "cn10k_ethdev.h" + +struct rte_flow * +cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow; + + flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); + if (!flow) + return NULL; + + return (struct rte_flow *)flow; +} + +int +cn10k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + + return cnxk_flow_destroy(eth_dev, flow, error); +} diff --git a/drivers/net/cnxk/cn10k_rte_flow.h b/drivers/net/cnxk/cn10k_rte_flow.h new file mode 100644 index 0000000..f64fcf2 --- /dev/null +++ b/drivers/net/cnxk/cn10k_rte_flow.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#ifndef __CN10K_RTE_FLOW_H__ +#define __CN10K_RTE_FLOW_H__ + +#include <rte_flow_driver.h> + +struct rte_flow *cn10k_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cn10k_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error); + +#endif /* __CN10K_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 2157dca..cf9f7c7 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ #include "cn9k_ethdev.h" +#include "cn9k_rte_flow.h" #include "cn9k_rx.h" #include "cn9k_tx.h" @@ -317,6 +318,20 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; } +static void +npc_flow_ops_override(void) +{ + static int init_once; + + if (init_once) + return; + init_once = 1; + + /* Update platform specific ops */ + cnxk_flow_ops.create = cn9k_flow_create; + cnxk_flow_ops.destroy = cn9k_flow_destroy; +} + static int cn9k_nix_remove(struct rte_pci_device *pci_dev) { @@ -342,6 +357,7 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) } nix_eth_dev_ops_override(); + npc_flow_ops_override(); /* Common probe */ rc = cnxk_nix_probe(pci_drv, pci_dev); diff --git a/drivers/net/cnxk/cn9k_rte_flow.c b/drivers/net/cnxk/cn9k_rte_flow.c new file mode 100644 index 0000000..24e1b92 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rte_flow.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#include <cnxk_rte_flow.h> +#include "cn9k_ethdev.h" +#include "cn9k_rte_flow.h" + +struct rte_flow * +cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow; + + flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); + if (!flow) + return NULL; + + return (struct rte_flow *)flow; +} + +int +cn9k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, + struct rte_flow_error *error) +{ + struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + + return cnxk_flow_destroy(eth_dev, flow, error); +} diff --git a/drivers/net/cnxk/cn9k_rte_flow.h b/drivers/net/cnxk/cn9k_rte_flow.h new file mode 100644 index 0000000..43d59e1 --- /dev/null +++ b/drivers/net/cnxk/cn9k_rte_flow.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ +#ifndef __CN9K_RTE_FLOW_H__ +#define __CN9K_RTE_FLOW_H__ + +#include <rte_flow_driver.h> + +struct rte_flow *cn9k_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cn9k_flow_destroy(struct rte_eth_dev *dev, struct rte_flow *flow, + struct rte_flow_error *error); + +#endif /* __CN9K_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4879ef9..b7869b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -233,6 +233,9 @@ cnxk_eth_txq_to_sp(void *__txq) /* Common ethdev ops */ extern struct eth_dev_ops cnxk_eth_dev_ops; +/* Common flow ops */ +extern struct rte_flow_ops cnxk_flow_ops; + /* Ops */ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c new file mode 100644 index 0000000..1695d4f --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -0,0 +1,330 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#include <cnxk_rte_flow.h> + +const struct cnxk_rte_flow_term_info term[] = { + [RTE_FLOW_ITEM_TYPE_ETH] = {ROC_NPC_ITEM_TYPE_ETH, + sizeof(struct rte_flow_item_eth)}, + [RTE_FLOW_ITEM_TYPE_VLAN] = {ROC_NPC_ITEM_TYPE_VLAN, + sizeof(struct rte_flow_item_vlan)}, + [RTE_FLOW_ITEM_TYPE_E_TAG] = {ROC_NPC_ITEM_TYPE_E_TAG, + sizeof(struct rte_flow_item_e_tag)}, + [RTE_FLOW_ITEM_TYPE_IPV4] = {ROC_NPC_ITEM_TYPE_IPV4, + sizeof(struct rte_flow_item_ipv4)}, + [RTE_FLOW_ITEM_TYPE_IPV6] = {ROC_NPC_ITEM_TYPE_IPV6, + sizeof(struct rte_flow_item_ipv6)}, + [RTE_FLOW_ITEM_TYPE_ARP_ETH_IPV4] = { + ROC_NPC_ITEM_TYPE_ARP_ETH_IPV4, + sizeof(struct rte_flow_item_arp_eth_ipv4)}, + [RTE_FLOW_ITEM_TYPE_MPLS] = {ROC_NPC_ITEM_TYPE_MPLS, + sizeof(struct rte_flow_item_mpls)}, + [RTE_FLOW_ITEM_TYPE_ICMP] = {ROC_NPC_ITEM_TYPE_ICMP, + sizeof(struct rte_flow_item_icmp)}, + [RTE_FLOW_ITEM_TYPE_UDP] = {ROC_NPC_ITEM_TYPE_UDP, + sizeof(struct rte_flow_item_udp)}, + [RTE_FLOW_ITEM_TYPE_TCP] = {ROC_NPC_ITEM_TYPE_TCP, + sizeof(struct rte_flow_item_tcp)}, + [RTE_FLOW_ITEM_TYPE_SCTP] = {ROC_NPC_ITEM_TYPE_SCTP, + sizeof(struct rte_flow_item_sctp)}, + [RTE_FLOW_ITEM_TYPE_ESP] = {ROC_NPC_ITEM_TYPE_ESP, + sizeof(struct rte_flow_item_esp)}, + [RTE_FLOW_ITEM_TYPE_GRE] = {ROC_NPC_ITEM_TYPE_GRE, + sizeof(struct rte_flow_item_gre)}, + [RTE_FLOW_ITEM_TYPE_NVGRE] = {ROC_NPC_ITEM_TYPE_NVGRE, + sizeof(struct rte_flow_item_nvgre)}, + [RTE_FLOW_ITEM_TYPE_VXLAN] = {ROC_NPC_ITEM_TYPE_VXLAN, + sizeof(struct rte_flow_item_vxlan)}, + [RTE_FLOW_ITEM_TYPE_GTPC] = {ROC_NPC_ITEM_TYPE_GTPC, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GTPU] = {ROC_NPC_ITEM_TYPE_GTPU, + sizeof(struct rte_flow_item_gtp)}, + [RTE_FLOW_ITEM_TYPE_GENEVE] = {ROC_NPC_ITEM_TYPE_GENEVE, + sizeof(struct rte_flow_item_geneve)}, + [RTE_FLOW_ITEM_TYPE_VXLAN_GPE] = { + ROC_NPC_ITEM_TYPE_VXLAN_GPE, + sizeof(struct rte_flow_item_vxlan_gpe)}, + [RTE_FLOW_ITEM_TYPE_IPV6_EXT] = {ROC_NPC_ITEM_TYPE_IPV6_EXT, + sizeof(struct rte_flow_item_ipv6_ext)}, + [RTE_FLOW_ITEM_TYPE_VOID] = {ROC_NPC_ITEM_TYPE_VOID, 0}, + [RTE_FLOW_ITEM_TYPE_ANY] = {ROC_NPC_ITEM_TYPE_ANY, 0}, + [RTE_FLOW_ITEM_TYPE_GRE_KEY] = {ROC_NPC_ITEM_TYPE_GRE_KEY, + sizeof(uint32_t)}, + [RTE_FLOW_ITEM_TYPE_HIGIG2] = { + ROC_NPC_ITEM_TYPE_HIGIG2, + sizeof(struct rte_flow_item_higig2_hdr)} +}; + +static int +cnxk_map_actions(struct rte_eth_dev *eth_dev, + const struct rte_flow_action actions[], + struct roc_npc_action in_actions[]) +{ + const struct rte_flow_action_count *act_count; + const struct rte_flow_action_queue *act_q; + int rq; + int i = 0; + + for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { + switch (actions->type) { + case RTE_FLOW_ACTION_TYPE_VOID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VOID; + break; + + case RTE_FLOW_ACTION_TYPE_MARK: + in_actions[i].type = ROC_NPC_ACTION_TYPE_MARK; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_FLAG: + in_actions[i].type = ROC_NPC_ACTION_TYPE_FLAG; + break; + + case RTE_FLOW_ACTION_TYPE_COUNT: + act_count = (const struct rte_flow_action_count *) + actions->conf; + + if (act_count->shared == 1) { + plt_npc_dbg("Shared counter is not supported"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_COUNT; + break; + + case RTE_FLOW_ACTION_TYPE_DROP: + in_actions[i].type = ROC_NPC_ACTION_TYPE_DROP; + break; + + case RTE_FLOW_ACTION_TYPE_PF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_PF; + break; + + case RTE_FLOW_ACTION_TYPE_VF: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VF; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_QUEUE: + act_q = (const struct rte_flow_action_queue *) + actions->conf; + rq = act_q->index; + if (rq >= eth_dev->data->nb_rx_queues) { + plt_npc_dbg("Invalid queue index"); + goto err_exit; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_QUEUE; + in_actions[i].conf = actions->conf; + break; + + case RTE_FLOW_ACTION_TYPE_RSS: + in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + break; + + case RTE_FLOW_ACTION_TYPE_SECURITY: + in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; + break; + default: + plt_npc_dbg("Action is not supported = %d", + actions->type); + goto err_exit; + } + i++; + } + in_actions[i].type = ROC_NPC_ACTION_TYPE_END; + return 0; + +err_exit: + return -EINVAL; +} + +static int +cnxk_map_flow_data(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct roc_npc_attr *in_attr, + struct roc_npc_item_info in_pattern[], + struct roc_npc_action in_actions[]) +{ + int i = 0; + + in_attr->priority = attr->priority; + in_attr->ingress = attr->ingress; + in_attr->egress = attr->egress; + + while (pattern->type != RTE_FLOW_ITEM_TYPE_END) { + in_pattern[i].spec = pattern->spec; + in_pattern[i].last = pattern->last; + in_pattern[i].mask = pattern->mask; + in_pattern[i].type = term[pattern->type].item_type; + in_pattern[i].size = term[pattern->type].item_size; + pattern++; + i++; + } + in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; + + return cnxk_map_actions(eth_dev, actions, in_actions); +} + +static int +cnxk_flow_validate(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow flow; + int rc; + + memset(&flow, 0, sizeof(flow)); + + rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return rc; + } + + return roc_npc_flow_parse(npc, &in_attr, in_pattern, in_actions, &flow); +} + +struct roc_npc_flow * +cnxk_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc_item_info in_pattern[ROC_NPC_ITEM_TYPE_END + 1]; + struct roc_npc_action in_actions[ROC_NPC_MAX_ACTION_COUNT]; + struct roc_npc *npc = &dev->npc; + struct roc_npc_attr in_attr; + struct roc_npc_flow *flow; + int errcode; + int rc; + + rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, + in_pattern, in_actions); + if (rc) { + rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, + NULL, "Failed to map flow data"); + return NULL; + } + + flow = roc_npc_flow_create(npc, &in_attr, in_pattern, in_actions, + &errcode); + if (errcode != 0) { + rte_flow_error_set(error, errcode, errcode, NULL, + roc_error_msg_get(errcode)); + return NULL; + } + + return flow; +} + +int +cnxk_flow_destroy(struct rte_eth_dev *eth_dev, struct roc_npc_flow *flow, + struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + int rc; + + rc = roc_npc_flow_destroy(npc, flow); + if (rc) + rte_flow_error_set(error, rc, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow Destroy failed"); + return rc; +} + +static int +cnxk_flow_flush(struct rte_eth_dev *eth_dev, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + int rc; + + rc = roc_npc_mcam_free_all_resources(npc); + if (rc) { + rte_flow_error_set(error, EIO, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Failed to flush filter"); + return -rte_errno; + } + + return 0; +} + +static int +cnxk_flow_query(struct rte_eth_dev *eth_dev, struct rte_flow *flow, + const struct rte_flow_action *action, void *data, + struct rte_flow_error *error) +{ + struct roc_npc_flow *in_flow = (struct roc_npc_flow *)flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + struct rte_flow_query_count *query = data; + const char *errmsg = NULL; + int errcode = ENOTSUP; + int rc; + + if (action->type != RTE_FLOW_ACTION_TYPE_COUNT) { + errmsg = "Only COUNT is supported in query"; + goto err_exit; + } + + if (in_flow->ctr_id == NPC_COUNTER_NONE) { + errmsg = "Counter is not available"; + goto err_exit; + } + + rc = roc_npc_mcam_read_counter(npc, in_flow->ctr_id, &query->hits); + if (rc != 0) { + errcode = EIO; + errmsg = "Error reading flow counter"; + goto err_exit; + } + query->hits_set = 1; + query->bytes_set = 0; + + if (query->reset) + rc = roc_npc_mcam_clear_counter(npc, in_flow->ctr_id); + if (rc != 0) { + errcode = EIO; + errmsg = "Error clearing flow counter"; + goto err_exit; + } + + return 0; + +err_exit: + rte_flow_error_set(error, errcode, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, errmsg); + return -rte_errno; +} + +static int +cnxk_flow_isolate(struct rte_eth_dev *eth_dev __rte_unused, + int enable __rte_unused, struct rte_flow_error *error) +{ + /* If we support, we need to un-install the default mcam + * entry for this port. + */ + + rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, + NULL, "Flow isolation not supported"); + + return -rte_errno; +} + +struct rte_flow_ops cnxk_flow_ops = { + .validate = cnxk_flow_validate, + .flush = cnxk_flow_flush, + .query = cnxk_flow_query, + .isolate = cnxk_flow_isolate, +}; diff --git a/drivers/net/cnxk/cnxk_rte_flow.h b/drivers/net/cnxk/cnxk_rte_flow.h new file mode 100644 index 0000000..bb23629 --- /dev/null +++ b/drivers/net/cnxk/cnxk_rte_flow.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ +#ifndef __CNXK_RTE_FLOW_H__ +#define __CNXK_RTE_FLOW_H__ + +#include <rte_flow_driver.h> +#include <rte_malloc.h> + +#include "cnxk_ethdev.h" +#include "roc_api.h" +#include "roc_npc_priv.h" + +struct cnxk_rte_flow_term_info { + uint16_t item_type; + uint16_t item_size; +}; + +struct roc_npc_flow *cnxk_flow_create(struct rte_eth_dev *dev, + const struct rte_flow_attr *attr, + const struct rte_flow_item pattern[], + const struct rte_flow_action actions[], + struct rte_flow_error *error); +int cnxk_flow_destroy(struct rte_eth_dev *dev, struct roc_npc_flow *flow, + struct rte_flow_error *error); + +#endif /* __CNXK_RTE_FLOW_H__ */ diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index 7d711a2..df953fd 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,10 +13,12 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_rte_flow.c', 'cnxk_stats.c') # CN9K sources += files('cn9k_ethdev.c', + 'cn9k_rte_flow.c', 'cn9k_rx.c', 'cn9k_rx_mseg.c', 'cn9k_rx_vec.c', @@ -25,6 +27,7 @@ sources += files('cn9k_ethdev.c', 'cn9k_tx_vec.c') # CN10K sources += files('cn10k_ethdev.c', + 'cn10k_rte_flow.c', 'cn10k_rx.c', 'cn10k_rx_mseg.c', 'cn10k_rx_vec.c', -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow Nithin Dabilpuram @ 2021-07-05 22:01 ` Thomas Monjalon 2021-07-07 9:29 ` Nithin Dabilpuram 0 siblings, 1 reply; 262+ messages in thread From: Thomas Monjalon @ 2021-07-05 22:01 UTC (permalink / raw) To: jerinj, Nithin Dabilpuram Cc: dev, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra 23/06/2021 06:46, Nithin Dabilpuram: > From: Kiran Kumar K <kirankumark@marvell.com> > > Adding initial version of rte_flow support for cnxk family device. > Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, flow_destroy? > flow_flush, flow_query, flow_isolate. > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> [...] > --- a/doc/guides/nics/features/cnxk.ini > +++ b/doc/guides/nics/features/cnxk.ini > @@ -39,3 +39,39 @@ Module EEPROM dump = Y > Linux = Y > ARMv8 = Y > Usage doc = Y > + > +[rte_flow items] > +any = Y > +arp_eth_ipv4 = Y > +esp = Y > +eth = Y > +e_tag = Y > +geneve = Y > +gre = Y > +gre_key = Y > +gtpc = Y > +gtpu = Y > +higig2 = Y > +icmp = Y > +ipv4 = Y > +ipv6 = Y > +ipv6_ext = Y > +mpls = Y > +nvgre = Y > +raw = Y raw item not found in PMD code. > +sctp = Y > +tcp = Y > +udp = Y > +vlan = Y > +vxlan = Y > +vxlan_gpe = Y > + > +[rte_flow actions] > +count = Y > +drop = Y > +flag = Y > +pf = Y > +port_id = Y port_id action not found in PMD code. > +queue = Y > +security = Y > +vf = Y I will remove raw and port_id while applying. ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow 2021-07-05 22:01 ` Thomas Monjalon @ 2021-07-07 9:29 ` Nithin Dabilpuram 0 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-07-07 9:29 UTC (permalink / raw) To: Thomas Monjalon Cc: jerinj, dev, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra On Tue, Jul 06, 2021 at 12:01:21AM +0200, Thomas Monjalon wrote: > 23/06/2021 06:46, Nithin Dabilpuram: > > From: Kiran Kumar K <kirankumark@marvell.com> > > > > Adding initial version of rte_flow support for cnxk family device. > > Supported rte_flow ops are flow_validate, flow_create, flow_crstroy, > > flow_destroy? > > > flow_flush, flow_query, flow_isolate. > > > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > [...] > > --- a/doc/guides/nics/features/cnxk.ini > > +++ b/doc/guides/nics/features/cnxk.ini > > @@ -39,3 +39,39 @@ Module EEPROM dump = Y > > Linux = Y > > ARMv8 = Y > > Usage doc = Y > > + > > +[rte_flow items] > > +any = Y > > +arp_eth_ipv4 = Y > > +esp = Y > > +eth = Y > > +e_tag = Y > > +geneve = Y > > +gre = Y > > +gre_key = Y > > +gtpc = Y > > +gtpu = Y > > +higig2 = Y > > +icmp = Y > > +ipv4 = Y > > +ipv6 = Y > > +ipv6_ext = Y > > +mpls = Y > > +nvgre = Y > > +raw = Y > > raw item not found in PMD code. > > > +sctp = Y > > +tcp = Y > > +udp = Y > > +vlan = Y > > +vxlan = Y > > +vxlan_gpe = Y > > + > > +[rte_flow actions] > > +count = Y > > +drop = Y > > +flag = Y > > +pf = Y > > +port_id = Y > > port_id action not found in PMD code. > > > +queue = Y > > +security = Y > > +vf = Y > > I will remove raw and port_id while applying. Ack, thanks. > > > ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 50/62] net/cnxk: add flow ops get operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (48 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram ` (12 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> This patch adds flow ops get operation to enable rte_flow_ops.A This patch also add support for flow dev dump API. Every flow rule added will be dumped in the below format. MCAM Index:1881 Interface :NIX-RX (0) Priority :1 NPC RX Action:0X00000000404001 ActionOp:NIX_RX_ACTIONOP_UCAST (1) PF_FUNC: 0X400 RQ Index:0X004 Match Id:0000 Flow Key Alg:0 NPC RX VTAG Action:0X00000000008100 VTAG0:relptr:0 lid:0X1 type:0 Patterns: NPC_PARSE_NIBBLE_CHAN:000 NPC_PARSE_NIBBLE_LA_LTYPE:LA_ETHER NPC_PARSE_NIBBLE_LB_LTYPE:NONE NPC_PARSE_NIBBLE_LC_LTYPE:LC_IP NPC_PARSE_NIBBLE_LD_LTYPE:LD_TCP NPC_PARSE_NIBBLE_LE_LTYPE:NONE LA_ETHER, hdr offset:0, len:0X6, key offset:0X8,\ Data:0X4AE124FC7FFF, Mask:0XFFFFFFFFFFFF LA_ETHER, hdr offset:0XC, len:0X2, key offset:0X4, Data:0XCA5A,\ Mask:0XFFFF LC_IP, hdr offset:0XC, len:0X8, key offset:0X10,\ Data:0X0A01010300000000, Mask:0XFFFFFFFF00000000 LD_TCP, hdr offset:0, len:0X4, key offset:0X18, Data:0X03450000,\ Mask:0XFFFF0000 MCAM Raw Data : DW0 :0000CA5A01202000 DW0_Mask:0000FFFF0FF0F000 DW1 :00004AE124FC7FFF DW1_Mask:0000FFFFFFFFFFFF DW2 :0A01010300000000 DW2_Mask:FFFFFFFF00000000 DW3 :0000000003450000 DW3_Mask:00000000FFFF0000 DW4 :0000000000000000 DW4_Mask:0000000000000000 DW5 :0000000000000000 DW5_Mask:0000000000000000 DW6 :0000000000000000 DW6_Mask:0000000000000000 Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- drivers/common/cnxk/roc_npc.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 3 +++ drivers/net/cnxk/cnxk_ethdev.h | 3 ++- drivers/net/cnxk/cnxk_ethdev_ops.c | 10 ++++++++++ drivers/net/cnxk/cnxk_rte_flow.c | 28 ++++++++++++++++++++++++++++ 5 files changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/common/cnxk/roc_npc.c b/drivers/common/cnxk/roc_npc.c index 8a76823..aff4eef 100644 --- a/drivers/common/cnxk/roc_npc.c +++ b/drivers/common/cnxk/roc_npc.c @@ -1009,6 +1009,8 @@ roc_npc_flow_create(struct roc_npc *roc_npc, const struct roc_npc_attr *attr, struct npc_flow_list *list; int rc; + npc->channel = roc_npc->channel; + flow = plt_zmalloc(sizeof(*flow), 0); if (flow == NULL) { *errcode = NPC_ERR_NO_MEM; diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 7088e67..d863810 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -819,6 +819,8 @@ cnxk_nix_configure(struct rte_eth_dev *eth_dev) goto fail_configure; } + dev->npc.channel = roc_nix_get_base_chan(nix); + nb_rxq = data->nb_rx_queues; nb_txq = data->nb_tx_queues; rc = -ENOMEM; @@ -1223,6 +1225,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, + .flow_ops_get = cnxk_nix_flow_ops_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index b7869b4..07c87ea 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -273,7 +273,8 @@ int cnxk_nix_rx_queue_intr_disable(struct rte_eth_dev *eth_dev, uint16_t rx_queue_id); int cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool); int cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt); - +int cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops); int cnxk_nix_configure(struct rte_eth_dev *eth_dev); int cnxk_nix_tx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, uint16_t nb_desc, uint16_t fp_tx_q_sz, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 305220a..36ca750 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -300,6 +300,16 @@ cnxk_nix_flow_ctrl_set(struct rte_eth_dev *eth_dev, } int +cnxk_nix_flow_ops_get(struct rte_eth_dev *eth_dev, + const struct rte_flow_ops **ops) +{ + RTE_SET_USED(eth_dev); + + *ops = &cnxk_flow_ops; + return 0; +} + +int cnxk_nix_mac_addr_set(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index 1695d4f..2dee19e 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -322,9 +322,37 @@ cnxk_flow_isolate(struct rte_eth_dev *eth_dev __rte_unused, return -rte_errno; } +static int +cnxk_flow_dev_dump(struct rte_eth_dev *eth_dev, struct rte_flow *flow, + FILE *file, struct rte_flow_error *error) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_npc *npc = &dev->npc; + + if (file == NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL, + "Invalid file"); + return -rte_errno; + } + + if (flow != NULL) { + rte_flow_error_set(error, EINVAL, + RTE_FLOW_ERROR_TYPE_HANDLE, + NULL, + "Invalid argument"); + return -EINVAL; + } + + roc_npc_flow_dump(file, npc); + + return 0; +} + struct rte_flow_ops cnxk_flow_ops = { .validate = cnxk_flow_validate, .flush = cnxk_flow_flush, .query = cnxk_flow_query, .isolate = cnxk_flow_isolate, + .dev_dump = cnxk_flow_dev_dump, }; -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 51/62] net/cnxk: add ethdev firmware version get 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (49 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 52/62] net/cnxk: add get register operation Nithin Dabilpuram ` (11 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> Add callback to get ethdev firmware version. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 2 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 19 +++++++++++++++++++ 6 files changed, 25 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index e2da5dd..d43b1c4 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -35,6 +35,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 89a3c37..8fb54d4 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -33,6 +33,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index b0e469f..2da2252 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -30,6 +30,7 @@ Packet type parsing = Y Basic stats = Y Stats per queue = Y Extended stats = Y +FW version = Y Module EEPROM dump = Y Linux = Y ARMv8 = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index d863810..26e4ddf 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1222,6 +1222,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .xstats_reset = cnxk_nix_xstats_reset, .xstats_get_by_id = cnxk_nix_xstats_get_by_id, .xstats_get_names_by_id = cnxk_nix_xstats_get_names_by_id, + .fw_version_get = cnxk_nix_fw_version_get, .rxq_info_get = cnxk_nix_rxq_info_get, .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 07c87ea..a25be8a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -313,6 +313,8 @@ int cnxk_nix_xstats_get_names_by_id(struct rte_eth_dev *eth_dev, int cnxk_nix_xstats_get_by_id(struct rte_eth_dev *eth_dev, const uint64_t *ids, uint64_t *values, unsigned int n); int cnxk_nix_xstats_reset(struct rte_eth_dev *eth_dev); +int cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size); void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo); void cnxk_nix_txq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 36ca750..228c624 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -639,6 +639,25 @@ cnxk_nix_pool_ops_supported(struct rte_eth_dev *eth_dev, const char *pool) return -ENOTSUP; } +int +cnxk_nix_fw_version_get(struct rte_eth_dev *eth_dev, char *fw_version, + size_t fw_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const char *str = roc_npc_profile_name_get(&dev->npc); + uint32_t size = strlen(str) + 1; + + if (fw_size > size) + fw_size = size; + + rte_strlcpy(fw_version, str, fw_size); + + if (fw_size < size) + return size; + + return 0; +} + void cnxk_nix_rxq_info_get(struct rte_eth_dev *eth_dev, uint16_t qid, struct rte_eth_rxq_info *qinfo) -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 52/62] net/cnxk: add get register operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (50 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram ` (10 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> With this patch implemented api to dump platform registers for debug purposes. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/cnxk.rst | 1 + doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 4 ++++ drivers/net/cnxk/cnxk_ethdev_ops.c | 25 +++++++++++++++++++++++++ 7 files changed, 34 insertions(+) diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst index cc71b22..e561321 100644 --- a/doc/guides/nics/cnxk.rst +++ b/doc/guides/nics/cnxk.rst @@ -32,6 +32,7 @@ Features of the CNXK Ethdev PMD are: - MTU update - Scatter-Gather IO support - Vector Poll mode driver +- Debug utilities - Context dump and error interrupt support - Support Rx interrupt Prerequisites diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index d43b1c4..246caf9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -37,6 +37,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 8fb54d4..6c6aeb0 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -35,6 +35,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 2da2252..8ec326a 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -32,6 +32,7 @@ Stats per queue = Y Extended stats = Y FW version = Y Module EEPROM dump = Y +Registers dump = Y Linux = Y ARMv8 = Y Usage doc = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 26e4ddf..b76442d 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1227,6 +1227,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .txq_info_get = cnxk_nix_txq_info_get, .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, + .get_reg = cnxk_nix_dev_get_reg, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index a25be8a..d51a6d7 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -328,6 +328,10 @@ void *cnxk_nix_fastpath_lookup_mem_get(void); int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev); +/* Debug */ +int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, + struct rte_dev_reg_info *regs); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 228c624..c879b25 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -697,3 +697,28 @@ cnxk_nix_tx_done_cleanup(void *txq, uint32_t free_cnt) return 0; } + +int +cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t *data = regs->data; + int rc = -ENOTSUP; + + if (data == NULL) { + rc = roc_nix_lf_get_reg_count(nix); + if (rc > 0) { + regs->length = rc; + regs->width = 8; + rc = 0; + } + return rc; + } + + if (!regs->length || + regs->length == (uint32_t)roc_nix_lf_get_reg_count(nix)) + return roc_nix_lf_reg_dump(nix, data); + + return rc; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 53/62] net/cnxk: support for RSS in rte flow 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (51 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 52/62] net/cnxk: add get register operation Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram ` (9 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> Added support for RSS action in rte flow code based on ROC. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + doc/guides/nics/features/cnxk_vf.ini | 1 + drivers/net/cnxk/cnxk_rte_flow.c | 74 +++++++++++++++++++++++++++++++---- 4 files changed, 69 insertions(+), 8 deletions(-) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 246caf9..eba4107 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -75,5 +75,6 @@ flag = Y pf = Y port_id = Y queue = Y +rss = Y security = Y vf = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 6c6aeb0..4871fac 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -73,5 +73,6 @@ flag = Y pf = Y port_id = Y queue = Y +rss = Y security = Y vf = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 8ec326a..81ee7cc 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -70,5 +70,6 @@ flag = Y pf = Y port_id = Y queue = Y +rss = Y security = Y vf = Y diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index 2dee19e..c61618a 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -56,14 +56,64 @@ const struct cnxk_rte_flow_term_info term[] = { }; static int -cnxk_map_actions(struct rte_eth_dev *eth_dev, +npc_rss_action_validate(struct rte_eth_dev *eth_dev, + const struct rte_flow_attr *attr, + const struct rte_flow_action *act) +{ + const struct rte_flow_action_rss *rss; + + rss = (const struct rte_flow_action_rss *)act->conf; + + if (attr->egress) { + plt_err("No support of RSS in egress"); + return -EINVAL; + } + + if (eth_dev->data->dev_conf.rxmode.mq_mode != ETH_MQ_RX_RSS) { + plt_err("multi-queue mode is disabled"); + return -ENOTSUP; + } + + if (!rss || !rss->queue_num) { + plt_err("no valid queues"); + return -EINVAL; + } + + if (rss->func != RTE_ETH_HASH_FUNCTION_DEFAULT) { + plt_err("non-default RSS hash functions are not supported"); + return -ENOTSUP; + } + + if (rss->key_len && rss->key_len > ROC_NIX_RSS_KEY_LEN) { + plt_err("RSS hash key too large"); + return -ENOTSUP; + } + + return 0; +} + +static void +npc_rss_flowkey_get(struct cnxk_eth_dev *eth_dev, + const struct roc_npc_action *rss_action, + uint32_t *flowkey_cfg) +{ + const struct roc_npc_action_rss *rss; + + rss = (const struct roc_npc_action_rss *)rss_action->conf; + + *flowkey_cfg = cnxk_rss_ethdev_to_nix(eth_dev, rss->types, rss->level); +} + +static int +cnxk_map_actions(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); const struct rte_flow_action_count *act_count; const struct rte_flow_action_queue *act_q; + int i = 0, rc = 0; int rq; - int i = 0; for (; actions->type != RTE_FLOW_ACTION_TYPE_END; actions++) { switch (actions->type) { @@ -117,7 +167,12 @@ cnxk_map_actions(struct rte_eth_dev *eth_dev, break; case RTE_FLOW_ACTION_TYPE_RSS: + rc = npc_rss_action_validate(eth_dev, attr, actions); + if (rc) + goto err_exit; in_actions[i].type = ROC_NPC_ACTION_TYPE_RSS; + in_actions[i].conf = actions->conf; + npc_rss_flowkey_get(dev, &in_actions[i], flowkey_cfg); break; case RTE_FLOW_ACTION_TYPE_SECURITY: @@ -144,7 +199,7 @@ cnxk_map_flow_data(struct rte_eth_dev *eth_dev, const struct rte_flow_action actions[], struct roc_npc_attr *in_attr, struct roc_npc_item_info in_pattern[], - struct roc_npc_action in_actions[]) + struct roc_npc_action in_actions[], uint32_t *flowkey_cfg) { int i = 0; @@ -163,7 +218,8 @@ cnxk_map_flow_data(struct rte_eth_dev *eth_dev, } in_pattern[i].type = ROC_NPC_ITEM_TYPE_END; - return cnxk_map_actions(eth_dev, actions, in_actions); + return cnxk_map_actions(eth_dev, attr, actions, in_actions, + flowkey_cfg); } static int @@ -179,12 +235,13 @@ cnxk_flow_validate(struct rte_eth_dev *eth_dev, struct roc_npc *npc = &dev->npc; struct roc_npc_attr in_attr; struct roc_npc_flow flow; + uint32_t flowkey_cfg = 0; int rc; memset(&flow, 0, sizeof(flow)); rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, &flowkey_cfg); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); @@ -206,11 +263,12 @@ cnxk_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, struct roc_npc *npc = &dev->npc; struct roc_npc_attr in_attr; struct roc_npc_flow *flow; - int errcode; + int errcode = 0; int rc; rc = cnxk_map_flow_data(eth_dev, attr, pattern, actions, &in_attr, - in_pattern, in_actions); + in_pattern, in_actions, + &npc->flowkey_cfg_state); if (rc) { rte_flow_error_set(error, 0, RTE_FLOW_ERROR_TYPE_ACTION_NUM, NULL, "Failed to map flow data"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 54/62] net/cnxk: register callback to get PTP status 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (52 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 55/62] net/cnxk: support base PTP timesync Nithin Dabilpuram ` (8 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Once PTP status is changed at H/W i.e. enable/disable then it is propagated to user via registered callback. So corresponding callback is registered to get PTP status. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 87 ++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cn10k_rx.h | 1 + drivers/net/cnxk/cn9k_ethdev.c | 73 ++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 +- drivers/net/cnxk/cnxk_ethdev.h | 5 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 2 + 7 files changed, 166 insertions(+), 5 deletions(-) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 0396ff6..bddb7fb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -268,6 +268,76 @@ cn10k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn10k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn10k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cannot be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -333,6 +403,7 @@ static int cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { struct rte_eth_dev *eth_dev; + struct cnxk_eth_dev *dev; int rc; if (RTE_CACHE_LINE_SIZE != 64) { @@ -354,15 +425,23 @@ cn10k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) if (rc) return rc; + /* Find eth dev allocated */ + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); + if (!eth_dev) + return -ENOENT; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { - eth_dev = rte_eth_dev_allocated(pci_dev->device.name); - if (!eth_dev) - return -ENOENT; - /* Setup callbacks for secondary process */ cn10k_eth_set_tx_function(eth_dev); cn10k_eth_set_rx_function(eth_dev); + return 0; } + + dev = cnxk_eth_pmd_priv(eth_dev); + + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn10k_nix_ptp_info_update_cb); + return 0; } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 7bb9dd8..29ee0ac 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -12,6 +12,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index cf9f7c7..63b13eb 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -277,6 +277,76 @@ cn9k_nix_configure(struct rte_eth_dev *eth_dev) return 0; } +/* Function to enable ptp config for VFs */ +static void +nix_ptp_enable_vf(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (nix_recalc_mtu(eth_dev)) + plt_err("Failed to set MTU size for ptp"); + + dev->scalar_ena = true; + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + + /* Setting up the function pointers as per new offload flags */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); +} + +static uint16_t +nix_ptp_vf_burst(void *queue, struct rte_mbuf **mbufs, uint16_t pkts) +{ + struct cn9k_eth_rxq *rxq = queue; + struct cnxk_eth_rxq_sp *rxq_sp; + struct rte_eth_dev *eth_dev; + + RTE_SET_USED(mbufs); + RTE_SET_USED(pkts); + + rxq_sp = cnxk_eth_rxq_to_sp(rxq); + eth_dev = rxq_sp->dev->eth_dev; + nix_ptp_enable_vf(eth_dev); + + return 0; +} + +static int +cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) +{ + struct cnxk_eth_dev *dev = (struct cnxk_eth_dev *)nix; + struct rte_eth_dev *eth_dev; + struct cn9k_eth_rxq *rxq; + int i; + + if (!dev) + return -EINVAL; + + eth_dev = dev->eth_dev; + if (!eth_dev) + return -EINVAL; + + dev->ptp_en = ptp_en; + + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { + rxq = eth_dev->data->rx_queues[i]; + rxq->mbuf_initializer = cnxk_nix_rxq_mbuf_setup(dev); + } + + if (roc_nix_is_vf_or_sdp(nix) && !(roc_nix_is_sdp(nix)) && + !(roc_nix_is_lbk(nix))) { + /* In case of VF, setting of MTU cannot be done directly in this + * function as this is running as part of MBOX request(PF->VF) + * and MTU setting also requires MBOX message to be + * sent(VF->PF) + */ + eth_dev->rx_pkt_burst = nix_ptp_vf_burst; + rte_mb(); + } + + return 0; +} + static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { @@ -396,6 +466,9 @@ cn9k_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) dev->hwcap = 0; + /* Register up msg callbacks for PTP information */ + roc_nix_ptp_info_cb_register(&dev->nix, cn9k_nix_ptp_info_update_cb); + /* Update HW erratas */ if (roc_model_is_cn96_a0() || roc_model_is_cn95_a0()) dev->cq_min_4k = 1; diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index bc04f5c..f4b3282 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -13,6 +13,7 @@ #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) +#define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b76442d..522f7ec 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -59,7 +59,7 @@ nix_enable_mseg_on_jumbo(struct cnxk_eth_rxq_sp *rxq) } } -static int +int nix_recalc_mtu(struct rte_eth_dev *eth_dev) { struct rte_eth_dev_data *data = eth_dev->data; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index d51a6d7..1c41dcb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -100,6 +100,7 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +#define CNXK_NIX_TIMESYNC_RX_OFFSET 8 #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -153,6 +154,7 @@ struct cnxk_eth_dev { uint16_t flags; uint8_t ptype_disable; bool scalar_ena; + bool ptp_en; /* Pointer back to rte */ struct rte_eth_dev *eth_dev; @@ -332,6 +334,9 @@ int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, int cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs); +/* Other private functions */ +int nix_recalc_mtu(struct rte_eth_dev *eth_dev); + /* Inlines */ static __rte_always_inline uint64_t cnxk_pktmbuf_detach(struct rte_mbuf *m) diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index c879b25..91de6b7 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -385,6 +385,8 @@ cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu) int rc = -EINVAL; uint32_t buffsz; + frame_size += CNXK_NIX_TIMESYNC_RX_OFFSET * dev->ptp_en; + /* Check if MTU is within the allowed range */ if ((frame_size - RTE_ETHER_CRC_LEN) < NIX_MIN_HW_FRS) { plt_err("MTU is lesser than minimum"); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 55/62] net/cnxk: support base PTP timesync 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (53 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram ` (7 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Base PTP timesync support is added for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 31 +++++ drivers/net/cnxk/cn10k_ethdev.h | 1 + drivers/net/cnxk/cn10k_rx.c | 32 ++--- drivers/net/cnxk/cn10k_rx.h | 61 +++++++--- drivers/net/cnxk/cn10k_rx_mseg.c | 2 +- drivers/net/cnxk/cn10k_rx_vec.c | 5 +- drivers/net/cnxk/cn10k_tx.c | 28 +++-- drivers/net/cnxk/cn10k_tx.h | 193 +++++++++++++++++++++++------ drivers/net/cnxk/cn10k_tx_mseg.c | 2 +- drivers/net/cnxk/cn10k_tx_vec.c | 3 +- drivers/net/cnxk/cn9k_ethdev.c | 34 +++++- drivers/net/cnxk/cn9k_ethdev.h | 1 + drivers/net/cnxk/cn9k_rx.c | 32 ++--- drivers/net/cnxk/cn9k_rx.h | 61 +++++++--- drivers/net/cnxk/cn9k_rx_mseg.c | 2 +- drivers/net/cnxk/cn9k_rx_vec.c | 5 +- drivers/net/cnxk/cn9k_tx.c | 28 +++-- drivers/net/cnxk/cn9k_tx.h | 253 ++++++++++++++++++++++++++++----------- drivers/net/cnxk/cn9k_tx_mseg.c | 2 +- drivers/net/cnxk/cn9k_tx_vec.c | 3 +- drivers/net/cnxk/cnxk_ethdev.c | 36 +++++- drivers/net/cnxk/cnxk_ethdev.h | 64 +++++++++- drivers/net/cnxk/cnxk_ptp.c | 169 ++++++++++++++++++++++++++ drivers/net/cnxk/meson.build | 1 + 24 files changed, 834 insertions(+), 215 deletions(-) create mode 100644 drivers/net/cnxk/cnxk_ptp.c diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index bddb7fb..5e0de13 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -30,6 +30,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -95,6 +98,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -121,6 +127,7 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; union nix_send_hdr_w0_u send_hdr_w0; + struct nix_send_mem_s *send_mem; union nix_send_sg_s sg_w0; RTE_SET_USED(dev); @@ -136,6 +143,22 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn10k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[0]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr_w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *)(txq->cmd + 2); + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } } else { /* 2(HDR) + 1(SG) + 1(IOVA) = 4/2 - 1 = 1 */ send_hdr_w0.sizem1 = 1; @@ -221,6 +244,7 @@ cn10k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -342,6 +366,7 @@ static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -349,6 +374,12 @@ cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn10k_ethdev.h b/drivers/net/cnxk/cn10k_ethdev.h index d39ca31..8b6e0f2 100644 --- a/drivers/net/cnxk/cn10k_ethdev.h +++ b/drivers/net/cnxk/cn10k_ethdev.h @@ -31,6 +31,7 @@ struct cn10k_eth_rxq { uint32_t available; uint16_t data_off; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index 0598111..c9744e2 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index 29ee0ac..c09ccdf 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -7,6 +7,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -250,6 +252,10 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn10k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -487,27 +493,44 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c index 9d283f7..b67d21f 100644 --- a/drivers/net/cnxk/cn10k_rx_mseg.c +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c index 0fa079c..1330235 100644 --- a/drivers/net/cnxk/cn10k_rx_vec.c +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -5,12 +5,15 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ struct rte_mbuf **rx_pkts, \ uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn10k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn10k_tx.c b/drivers/net/cnxk/cn10k_tx.c index e6eb101..18694dc 100644 --- a/drivers/net/cnxk/cn10k_tx.c +++ b/drivers/net/cnxk/cn10k_tx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -24,12 +24,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TSP] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -42,25 +43,25 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -68,7 +69,8 @@ cn10k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn10k_tx.h b/drivers/net/cnxk/cn10k_tx.h index b74df10..8b1446f 100644 --- a/drivers/net/cnxk/cn10k_tx.h +++ b/drivers/net/cnxk/cn10k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -49,8 +51,12 @@ static __rte_always_inline int cn10k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline uint8_t @@ -380,6 +386,45 @@ cn10k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, uintptr_t lmt_addr, *(rte_iova_t *)(lmt_addr + 8) = *(rte_iova_t *)(sg + 1); } +static __rte_always_inline void +cn10k_nix_xmit_prepare_tstamp(uintptr_t lmt_addr, const uint64_t *cmd, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + struct nix_send_ext_s *send_hdr_ext = + (struct nix_send_ext_s *)lmt_addr + 16; + uint64_t *lmt = (uint64_t *)lmt_addr; + uint16_t off = (no_segdw - 1) << 1; + struct nix_send_mem_s *send_mem; + + send_mem = (struct nix_send_mem_s *)(lmt + off); + send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + send_hdr_ext->w0.tstmp = 1; + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + lmt[off] = cmd[2]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.subdc = NIX_SUBDC_MEM; + send_mem->w0.alg = NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + send_mem->addr = + (rte_iova_t)(((uint64_t *)cmd[3]) + is_ol_tstamp); + } +} + static __rte_always_inline uint16_t cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) { @@ -445,7 +490,7 @@ cn10k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -487,6 +532,8 @@ cn10k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, cn10k_nix_xmit_prepare(tx_pkts[i], cmd, lmt_addr, flags, lso_tun_fmt); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); } @@ -576,6 +623,9 @@ cn10k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, /* Store sg list directly on lmt line */ segdw = cn10k_nix_prepare_mseg(tx_pkts[i], (uint64_t *)lmt_addr, flags); + cn10k_nix_xmit_prepare_tstamp(lmt_addr, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); lmt_addr += (1ULL << ROC_LMT_LINE_SIZE_LOG2); data128 |= (((__uint128_t)(segdw - 1)) << shft); shft += 3; @@ -1406,75 +1456,140 @@ cn10k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ #define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_tx_mseg.c b/drivers/net/cnxk/cn10k_tx_mseg.c index 6ae6907..33f6754 100644 --- a/drivers/net/cnxk/cn10k_tx_mseg.c +++ b/drivers/net/cnxk/cn10k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn10k_tx_vec.c b/drivers/net/cnxk/cn10k_tx_vec.c index 42baeb5..7453f3b 100644 --- a/drivers/net/cnxk/cn10k_tx_vec.c +++ b/drivers/net/cnxk/cn10k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn10k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd,\ diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 63b13eb..0d63604 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -30,6 +30,9 @@ nix_rx_offload_flags(struct rte_eth_dev *eth_dev) if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) flags |= NIX_RX_MULTI_SEG_F; + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_RX_OFFLOAD_TSTAMP_F; + if (!dev->ptype_disable) flags |= NIX_RX_OFFLOAD_PTYPE_F; @@ -95,6 +98,9 @@ nix_tx_offload_flags(struct rte_eth_dev *eth_dev) flags |= (NIX_TX_OFFLOAD_TSO_F | NIX_TX_OFFLOAD_OL3_OL4_CSUM_F | NIX_TX_OFFLOAD_L3_L4_CSUM_F); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP)) + flags |= NIX_TX_OFFLOAD_TSTAMP_F; + return flags; } @@ -121,10 +127,9 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, { struct nix_send_ext_s *send_hdr_ext; struct nix_send_hdr_s *send_hdr; + struct nix_send_mem_s *send_mem; union nix_send_sg_s *sg; - RTE_SET_USED(dev); - /* Initialize the fields based on basic single segment packet */ memset(&txq->cmd, 0, sizeof(txq->cmd)); @@ -135,6 +140,23 @@ nix_form_default_desc(struct cnxk_eth_dev *dev, struct cn9k_eth_txq *txq, send_hdr_ext = (struct nix_send_ext_s *)&txq->cmd[2]; send_hdr_ext->w0.subdc = NIX_SUBDC_EXT; + if (dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F) { + /* Default: one seg packet would have: + * 2(HDR) + 2(EXT) + 1(SG) + 1(IOVA) + 2(MEM) + * => 8/2 - 1 = 3 + */ + send_hdr->w0.sizem1 = 3; + send_hdr_ext->w0.tstmp = 1; + + /* To calculate the offset for send_mem, + * send_hdr->w0.sizem1 * 2 + */ + send_mem = (struct nix_send_mem_s *) + (txq->cmd + (send_hdr->w0.sizem1 << 1)); + send_mem->w0.cn9k.subdc = NIX_SUBDC_MEM; + send_mem->w0.cn9k.alg = NIX_SENDMEMALG_SETTSTMP; + send_mem->addr = dev->tstamp.tx_tstamp_iova; + } sg = (union nix_send_sg_s *)&txq->cmd[4]; } else { send_hdr = (struct nix_send_hdr_s *)&txq->cmd[0]; @@ -219,6 +241,7 @@ cn9k_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, rxq->wdata = cq->wdata; rxq->head = cq->head; rxq->qmask = cq->qmask; + rxq->tstamp = &dev->tstamp; /* Data offset from data to start of mbuf is first_skip */ rxq->data_off = rq->first_skip; @@ -351,6 +374,7 @@ static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; int rc; /* Common eth dev start */ @@ -358,6 +382,12 @@ cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) if (rc) return rc; + /* Update VF about data off shifted by 8 bytes if PTP already + * enabled in PF owning this VF + */ + if (dev->ptp_en && (!roc_nix_is_pf(nix) && (!roc_nix_is_sdp(nix)))) + nix_ptp_enable_vf(eth_dev); + /* Setting up the rx[tx]_offload_flags due to change * in rx[tx]_offloads. */ diff --git a/drivers/net/cnxk/cn9k_ethdev.h b/drivers/net/cnxk/cn9k_ethdev.h index f8344e3..3d4a206 100644 --- a/drivers/net/cnxk/cn9k_ethdev.h +++ b/drivers/net/cnxk/cn9k_ethdev.h @@ -29,6 +29,7 @@ struct cn9k_eth_rxq { uint32_t qmask; uint32_t available; uint16_t rq; + struct cnxk_timesync_info *tstamp; } __plt_cache_aligned; /* Rx and Tx routines */ diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 01eb21f..a15428d 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_PTYPE_F)] @@ -34,31 +35,34 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2] = { -#define R(name, f3, f2, f1, f0, flags) \ - [f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { +#define R(name, f4, f3, f2, f1, f0, flags) \ + [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R }; - if (dev->scalar_ena) + /* For PTP enabled, scalar rx function should be chosen as most of the + * PTP apps are implemented to rx burst 1 pkt. + */ + if (dev->scalar_ena || dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) pick_rx_func(eth_dev, nix_eth_rx_burst); else pick_rx_func(eth_dev, nix_eth_rx_vec_burst); @@ -69,6 +73,6 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index f4b3282..c5ad5db 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -8,6 +8,8 @@ #include <rte_ether.h> #include <rte_vect.h> +#include <cnxk_ethdev.h> + #define NIX_RX_OFFLOAD_NONE (0) #define NIX_RX_OFFLOAD_RSS_F BIT(0) #define NIX_RX_OFFLOAD_PTYPE_F BIT(1) @@ -253,6 +255,10 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, cn9k_nix_cqe_to_mbuf(cq, cq->tag, mbuf, lookup_mem, mbuf_init, flags); + cnxk_nix_mbuf_to_tstamp(mbuf, rxq->tstamp, + (flags & NIX_RX_OFFLOAD_TSTAMP_F), + (uint64_t *)((uint8_t *)mbuf + data_off) + ); rx_pkts[packets++] = mbuf; roc_prefetch_store_keep(mbuf); head++; @@ -489,27 +495,44 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define PTYPE_F NIX_RX_OFFLOAD_PTYPE_F #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F +#define TS_F NIX_RX_OFFLOAD_TSTAMP_F -/* [MARK] [CKSUM] [PTYPE] [RSS] */ -#define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F)\ -R(mark_cksum_ptype_rss, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F) +/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +#define NIX_RX_FASTPATH_MODES \ +R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ +R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ +R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ +R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ +R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c index 6ad8c1d..3b26962 100644 --- a/drivers/net/cnxk/cn9k_rx_mseg.c +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c index 997177f..b19c7f3 100644 --- a/drivers/net/cnxk/cn9k_rx_vec.c +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -5,10 +5,13 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f3, f2, f1, f0, flags) \ +#define R(name, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ + /* TSTMP is not supported by vector */ \ + if ((flags) & NIX_RX_OFFLOAD_TSTAMP_F) \ + return 0; \ return cn9k_nix_recv_pkts_vector(rx_queue, rx_pkts, pkts, \ (flags)); \ } diff --git a/drivers/net/cnxk/cn9k_tx.c b/drivers/net/cnxk/cn9k_tx.c index 2ff9720..b802606 100644 --- a/drivers/net/cnxk/cn9k_tx.c +++ b/drivers/net/cnxk/cn9k_tx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts) \ { \ @@ -23,12 +23,13 @@ NIX_TX_FASTPATH_MODES static inline void pick_tx_func(struct rte_eth_dev *eth_dev, - const eth_tx_burst_t tx_burst[2][2][2][2][2]) + const eth_tx_burst_t tx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ + /* [TS] [TSO] [NOFF] [VLAN] [OL3_OL4_CSUM] [IL3_IL4_CSUM] */ eth_dev->tx_pkt_burst = tx_burst + [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSTAMP_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_TSO_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(dev->tx_offload_flags & NIX_TX_OFFLOAD_VLAN_QINQ_F)] @@ -41,25 +42,25 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, + const eth_tx_burst_t nix_eth_tx_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, + const eth_tx_burst_t nix_eth_tx_burst_mseg[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_mseg_##name, NIX_TX_FASTPATH_MODES #undef T }; - const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2] = { -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, + const eth_tx_burst_t nix_eth_tx_vec_burst[2][2][2][2][2][2] = { +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_xmit_pkts_vec_##name, NIX_TX_FASTPATH_MODES #undef T @@ -67,7 +68,8 @@ cn9k_eth_set_tx_function(struct rte_eth_dev *eth_dev) if (dev->scalar_ena || (dev->tx_offload_flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F))) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | + NIX_TX_OFFLOAD_TSO_F))) pick_tx_func(eth_dev, nix_eth_tx_burst); else pick_tx_func(eth_dev, nix_eth_tx_vec_burst); diff --git a/drivers/net/cnxk/cn9k_tx.h b/drivers/net/cnxk/cn9k_tx.h index 7b0d536..1899d66 100644 --- a/drivers/net/cnxk/cn9k_tx.h +++ b/drivers/net/cnxk/cn9k_tx.h @@ -12,6 +12,7 @@ #define NIX_TX_OFFLOAD_VLAN_QINQ_F BIT(2) #define NIX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define NIX_TX_OFFLOAD_TSO_F BIT(4) +#define NIX_TX_OFFLOAD_TSTAMP_F BIT(5) /* Flags to control xmit_prepare function. * Defining it from backwards to denote its been @@ -24,7 +25,8 @@ NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) #define NIX_TX_NEED_EXT_HDR \ - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F) + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSTAMP_F | \ + NIX_TX_OFFLOAD_TSO_F) #define NIX_XMIT_FC_OR_RETURN(txq, pkts) \ do { \ @@ -46,8 +48,12 @@ static __rte_always_inline int cn9k_nix_tx_ext_subs(const uint16_t flags) { - return (flags & - (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) ? 1 : 0; + return (flags & NIX_TX_OFFLOAD_TSTAMP_F) + ? 2 + : ((flags & + (NIX_TX_OFFLOAD_VLAN_QINQ_F | NIX_TX_OFFLOAD_TSO_F)) + ? 1 + : 0); } static __rte_always_inline void @@ -283,6 +289,41 @@ cn9k_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags, } static __rte_always_inline void +cn9k_nix_xmit_prepare_tstamp(uint64_t *cmd, const uint64_t *send_mem_desc, + const uint64_t ol_flags, const uint16_t no_segdw, + const uint16_t flags) +{ + if (flags & NIX_TX_OFFLOAD_TSTAMP_F) { + struct nix_send_mem_s *send_mem; + uint16_t off = (no_segdw - 1) << 1; + const uint8_t is_ol_tstamp = !(ol_flags & PKT_TX_IEEE1588_TMST); + + send_mem = (struct nix_send_mem_s *)(cmd + off); + if (flags & NIX_TX_MULTI_SEG_F) { + /* Retrieving the default desc values */ + cmd[off] = send_mem_desc[6]; + + /* Using compiler barier to avoid voilation of C + * aliasing rules. + */ + rte_compiler_barrier(); + } + + /* Packets for which PKT_TX_IEEE1588_TMST is not set, tx tstamp + * should not be recorded, hence changing the alg type to + * NIX_SENDMEMALG_SET and also changing send mem addr field to + * next 8 bytes as it corrpt the actual tx tstamp registered + * address. + */ + send_mem->w0.cn9k.alg = + NIX_SENDMEMALG_SETTSTMP - (is_ol_tstamp); + + send_mem->addr = (rte_iova_t)((uint64_t *)send_mem_desc[7] + + (is_ol_tstamp)); + } +} + +static __rte_always_inline void cn9k_nix_xmit_one(uint64_t *cmd, void *lmt_addr, const rte_iova_t io_addr, const uint32_t flags) { @@ -380,7 +421,7 @@ cn9k_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) /* Roundup extra dwords to multiple of 2 */ segdw = (segdw >> 1) + (segdw & 0x1); /* Default dwords */ - segdw += (off >> 1) + 1; + segdw += (off >> 1) + 1 + !!(flags & NIX_TX_OFFLOAD_TSTAMP_F); send_hdr->w0.sizem1 = segdw - 1; return segdw; @@ -447,6 +488,8 @@ cn9k_nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, 4, flags); cn9k_nix_xmit_one(cmd, lmt_addr, io_addr, flags); } @@ -488,6 +531,9 @@ cn9k_nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, for (i = 0; i < pkts; i++) { cn9k_nix_xmit_prepare(tx_pkts[i], cmd, flags, lso_tun_fmt); segdw = cn9k_nix_prepare_mseg(tx_pkts[i], cmd, flags); + cn9k_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], + tx_pkts[i]->ol_flags, segdw, + flags); cn9k_nix_xmit_mseg_one(cmd, lmt_addr, io_addr, segdw); } @@ -1241,75 +1287,140 @@ cn9k_nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F #define NOFF_F NIX_TX_OFFLOAD_MBUF_NOFF_F #define TSO_F NIX_TX_OFFLOAD_TSO_F +#define TSP_F NIX_TX_OFFLOAD_TSTAMP_F -/* [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ -#define NIX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 0, 0, 0, 4, \ - NIX_TX_OFFLOAD_NONE) \ -T(l3l4csum, 0, 0, 0, 0, 1, 4, \ - L3L4CSUM_F) \ -T(ol3ol4csum, 0, 0, 0, 1, 0, 4, \ - OL3OL4CSUM_F) \ -T(ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 4, \ - OL3OL4CSUM_F | L3L4CSUM_F) \ -T(vlan, 0, 0, 1, 0, 0, 6, \ - VLAN_F) \ -T(vlan_l3l4csum, 0, 0, 1, 0, 1, 6, \ - VLAN_F | L3L4CSUM_F) \ -T(vlan_ol3ol4csum, 0, 0, 1, 1, 0, 6, \ - VLAN_F | OL3OL4CSUM_F) \ -T(vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 6, \ - VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff, 0, 1, 0, 0, 0, 4, \ - NOFF_F) \ -T(noff_l3l4csum, 0, 1, 0, 0, 1, 4, \ - NOFF_F | L3L4CSUM_F) \ -T(noff_ol3ol4csum, 0, 1, 0, 1, 0, 4, \ - NOFF_F | OL3OL4CSUM_F) \ -T(noff_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 4, \ - NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(noff_vlan, 0, 1, 1, 0, 0, 6, \ - NOFF_F | VLAN_F) \ -T(noff_vlan_l3l4csum, 0, 1, 1, 0, 1, 6, \ - NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(noff_vlan_ol3ol4csum, 0, 1, 1, 1, 0, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 6, \ - NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso, 1, 0, 0, 0, 0, 6, \ - TSO_F) \ -T(tso_l3l4csum, 1, 0, 0, 0, 1, 6, \ - TSO_F | L3L4CSUM_F) \ -T(tso_ol3ol4csum, 1, 0, 0, 1, 0, 6, \ - TSO_F | OL3OL4CSUM_F) \ -T(tso_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 6, \ - TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_vlan, 1, 0, 1, 0, 0, 6, \ - TSO_F | VLAN_F) \ -T(tso_vlan_l3l4csum, 1, 0, 1, 0, 1, 6, \ - TSO_F | VLAN_F | L3L4CSUM_F) \ -T(tso_vlan_ol3ol4csum, 1, 0, 1, 1, 0, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 6, \ - TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff, 1, 1, 0, 0, 0, 6, \ - TSO_F | NOFF_F) \ -T(tso_noff_l3l4csum, 1, 1, 0, 0, 1, 6, \ - TSO_F | NOFF_F | L3L4CSUM_F) \ -T(tso_noff_ol3ol4csum, 1, 1, 0, 1, 0, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F) \ -T(tso_noff_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 6, \ - TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ -T(tso_noff_vlan, 1, 1, 1, 0, 0, 6, \ - TSO_F | NOFF_F | VLAN_F) \ -T(tso_noff_vlan_l3l4csum, 1, 1, 1, 0, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 0, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ -T(tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 6, \ - TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) +/* [TSP] [TSO] [NOFF] [VLAN] [OL3OL4CSUM] [L3L4CSUM] */ +#define NIX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 0, 0, 4, \ + NIX_TX_OFFLOAD_NONE) \ +T(l3l4csum, 0, 0, 0, 0, 0, 1, 4, \ + L3L4CSUM_F) \ +T(ol3ol4csum, 0, 0, 0, 0, 1, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3ol4csum_l3l4csum, 0, 0, 0, 0, 1, 1, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(vlan, 0, 0, 0, 1, 0, 0, 6, \ + VLAN_F) \ +T(vlan_l3l4csum, 0, 0, 0, 1, 0, 1, 6, \ + VLAN_F | L3L4CSUM_F) \ +T(vlan_ol3ol4csum, 0, 0, 0, 1, 1, 0, 6, \ + VLAN_F | OL3OL4CSUM_F) \ +T(vlan_ol3ol4csum_l3l4csum, 0, 0, 0, 1, 1, 1, 6, \ + VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff, 0, 0, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_l3l4csum, 0, 0, 1, 0, 0, 1, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum, 0, 0, 1, 0, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum, 0, 0, 1, 0, 1, 1, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_vlan, 0, 0, 1, 1, 0, 0, 6, \ + NOFF_F | VLAN_F) \ +T(noff_vlan_l3l4csum, 0, 0, 1, 1, 0, 1, 6, \ + NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(noff_vlan_ol3ol4csum, 0, 0, 1, 1, 1, 0, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(noff_vlan_ol3ol4csum_l3l4csum, 0, 0, 1, 1, 1, 1, 6, \ + NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso, 0, 1, 0, 0, 0, 0, 6, \ + TSO_F) \ +T(tso_l3l4csum, 0, 1, 0, 0, 0, 1, 6, \ + TSO_F | L3L4CSUM_F) \ +T(tso_ol3ol4csum, 0, 1, 0, 0, 1, 0, 6, \ + TSO_F | OL3OL4CSUM_F) \ +T(tso_ol3ol4csum_l3l4csum, 0, 1, 0, 0, 1, 1, 6, \ + TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_vlan, 0, 1, 0, 1, 0, 0, 6, \ + TSO_F | VLAN_F) \ +T(tso_vlan_l3l4csum, 0, 1, 0, 1, 0, 1, 6, \ + TSO_F | VLAN_F | L3L4CSUM_F) \ +T(tso_vlan_ol3ol4csum, 0, 1, 0, 1, 1, 0, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_vlan_ol3ol4csum_l3l4csum, 0, 1, 0, 1, 1, 1, 6, \ + TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff, 0, 1, 1, 0, 0, 0, 6, \ + TSO_F | NOFF_F) \ +T(tso_noff_l3l4csum, 0, 1, 1, 0, 0, 1, 6, \ + TSO_F | NOFF_F | L3L4CSUM_F) \ +T(tso_noff_ol3ol4csum, 0, 1, 1, 0, 1, 0, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(tso_noff_ol3ol4csum_l3l4csum, 0, 1, 1, 0, 1, 1, 6, \ + TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(tso_noff_vlan, 0, 1, 1, 1, 0, 0, 6, \ + TSO_F | NOFF_F | VLAN_F) \ +T(tso_noff_vlan_l3l4csum, 0, 1, 1, 1, 0, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum, 0, 1, 1, 1, 1, 0, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(tso_noff_vlan_ol3ol4csum_l3l4csum, 0, 1, 1, 1, 1, 1, 6, \ + TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts, 1, 0, 0, 0, 0, 0, 8, \ + TSP_F) \ +T(ts_l3l4csum, 1, 0, 0, 0, 0, 1, 8, \ + TSP_F | L3L4CSUM_F) \ +T(ts_ol3ol4csum, 1, 0, 0, 0, 1, 0, 8, \ + TSP_F | OL3OL4CSUM_F) \ +T(ts_ol3ol4csum_l3l4csum, 1, 0, 0, 0, 1, 1, 8, \ + TSP_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_vlan, 1, 0, 0, 1, 0, 0, 8, \ + TSP_F | VLAN_F) \ +T(ts_vlan_l3l4csum, 1, 0, 0, 1, 0, 1, 8, \ + TSP_F | VLAN_F | L3L4CSUM_F) \ +T(ts_vlan_ol3ol4csum, 1, 0, 0, 1, 1, 0, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_vlan_ol3ol4csum_l3l4csum, 1, 0, 0, 1, 1, 1, 8, \ + TSP_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff, 1, 0, 1, 0, 0, 0, 8, \ + TSP_F | NOFF_F) \ +T(ts_noff_l3l4csum, 1, 0, 1, 0, 0, 1, 8, \ + TSP_F | NOFF_F | L3L4CSUM_F) \ +T(ts_noff_ol3ol4csum, 1, 0, 1, 0, 1, 0, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_noff_ol3ol4csum_l3l4csum, 1, 0, 1, 0, 1, 1, 8, \ + TSP_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_noff_vlan, 1, 0, 1, 1, 0, 0, 8, \ + TSP_F | NOFF_F | VLAN_F) \ +T(ts_noff_vlan_l3l4csum, 1, 0, 1, 1, 0, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum, 1, 0, 1, 1, 1, 0, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_noff_vlan_ol3ol4csum_l3l4csum, 1, 0, 1, 1, 1, 1, 8, \ + TSP_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso, 1, 1, 0, 0, 0, 0, 8, \ + TSP_F | TSO_F) \ +T(ts_tso_l3l4csum, 1, 1, 0, 0, 0, 1, 8, \ + TSP_F | TSO_F | L3L4CSUM_F) \ +T(ts_tso_ol3ol4csum, 1, 1, 0, 0, 1, 0, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F) \ +T(ts_tso_ol3ol4csum_l3l4csum, 1, 1, 0, 0, 1, 1, 8, \ + TSP_F | TSO_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_vlan, 1, 1, 0, 1, 0, 0, 8, \ + TSP_F | TSO_F | VLAN_F) \ +T(ts_tso_vlan_l3l4csum, 1, 1, 0, 1, 0, 1, 8, \ + TSP_F | TSO_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum, 1, 1, 0, 1, 1, 0, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_vlan_ol3ol4csum_l3l4csum, 1, 1, 0, 1, 1, 1, 8, \ + TSP_F | TSO_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff, 1, 1, 1, 0, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F) \ +T(ts_tso_noff_l3l4csum, 1, 1, 1, 0, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | L3L4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum, 1, 1, 1, 0, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan, 1, 1, 1, 1, 0, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F) \ +T(ts_tso_noff_vlan_l3l4csum, 1, 1, 1, 1, 0, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | L3L4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum, 1, 1, 1, 1, 1, 0, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F) \ +T(ts_tso_noff_vlan_ol3ol4csum_l3l4csum, 1, 1, 1, 1, 1, 1, 8, \ + TSP_F | TSO_F | NOFF_F | VLAN_F | OL3OL4CSUM_F | L3L4CSUM_F) -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_xmit_pkts_##name( \ void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_tx_mseg.c b/drivers/net/cnxk/cn9k_tx_mseg.c index 65c5f36..f3c427c 100644 --- a/drivers/net/cnxk/cn9k_tx_mseg.c +++ b/drivers/net/cnxk/cn9k_tx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_mseg_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ diff --git a/drivers/net/cnxk/cn9k_tx_vec.c b/drivers/net/cnxk/cn9k_tx_vec.c index 21ffc2c..a6e7c9e 100644 --- a/drivers/net/cnxk/cn9k_tx_vec.c +++ b/drivers/net/cnxk/cn9k_tx_vec.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_tx.h" -#define T(name, f4, f3, f2, f1, f0, sz, flags) \ +#define T(name, f5, f4, f3, f2, f1, f0, sz, flags) \ uint16_t __rte_noinline __rte_hot \ cn9k_nix_xmit_pkts_vec_##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, \ @@ -15,6 +15,7 @@ \ /* VLAN, TSTMP, TSO is not supported by vec */ \ if ((flags) & NIX_TX_OFFLOAD_VLAN_QINQ_F || \ + (flags) & NIX_TX_OFFLOAD_TSTAMP_F || \ (flags) & NIX_TX_OFFLOAD_TSO_F) \ return 0; \ return cn9k_nix_xmit_pkts_vector(tx_queue, tx_pkts, pkts, cmd, \ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index 522f7ec..fc91346 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -150,7 +150,8 @@ cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev) offsetof(struct rte_mbuf, data_off) != 6); mb_def.nb_segs = 1; - mb_def.data_off = RTE_PKTMBUF_HEADROOM; + mb_def.data_off = RTE_PKTMBUF_HEADROOM + + (dev->ptp_en * CNXK_NIX_TIMESYNC_RX_OFFSET); mb_def.port = port_id; rte_mbuf_refcnt_set(&mb_def, 1); @@ -356,6 +357,18 @@ cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, eth_dev->data->rx_queues[qid] = rxq_sp + 1; eth_dev->data->rx_queue_state[qid] = RTE_ETH_QUEUE_STATE_STOPPED; + /* Calculating delta and freq mult between PTP HI clock and tsc. + * These are needed in deriving raw clock value from tsc counter. + * read_clock eth op returns raw clock value. + */ + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) { + rc = cnxk_nix_tsc_convert(dev); + if (rc) { + plt_err("Failed to calculate delta and freq mult"); + goto rq_fini; + } + } + return 0; rq_fini: rc |= roc_nix_rq_fini(rq); @@ -1124,7 +1137,7 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); int rc, i; - if (eth_dev->data->nb_rx_queues != 0) { + if (eth_dev->data->nb_rx_queues != 0 && !dev->ptp_en) { rc = nix_recalc_mtu(eth_dev); if (rc) return rc; @@ -1169,6 +1182,25 @@ cnxk_nix_dev_start(struct rte_eth_dev *eth_dev) } } + /* Enable PTP if it is requested by the user or already + * enabled on PF owning this VF + */ + memset(&dev->tstamp, 0, sizeof(struct cnxk_timesync_info)); + if ((dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) || dev->ptp_en) + cnxk_eth_dev_ops.timesync_enable(eth_dev); + else + cnxk_eth_dev_ops.timesync_disable(eth_dev); + + if (dev->rx_offloads & DEV_RX_OFFLOAD_TIMESTAMP) { + rc = rte_mbuf_dyn_rx_timestamp_register + (&dev->tstamp.tstamp_dynfield_offset, + &dev->tstamp.rx_tstamp_dynflag); + if (rc != 0) { + plt_err("Failed to register Rx timestamp field/flag"); + goto rx_disable; + } + } + cnxk_nix_toggle_flag_link_cfg(dev, false); return 0; diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 1c41dcb..de6d533 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -13,6 +13,7 @@ #include <rte_mbuf.h> #include <rte_mbuf_pool_ops.h> #include <rte_mempool.h> +#include <rte_time.h> #include "roc_api.h" @@ -75,7 +76,7 @@ (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ - DEV_RX_OFFLOAD_RSS_HASH) + DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP) #define RSS_IPV4_ENABLE \ (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ @@ -100,7 +101,10 @@ /* Default mark value used when none is provided. */ #define CNXK_FLOW_ACTION_FLAG_DEFAULT 0xffff +/* Default cycle counter mask */ +#define CNXK_CYCLECOUNTER_MASK 0xffffffffffffffffULL #define CNXK_NIX_TIMESYNC_RX_OFFSET 8 + #define PTYPE_NON_TUNNEL_WIDTH 16 #define PTYPE_TUNNEL_WIDTH 12 #define PTYPE_NON_TUNNEL_ARRAY_SZ BIT(PTYPE_NON_TUNNEL_WIDTH) @@ -130,6 +134,16 @@ struct cnxk_eth_qconf { uint8_t valid; }; +struct cnxk_timesync_info { + uint64_t rx_tstamp_dynflag; + rte_iova_t tx_tstamp_iova; + uint64_t *tx_tstamp; + uint64_t rx_tstamp; + int tstamp_dynfield_offset; + uint8_t tx_ready; + uint8_t rx_ready; +} __plt_cache_aligned; + struct cnxk_eth_dev { /* ROC NIX */ struct roc_nix nix; @@ -188,6 +202,14 @@ struct cnxk_eth_dev { /* Flow control configuration */ struct cnxk_fc_cfg fc_cfg; + /* PTP Counters */ + struct cnxk_timesync_info tstamp; + struct rte_timecounter systime_tc; + struct rte_timecounter rx_tstamp_tc; + struct rte_timecounter tx_tstamp_tc; + double clk_freq_mult; + uint64_t clk_delta; + /* Rx burst for cleanup(Only Primary) */ eth_rx_burst_t rx_pkt_burst_no_offload; @@ -288,6 +310,9 @@ int cnxk_nix_rx_queue_setup(struct rte_eth_dev *eth_dev, uint16_t qid, int cnxk_nix_tx_queue_start(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); @@ -404,4 +429,41 @@ cnxk_nix_prefree_seg(struct rte_mbuf *m) return 1; } +static inline rte_mbuf_timestamp_t * +cnxk_nix_timestamp_dynfield(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *info) +{ + return RTE_MBUF_DYNFIELD(mbuf, info->tstamp_dynfield_offset, + rte_mbuf_timestamp_t *); +} + +static __rte_always_inline void +cnxk_nix_mbuf_to_tstamp(struct rte_mbuf *mbuf, + struct cnxk_timesync_info *tstamp, bool ts_enable, + uint64_t *tstamp_ptr) +{ + if (ts_enable && + (mbuf->data_off == + RTE_PKTMBUF_HEADROOM + CNXK_NIX_TIMESYNC_RX_OFFSET)) { + mbuf->pkt_len -= CNXK_NIX_TIMESYNC_RX_OFFSET; + + /* Reading the rx timestamp inserted by CGX, viz at + * starting of the packet data. + */ + *cnxk_nix_timestamp_dynfield(mbuf, tstamp) = + rte_be_to_cpu_64(*tstamp_ptr); + /* PKT_RX_IEEE1588_TMST flag needs to be set only in case + * PTP packets are received. + */ + if (mbuf->packet_type == RTE_PTYPE_L2_ETHER_TIMESYNC) { + tstamp->rx_tstamp = + *cnxk_nix_timestamp_dynfield(mbuf, tstamp); + tstamp->rx_ready = 1; + mbuf->ol_flags |= PKT_RX_IEEE1588_PTP | + PKT_RX_IEEE1588_TMST | + tstamp->rx_tstamp_dynflag; + } + } +} + #endif /* __CNXK_ETHDEV_H__ */ diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c new file mode 100644 index 0000000..fc317965 --- /dev/null +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -0,0 +1,169 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include "cnxk_ethdev.h" + +/* This function calculates two parameters "clk_freq_mult" and + * "clk_delta" which is useful in deriving PTP HI clock from + * timestamp counter (tsc) value. + */ +int +cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) +{ + uint64_t ticks_base = 0, ticks = 0, tsc = 0, t_freq; + struct roc_nix *nix = &dev->nix; + int rc, val; + + /* Calculating the frequency at which PTP HI clock is running */ + rc = roc_nix_ptp_clock_read(nix, &ticks_base, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + rte_delay_ms(100); + + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, false); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + t_freq = (ticks - ticks_base) * 10; + + /* Calculating the freq multiplier viz the ratio between the + * frequency at which PTP HI clock works and tsc clock runs + */ + dev->clk_freq_mult = + (double)pow(10, floor(log10(t_freq))) / rte_get_timer_hz(); + + val = false; +#ifdef RTE_ARM_EAL_RDTSC_USE_PMU + val = true; +#endif + rc = roc_nix_ptp_clock_read(nix, &ticks, &tsc, val); + if (rc) { + plt_err("Failed to read the raw clock value: %d", rc); + goto fail; + } + + /* Calculating delta between PTP HI clock and tsc */ + dev->clk_delta = ((uint64_t)(ticks / dev->clk_freq_mult) - tsc); + +fail: + return rc; +} + +int +cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + struct roc_nix *nix = &dev->nix; + const struct rte_memzone *ts; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be enabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) { + plt_err("PTP cannot be enabled for VF/SDP/LBK"); + return -EINVAL; + } + + if (dev->ptp_en) + return rc; + + if (dev->ptype_disable) { + plt_err("Ptype offload is disabled, it should be enabled"); + return -EINVAL; + } + + if (dev->npc.switch_header_type == ROC_PRIV_FLAGS_HIGIG) { + plt_err("Both PTP and switch header cannot be enabled"); + return -EINVAL; + } + + /* Allocating a iova address for tx tstamp */ + ts = rte_eth_dma_zone_reserve(eth_dev, "cnxk_ts", 0, 128, 128, 0); + if (ts == NULL) { + plt_err("Failed to allocate mem for tx tstamp addr"); + return -ENOMEM; + } + + tstamp->tx_tstamp_iova = ts->iova; + tstamp->tx_tstamp = ts->addr; + + rc = rte_mbuf_dyn_rx_timestamp_register(&tstamp->tstamp_dynfield_offset, + &tstamp->rx_tstamp_dynflag); + if (rc) { + plt_err("Failed to register Rx timestamp field/flag"); + goto error; + } + + /* System time should be already on by default */ + memset(&dev->systime_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->rx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + memset(&dev->tx_tstamp_tc, 0, sizeof(struct rte_timecounter)); + + dev->systime_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->rx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + dev->tx_tstamp_tc.cc_mask = CNXK_CYCLECOUNTER_MASK; + + dev->rx_offloads |= DEV_RX_OFFLOAD_TIMESTAMP; + + rc = roc_nix_ptp_rx_ena_dis(nix, true); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, true); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, false); + goto error; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) { + plt_err("Failed to set MTU size for ptp"); + goto error; + } + + return rc; + +error: + rte_eth_dma_zone_free(eth_dev, "cnxk_ts", 0); + dev->tstamp.tx_tstamp_iova = 0; + dev->tstamp.tx_tstamp = NULL; + return rc; +} + +int +cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t rx_offloads = DEV_RX_OFFLOAD_TIMESTAMP; + struct roc_nix *nix = &dev->nix; + int rc = 0; + + /* If we are VF/SDP/LBK, ptp cannot not be disabled */ + if (roc_nix_is_vf_or_sdp(nix) || roc_nix_is_lbk(nix)) + return -EINVAL; + + if (!dev->ptp_en) + return rc; + + dev->rx_offloads &= ~rx_offloads; + + rc = roc_nix_ptp_rx_ena_dis(nix, false); + if (!rc) { + rc = roc_nix_ptp_tx_ena_dis(nix, false); + if (rc) { + roc_nix_ptp_rx_ena_dis(nix, true); + return rc; + } + } + + rc = nix_recalc_mtu(eth_dev); + if (rc) + plt_err("Failed to set MTU size for ptp"); + + return rc; +} diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build index df953fd..2071d0d 100644 --- a/drivers/net/cnxk/meson.build +++ b/drivers/net/cnxk/meson.build @@ -13,6 +13,7 @@ sources = files('cnxk_ethdev.c', 'cnxk_ethdev_devargs.c', 'cnxk_link.c', 'cnxk_lookup.c', + 'cnxk_ptp.c', 'cnxk_rte_flow.c', 'cnxk_stats.c') -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 56/62] net/cnxk: add timesync enable/disable operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (54 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 55/62] net/cnxk: support base PTP timesync Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram ` (6 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements timesync enable/disable operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cn10k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ drivers/net/cnxk/cn9k_ethdev.c | 50 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/drivers/net/cnxk/cn10k_ethdev.c b/drivers/net/cnxk/cn10k_ethdev.c index 5e0de13..b079edb 100644 --- a/drivers/net/cnxk/cn10k_ethdev.c +++ b/drivers/net/cnxk/cn10k_ethdev.c @@ -363,6 +363,54 @@ cn10k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn10k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn10k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn10k_eth_set_rx_function(eth_dev); + cn10k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn10k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -408,6 +456,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn10k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn10k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn10k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn10k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn10k_nix_timesync_disable; } static void diff --git a/drivers/net/cnxk/cn9k_ethdev.c b/drivers/net/cnxk/cn9k_ethdev.c index 0d63604..994fdb7 100644 --- a/drivers/net/cnxk/cn9k_ethdev.c +++ b/drivers/net/cnxk/cn9k_ethdev.c @@ -371,6 +371,54 @@ cn9k_nix_ptp_info_update_cb(struct roc_nix *nix, bool ptp_en) } static int +cn9k_nix_timesync_enable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_enable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags |= NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags |= NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int +cn9k_nix_timesync_disable(struct rte_eth_dev *eth_dev) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int i, rc; + + rc = cnxk_nix_timesync_disable(eth_dev); + if (rc) + return rc; + + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_TSTAMP_F; + dev->tx_offload_flags &= ~NIX_TX_OFFLOAD_TSTAMP_F; + + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) + nix_form_default_desc(dev, eth_dev->data->tx_queues[i], i); + + /* Setting up the rx[tx]_offload_flags due to change + * in rx[tx]_offloads. + */ + cn9k_eth_set_rx_function(eth_dev); + cn9k_eth_set_tx_function(eth_dev); + return 0; +} + +static int cn9k_nix_dev_start(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); @@ -416,6 +464,8 @@ nix_eth_dev_ops_override(void) cnxk_eth_dev_ops.tx_queue_stop = cn9k_nix_tx_queue_stop; cnxk_eth_dev_ops.dev_start = cn9k_nix_dev_start; cnxk_eth_dev_ops.dev_ptypes_set = cn9k_nix_ptypes_set; + cnxk_eth_dev_ops.timesync_enable = cn9k_nix_timesync_enable; + cnxk_eth_dev_ops.timesync_disable = cn9k_nix_timesync_disable; } static void -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 57/62] net/cnxk: add Rx/Tx timestamp read operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (55 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram ` (5 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements Rx/Tx timestamp read operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 +++++ drivers/net/cnxk/cnxk_ptp.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index fc91346..cb583b4 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1260,6 +1260,8 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .tx_done_cleanup = cnxk_nix_tx_done_cleanup, .flow_ops_get = cnxk_nix_flow_ops_get, .get_reg = cnxk_nix_dev_get_reg, + .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, + .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index de6d533..76df84a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -312,6 +312,11 @@ int cnxk_nix_tx_queue_stop(struct rte_eth_dev *eth_dev, uint16_t qid); int cnxk_nix_dev_start(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev); int cnxk_nix_timesync_disable(struct rte_eth_dev *eth_dev); +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, + uint32_t flags); +int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index fc317965..7b00f87 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,44 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp, uint32_t flags) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + PLT_SET_USED(flags); + + if (!tstamp->rx_ready) + return -EINVAL; + + ns = rte_timecounter_update(&dev->rx_tstamp_tc, tstamp->rx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + tstamp->rx_ready = 0; + return 0; +} + +int +cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, + struct timespec *timestamp) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct cnxk_timesync_info *tstamp = &dev->tstamp; + uint64_t ns; + + if (*tstamp->tx_tstamp == 0) + return -EINVAL; + + ns = rte_timecounter_update(&dev->tx_tstamp_tc, *tstamp->tx_tstamp); + *timestamp = rte_ns_to_timespec(ns); + *tstamp->tx_tstamp = 0; + rte_wmb(); + + return 0; +} + +int cnxk_nix_timesync_enable(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 58/62] net/cnxk: add time read/write/adjust operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (56 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 59/62] net/cnxk: add read clock operation Nithin Dabilpuram ` (4 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements read/write/adjust time operations for cn9k and cn10k platforms. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- drivers/net/cnxk/cnxk_ethdev.c | 3 ++ drivers/net/cnxk/cnxk_ethdev.h | 5 ++++ drivers/net/cnxk/cnxk_ptp.c | 63 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+) diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index cb583b4..c8bbb7a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1262,6 +1262,9 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .get_reg = cnxk_nix_dev_get_reg, .timesync_read_rx_timestamp = cnxk_nix_timesync_read_rx_timestamp, .timesync_read_tx_timestamp = cnxk_nix_timesync_read_tx_timestamp, + .timesync_read_time = cnxk_nix_timesync_read_time, + .timesync_write_time = cnxk_nix_timesync_write_time, + .timesync_adjust_time = cnxk_nix_timesync_adjust_time, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 76df84a..4214365 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -317,6 +317,11 @@ int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, uint32_t flags); int cnxk_nix_timesync_read_tx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp); +int cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, + struct timespec *ts); +int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts); +int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 7b00f87..52f6eb1 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -56,6 +56,69 @@ cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev) } int +cnxk_nix_timesync_read_time(struct rte_eth_dev *eth_dev, struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint64_t clock, ns; + int rc; + + rc = roc_nix_ptp_clock_read(nix, &clock, NULL, false); + if (rc) + return rc; + + ns = rte_timecounter_update(&dev->systime_tc, clock); + *ts = rte_ns_to_timespec(ns); + return 0; +} + +int +cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, + const struct timespec *ts) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint64_t ns; + + ns = rte_timespec_to_ns(ts); + /* Set the time counters to a new value. */ + dev->systime_tc.nsec = ns; + dev->rx_tstamp_tc.nsec = ns; + dev->tx_tstamp_tc.nsec = ns; + + return 0; +} + +int +cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + int rc; + + /* Adjust the frequent to make tics increments in 10^9 tics per sec */ + if (delta < ROC_NIX_PTP_FREQ_ADJUST && + delta > -ROC_NIX_PTP_FREQ_ADJUST) { + rc = roc_nix_ptp_sync_time_adjust(nix, delta); + if (rc) + return rc; + + /* Since the frequency of PTP comp register is tuned, delta and + * freq mult calculation for deriving PTP_HI from timestamp + * counter should be done again. + */ + rc = cnxk_nix_tsc_convert(dev); + if (rc) + plt_err("Failed to calculate delta and freq mult"); + } + + dev->systime_tc.nsec += delta; + dev->rx_tstamp_tc.nsec += delta; + dev->tx_tstamp_tc.nsec += delta; + + return 0; +} + +int cnxk_nix_timesync_read_rx_timestamp(struct rte_eth_dev *eth_dev, struct timespec *timestamp, uint32_t flags) { -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 59/62] net/cnxk: add read clock operation 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (57 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram @ 2021-06-23 4:46 ` Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram ` (3 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:46 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch implements read raw clock operation for cn9k and cn10k. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 ++ drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 1 + drivers/net/cnxk/cnxk_ptp.c | 17 +++++++++++++++++ 4 files changed, 21 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index eba4107..5874531 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -32,6 +32,8 @@ L4 checksum offload = Y Inner L3 checksum = Y Inner L4 checksum = Y Packet type parsing = Y +Timesync = Y +Timestamp offload = Y Basic stats = Y Stats per queue = Y Extended stats = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index c8bbb7a..b2a8f3a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1265,6 +1265,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_read_time = cnxk_nix_timesync_read_time, .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, + .read_clock = cnxk_nix_read_clock, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 4214365..fa6f16f 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -323,6 +323,7 @@ int cnxk_nix_timesync_write_time(struct rte_eth_dev *eth_dev, const struct timespec *ts); int cnxk_nix_timesync_adjust_time(struct rte_eth_dev *eth_dev, int64_t delta); int cnxk_nix_tsc_convert(struct cnxk_eth_dev *dev); +int cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock); uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); diff --git a/drivers/net/cnxk/cnxk_ptp.c b/drivers/net/cnxk/cnxk_ptp.c index 52f6eb1..449489f 100644 --- a/drivers/net/cnxk/cnxk_ptp.c +++ b/drivers/net/cnxk/cnxk_ptp.c @@ -4,6 +4,23 @@ #include "cnxk_ethdev.h" +int +cnxk_nix_read_clock(struct rte_eth_dev *eth_dev, uint64_t *clock) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + /* This API returns the raw PTP HI clock value. Since LFs do not + * have direct access to PTP registers and it requires mbox msg + * to AF for this value. In fastpath reading this value for every + * packet (which involes mbox call) becomes very expensive, hence + * we should be able to derive PTP HI clock value from tsc by + * using freq_mult and clk_delta calculated during configure stage. + */ + *clock = (rte_get_tsc_cycles() + dev->clk_delta) * dev->clk_freq_mult; + + return 0; +} + /* This function calculates two parameters "clk_freq_mult" and * "clk_delta" which is useful in deriving PTP HI clock from * timestamp counter (tsc) value. -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 60/62] net/cnxk: added RETA and RSS hash operations 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (58 preceding siblings ...) 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 59/62] net/cnxk: add read clock operation Nithin Dabilpuram @ 2021-06-23 4:47 ` Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 61/62] net/cnxk: support multicast filter Nithin Dabilpuram ` (2 subsequent siblings) 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:47 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satha Rao <skoteshwar@marvell.com> This patch will implement RETA and RSS hash apis. Also added device argument to lock rx context. Signed-off-by: Satha Rao <skoteshwar@marvell.com> --- doc/guides/nics/features/cnxk.ini | 2 + doc/guides/nics/features/cnxk_vec.ini | 2 + doc/guides/nics/features/cnxk_vf.ini | 2 + drivers/net/cnxk/cnxk_ethdev.c | 4 ++ drivers/net/cnxk/cnxk_ethdev.h | 10 +++ drivers/net/cnxk/cnxk_ethdev_devargs.c | 4 ++ drivers/net/cnxk/cnxk_ethdev_ops.c | 121 +++++++++++++++++++++++++++++++++ 7 files changed, 145 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 5874531..9945af9 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -23,6 +23,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 4871fac..77d63c4 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -22,6 +22,8 @@ Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Flow control = Y Jumbo frame = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 81ee7cc..59acfdb 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -19,6 +19,8 @@ Queue start/stop = Y MTU update = Y TSO = Y RSS hash = Y +RSS key update = Y +RSS reta update = Y Inner RSS = Y Jumbo frame = Y Scattered Rx = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b2a8f3a..abd9bf1 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1266,6 +1266,10 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .timesync_write_time = cnxk_nix_timesync_write_time, .timesync_adjust_time = cnxk_nix_timesync_adjust_time, .read_clock = cnxk_nix_read_clock, + .reta_update = cnxk_nix_reta_update, + .reta_query = cnxk_nix_reta_query, + .rss_hash_update = cnxk_nix_rss_hash_update, + .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, }; static int diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index fa6f16f..cd08e3a 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -330,6 +330,16 @@ uint64_t cnxk_nix_rxq_mbuf_setup(struct cnxk_eth_dev *dev); /* RSS */ uint32_t cnxk_rss_ethdev_to_nix(struct cnxk_eth_dev *dev, uint64_t ethdev_rss, uint8_t rss_level); +int cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size); +int cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); +int cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf); /* Link */ void cnxk_nix_toggle_flag_link_cfg(struct cnxk_eth_dev *dev, bool set); diff --git a/drivers/net/cnxk/cnxk_ethdev_devargs.c b/drivers/net/cnxk/cnxk_ethdev_devargs.c index 7fd06eb..c76b628 100644 --- a/drivers/net/cnxk/cnxk_ethdev_devargs.c +++ b/drivers/net/cnxk/cnxk_ethdev_devargs.c @@ -109,6 +109,7 @@ parse_switch_header_type(const char *key, const char *value, void *extra_args) #define CNXK_FLOW_MAX_PRIORITY "flow_max_priority" #define CNXK_SWITCH_HEADER_TYPE "switch_header" #define CNXK_RSS_TAG_AS_XOR "tag_as_xor" +#define CNXK_LOCK_RX_CTX "lock_rx_ctx" int cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) @@ -120,6 +121,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) uint16_t flow_max_priority = 3; uint16_t rss_tag_as_xor = 0; uint16_t scalar_enable = 0; + uint8_t lock_rx_ctx = 0; struct rte_kvargs *kvlist; if (devargs == NULL) @@ -143,6 +145,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) &parse_switch_header_type, &switch_header_type); rte_kvargs_process(kvlist, CNXK_RSS_TAG_AS_XOR, &parse_flag, &rss_tag_as_xor); + rte_kvargs_process(kvlist, CNXK_LOCK_RX_CTX, &parse_flag, &lock_rx_ctx); rte_kvargs_free(kvlist); null_devargs: @@ -150,6 +153,7 @@ cnxk_ethdev_parse_devargs(struct rte_devargs *devargs, struct cnxk_eth_dev *dev) dev->nix.rss_tag_as_xor = !!rss_tag_as_xor; dev->nix.max_sqb_count = sqb_count; dev->nix.reta_sz = reta_sz; + dev->nix.lock_rx_ctx = lock_rx_ctx; dev->npc.flow_prealloc_size = flow_prealloc_size; dev->npc.flow_max_priority = flow_max_priority; dev->npc.switch_header_type = switch_header_type; diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index 91de6b7..d257763 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -724,3 +724,124 @@ cnxk_nix_dev_get_reg(struct rte_eth_dev *eth_dev, struct rte_dev_reg_info *regs) return rc; } + +int +cnxk_nix_reta_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int i, j, rc = -EINVAL, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta[idx] = reta_conf[i].reta[j]; + idx++; + } + } + + return roc_nix_rss_reta_set(nix, 0, reta); + +fail: + return rc; +} + +int +cnxk_nix_reta_query(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_reta_entry64 *reta_conf, + uint16_t reta_size) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + uint16_t reta[ROC_NIX_RSS_RETA_MAX]; + struct roc_nix *nix = &dev->nix; + int rc = -EINVAL, i, j, idx = 0; + + if (reta_size != dev->nix.reta_sz) { + plt_err("Size of hash lookup table configured (%d) does not " + "match the number hardware can supported (%d)", + reta_size, dev->nix.reta_sz); + goto fail; + } + + rc = roc_nix_rss_reta_get(nix, 0, reta); + if (rc) + goto fail; + + /* Copy RETA table */ + for (i = 0; i < (int)(dev->nix.reta_sz / RTE_RETA_GROUP_SIZE); i++) { + for (j = 0; j < RTE_RETA_GROUP_SIZE; j++) { + if ((reta_conf[i].mask >> j) & 0x01) + reta_conf[i].reta[j] = reta[idx]; + idx++; + } + } + + return 0; + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_update(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct roc_nix *nix = &dev->nix; + uint8_t rss_hash_level; + uint32_t flowkey_cfg; + int rc = -EINVAL; + uint8_t alg_idx; + + if (rss_conf->rss_key && rss_conf->rss_key_len != ROC_NIX_RSS_KEY_LEN) { + plt_err("Hash key size mismatch %d vs %d", + rss_conf->rss_key_len, ROC_NIX_RSS_KEY_LEN); + goto fail; + } + + if (rss_conf->rss_key) + roc_nix_rss_key_set(nix, rss_conf->rss_key); + + rss_hash_level = ETH_RSS_LEVEL(rss_conf->rss_hf); + if (rss_hash_level) + rss_hash_level -= 1; + flowkey_cfg = + cnxk_rss_ethdev_to_nix(dev, rss_conf->rss_hf, rss_hash_level); + + rc = roc_nix_rss_flowkey_set(nix, &alg_idx, flowkey_cfg, + ROC_NIX_RSS_GROUP_DEFAULT, + ROC_NIX_RSS_MCAM_IDX_DEFAULT); + if (rc) { + plt_err("Failed to set RSS hash function rc=%d", rc); + return rc; + } + +fail: + return rc; +} + +int +cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, + struct rte_eth_rss_conf *rss_conf) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + + if (rss_conf->rss_key) + roc_nix_rss_key_get(&dev->nix, rss_conf->rss_key); + + rss_conf->rss_key_len = ROC_NIX_RSS_KEY_LEN; + rss_conf->rss_hf = dev->ethdev_rss_hf; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 61/62] net/cnxk: support multicast filter 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (59 preceding siblings ...) 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram @ 2021-06-23 4:47 ` Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 62/62] net/cnxk: support marking and VLAN tagging Nithin Dabilpuram 2021-06-25 14:01 ` [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:47 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Sunil Kumar Kori <skori@marvell.com> Patch adds multicast filter support for cn9k and cn10k platforms. CGX DMAC filter table(32 entries) is divided among all LMACs connected to it i.e. if CGX has 4 LMACs then each LMAC can have up to 8 filters. If CGX has 1 LMAC then it can have up to 32 filters. Above mentioned filter table is used to install unicast and multicast DMAC address filters. Unicast filters are installed via rte_eth_dev_mac_addr_add API while multicast filters are installed via rte_eth_dev_set_mc_addr_list API. So in total, supported MAC filters are equal to DMAC filters plus mcast filters. Signed-off-by: Sunil Kumar Kori <skori@marvell.com> --- doc/guides/nics/features/cnxk.ini | 1 + doc/guides/nics/features/cnxk_vec.ini | 1 + drivers/net/cnxk/cnxk_ethdev.c | 2 ++ drivers/net/cnxk/cnxk_ethdev.h | 4 +++ drivers/net/cnxk/cnxk_ethdev_ops.c | 65 +++++++++++++++++++++++++++++++++++ 5 files changed, 73 insertions(+) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index 9945af9..c686ceb 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -22,6 +22,7 @@ TSO = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 77d63c4..018d09e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -21,6 +21,7 @@ MTU update = Y Promiscuous mode = Y Allmulticast mode = Y Unicast MAC filter = Y +Multicast MAC filter = Y RSS hash = Y RSS key update = Y RSS reta update = Y diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index abd9bf1..b7872cc 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1270,6 +1270,7 @@ struct eth_dev_ops cnxk_eth_dev_ops = { .reta_query = cnxk_nix_reta_query, .rss_hash_update = cnxk_nix_rss_hash_update, .rss_hash_conf_get = cnxk_nix_rss_hash_conf_get, + .set_mc_addr_list = cnxk_nix_mc_addr_list_configure, }; static int @@ -1334,6 +1335,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) } dev->max_mac_entries = max_entries; + dev->dmac_filter_count = 1; /* Get mac address */ rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index cd08e3a..c07e988 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -162,6 +162,7 @@ struct cnxk_eth_dev { uint8_t configured; /* Max macfilter entries */ + uint8_t dmac_filter_count; uint8_t max_mac_entries; bool dmac_filter_enable; @@ -265,6 +266,9 @@ int cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev); int cnxk_nix_remove(struct rte_pci_device *pci_dev); int cnxk_nix_mtu_set(struct rte_eth_dev *eth_dev, uint16_t mtu); +int cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr); int cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, uint32_t index, uint32_t pool); diff --git a/drivers/net/cnxk/cnxk_ethdev_ops.c b/drivers/net/cnxk/cnxk_ethdev_ops.c index d257763..b6cc528 100644 --- a/drivers/net/cnxk/cnxk_ethdev_ops.c +++ b/drivers/net/cnxk/cnxk_ethdev_ops.c @@ -359,6 +359,7 @@ cnxk_nix_mac_addr_add(struct rte_eth_dev *eth_dev, struct rte_ether_addr *addr, roc_nix_npc_promisc_ena_dis(nix, true); dev->dmac_filter_enable = true; eth_dev->data->promiscuous = false; + dev->dmac_filter_count++; return 0; } @@ -373,6 +374,8 @@ cnxk_nix_mac_addr_del(struct rte_eth_dev *eth_dev, uint32_t index) rc = roc_nix_mac_addr_del(nix, index); if (rc) plt_err("Failed to delete mac address, rc=%d", rc); + + dev->dmac_filter_count--; } int @@ -845,3 +848,65 @@ cnxk_nix_rss_hash_conf_get(struct rte_eth_dev *eth_dev, return 0; } + +int +cnxk_nix_mc_addr_list_configure(struct rte_eth_dev *eth_dev, + struct rte_ether_addr *mc_addr_set, + uint32_t nb_mc_addr) +{ + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_ether_addr null_mac_addr; + struct roc_nix *nix = &dev->nix; + int rc, index; + uint32_t i; + + memset(&null_mac_addr, 0, sizeof(null_mac_addr)); + + /* All configured multicast filters should be flushed first */ + for (i = 0; i < dev->max_mac_entries; i++) { + if (rte_is_multicast_ether_addr(&data->mac_addrs[i])) { + rc = roc_nix_mac_addr_del(nix, i); + if (rc) { + plt_err("Failed to flush mcast address, rc=%d", + rc); + return rc; + } + + dev->dmac_filter_count--; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&null_mac_addr, + &data->mac_addrs[i]); + } + } + + if (!mc_addr_set || !nb_mc_addr) + return 0; + + /* Check for available space */ + if (nb_mc_addr > + ((uint32_t)(dev->max_mac_entries - dev->dmac_filter_count))) { + plt_err("No space is available to add multicast filters"); + return -ENOSPC; + } + + /* Multicast addresses are to be installed */ + for (i = 0; i < nb_mc_addr; i++) { + index = roc_nix_mac_addr_add(nix, mc_addr_set[i].addr_bytes); + if (index < 0) { + plt_err("Failed to add mcast mac address, rc=%d", + index); + return index; + } + + dev->dmac_filter_count++; + /* Update address in NIC data structure */ + rte_ether_addr_copy(&mc_addr_set[i], &data->mac_addrs[index]); + } + + roc_nix_npc_promisc_ena_dis(nix, true); + dev->dmac_filter_enable = true; + eth_dev->data->promiscuous = false; + + return 0; +} -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* [dpdk-dev] [PATCH v4 62/62] net/cnxk: support marking and VLAN tagging 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (60 preceding siblings ...) 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 61/62] net/cnxk: support multicast filter Nithin Dabilpuram @ 2021-06-23 4:47 ` Nithin Dabilpuram 2021-06-25 14:01 ` [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 62 siblings, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-06-23 4:47 UTC (permalink / raw) To: dev Cc: jerinj, skori, skoteshwar, pbhagavatula, kirankumark, psatheesh, asekhar, hkalra From: Satheesh Paul <psatheesh@marvell.com> This patch adds support for mark, flag, VLAN pop and push flow actions. Signed-off-by: Satheesh Paul <psatheesh@marvell.com> --- doc/guides/nics/features/cnxk.ini | 5 + doc/guides/nics/features/cnxk_vec.ini | 5 + doc/guides/nics/features/cnxk_vf.ini | 5 + drivers/net/cnxk/cn10k_rte_flow.c | 42 ++++++++ drivers/net/cnxk/cn10k_rx.c | 27 +++--- drivers/net/cnxk/cn10k_rx.h | 175 +++++++++++++++++++++++++++------- drivers/net/cnxk/cn10k_rx_mseg.c | 2 +- drivers/net/cnxk/cn10k_rx_vec.c | 2 +- drivers/net/cnxk/cn9k_rte_flow.c | 42 ++++++++ drivers/net/cnxk/cn9k_rx.c | 27 +++--- drivers/net/cnxk/cn9k_rx.h | 175 +++++++++++++++++++++++++++------- drivers/net/cnxk/cn9k_rx_mseg.c | 2 +- drivers/net/cnxk/cn9k_rx_vec.c | 2 +- drivers/net/cnxk/cnxk_ethdev.c | 1 + drivers/net/cnxk/cnxk_ethdev.h | 3 +- drivers/net/cnxk/cnxk_rte_flow.c | 17 ++++ 16 files changed, 433 insertions(+), 99 deletions(-) diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini index c686ceb..337ad24 100644 --- a/doc/guides/nics/features/cnxk.ini +++ b/doc/guides/nics/features/cnxk.ini @@ -77,6 +77,11 @@ vxlan_gpe = Y count = Y drop = Y flag = Y +mark = Y +of_pop_vlan = Y +of_push_vlan = Y +of_set_vlan_pcp = Y +of_set_vlan_vid = Y pf = Y port_id = Y queue = Y diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini index 018d09e..d55b61e 100644 --- a/doc/guides/nics/features/cnxk_vec.ini +++ b/doc/guides/nics/features/cnxk_vec.ini @@ -73,6 +73,11 @@ vxlan_gpe = Y count = Y drop = Y flag = Y +mark = Y +of_pop_vlan = Y +of_push_vlan = Y +of_set_vlan_pcp = Y +of_set_vlan_vid = Y pf = Y port_id = Y queue = Y diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini index 59acfdb..1bea00c 100644 --- a/doc/guides/nics/features/cnxk_vf.ini +++ b/doc/guides/nics/features/cnxk_vf.ini @@ -69,6 +69,11 @@ vxlan_gpe = Y count = Y drop = Y flag = Y +mark = Y +of_pop_vlan = Y +of_push_vlan = Y +of_set_vlan_pcp = Y +of_set_vlan_vid = Y pf = Y port_id = Y queue = Y diff --git a/drivers/net/cnxk/cn10k_rte_flow.c b/drivers/net/cnxk/cn10k_rte_flow.c index 65893cc..b04de6a 100644 --- a/drivers/net/cnxk/cn10k_rte_flow.c +++ b/drivers/net/cnxk/cn10k_rte_flow.c @@ -4,6 +4,7 @@ #include <cnxk_rte_flow.h> #include "cn10k_rte_flow.h" #include "cn10k_ethdev.h" +#include "cn10k_rx.h" struct rte_flow * cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, @@ -11,12 +12,29 @@ cn10k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], struct rte_flow_error *error) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; struct roc_npc_flow *flow; flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); if (!flow) return NULL; + mark_actions = roc_npc_mark_actions_get(npc); + + if (mark_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn10k_eth_set_rx_function(eth_dev); + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + + if (vtag_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn10k_eth_set_rx_function(eth_dev); + } + return (struct rte_flow *)flow; } @@ -25,6 +43,30 @@ cn10k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, struct rte_flow_error *error) { struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; + + mark_actions = roc_npc_mark_actions_get(npc); + if (mark_actions) { + mark_actions = roc_npc_mark_actions_sub_return(npc, 1); + if (mark_actions == 0) { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn10k_eth_set_rx_function(eth_dev); + } + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + if (vtag_actions) { + if (flow->nix_intf == ROC_NPC_INTF_RX) { + vtag_actions = roc_npc_vtag_actions_sub_return(npc, 1); + if (vtag_actions == 0) { + dev->rx_offload_flags &= + ~NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn10k_eth_set_rx_function(eth_dev); + } + } + } return cnxk_flow_destroy(eth_dev, flow, error); } diff --git a/drivers/net/cnxk/cn10k_rx.c b/drivers/net/cnxk/cn10k_rx.c index c9744e2..5c956c0 100644 --- a/drivers/net/cnxk/cn10k_rx.c +++ b/drivers/net/cnxk/cn10k_rx.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [VLAN] [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] @@ -35,25 +36,25 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn10k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R @@ -73,6 +74,6 @@ cn10k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn10k_rx.h b/drivers/net/cnxk/cn10k_rx.h index c09ccdf..1cc37cb 100644 --- a/drivers/net/cnxk/cn10k_rx.h +++ b/drivers/net/cnxk/cn10k_rx.h @@ -15,6 +15,7 @@ #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) +#define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(5) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -179,6 +180,17 @@ cn10k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + if (flag & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + if (rx->vtag0_gone) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + mbuf->vlan_tci = rx->vtag0_tci; + } + if (rx->vtag1_gone) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = rx->vtag1_tci; + } + } + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->match_id, ol_flags, mbuf); @@ -273,6 +285,28 @@ cn10k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, #if defined(RTE_ARCH_ARM64) +static __rte_always_inline uint64_t +nix_vlan_update(const uint64_t w2, uint64_t ol_flags, uint8x16_t *f) +{ + if (w2 & BIT_ULL(21) /* vtag0_gone */) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + *f = vsetq_lane_u16((uint16_t)(w2 >> 32), *f, 5); + } + + return ol_flags; +} + +static __rte_always_inline uint64_t +nix_qinq_update(const uint64_t w2, uint64_t ol_flags, struct rte_mbuf *mbuf) +{ + if (w2 & BIT_ULL(23) /* vtag1_gone */) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = (uint16_t)(w2 >> 48); + } + + return ol_flags; +} + static __rte_always_inline uint16_t cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, const uint16_t flags) @@ -397,6 +431,23 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); } + if (flags & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + uint64_t cq0_w2 = *(uint64_t *)(cq0 + CQE_SZ(0) + 16); + uint64_t cq1_w2 = *(uint64_t *)(cq0 + CQE_SZ(1) + 16); + uint64_t cq2_w2 = *(uint64_t *)(cq0 + CQE_SZ(2) + 16); + uint64_t cq3_w2 = *(uint64_t *)(cq0 + CQE_SZ(3) + 16); + + ol_flags0 = nix_vlan_update(cq0_w2, ol_flags0, &f0); + ol_flags1 = nix_vlan_update(cq1_w2, ol_flags1, &f1); + ol_flags2 = nix_vlan_update(cq2_w2, ol_flags2, &f2); + ol_flags3 = nix_vlan_update(cq3_w2, ol_flags3, &f3); + + ol_flags0 = nix_qinq_update(cq0_w2, ol_flags0, mbuf0); + ol_flags1 = nix_qinq_update(cq1_w2, ol_flags1, mbuf1); + ol_flags2 = nix_qinq_update(cq2_w2, ol_flags2, mbuf2); + ol_flags3 = nix_qinq_update(cq3_w2, ol_flags3, mbuf3); + } + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { ol_flags0 = nix_update_match_id( *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, @@ -494,43 +545,99 @@ cn10k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F -/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +/* [RX_VLAN_F] [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ -R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts, 1, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ -R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ -R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ -R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ -R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ -R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ -R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ -R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ -R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) +R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan, 1, 0, 0, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 1, 0, 0, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 1, 0, 0, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 1, 0, 0, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 1, 0, 0, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 1, 0, 0, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 1, 0, 0, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_mark, 1, 0, 1, 0, 0, 0, RX_VLAN_F | MARK_F) \ +R(vlan_mark_rss, 1, 0, 1, 0, 0, 1, RX_VLAN_F | MARK_F | RSS_F) \ +R(vlan_mark_ptype, 1, 0, 1, 0, 1, 0, RX_VLAN_F | MARK_F | PTYPE_F)\ +R(vlan_mark_ptype_rss, 1, 0, 1, 0, 1, 1, \ + RX_VLAN_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_mark_cksum, 1, 0, 1, 1, 0, 0, RX_VLAN_F | MARK_F | CKSUM_F)\ +R(vlan_mark_cksum_rss, 1, 0, 1, 1, 0, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_mark_cksum_ptype, 1, 0, 1, 1, 1, 0, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_mark_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts, 1, 1, 0, 0, 0, 0, RX_VLAN_F | TS_F) \ +R(vlan_ts_rss, 1, 1, 0, 0, 0, 1, RX_VLAN_F | TS_F | RSS_F) \ +R(vlan_ts_ptype, 1, 1, 0, 0, 1, 0, RX_VLAN_F | TS_F | PTYPE_F) \ +R(vlan_ts_ptype_rss, 1, 1, 0, 0, 1, 1, \ + RX_VLAN_F | TS_F | PTYPE_F | RSS_F) \ +R(vlan_ts_cksum, 1, 1, 0, 1, 0, 0, RX_VLAN_F | TS_F | CKSUM_F) \ +R(vlan_ts_cksum_rss, 1, 1, 0, 1, 0, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | RSS_F) \ +R(vlan_ts_cksum_ptype, 1, 1, 0, 1, 1, 0, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark, 1, 1, 1, 0, 0, 0, RX_VLAN_F | TS_F | MARK_F) \ +R(vlan_ts_mark_rss, 1, 1, 1, 0, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | RSS_F) \ +R(vlan_ts_mark_ptype, 1, 1, 1, 0, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F) \ +R(vlan_ts_mark_ptype_rss, 1, 1, 1, 0, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark_cksum, 1, 1, 1, 1, 0, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F) \ +R(vlan_ts_mark_cksum_rss, 1, 1, 1, 1, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_ts_mark_cksum_ptype, 1, 1, 1, 1, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn10k_rx_mseg.c b/drivers/net/cnxk/cn10k_rx_mseg.c index b67d21f..3340771 100644 --- a/drivers/net/cnxk/cn10k_rx_mseg.c +++ b/drivers/net/cnxk/cn10k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn10k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn10k_rx_vec.c b/drivers/net/cnxk/cn10k_rx_vec.c index 1330235..65ffa97 100644 --- a/drivers/net/cnxk/cn10k_rx_vec.c +++ b/drivers/net/cnxk/cn10k_rx_vec.c @@ -5,7 +5,7 @@ #include "cn10k_ethdev.h" #include "cn10k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot \ cn10k_nix_recv_pkts_vec_##name(void *rx_queue, \ struct rte_mbuf **rx_pkts, \ diff --git a/drivers/net/cnxk/cn9k_rte_flow.c b/drivers/net/cnxk/cn9k_rte_flow.c index 24e1b92..b94d29e 100644 --- a/drivers/net/cnxk/cn9k_rte_flow.c +++ b/drivers/net/cnxk/cn9k_rte_flow.c @@ -4,6 +4,7 @@ #include <cnxk_rte_flow.h> #include "cn9k_ethdev.h" #include "cn9k_rte_flow.h" +#include "cn9k_rx.h" struct rte_flow * cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, @@ -11,12 +12,29 @@ cn9k_flow_create(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, const struct rte_flow_action actions[], struct rte_flow_error *error) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; struct roc_npc_flow *flow; flow = cnxk_flow_create(eth_dev, attr, pattern, actions, error); if (!flow) return NULL; + mark_actions = roc_npc_mark_actions_get(npc); + + if (mark_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn9k_eth_set_rx_function(eth_dev); + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + + if (vtag_actions) { + dev->rx_offload_flags |= NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn9k_eth_set_rx_function(eth_dev); + } + return (struct rte_flow *)flow; } @@ -25,6 +43,30 @@ cn9k_flow_destroy(struct rte_eth_dev *eth_dev, struct rte_flow *rte_flow, struct rte_flow_error *error) { struct roc_npc_flow *flow = (struct roc_npc_flow *)rte_flow; + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + int mark_actions = 0, vtag_actions = 0; + struct roc_npc *npc = &dev->npc; + + mark_actions = roc_npc_mark_actions_get(npc); + if (mark_actions) { + mark_actions = roc_npc_mark_actions_sub_return(npc, 1); + if (mark_actions == 0) { + dev->rx_offload_flags &= ~NIX_RX_OFFLOAD_MARK_UPDATE_F; + cn9k_eth_set_rx_function(eth_dev); + } + } + + vtag_actions = roc_npc_vtag_actions_get(npc); + if (vtag_actions) { + if (flow->nix_intf == ROC_NPC_INTF_RX) { + vtag_actions = roc_npc_vtag_actions_sub_return(npc, 1); + if (vtag_actions == 0) { + dev->rx_offload_flags &= + ~NIX_RX_OFFLOAD_VLAN_STRIP_F; + cn9k_eth_set_rx_function(eth_dev); + } + } + } return cnxk_flow_destroy(eth_dev, flow, error); } diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index a15428d..0acedd0 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ @@ -17,12 +17,13 @@ NIX_RX_FASTPATH_MODES static inline void pick_rx_func(struct rte_eth_dev *eth_dev, - const eth_rx_burst_t rx_burst[2][2][2][2][2]) + const eth_rx_burst_t rx_burst[2][2][2][2][2][2]) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - /* [TSP] [MARK] [CKSUM] [PTYPE] [RSS] */ + /* [TSP] [MARK] [VLAN] [CKSUM] [PTYPE] [RSS] */ eth_dev->rx_pkt_burst = rx_burst + [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_VLAN_STRIP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_TSTAMP_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_MARK_UPDATE_F)] [!!(dev->rx_offload_flags & NIX_RX_OFFLOAD_CHECKSUM_F)] @@ -35,25 +36,25 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); - const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, NIX_RX_FASTPATH_MODES #undef R }; - const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2] = { -#define R(name, f4, f3, f2, f1, f0, flags) \ - [f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, + const eth_rx_burst_t nix_eth_rx_vec_burst[2][2][2][2][2][2] = { +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ + [f5][f4][f3][f2][f1][f0] = cn9k_nix_recv_pkts_vec_##name, NIX_RX_FASTPATH_MODES #undef R @@ -73,6 +74,6 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) /* Copy multi seg version with no offload for tear down sequence */ if (rte_eal_process_type() == RTE_PROC_PRIMARY) dev->rx_pkt_burst_no_offload = - nix_eth_rx_burst_mseg[0][0][0][0][0]; + nix_eth_rx_burst_mseg[0][0][0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index c5ad5db..10ef5c6 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -16,6 +16,7 @@ #define NIX_RX_OFFLOAD_CHECKSUM_F BIT(2) #define NIX_RX_OFFLOAD_MARK_UPDATE_F BIT(3) #define NIX_RX_OFFLOAD_TSTAMP_F BIT(4) +#define NIX_RX_OFFLOAD_VLAN_STRIP_F BIT(5) /* Flags to control cqe_to_mbuf conversion function. * Defining it from backwards to denote its been @@ -181,6 +182,17 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, if (flag & NIX_RX_OFFLOAD_CHECKSUM_F) ol_flags |= nix_rx_olflags_get(lookup_mem, w1); + if (flag & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + if (rx->cn9k.vtag0_gone) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + mbuf->vlan_tci = rx->cn9k.vtag0_tci; + } + if (rx->cn9k.vtag1_gone) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = rx->cn9k.vtag1_tci; + } + } + if (flag & NIX_RX_OFFLOAD_MARK_UPDATE_F) ol_flags = nix_update_match_id(rx->cn9k.match_id, ol_flags, mbuf); @@ -276,6 +288,28 @@ cn9k_nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, #if defined(RTE_ARCH_ARM64) +static __rte_always_inline uint64_t +nix_vlan_update(const uint64_t w2, uint64_t ol_flags, uint8x16_t *f) +{ + if (w2 & BIT_ULL(21) /* vtag0_gone */) { + ol_flags |= PKT_RX_VLAN | PKT_RX_VLAN_STRIPPED; + *f = vsetq_lane_u16((uint16_t)(w2 >> 32), *f, 5); + } + + return ol_flags; +} + +static __rte_always_inline uint64_t +nix_qinq_update(const uint64_t w2, uint64_t ol_flags, struct rte_mbuf *mbuf) +{ + if (w2 & BIT_ULL(23) /* vtag1_gone */) { + ol_flags |= PKT_RX_QINQ | PKT_RX_QINQ_STRIPPED; + mbuf->vlan_tci_outer = (uint16_t)(w2 >> 48); + } + + return ol_flags; +} + static __rte_always_inline uint16_t cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, const uint16_t flags) @@ -400,6 +434,23 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, ol_flags3 |= nix_rx_olflags_get(lookup_mem, cq3_w1); } + if (flags & NIX_RX_OFFLOAD_VLAN_STRIP_F) { + uint64_t cq0_w2 = *(uint64_t *)(cq0 + CQE_SZ(0) + 16); + uint64_t cq1_w2 = *(uint64_t *)(cq0 + CQE_SZ(1) + 16); + uint64_t cq2_w2 = *(uint64_t *)(cq0 + CQE_SZ(2) + 16); + uint64_t cq3_w2 = *(uint64_t *)(cq0 + CQE_SZ(3) + 16); + + ol_flags0 = nix_vlan_update(cq0_w2, ol_flags0, &f0); + ol_flags1 = nix_vlan_update(cq1_w2, ol_flags1, &f1); + ol_flags2 = nix_vlan_update(cq2_w2, ol_flags2, &f2); + ol_flags3 = nix_vlan_update(cq3_w2, ol_flags3, &f3); + + ol_flags0 = nix_qinq_update(cq0_w2, ol_flags0, mbuf0); + ol_flags1 = nix_qinq_update(cq1_w2, ol_flags1, mbuf1); + ol_flags2 = nix_qinq_update(cq2_w2, ol_flags2, mbuf2); + ol_flags3 = nix_qinq_update(cq3_w2, ol_flags3, mbuf3); + } + if (flags & NIX_RX_OFFLOAD_MARK_UPDATE_F) { ol_flags0 = nix_update_match_id( *(uint16_t *)(cq0 + CQE_SZ(0) + 38), ol_flags0, @@ -496,43 +547,99 @@ cn9k_nix_recv_pkts_vector(void *rx_queue, struct rte_mbuf **rx_pkts, #define CKSUM_F NIX_RX_OFFLOAD_CHECKSUM_F #define MARK_F NIX_RX_OFFLOAD_MARK_UPDATE_F #define TS_F NIX_RX_OFFLOAD_TSTAMP_F +#define RX_VLAN_F NIX_RX_OFFLOAD_VLAN_STRIP_F -/* [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ +/* [RX_VLAN_F] [TS] [MARK] [CKSUM] [PTYPE] [RSS] */ #define NIX_RX_FASTPATH_MODES \ -R(no_offload, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ -R(rss, 0, 0, 0, 0, 1, RSS_F) \ -R(ptype, 0, 0, 0, 1, 0, PTYPE_F) \ -R(ptype_rss, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ -R(cksum, 0, 0, 1, 0, 0, CKSUM_F) \ -R(cksum_rss, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ -R(cksum_ptype, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ -R(cksum_ptype_rss, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ -R(mark, 0, 1, 0, 0, 0, MARK_F) \ -R(mark_rss, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ -R(mark_ptype, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ -R(mark_ptype_rss, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ -R(mark_cksum, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ -R(mark_cksum_rss, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ -R(mark_cksum_ptype, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ -R(mark_cksum_ptype_rss, 0, 1, 1, 1, 1, MARK_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts, 1, 0, 0, 0, 0, TS_F) \ -R(ts_rss, 1, 0, 0, 0, 1, TS_F | RSS_F) \ -R(ts_ptype, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ -R(ts_ptype_rss, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ -R(ts_cksum, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ -R(ts_cksum_rss, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ -R(ts_cksum_ptype, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ -R(ts_cksum_ptype_rss, 1, 0, 1, 1, 1, TS_F | CKSUM_F | PTYPE_F | RSS_F)\ -R(ts_mark, 1, 1, 0, 0, 0, TS_F | MARK_F) \ -R(ts_mark_rss, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ -R(ts_mark_ptype, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ -R(ts_mark_ptype_rss, 1, 1, 0, 1, 1, TS_F | MARK_F | PTYPE_F | RSS_F)\ -R(ts_mark_cksum, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ -R(ts_mark_cksum_rss, 1, 1, 1, 0, 1, TS_F | MARK_F | CKSUM_F | RSS_F)\ -R(ts_mark_cksum_ptype, 1, 1, 1, 1, 0, TS_F | MARK_F | CKSUM_F | PTYPE_F)\ -R(ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) +R(no_offload, 0, 0, 0, 0, 0, 0, NIX_RX_OFFLOAD_NONE) \ +R(rss, 0, 0, 0, 0, 0, 1, RSS_F) \ +R(ptype, 0, 0, 0, 0, 1, 0, PTYPE_F) \ +R(ptype_rss, 0, 0, 0, 0, 1, 1, PTYPE_F | RSS_F) \ +R(cksum, 0, 0, 0, 1, 0, 0, CKSUM_F) \ +R(cksum_rss, 0, 0, 0, 1, 0, 1, CKSUM_F | RSS_F) \ +R(cksum_ptype, 0, 0, 0, 1, 1, 0, CKSUM_F | PTYPE_F) \ +R(cksum_ptype_rss, 0, 0, 0, 1, 1, 1, CKSUM_F | PTYPE_F | RSS_F) \ +R(mark, 0, 0, 1, 0, 0, 0, MARK_F) \ +R(mark_rss, 0, 0, 1, 0, 0, 1, MARK_F | RSS_F) \ +R(mark_ptype, 0, 0, 1, 0, 1, 0, MARK_F | PTYPE_F) \ +R(mark_ptype_rss, 0, 0, 1, 0, 1, 1, MARK_F | PTYPE_F | RSS_F) \ +R(mark_cksum, 0, 0, 1, 1, 0, 0, MARK_F | CKSUM_F) \ +R(mark_cksum_rss, 0, 0, 1, 1, 0, 1, MARK_F | CKSUM_F | RSS_F) \ +R(mark_cksum_ptype, 0, 0, 1, 1, 1, 0, MARK_F | CKSUM_F | PTYPE_F) \ +R(mark_cksum_ptype_rss, 0, 0, 1, 1, 1, 1, \ + MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts, 0, 1, 0, 0, 0, 0, TS_F) \ +R(ts_rss, 0, 1, 0, 0, 0, 1, TS_F | RSS_F) \ +R(ts_ptype, 0, 1, 0, 0, 1, 0, TS_F | PTYPE_F) \ +R(ts_ptype_rss, 0, 1, 0, 0, 1, 1, TS_F | PTYPE_F | RSS_F) \ +R(ts_cksum, 0, 1, 0, 1, 0, 0, TS_F | CKSUM_F) \ +R(ts_cksum_rss, 0, 1, 0, 1, 0, 1, TS_F | CKSUM_F | RSS_F) \ +R(ts_cksum_ptype, 0, 1, 0, 1, 1, 0, TS_F | CKSUM_F | PTYPE_F) \ +R(ts_cksum_ptype_rss, 0, 1, 0, 1, 1, 1, \ + TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(ts_mark, 0, 1, 1, 0, 0, 0, TS_F | MARK_F) \ +R(ts_mark_rss, 0, 1, 1, 0, 0, 1, TS_F | MARK_F | RSS_F) \ +R(ts_mark_ptype, 0, 1, 1, 0, 1, 0, TS_F | MARK_F | PTYPE_F) \ +R(ts_mark_ptype_rss, 0, 1, 1, 0, 1, 1, \ + TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(ts_mark_cksum, 0, 1, 1, 1, 0, 0, TS_F | MARK_F | CKSUM_F) \ +R(ts_mark_cksum_rss, 0, 1, 1, 1, 0, 1, \ + TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(ts_mark_cksum_ptype, 0, 1, 1, 1, 1, 0, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(ts_mark_cksum_ptype_rss, 0, 1, 1, 1, 1, 1, \ + TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan, 1, 0, 0, 0, 0, 0, RX_VLAN_F) \ +R(vlan_rss, 1, 0, 0, 0, 0, 1, RX_VLAN_F | RSS_F) \ +R(vlan_ptype, 1, 0, 0, 0, 1, 0, RX_VLAN_F | PTYPE_F) \ +R(vlan_ptype_rss, 1, 0, 0, 0, 1, 1, RX_VLAN_F | PTYPE_F | RSS_F) \ +R(vlan_cksum, 1, 0, 0, 1, 0, 0, RX_VLAN_F | CKSUM_F) \ +R(vlan_cksum_rss, 1, 0, 0, 1, 0, 1, RX_VLAN_F | CKSUM_F | RSS_F) \ +R(vlan_cksum_ptype, 1, 0, 0, 1, 1, 0, \ + RX_VLAN_F | CKSUM_F | PTYPE_F) \ +R(vlan_cksum_ptype_rss, 1, 0, 0, 1, 1, 1, \ + RX_VLAN_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_mark, 1, 0, 1, 0, 0, 0, RX_VLAN_F | MARK_F) \ +R(vlan_mark_rss, 1, 0, 1, 0, 0, 1, RX_VLAN_F | MARK_F | RSS_F) \ +R(vlan_mark_ptype, 1, 0, 1, 0, 1, 0, RX_VLAN_F | MARK_F | PTYPE_F)\ +R(vlan_mark_ptype_rss, 1, 0, 1, 0, 1, 1, \ + RX_VLAN_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_mark_cksum, 1, 0, 1, 1, 0, 0, RX_VLAN_F | MARK_F | CKSUM_F)\ +R(vlan_mark_cksum_rss, 1, 0, 1, 1, 0, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_mark_cksum_ptype, 1, 0, 1, 1, 1, 0, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_mark_cksum_ptype_rss, 1, 0, 1, 1, 1, 1, \ + RX_VLAN_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts, 1, 1, 0, 0, 0, 0, RX_VLAN_F | TS_F) \ +R(vlan_ts_rss, 1, 1, 0, 0, 0, 1, RX_VLAN_F | TS_F | RSS_F) \ +R(vlan_ts_ptype, 1, 1, 0, 0, 1, 0, RX_VLAN_F | TS_F | PTYPE_F) \ +R(vlan_ts_ptype_rss, 1, 1, 0, 0, 1, 1, \ + RX_VLAN_F | TS_F | PTYPE_F | RSS_F) \ +R(vlan_ts_cksum, 1, 1, 0, 1, 0, 0, RX_VLAN_F | TS_F | CKSUM_F) \ +R(vlan_ts_cksum_rss, 1, 1, 0, 1, 0, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | RSS_F) \ +R(vlan_ts_cksum_ptype, 1, 1, 0, 1, 1, 0, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_cksum_ptype_rss, 1, 1, 0, 1, 1, 1, \ + RX_VLAN_F | TS_F | CKSUM_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark, 1, 1, 1, 0, 0, 0, RX_VLAN_F | TS_F | MARK_F) \ +R(vlan_ts_mark_rss, 1, 1, 1, 0, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | RSS_F) \ +R(vlan_ts_mark_ptype, 1, 1, 1, 0, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F) \ +R(vlan_ts_mark_ptype_rss, 1, 1, 1, 0, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | PTYPE_F | RSS_F) \ +R(vlan_ts_mark_cksum, 1, 1, 1, 1, 0, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F) \ +R(vlan_ts_mark_cksum_rss, 1, 1, 1, 1, 0, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | RSS_F) \ +R(vlan_ts_mark_cksum_ptype, 1, 1, 1, 1, 1, 0, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F) \ +R(vlan_ts_mark_cksum_ptype_rss, 1, 1, 1, 1, 1, 1, \ + RX_VLAN_F | TS_F | MARK_F | CKSUM_F | PTYPE_F | RSS_F) -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts); \ \ diff --git a/drivers/net/cnxk/cn9k_rx_mseg.c b/drivers/net/cnxk/cn9k_rx_mseg.c index 3b26962..d7e19b1 100644 --- a/drivers/net/cnxk/cn9k_rx_mseg.c +++ b/drivers/net/cnxk/cn9k_rx_mseg.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_mseg_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cn9k_rx_vec.c b/drivers/net/cnxk/cn9k_rx_vec.c index b19c7f3..e61c222 100644 --- a/drivers/net/cnxk/cn9k_rx_vec.c +++ b/drivers/net/cnxk/cn9k_rx_vec.c @@ -5,7 +5,7 @@ #include "cn9k_ethdev.h" #include "cn9k_rx.h" -#define R(name, f4, f3, f2, f1, f0, flags) \ +#define R(name, f5, f4, f3, f2, f1, f0, flags) \ uint16_t __rte_noinline __rte_hot cn9k_nix_recv_pkts_vec_##name( \ void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c index b7872cc..7adab46 100644 --- a/drivers/net/cnxk/cnxk_ethdev.c +++ b/drivers/net/cnxk/cnxk_ethdev.c @@ -1299,6 +1299,7 @@ cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) /* Initialize base roc nix */ nix->pci_dev = pci_dev; + nix->hw_vlan_ins = true; rc = roc_nix_dev_init(nix); if (rc) { plt_err("Failed to initialize roc nix rc=%d", rc); diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index c07e988..67b1f42 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -76,7 +76,8 @@ (DEV_RX_OFFLOAD_CHECKSUM | DEV_RX_OFFLOAD_SCTP_CKSUM | \ DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | DEV_RX_OFFLOAD_SCATTER | \ DEV_RX_OFFLOAD_JUMBO_FRAME | DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \ - DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP) + DEV_RX_OFFLOAD_RSS_HASH | DEV_RX_OFFLOAD_TIMESTAMP | \ + DEV_RX_OFFLOAD_VLAN_STRIP) #define RSS_IPV4_ENABLE \ (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_UDP | \ diff --git a/drivers/net/cnxk/cnxk_rte_flow.c b/drivers/net/cnxk/cnxk_rte_flow.c index c61618a..213125b 100644 --- a/drivers/net/cnxk/cnxk_rte_flow.c +++ b/drivers/net/cnxk/cnxk_rte_flow.c @@ -178,6 +178,23 @@ cnxk_map_actions(struct rte_eth_dev *eth_dev, const struct rte_flow_attr *attr, case RTE_FLOW_ACTION_TYPE_SECURITY: in_actions[i].type = ROC_NPC_ACTION_TYPE_SEC; break; + case RTE_FLOW_ACTION_TYPE_OF_POP_VLAN: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VLAN_STRIP; + break; + case RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID: + in_actions[i].type = ROC_NPC_ACTION_TYPE_VLAN_INSERT; + in_actions[i].conf = actions->conf; + break; + case RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN: + in_actions[i].type = + ROC_NPC_ACTION_TYPE_VLAN_ETHTYPE_INSERT; + in_actions[i].conf = actions->conf; + break; + case RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_PCP: + in_actions[i].type = + ROC_NPC_ACTION_TYPE_VLAN_PCP_INSERT; + in_actions[i].conf = actions->conf; + break; default: plt_npc_dbg("Action is not supported = %d", actions->type); -- 2.8.4 ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram ` (61 preceding siblings ...) 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 62/62] net/cnxk: support marking and VLAN tagging Nithin Dabilpuram @ 2021-06-25 14:01 ` Jerin Jacob 2021-07-06 16:20 ` Thomas Monjalon 62 siblings, 1 reply; 262+ messages in thread From: Jerin Jacob @ 2021-06-25 14:01 UTC (permalink / raw) To: Nithin Dabilpuram Cc: dpdk-dev, Jerin Jacob, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra, Thomas Monjalon On Wed, Jun 23, 2021 at 10:17 AM Nithin Dabilpuram <ndabilpuram@marvell.com> wrote: > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > driver when code is ready and 'net/octeontx2' will be deprecated. Series applied to dpdk-next-net-mrvl/for-next-net. Thanks. Change the state to "Awaiting Upstream" for the main tree. > > Harman Kalra (1): > common/cnxk: allocate lmt region in userspace > > Jerin Jacob (7): > common/cnxk: fix batch alloc completion poll logic > net/cnxk: add Rx burst for cn9k > net/cnxk: add Rx vector version for cn9k > net/cnxk: add Tx burst for cn9k > net/cnxk: add Rx burst for cn10k > net/cnxk: add Rx vector version for cn10k > net/cnxk: add Tx burst for cn10k > > Kiran Kumar K (2): > net/cnxk: add support to configure npc > net/cnxk: support initial version of rte flow > > Nithin Dabilpuram (18): > common/cnxk: change model API to not use camel case > net/cnxk: add build infra and common probe > net/cnxk: add platform specific probe and remove > net/cnxk: add common devargs parsing function > net/cnxk: support common dev infos get > net/cnxk: add device configuration operation > net/cnxk: support link status update > net/cnxk: add Rx queue setup and release > net/cnxk: add Tx queue setup and release > net/cnxk: support packet type > net/cnxk: support queue start and stop > net/cnxk: add Rx multi-segmented version for cn9k > net/cnxk: add Tx multi-segment version for cn9k > net/cnxk: add Tx vector version for cn9k > net/cnxk: add Rx multi-segment version for cn10k > net/cnxk: add Tx multi-segment version for cn10k > net/cnxk: add Tx vector version for cn10k > net/cnxk: add device start and stop operations > > Satha Rao (8): > common/cnxk: add support to lock NIX RQ contexts > common/cnxk: add provision to enable RED on RQ > net/cnxk: add port/queue stats > net/cnxk: add xstats apis > net/cnxk: add rxq/txq info get operations > net/cnxk: add ethdev firmware version get > net/cnxk: add get register operation > net/cnxk: added RETA and RSS hash operations > > Satheesh Paul (6): > common/cnxk: add support to dump flow entries > common/cnxk: support for mark and flag flow actions > common/cnxk: support for VLAN push and pop flow actions > net/cnxk: add flow ops get operation > net/cnxk: support for RSS in rte flow > net/cnxk: support marking and VLAN tagging > > Sunil Kumar Kori (20): > net/cnxk: add MAC address set ops > net/cnxk: add MTU set device operation > net/cnxk: add promiscuous mode enable and disable > net/cnxk: support DMAC filter > net/cnxk: add all multicast enable/disable ethops > net/cnxk: add Rx/Tx burst mode get ops > net/cnxk: add flow ctrl set/get ops > net/cnxk: add link up/down operations > net/cnxk: add EEPROM module info get operations > net/cnxk: add Rx queue interrupt enable/disable ops > net/cnxk: add validation API for mempool ops > net/cnxk: add device close and reset operations > net/cnxk: add pending Tx mbuf cleanup operation > net/cnxk: register callback to get PTP status > net/cnxk: support base PTP timesync > net/cnxk: add timesync enable/disable operations > net/cnxk: add Rx/Tx timestamp read operations > net/cnxk: add time read/write/adjust operations > net/cnxk: add read clock operation > net/cnxk: support multicast filter > > -- > > v4: > - Fixed build issue with gcc 4.8 > - Shortened subject lines of few commits > - Removed camel case for model API > - Updated rte_flow features in cnxk_vec.ini and cnxk_vf.ini > - Added CC stable to "fix batch alloc.." patch > - Squashed cn98xx flow create related common patch to > VLAN push and pop flow actions patch. > - Changed INTERNAL to DPDK_21 in version.map > > v3: > - Updated release notes > - Removed RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS flag and add support for queue > stats in xstats > - Fixed issue with LSO format indices > - Removed mbox sync changes patch from this series > - Fixed documentation issues > - Removed repetitive code in fast path SIMD > - Optimize cn10k LMTST logic > - Make rte_flow_create implementation specific > to handle VLAN Stripping and MARK actions/offloads > - Use rte_atomic_thread_fence() instead of rte_rmb() > - Handle other comments from Jerin. > - Merged rte flow dump API patch to flow ops get patch > - Added marking and vlan tagging support. > - Fixed some checkpatch and git check log issues. > > v2: > - Fixed issue with flow validate and flow create for 98xx > - Fixed issue batch alloc logic > - Fix lmtline allocation to be cached > - Sync Inline IPSec Rx mbox with kernel > - Add support for mark and flag flow actions > - Add reta key and hash update ops > - Added PTP and multicast filter support > > MAINTAINERS | 5 +- > doc/guides/nics/cnxk.rst | 232 +++++ > doc/guides/nics/features/cnxk.ini | 90 ++ > doc/guides/nics/features/cnxk_vec.ini | 86 ++ > doc/guides/nics/features/cnxk_vf.ini | 82 ++ > doc/guides/nics/index.rst | 1 + > doc/guides/platform/cnxk.rst | 3 + > doc/guides/rel_notes/release_21_08.rst | 5 + > drivers/common/cnxk/hw/npc.h | 2 + > drivers/common/cnxk/meson.build | 1 + > drivers/common/cnxk/roc_api.h | 2 + > drivers/common/cnxk/roc_dev.c | 98 +- > drivers/common/cnxk/roc_dev_priv.h | 1 + > drivers/common/cnxk/roc_mbox.h | 3 + > drivers/common/cnxk/roc_model.h | 12 +- > drivers/common/cnxk/roc_nix.h | 39 +- > drivers/common/cnxk/roc_nix_queue.c | 52 + > drivers/common/cnxk/roc_nix_rss.c | 51 +- > drivers/common/cnxk/roc_nix_tm_utils.c | 86 +- > drivers/common/cnxk/roc_npa.c | 10 +- > drivers/common/cnxk/roc_npa.h | 35 +- > drivers/common/cnxk/roc_npc.c | 296 +++++- > drivers/common/cnxk/roc_npc.h | 39 +- > drivers/common/cnxk/roc_npc_mcam.c | 2 +- > drivers/common/cnxk/roc_npc_mcam_dump.c | 611 ++++++++++++ > drivers/common/cnxk/roc_npc_priv.h | 3 +- > drivers/common/cnxk/roc_npc_utils.c | 4 + > drivers/common/cnxk/roc_platform.h | 13 + > drivers/common/cnxk/version.map | 7 + > drivers/net/cnxk/cn10k_ethdev.c | 551 +++++++++++ > drivers/net/cnxk/cn10k_ethdev.h | 41 + > drivers/net/cnxk/cn10k_rte_flow.c | 72 ++ > drivers/net/cnxk/cn10k_rte_flow.h | 17 + > drivers/net/cnxk/cn10k_rx.c | 79 ++ > drivers/net/cnxk/cn10k_rx.h | 653 +++++++++++++ > drivers/net/cnxk/cn10k_rx_mseg.c | 17 + > drivers/net/cnxk/cn10k_rx_vec.c | 22 + > drivers/net/cnxk/cn10k_tx.c | 82 ++ > drivers/net/cnxk/cn10k_tx.h | 1605 +++++++++++++++++++++++++++++++ > drivers/net/cnxk/cn10k_tx_mseg.c | 25 + > drivers/net/cnxk/cn10k_tx_vec.c | 26 + > drivers/net/cnxk/cn9k_ethdev.c | 574 +++++++++++ > drivers/net/cnxk/cn9k_ethdev.h | 39 + > drivers/net/cnxk/cn9k_rte_flow.c | 72 ++ > drivers/net/cnxk/cn9k_rte_flow.h | 17 + > drivers/net/cnxk/cn9k_rx.c | 79 ++ > drivers/net/cnxk/cn9k_rx.h | 655 +++++++++++++ > drivers/net/cnxk/cn9k_rx_mseg.c | 17 + > drivers/net/cnxk/cn9k_rx_vec.c | 20 + > drivers/net/cnxk/cn9k_tx.c | 81 ++ > drivers/net/cnxk/cn9k_tx.h | 1436 +++++++++++++++++++++++++++ > drivers/net/cnxk/cn9k_tx_mseg.c | 25 + > drivers/net/cnxk/cn9k_tx_vec.c | 26 + > drivers/net/cnxk/cnxk_ethdev.c | 1540 +++++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 495 ++++++++++ > drivers/net/cnxk/cnxk_ethdev_devargs.c | 173 ++++ > drivers/net/cnxk/cnxk_ethdev_ops.c | 912 ++++++++++++++++++ > drivers/net/cnxk/cnxk_link.c | 113 +++ > drivers/net/cnxk/cnxk_lookup.c | 326 +++++++ > drivers/net/cnxk/cnxk_ptp.c | 287 ++++++ > drivers/net/cnxk/cnxk_rte_flow.c | 433 +++++++++ > drivers/net/cnxk/cnxk_rte_flow.h | 27 + > drivers/net/cnxk/cnxk_stats.c | 320 ++++++ > drivers/net/cnxk/meson.build | 48 + > drivers/net/cnxk/version.map | 3 + > drivers/net/meson.build | 1 + > 66 files changed, 12678 insertions(+), 102 deletions(-) > create mode 100644 doc/guides/nics/cnxk.rst > create mode 100644 doc/guides/nics/features/cnxk.ini > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > create mode 100644 drivers/common/cnxk/roc_npc_mcam_dump.c > create mode 100644 drivers/net/cnxk/cn10k_ethdev.c > create mode 100644 drivers/net/cnxk/cn10k_ethdev.h > create mode 100644 drivers/net/cnxk/cn10k_rte_flow.c > create mode 100644 drivers/net/cnxk/cn10k_rte_flow.h > create mode 100644 drivers/net/cnxk/cn10k_rx.c > create mode 100644 drivers/net/cnxk/cn10k_rx.h > create mode 100644 drivers/net/cnxk/cn10k_rx_mseg.c > create mode 100644 drivers/net/cnxk/cn10k_rx_vec.c > create mode 100644 drivers/net/cnxk/cn10k_tx.c > create mode 100644 drivers/net/cnxk/cn10k_tx.h > create mode 100644 drivers/net/cnxk/cn10k_tx_mseg.c > create mode 100644 drivers/net/cnxk/cn10k_tx_vec.c > create mode 100644 drivers/net/cnxk/cn9k_ethdev.c > create mode 100644 drivers/net/cnxk/cn9k_ethdev.h > create mode 100644 drivers/net/cnxk/cn9k_rte_flow.c > create mode 100644 drivers/net/cnxk/cn9k_rte_flow.h > create mode 100644 drivers/net/cnxk/cn9k_rx.c > create mode 100644 drivers/net/cnxk/cn9k_rx.h > create mode 100644 drivers/net/cnxk/cn9k_rx_mseg.c > create mode 100644 drivers/net/cnxk/cn9k_rx_vec.c > create mode 100644 drivers/net/cnxk/cn9k_tx.c > create mode 100644 drivers/net/cnxk/cn9k_tx.h > create mode 100644 drivers/net/cnxk/cn9k_tx_mseg.c > create mode 100644 drivers/net/cnxk/cn9k_tx_vec.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > create mode 100644 drivers/net/cnxk/cnxk_ethdev_devargs.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev_ops.c > create mode 100644 drivers/net/cnxk/cnxk_link.c > create mode 100644 drivers/net/cnxk/cnxk_lookup.c > create mode 100644 drivers/net/cnxk/cnxk_ptp.c > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.c > create mode 100644 drivers/net/cnxk/cnxk_rte_flow.h > create mode 100644 drivers/net/cnxk/cnxk_stats.c > create mode 100644 drivers/net/cnxk/meson.build > create mode 100644 drivers/net/cnxk/version.map > > -- > 2.8.4 > ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver 2021-06-25 14:01 ` [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver Jerin Jacob @ 2021-07-06 16:20 ` Thomas Monjalon 2021-07-06 16:31 ` Jerin Jacob 2021-07-07 9:26 ` Nithin Dabilpuram 0 siblings, 2 replies; 262+ messages in thread From: Thomas Monjalon @ 2021-07-06 16:20 UTC (permalink / raw) To: Jerin Jacob Cc: Nithin Dabilpuram, dev, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra, Jerin Jacob, david.marchand 25/06/2021 16:01, Jerin Jacob: > On Wed, Jun 23, 2021 at 10:17 AM Nithin Dabilpuram > <ndabilpuram@marvell.com> wrote: > > > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > > driver when code is ready and 'net/octeontx2' will be deprecated. > > Series applied to dpdk-next-net-mrvl/for-next-net. Thanks. > Change the state to "Awaiting Upstream" for the main tree. I had to rework the file meson.build in every commits. Please next time, run devtools/check-meson.py (no it is not integrated in the CI yet) Also, please take care of the uppercases in your product names. ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver 2021-07-06 16:20 ` Thomas Monjalon @ 2021-07-06 16:31 ` Jerin Jacob 2021-07-07 9:26 ` Nithin Dabilpuram 1 sibling, 0 replies; 262+ messages in thread From: Jerin Jacob @ 2021-07-06 16:31 UTC (permalink / raw) To: Thomas Monjalon Cc: Jerin Jacob, Nithin Dabilpuram, dpdk-dev, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra, David Marchand On Tue, Jul 6, 2021 at 9:51 PM Thomas Monjalon <thomas@monjalon.net> wrote: > > 25/06/2021 16:01, Jerin Jacob: > > On Wed, Jun 23, 2021 at 10:17 AM Nithin Dabilpuram > > <ndabilpuram@marvell.com> wrote: > > > > > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > > > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > > > driver when code is ready and 'net/octeontx2' will be deprecated. > > > > Series applied to dpdk-next-net-mrvl/for-next-net. Thanks. > > Change the state to "Awaiting Upstream" for the main tree. > > I had to rework the file meson.build in every commits. > Please next time, run devtools/check-meson.py > (no it is not integrated in the CI yet) Thanks for the info. I was not aware of this script. I will add it to my local check. > > Also, please take care of the uppercases in your product names. > > ^ permalink raw reply [flat|nested] 262+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver 2021-07-06 16:20 ` Thomas Monjalon 2021-07-06 16:31 ` Jerin Jacob @ 2021-07-07 9:26 ` Nithin Dabilpuram 1 sibling, 0 replies; 262+ messages in thread From: Nithin Dabilpuram @ 2021-07-07 9:26 UTC (permalink / raw) To: Thomas Monjalon Cc: Jerin Jacob, dev, Sunil Kumar Kori, Satha Koteswara Rao Kottidi, Pavan Nikhilesh, Kiran Kumar K, Satheesh Paul, Ashwin Sekhar Thalakalath Kottilveetil, Harman Kalra, Jerin Jacob, david.marchand On Tue, Jul 06, 2021 at 06:20:58PM +0200, Thomas Monjalon wrote: > 25/06/2021 16:01, Jerin Jacob: > > On Wed, Jun 23, 2021 at 10:17 AM Nithin Dabilpuram > > <ndabilpuram@marvell.com> wrote: > > > > > > This patchset adds support for Marvell CN106XX SoC based on 'common/cnxk' > > > driver. In future, CN9K a.k.a octeontx2 will also be supported by same > > > driver when code is ready and 'net/octeontx2' will be deprecated. > > > > Series applied to dpdk-next-net-mrvl/for-next-net. Thanks. > > Change the state to "Awaiting Upstream" for the main tree. > > I had to rework the file meson.build in every commits. > Please next time, run devtools/check-meson.py > (no it is not integrated in the CI yet) Sorry for the inconvenience. Missed to run this script and follow other PMD's with regards to meson.build indentation. > > Also, please take care of the uppercases in your product names. > > ^ permalink raw reply [flat|nested] 262+ messages in thread
end of thread, other threads:[~2021-07-07 9:29 UTC | newest] Thread overview: 262+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-06 15:33 [dpdk-dev] [PATCH 00/44] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 01/44] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 02/44] net/cnxk: add platform specific probe and remove Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 03/44] net/cnxk: add common devargs parsing function Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 04/44] net/cnxk: add common dev infos get support Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 05/44] net/cnxk: add device configuration operation Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 06/44] net/cnxk: add link status update support Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 07/44] net/cnxk: add Rx queue setup and release Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 08/44] net/cnxk: add Tx " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 09/44] net/cnxk: add packet type support Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 10/44] net/cnxk: add queue start and stop support Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 11/44] net/cnxk: add Rx support for cn9k Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 13/44] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 14/44] net/cnxk: add Tx support " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 15/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 16/44] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 17/44] net/cnxk: add Rx support for cn10k Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 18/44] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 19/44] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 20/44] net/cnxk: add Tx support " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 21/44] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 22/44] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 23/44] net/cnxk: add device start and stop operations Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 24/44] net/cnxk: add MAC address set ops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 25/44] net/cnxk: add MTU set device operation Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 26/44] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 27/44] net/cnxk: add DMAC filter support Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 28/44] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 29/44] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 30/44] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 31/44] net/cnxk: add link up/down operations Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 32/44] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 33/44] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 34/44] net/cnxk: add validation API for mempool ops Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 35/44] net/cnxk: add port/queue stats Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 36/44] net/cnxk: add xstats apis Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 37/44] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 38/44] net/cnxk: add device close and reset operations Nithin Dabilpuram 2021-03-06 15:33 ` [dpdk-dev] [PATCH 39/44] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 40/44] net/cnxk: add support to configure npc Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 41/44] net/cnxk: add initial version of rte flow support Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 42/44] net/cnxk: add filter ctrl operation Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 43/44] net/cnxk: add ethdev firmware version get Nithin Dabilpuram 2021-03-06 15:34 ` [dpdk-dev] [PATCH 44/44] net/cnxk: add get register operation Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-07 18:25 ` Stephen Hemminger 2021-06-08 3:47 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 02/62] common/cnxk: update Rx inline IPsec mbox message format Nithin Dabilpuram 2021-06-08 12:26 ` Andrew Rybchenko 2021-06-09 11:02 ` Nithin Dabilpuram 2021-06-14 3:30 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 03/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 04/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 05/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 06/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram 2021-06-14 3:32 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 07/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-06-09 1:38 ` Huisong Li 2021-06-09 10:45 ` Nithin Dabilpuram 2021-06-14 3:52 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram 2021-06-15 12:26 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram 2021-06-14 4:22 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram 2021-06-15 12:29 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 14/62] net/cnxk: add link status update support Nithin Dabilpuram 2021-06-15 12:31 ` Jerin Jacob 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 16/62] net/cnxk: add Tx " Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 17/62] net/cnxk: add packet type support Nithin Dabilpuram 2021-06-07 17:58 ` [dpdk-dev] [PATCH v2 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 22/62] net/cnxk: add Tx support " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 28/62] net/cnxk: add Tx support " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram 2021-06-15 12:40 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 44/62] net/cnxk: add xstats apis Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram 2021-06-15 12:45 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram 2021-06-15 12:47 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 52/62] net/cnxk: add get register operation Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 53/62] net/cnxk: support for rss in rte_flow Nithin Dabilpuram 2021-06-15 14:38 ` Jerin Jacob 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram 2021-06-08 12:04 ` Pavan Nikhilesh Bhagavatula 2021-06-09 11:06 ` Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 59/62] net/cnxk: add read clock operation Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 60/62] net/cnxk: support for rte flow dev dump API Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 61/62] net/cnxk: added reta and rss_hash operations Nithin Dabilpuram 2021-06-07 17:59 ` [dpdk-dev] [PATCH v2 62/62] net/cnxk: add multicast filter support Nithin Dabilpuram 2021-06-14 3:27 ` [dpdk-dev] [PATCH v2 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 " Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 07/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 08/62] common/cnxk: fix flow create on CN98xx Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 12/62] net/cnxk: add common dev infos get support Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 14/62] net/cnxk: add link status update support Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 16/62] net/cnxk: add Tx " Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 17/62] net/cnxk: add packet type support Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 18/62] net/cnxk: add queue start and stop support Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 19/62] net/cnxk: add Rx support for cn9k Nithin Dabilpuram 2021-06-18 10:36 ` [dpdk-dev] [PATCH v3 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 22/62] net/cnxk: add Tx support " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 25/62] net/cnxk: add Rx support for cn10k Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 28/62] net/cnxk: add Tx support " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 35/62] net/cnxk: add DMAC filter support Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 44/62] net/cnxk: add xstats apis Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 49/62] net/cnxk: add initial version of rte flow support Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 52/62] net/cnxk: add get register operation Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 55/62] net/cnxk: add base PTP timesync support Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 59/62] net/cnxk: add read clock operation Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 61/62] net/cnxk: add multicast filter support Nithin Dabilpuram 2021-06-18 10:37 ` [dpdk-dev] [PATCH v3 62/62] net/cnxk: add marking and VLAN tagging support Nithin Dabilpuram 2021-06-21 13:41 ` [dpdk-dev] [PATCH v3 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 01/62] common/cnxk: add support to lock NIX RQ contexts Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 02/62] common/cnxk: fix batch alloc completion poll logic Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 03/62] common/cnxk: add support to dump flow entries Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 04/62] common/cnxk: support for mark and flag flow actions Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 05/62] common/cnxk: allocate lmt region in userspace Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 06/62] common/cnxk: add provision to enable RED on RQ Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 07/62] common/cnxk: change model API to not use camel case Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 08/62] common/cnxk: support for VLAN push and pop flow actions Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 09/62] net/cnxk: add build infra and common probe Nithin Dabilpuram 2021-07-05 21:55 ` Thomas Monjalon 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 10/62] net/cnxk: add platform specific probe and remove Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 11/62] net/cnxk: add common devargs parsing function Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 12/62] net/cnxk: support common dev infos get Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 13/62] net/cnxk: add device configuration operation Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 14/62] net/cnxk: support link status update Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 15/62] net/cnxk: add Rx queue setup and release Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 16/62] net/cnxk: add Tx " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 17/62] net/cnxk: support packet type Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 18/62] net/cnxk: support queue start and stop Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 19/62] net/cnxk: add Rx burst for cn9k Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 20/62] net/cnxk: add Rx multi-segmented version " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 21/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 22/62] net/cnxk: add Tx burst " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 23/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 24/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 25/62] net/cnxk: add Rx burst for cn10k Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 26/62] net/cnxk: add Rx multi-segment version " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 27/62] net/cnxk: add Rx vector " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 28/62] net/cnxk: add Tx burst " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 29/62] net/cnxk: add Tx multi-segment version " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 30/62] net/cnxk: add Tx vector " Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 31/62] net/cnxk: add device start and stop operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 32/62] net/cnxk: add MAC address set ops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 33/62] net/cnxk: add MTU set device operation Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 34/62] net/cnxk: add promiscuous mode enable and disable Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 35/62] net/cnxk: support DMAC filter Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 36/62] net/cnxk: add all multicast enable/disable ethops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 37/62] net/cnxk: add Rx/Tx burst mode get ops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 38/62] net/cnxk: add flow ctrl set/get ops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 39/62] net/cnxk: add link up/down operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 40/62] net/cnxk: add EEPROM module info get operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 41/62] net/cnxk: add Rx queue interrupt enable/disable ops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 42/62] net/cnxk: add validation API for mempool ops Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 43/62] net/cnxk: add port/queue stats Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 44/62] net/cnxk: add xstats apis Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 45/62] net/cnxk: add rxq/txq info get operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 46/62] net/cnxk: add device close and reset operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 47/62] net/cnxk: add pending Tx mbuf cleanup operation Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 48/62] net/cnxk: add support to configure npc Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 49/62] net/cnxk: support initial version of rte flow Nithin Dabilpuram 2021-07-05 22:01 ` Thomas Monjalon 2021-07-07 9:29 ` Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 50/62] net/cnxk: add flow ops get operation Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 51/62] net/cnxk: add ethdev firmware version get Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 52/62] net/cnxk: add get register operation Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 53/62] net/cnxk: support for RSS in rte flow Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 54/62] net/cnxk: register callback to get PTP status Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 55/62] net/cnxk: support base PTP timesync Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 56/62] net/cnxk: add timesync enable/disable operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 57/62] net/cnxk: add Rx/Tx timestamp read operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 58/62] net/cnxk: add time read/write/adjust operations Nithin Dabilpuram 2021-06-23 4:46 ` [dpdk-dev] [PATCH v4 59/62] net/cnxk: add read clock operation Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 60/62] net/cnxk: added RETA and RSS hash operations Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 61/62] net/cnxk: support multicast filter Nithin Dabilpuram 2021-06-23 4:47 ` [dpdk-dev] [PATCH v4 62/62] net/cnxk: support marking and VLAN tagging Nithin Dabilpuram 2021-06-25 14:01 ` [dpdk-dev] [PATCH v4 00/62] Marvell CNXK Ethdev Driver Jerin Jacob 2021-07-06 16:20 ` Thomas Monjalon 2021-07-06 16:31 ` Jerin Jacob 2021-07-07 9:26 ` Nithin Dabilpuram
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).