From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 973F4A0C43; Wed, 9 Jun 2021 03:38:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1E94A40689; Wed, 9 Jun 2021 03:38:53 +0200 (CEST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 710794067A for ; Wed, 9 Jun 2021 03:38:50 +0200 (CEST) Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4G08nn6hF8zYsVm; Wed, 9 Jun 2021 09:35:57 +0800 (CST) Received: from dggema767-chm.china.huawei.com (10.1.198.209) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Wed, 9 Jun 2021 09:38:47 +0800 Received: from [10.66.74.184] (10.66.74.184) by dggema767-chm.china.huawei.com (10.1.198.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Wed, 9 Jun 2021 09:38:47 +0800 To: Nithin Dabilpuram , CC: , , , , , , References: <20210306153404.10781-1-ndabilpuram@marvell.com> <20210607175943.31690-1-ndabilpuram@marvell.com> <20210607175943.31690-10-ndabilpuram@marvell.com> From: Huisong Li Message-ID: Date: Wed, 9 Jun 2021 09:38:04 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210607175943.31690-10-ndabilpuram@marvell.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.66.74.184] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggema767-chm.china.huawei.com (10.1.198.209) X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 在 2021/6/8 1:58, Nithin Dabilpuram 写道: > Add build infrastructure and common probe and remove for cnxk driver > which is used by both CN10K and CN9K SoC. > > Signed-off-by: Nithin Dabilpuram > --- > MAINTAINERS | 3 + > doc/guides/nics/cnxk.rst | 29 +++++ > doc/guides/nics/features/cnxk.ini | 9 ++ > doc/guides/nics/features/cnxk_vec.ini | 9 ++ > doc/guides/nics/features/cnxk_vf.ini | 9 ++ > doc/guides/nics/index.rst | 1 + > doc/guides/platform/cnxk.rst | 3 + > drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++ > drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++ > drivers/net/cnxk/meson.build | 21 ++++ > drivers/net/cnxk/version.map | 3 + > drivers/net/meson.build | 1 + > 12 files changed, 364 insertions(+) > create mode 100644 doc/guides/nics/cnxk.rst > create mode 100644 doc/guides/nics/features/cnxk.ini > create mode 100644 doc/guides/nics/features/cnxk_vec.ini > create mode 100644 doc/guides/nics/features/cnxk_vf.ini > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h > create mode 100644 drivers/net/cnxk/meson.build > create mode 100644 drivers/net/cnxk/version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index 5877a16..2be220e 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -746,6 +746,9 @@ M: Sunil Kumar Kori > M: Satha Rao > T: git://dpdk.org/next/dpdk-next-net-mrvl > F: drivers/common/cnxk/ > +F: drivers/net/cnxk/ > +F: doc/guides/nics/cnxk.rst > +F: doc/guides/nics/features/cnxk*.ini > F: doc/guides/platform/cnxk.rst > > Marvell mvpp2 > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst > new file mode 100644 > index 0000000..ca21842 > --- /dev/null > +++ b/doc/guides/nics/cnxk.rst > @@ -0,0 +1,29 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(C) 2021 Marvell. > + > +CNXK Poll Mode driver > +===================== > + > +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver > +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K** > +SoC family as well as for their virtual functions (VF) in SR-IOV context. > + > +More information can be found at `Marvell Official Website > +`_. > + > +Features > +-------- > + > +Features of the CNXK Ethdev PMD are: > + > +Prerequisites > +------------- > + > +See :doc:`../platform/cnxk` for setup information. > + > + > +Driver compilation and testing > +------------------------------ > + > +Refer to the document :ref:`compiling and testing a PMD for a NIC ` > +for details. > diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini > new file mode 100644 > index 0000000..2c23464 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini > new file mode 100644 > index 0000000..de78516 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk_vec.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk_vec' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini > new file mode 100644 > index 0000000..9c96351 > --- /dev/null > +++ b/doc/guides/nics/features/cnxk_vf.ini > @@ -0,0 +1,9 @@ > +; > +; Supported features of the 'cnxk_vf' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Linux = Y > +ARMv8 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 799697c..c1a04d9 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -19,6 +19,7 @@ Network Interface Controller Drivers > axgbe > bnx2x > bnxt > + cnxk > cxgbe > dpaa > dpaa2 > diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst > index cebb3d0..b506c11 100644 > --- a/doc/guides/platform/cnxk.rst > +++ b/doc/guides/platform/cnxk.rst > @@ -142,6 +142,9 @@ HW Offload Drivers > > This section lists dataplane H/W block(s) available in cnxk SoC. > > +#. **Ethdev Driver** > + See :doc:`../nics/cnxk` for NIX Ethdev driver information. > + > #. **Mempool Driver** > See :doc:`../mempool/cnxk` for NPA mempool driver information. > > diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c > new file mode 100644 > index 0000000..6717410 > --- /dev/null > +++ b/drivers/net/cnxk/cnxk_ethdev.c > @@ -0,0 +1,219 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#include > + > +/* CNXK platform independent eth dev ops */ > +struct eth_dev_ops cnxk_eth_dev_ops; > + > +static int > +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + struct roc_nix *nix = &dev->nix; > + struct rte_pci_device *pci_dev; > + int rc, max_entries; > + > + eth_dev->dev_ops = &cnxk_eth_dev_ops; > + > + /* For secondary processes, the primary has done all the work */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); > + rte_eth_copy_pci_info(eth_dev, pci_dev); > + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS; Hi, It is recommended not to use this flag when add new driver. Queue stats will be moved to xstats and be filled by PMDs. Please check the following patch: commit f30e69b41f949cd4a9afb6ff39de196e661708e2 Author: Ferruh Yigit Date:   Wed Oct 14 03:26:47 2020 +0100     ethdev: add device flag to bypass auto-filled queue xstats     Queue stats are stored in 'struct rte_eth_stats' as array and array size     is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag.     As a result of technical board discussion, decided to remove the queue     statistics from 'struct rte_eth_stats' in the long term.     Instead PMDs should represent the queue statistics via xstats, this     gives more flexibility on the number of the queues supported.     Currently queue stats in the xstats are filled by ethdev layer, using     some basic stats, when queue stats removed from basic stats the     responsibility to fill the relevant xstats will be pushed to the PMDs.     During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS'     device flag is created. Initially all PMDs using xstats set this flag.     The PMDs implemented queue stats in the xstats should clear the flag.     When all PMDs switch to the xstats for the queue stats, queue stats     related fields from 'struct rte_eth_stats' will be removed, as well as     'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag.     Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be     removed.     Signed-off-by: Ferruh Yigit     Acked-by: Haiyue Wang     Acked-by: Xiao Wang     Acked-by: Thomas Monjalon > + > + /* Initialize base roc nix */ > + nix->pci_dev = pci_dev; > + rc = roc_nix_dev_init(nix); > + if (rc) { > + plt_err("Failed to initialize roc nix rc=%d", rc); > + goto error; > + } > + > + dev->eth_dev = eth_dev; > + > + /* For vfs, returned max_entries will be 0. but to keep default mac > + * address, one entry must be allocated. so setting up to 1. > + */ > + if (roc_nix_is_vf_or_sdp(nix)) > + max_entries = 1; > + else > + max_entries = roc_nix_mac_max_entries_get(nix); > + > + if (max_entries <= 0) { > + plt_err("Failed to get max entries for mac addr"); > + rc = -ENOTSUP; > + goto dev_fini; > + } > + > + eth_dev->data->mac_addrs = > + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0); > + if (eth_dev->data->mac_addrs == NULL) { > + plt_err("Failed to allocate memory for mac addr"); > + rc = -ENOMEM; > + goto dev_fini; > + } > + > + dev->max_mac_entries = max_entries; > + > + /* Get mac address */ > + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr); > + if (rc) { > + plt_err("Failed to get mac addr, rc=%d", rc); > + goto free_mac_addrs; > + } > + > + /* Update the mac address */ > + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN); > + > + if (!roc_nix_is_vf_or_sdp(nix)) { > + /* Sync same MAC address to CGX/RPM table */ > + rc = roc_nix_mac_addr_set(nix, dev->mac_addr); > + if (rc) { > + plt_err("Failed to set mac addr, rc=%d", rc); > + goto free_mac_addrs; > + } > + } > + > + /* Initialize roc npc */ > + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64 > + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64, > + eth_dev->data->port_id, roc_nix_get_pf(nix), > + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap, > + dev->rx_offload_capa, dev->tx_offload_capa); > + return 0; > + > +free_mac_addrs: > + rte_free(eth_dev->data->mac_addrs); > +dev_fini: > + roc_nix_dev_fini(nix); > +error: > + plt_err("Failed to init nix eth_dev rc=%d", rc); > + return rc; > +} > + > +static int > +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close) > +{ > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); > + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops; > + struct roc_nix *nix = &dev->nix; > + int rc, i; > + > + /* Nothing to be done for secondary processes */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + roc_nix_npc_rx_ena_dis(nix, false); > + > + /* Free up SQs */ > + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) { > + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]); > + eth_dev->data->tx_queues[i] = NULL; > + } > + eth_dev->data->nb_tx_queues = 0; > + > + /* Free up RQ's and CQ's */ > + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) { > + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]); > + eth_dev->data->rx_queues[i] = NULL; > + } > + eth_dev->data->nb_rx_queues = 0; > + > + /* Free tm resources */ > + roc_nix_tm_fini(nix); > + > + /* Unregister queue irqs */ > + roc_nix_unregister_queue_irqs(nix); > + > + /* Unregister cq irqs */ > + if (eth_dev->data->dev_conf.intr_conf.rxq) > + roc_nix_unregister_cq_irqs(nix); > + > + /* Free nix lf resources */ > + rc = roc_nix_lf_free(nix); > + if (rc) > + plt_err("Failed to free nix lf, rc=%d", rc); > + > + rte_free(eth_dev->data->mac_addrs); > + eth_dev->data->mac_addrs = NULL; > + > + /* Check if mbox close is needed */ > + if (!mbox_close) > + return 0; > + > + rc = roc_nix_dev_fini(nix); > + /* Can be freed later by PMD if NPA LF is in use */ > + if (rc == -EAGAIN) { > + eth_dev->data->dev_private = NULL; > + return 0; > + } else if (rc) { > + plt_err("Failed in nix dev fini, rc=%d", rc); > + } > + > + return rc; > +} > + > +int > +cnxk_nix_remove(struct rte_pci_device *pci_dev) > +{ > + struct rte_eth_dev *eth_dev; > + struct roc_nix *nix; > + int rc = -EINVAL; > + > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name); > + if (eth_dev) { > + /* Cleanup eth dev */ > + rc = cnxk_eth_dev_uninit(eth_dev, true); > + if (rc) > + return rc; > + > + rte_eth_dev_release_port(eth_dev); > + } > + > + /* Nothing to be done for secondary processes */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) > + return 0; > + > + /* Check if this device is hosting common resource */ > + nix = roc_idev_npa_nix_get(); > + if (nix->pci_dev != pci_dev) > + return 0; > + > + /* Try nix fini now */ > + rc = roc_nix_dev_fini(nix); > + if (rc == -EAGAIN) { > + plt_info("%s: common resource in use by other devices", > + pci_dev->name); > + goto exit; > + } else if (rc) { > + plt_err("Failed in nix dev fini, rc=%d", rc); > + goto exit; > + } > + > + /* Free device pointer as rte_ethdev does not have it anymore */ > + rte_free(nix); > +exit: > + return rc; > +} > + > +int > +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) > +{ > + int rc; > + > + RTE_SET_USED(pci_drv); > + > + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev), > + cnxk_eth_dev_init); > + > + /* On error on secondary, recheck if port exists in primary or > + * in mid of detach state. > + */ > + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc) > + if (!rte_eth_dev_allocated(pci_dev->device.name)) > + return 0; > + return rc; > +} > diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h > new file mode 100644 > index 0000000..0460d1e > --- /dev/null > +++ b/drivers/net/cnxk/cnxk_ethdev.h > @@ -0,0 +1,57 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2021 Marvell. > + */ > +#ifndef __CNXK_ETHDEV_H__ > +#define __CNXK_ETHDEV_H__ > + > +#include > +#include > + > +#include > +#include > + > +#include "roc_api.h" > + > +#define CNXK_ETH_DEV_PMD_VERSION "1.0" > + > +struct cnxk_eth_dev { > + /* ROC NIX */ > + struct roc_nix nix; > + > + /* Max macfilter entries */ > + uint8_t max_mac_entries; > + > + uint16_t flags; > + > + /* Pointer back to rte */ > + struct rte_eth_dev *eth_dev; > + > + /* HW capabilities / Limitations */ > + union { > + uint64_t hwcap; > + }; > + > + /* Rx and Tx offload capabilities */ > + uint64_t rx_offload_capa; > + uint64_t tx_offload_capa; > + uint32_t speed_capa; > + > + /* Default mac address */ > + uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; > +}; > + > +static inline struct cnxk_eth_dev * > +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev) > +{ > + return eth_dev->data->dev_private; > +} > + > +/* Common ethdev ops */ > +extern struct eth_dev_ops cnxk_eth_dev_ops; > + > +/* Ops */ > +int cnxk_nix_probe(struct rte_pci_driver *pci_drv, > + struct rte_pci_device *pci_dev); > +int cnxk_nix_remove(struct rte_pci_device *pci_dev); > + > +#endif /* __CNXK_ETHDEV_H__ */ > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build > new file mode 100644 > index 0000000..77b2f18 > --- /dev/null > +++ b/drivers/net/cnxk/meson.build > @@ -0,0 +1,21 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(C) 2021 Marvell. > +# > + > +if not dpdk_conf.get('RTE_ARCH_64') > + build = false > + reason = 'only supported on 64-bit' > + subdir_done() > +endif > + > +sources = files('cnxk_ethdev.c') > + > +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security'] > +deps += ['common_cnxk', 'mempool_cnxk'] > + > +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing'] > +foreach flag: extra_flags > + if cc.has_argument(flag) > + cflags += flag > + endif > +endforeach > diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map > new file mode 100644 > index 0000000..ee80c51 > --- /dev/null > +++ b/drivers/net/cnxk/version.map > @@ -0,0 +1,3 @@ > +INTERNAL { > + local: *; > +}; > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index c8b5ce2..5b066fd 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -12,6 +12,7 @@ drivers = [ > 'bnx2x', > 'bnxt', > 'bonding', > + 'cnxk', > 'cxgbe', > 'dpaa', > 'dpaa2',