Re: [dpdk-dev] [PATCH 2/4] dma/cnxk: create and initialize dmadev on pci probe

[Jerin Jacob <jerinjacobk@gmail.com>]

On Tue, Oct 26, 2021 at 9:43 AM Radha Mohan Chintakuntla
<radhac@marvell.com> wrote:
>
> This patch creates and initializes a dmadev device on pci probe.
>
> Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
> ---
>  MAINTAINERS                    |   7 +-
>  doc/guides/dmadevs/cnxk.rst    |  53 +++++++++++++++
>  doc/guides/dmadevs/index.rst   |   1 +
>  drivers/dma/cnxk/cnxk_dmadev.c | 119 +++++++++++++++++++++++++++++++++
>  drivers/dma/cnxk/cnxk_dmadev.h |  11 +++
>  drivers/dma/cnxk/meson.build   |   7 ++
>  drivers/dma/meson.build        |   1 +
>  7 files changed, 198 insertions(+), 1 deletion(-)
>  create mode 100644 doc/guides/dmadevs/cnxk.rst
>  create mode 100644 drivers/dma/cnxk/cnxk_dmadev.c
>  create mode 100644 drivers/dma/cnxk/cnxk_dmadev.h
>  create mode 100644 drivers/dma/cnxk/meson.build
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index be2c9b6815..cdc2d98a6b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1186,7 +1186,6 @@ F: drivers/compress/zlib/
>  F: doc/guides/compressdevs/zlib.rst
>  F: doc/guides/compressdevs/features/zlib.ini
>
> -

Unwanted line deletion.

>  DMAdev Drivers
>  --------------
>
> @@ -1202,6 +1201,12 @@ M: Conor Walsh <conor.walsh@intel.com>
>  F: drivers/dma/ioat/
>  F: doc/guides/dmadevs/ioat.rst
>
> +Marvell CNXK DPI DMA
> +M: Radha Mohan Chintakuntla <radhac@marvell.com>
> +M: Veerasenareddy Burru <vburru@marvell.com>
> +F: drivers/dma/cnxk/
> +F: doc/guides/dmadevs/cnxk.rst
> +
>
>  RegEx Drivers
>  -------------
> diff --git a/doc/guides/dmadevs/cnxk.rst b/doc/guides/dmadevs/cnxk.rst
> new file mode 100644
> index 0000000000..8ae7c1f8cd
> --- /dev/null
> +++ b/doc/guides/dmadevs/cnxk.rst
> @@ -0,0 +1,53 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2021 Marvell International Ltd.
> +
> +.. include:: <isonum.txt>
> +


Please link top-level doc/guides/platform/cnxk.rst documentation file
to this file.


> +CNXK DMA Device Driver
> +======================
> +
> +The ``cnxk`` dmadev driver provides a poll-mode driver (PMD) for Marvell DPI DMA
> +Hardware Accelerator block found in OCTEONTX2 and OCTEONTX3 family of SoCs. Each
> +DMA queue is exposed as a VF function when SRIOV is enabled.
> +
> +The block supports following modes of DMA transfers
> +
> +#. Internal - DMA within SoC DRAM to DRAM
> +
> +#. Inbound  - Host DRAM to SoC DRAM when SoC is in PCIe Endpoint
> +
> +#. Outbound - SoC DRAM to Host DRAM when SoC is in PCIe Endpoint
> +
> +Device Setup
> +-------------
> +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the
> +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma``
> +will show all the CNXK DMA devices.
> +
> +Devices using VFIO drivers
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The HW devices to be used will need to be bound to a user-space IO driver for use.
> +The ``dpdk-devbind.py`` script can be used to view the state of the devices
> +and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``.
> +For example::
> +
> +     $ dpdk-devbind.py -b vfio-pci 0000:05:00.1
> +
> +Device Probing and Initialization
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +To use the devices from an application, the dmadev API can be used.
> +CNXK DMA device configuration requirements:
> +
> +* Only one ``vchan`` is supported per device.
> +* CNXK DMA devices do not support silent mode.
> +
> +Once configured, the device can then be made ready for use by calling the
> + ``rte_dma_start()`` API.
> +
> +Performing Data Copies
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Refer to the :ref:`Enqueue / Dequeue APIs <dmadev_enqueue_dequeue>` section of the dmadev library
> +documentation for details on operation enqueue and submission API usage.
> diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
> index 20476039a5..227fa00c68 100644
> --- a/doc/guides/dmadevs/index.rst
> +++ b/doc/guides/dmadevs/index.rst
> @@ -11,5 +11,6 @@ an application through DMA API.
>     :maxdepth: 2
>     :numbered:
>
> +   cnxk
>     idxd
>     ioat
> diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
> new file mode 100644
> index 0000000000..620766743d
> --- /dev/null
> +++ b/drivers/dma/cnxk/cnxk_dmadev.c
> @@ -0,0 +1,119 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright (C) 2021 Marvell International Ltd.
> + */
> +
> +#include <string.h>
> +#include <unistd.h>
> +
> +#include <rte_bus.h>
> +#include <rte_bus_pci.h>
> +#include <rte_common.h>
> +#include <rte_eal.h>
> +#include <rte_lcore.h>
> +#include <rte_mempool.h>
> +#include <rte_pci.h>
> +#include <rte_dmadev.h>
> +#include <rte_dmadev_pmd.h>
> +
> +#include <roc_api.h>
> +#include <cnxk_dmadev.h>
> +
> +static int
> +cnxk_dmadev_probe(struct rte_pci_driver *pci_drv __rte_unused,
> +                 struct rte_pci_device *pci_dev)
> +{
> +       struct cnxk_dpi_vf_s *dpivf = NULL;
> +       char name[RTE_DEV_NAME_MAX_LEN];
> +       struct rte_dma_dev *dmadev;
> +       struct roc_dpi *rdpi = NULL;
> +       int rc;
> +
> +       if (!pci_dev->mem_resource[0].addr)
> +               return -ENODEV;
> +
> +       rc = roc_plt_init();
> +       if (rc) {
> +               plt_err("Failed to initialize platform model, rc=%d", rc);
> +               return rc;
> +       }
> +       memset(name, 0, sizeof(name));
> +       rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
> +
> +       dmadev = rte_dma_pmd_allocate(name, pci_dev->device.numa_node,
> +                                     sizeof(*dpivf));
> +       if (dmadev == NULL) {
> +               plt_err("dma device allocation failed for %s", name);
> +               return -ENOMEM;
> +       }
> +
> +       dpivf = dmadev->data->dev_private;
> +
> +       dmadev->device = &pci_dev->device;
> +       dmadev->fp_obj->dev_private = dpivf;
> +
> +       rdpi = &dpivf->rdpi;
> +
> +       rdpi->pci_dev = pci_dev;
> +       rc = roc_dpi_dev_init(rdpi);
> +       if (rc < 0)
> +               goto err_out_free;
> +
> +       return 0;
> +
> +err_out_free:
> +       if (dmadev)
> +               rte_dma_pmd_release(name);
> +
> +       return rc;
> +}
> +
> +static int
> +cnxk_dmadev_remove(struct rte_pci_device *pci_dev)
> +{
> +       char name[RTE_DEV_NAME_MAX_LEN];
> +       struct rte_dma_dev *dmadev;
> +       struct cnxk_dpi_vf_s *dpivf;
> +       int dev_id;
> +
> +       memset(name, 0, sizeof(name));
> +       rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
> +
> +       dev_id = rte_dma_get_dev_id_by_name(name);
> +       if (dev_id < 0) {
> +               plt_err("Invalid device ID");
> +               return -EINVAL;
> +       }
> +
> +       dmadev = &rte_dma_devices[dev_id];
> +       if (!dmadev) {
> +               plt_err("dmadev with name %s not found\n", name);
> +               return -ENODEV;
> +       }
> +
> +       dpivf = dmadev->fp_obj->dev_private;
> +       roc_dpi_queue_stop(&dpivf->rdpi);
> +       roc_dpi_dev_fini(&dpivf->rdpi);
> +
> +       return rte_dma_pmd_release(name);
> +}
> +
> +static const struct rte_pci_id cnxk_dma_pci_map[] = {
> +       {
> +               RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM,
> +                              PCI_DEVID_CNXK_DPI_VF)
> +       },
> +       {
> +               .vendor_id = 0,
> +       },
> +};
> +
> +static struct rte_pci_driver cnxk_dmadev = {
> +       .id_table  = cnxk_dma_pci_map,
> +       .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA,
> +       .probe     = cnxk_dmadev_probe,
> +       .remove    = cnxk_dmadev_remove,
> +};
> +
> +RTE_PMD_REGISTER_PCI(cnxk_dmadev_pci_driver, cnxk_dmadev);
> +RTE_PMD_REGISTER_PCI_TABLE(cnxk_dmadev_pci_driver, cnxk_dma_pci_map);
> +RTE_PMD_REGISTER_KMOD_DEP(cnxk_dmadev_pci_driver, "vfio-pci");
> diff --git a/drivers/dma/cnxk/cnxk_dmadev.h b/drivers/dma/cnxk/cnxk_dmadev.h
> new file mode 100644
> index 0000000000..9e0bb7b2ce
> --- /dev/null
> +++ b/drivers/dma/cnxk/cnxk_dmadev.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(C) 2021 Marvell International Ltd.
> + */
> +#ifndef _CNXK_DMADEV_H_
> +#define _CNXK_DMADEV_H_
> +
> +struct cnxk_dpi_vf_s {
> +       struct roc_dpi rdpi;
> +};
> +
> +#endif
> diff --git a/drivers/dma/cnxk/meson.build b/drivers/dma/cnxk/meson.build
> new file mode 100644
> index 0000000000..9489d6e6dc
> --- /dev/null
> +++ b/drivers/dma/cnxk/meson.build
> @@ -0,0 +1,7 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(C) 2021 Marvell International Ltd.
> +#
> +
> +deps += ['bus_pci', 'common_cnxk', 'dmadev']
> +sources = files('cnxk_dmadev.c')
> +headers = files('cnxk_dmadev.h')
> diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
> index a69418ce9b..c562c8b429 100644
> --- a/drivers/dma/meson.build
> +++ b/drivers/dma/meson.build
> @@ -2,6 +2,7 @@
>  # Copyright 2021 HiSilicon Limited
>
>  drivers = [
> +        'cnxk',
>          'idxd',
>          'ioat',
>          'skeleton',
> --
> 2.17.1
>