From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E4E4BA0C54; Fri, 3 Sep 2021 12:50:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 981D8410F6; Fri, 3 Sep 2021 12:50:19 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id ABD50410E0 for ; Fri, 3 Sep 2021 12:50:16 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10095"; a="216247355" X-IronPort-AV: E=Sophos;i="5.85,265,1624345200"; d="scan'208";a="216247355" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Sep 2021 03:50:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,265,1624345200"; d="scan'208";a="521643384" Received: from silpixa00401122.ir.intel.com ([10.55.128.10]) by fmsmga004.fm.intel.com with ESMTP; 03 Sep 2021 03:50:14 -0700 From: Kevin Laatz To: dev@dpdk.org Cc: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com, Kevin Laatz Date: Fri, 3 Sep 2021 10:49:48 +0000 Message-Id: <20210903105001.1179328-4-kevin.laatz@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210903105001.1179328-1-kevin.laatz@intel.com> References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210903105001.1179328-1-kevin.laatz@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 03/16] dma/idxd: add skeleton for VFIO based DSA device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probe/remove skeleton code for DSA device bound to the vfio pci driver. Relevant documentation and MAINTAINERS update also included. Signed-off-by: Bruce Richardson Signed-off-by: Kevin Laatz --- MAINTAINERS | 10 +++++ doc/guides/dmadevs/idxd.rst | 58 ++++++++++++++++++++++++++ doc/guides/rel_notes/release_21_11.rst | 5 +++ drivers/dma/idxd/idxd_internal.h | 27 ++++++++++++ drivers/dma/idxd/idxd_pci.c | 55 ++++++++++++++++++++++++ drivers/dma/idxd/meson.build | 7 ++++ drivers/dma/idxd/version.map | 3 ++ drivers/dma/meson.build | 1 + 8 files changed, 166 insertions(+) create mode 100644 doc/guides/dmadevs/idxd.rst create mode 100644 drivers/dma/idxd/idxd_internal.h create mode 100644 drivers/dma/idxd/idxd_pci.c create mode 100644 drivers/dma/idxd/meson.build create mode 100644 drivers/dma/idxd/version.map diff --git a/MAINTAINERS b/MAINTAINERS index c057a090d6..b4c614a229 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1199,6 +1199,16 @@ F: doc/guides/compressdevs/zlib.rst F: doc/guides/compressdevs/features/zlib.ini +DMAdev Drivers +-------------- + +Intel IDXD - EXPERIMENTAL +M: Bruce Richardson +M: Kevin Laatz +F: drivers/dma/idxd/ +F: doc/guides/dmadevs/idxd.rst + + RegEx Drivers ------------- diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst new file mode 100644 index 0000000000..924700d17e --- /dev/null +++ b/doc/guides/dmadevs/idxd.rst @@ -0,0 +1,58 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2021 Intel Corporation. + +.. include:: + +IDXD DMA Device Driver +====================== + +The ``idxd`` dmadev driver provides a poll-mode driver (PMD) for Intel\ |reg| +Data Streaming Accelerator `(Intel DSA) +`_. +This PMD can be used in conjunction with Intel\ |reg| DSA devices to offload +data operations, such as data copies, to hardware, freeing up CPU cycles for +other tasks. + +Hardware Requirements +---------------------- + +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` +will show all the DMA devices on the system, including IDXD supported devices. +Intel\ |reg| DSA devices, are currently (at time of writing) appearing +as devices with type “0b25”, due to the absence of pci-id database entries for +them at this point. + +Compilation +------------ + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. No additional compilation steps are necessary. + +Device Setup +------------- + +Devices using VFIO/UIO drivers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The HW devices to be used will need to be bound to a user-space IO driver for use. +The ``dpdk-devbind.py`` script can be used to view the state of the devices +and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``. +For example:: + + $ dpdk-devbind.py -b vfio-pci 6a:01.0 + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For devices bound to a suitable DPDK-supported VFIO/UIO driver, the HW devices will +be found as part of the device scan done at application initialization time without +the need to pass parameters to the application. + +For Intel\ |reg| DSA devices, DPDK will automatically configure the device with the +maximum number of workqueues available on it, partitioning all resources equally +among the queues. +If fewer workqueues are required, then the ``max_queues`` parameter may be passed to +the device driver on the EAL commandline, via the ``allowlist`` or ``-a`` flag e.g.:: + + $ dpdk-test -a ,max_queues=4 diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 78b9691bf3..f7b0fcf3fd 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -60,6 +60,11 @@ New Features The dmadev library provides a DMA device framework for management and provision of hardware and software DMA devices. +* **Added IDXD dmadev driver implementation.** + + The IDXD dmadev driver provide device drivers for the Intel DSA devices. + This device driver can be used through the generic dmadev API. + Removed Items ------------- diff --git a/drivers/dma/idxd/idxd_internal.h b/drivers/dma/idxd/idxd_internal.h new file mode 100644 index 0000000000..c6a7dcd72f --- /dev/null +++ b/drivers/dma/idxd/idxd_internal.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IDXD_INTERNAL_H_ +#define _IDXD_INTERNAL_H_ + +/** + * @file idxd_internal.h + * + * Internal data structures for the idxd/DSA driver for dmadev + * + * @warning + * @b EXPERIMENTAL: these structures and APIs may change without prior notice + */ + +extern int idxd_pmd_logtype; + +#define IDXD_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + idxd_pmd_logtype, "IDXD: %s(): " fmt "\n", __func__, ##args) + +#define IDXD_PMD_DEBUG(fmt, args...) IDXD_PMD_LOG(DEBUG, fmt, ## args) +#define IDXD_PMD_INFO(fmt, args...) IDXD_PMD_LOG(INFO, fmt, ## args) +#define IDXD_PMD_ERR(fmt, args...) IDXD_PMD_LOG(ERR, fmt, ## args) +#define IDXD_PMD_WARN(fmt, args...) IDXD_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IDXD_INTERNAL_H_ */ diff --git a/drivers/dma/idxd/idxd_pci.c b/drivers/dma/idxd/idxd_pci.c new file mode 100644 index 0000000000..79e4aadcab --- /dev/null +++ b/drivers/dma/idxd/idxd_pci.c @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include + +#include "idxd_internal.h" + +#define IDXD_VENDOR_ID 0x8086 +#define IDXD_DEVICE_ID_SPR 0x0B25 + +#define IDXD_PMD_DMADEV_NAME_PCI dmadev_idxd_pci + +const struct rte_pci_id pci_id_idxd_map[] = { + { RTE_PCI_DEVICE(IDXD_VENDOR_ID, IDXD_DEVICE_ID_SPR) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static int +idxd_dmadev_probe_pci(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + int ret = 0; + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IDXD_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + dev->device.driver = &drv->driver; + + return ret; +} + +static int +idxd_dmadev_remove_pci(struct rte_pci_device *dev) +{ + char name[PCI_PRI_STR_SIZE]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IDXD_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + return 0; +} + +struct rte_pci_driver idxd_pmd_drv_pci = { + .id_table = pci_id_idxd_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING, + .probe = idxd_dmadev_probe_pci, + .remove = idxd_dmadev_remove_pci, +}; + +RTE_PMD_REGISTER_PCI(IDXD_PMD_DMADEV_NAME_PCI, idxd_pmd_drv_pci); +RTE_PMD_REGISTER_PCI_TABLE(IDXD_PMD_DMADEV_NAME_PCI, pci_id_idxd_map); +RTE_PMD_REGISTER_KMOD_DEP(IDXD_PMD_DMADEV_NAME_PCI, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(dmadev_idxd_pci, "max_queues=0"); diff --git a/drivers/dma/idxd/meson.build b/drivers/dma/idxd/meson.build new file mode 100644 index 0000000000..24c0f3106e --- /dev/null +++ b/drivers/dma/idxd/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2021 Intel Corporation + +deps += ['bus_pci'] +sources = files( + 'idxd_pci.c' +) \ No newline at end of file diff --git a/drivers/dma/idxd/version.map b/drivers/dma/idxd/version.map new file mode 100644 index 0000000000..4a76d1d52d --- /dev/null +++ b/drivers/dma/idxd/version.map @@ -0,0 +1,3 @@ +DPDK_21 { + local: *; +}; diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build index 0c2c34cd00..0b01b6a8ab 100644 --- a/drivers/dma/meson.build +++ b/drivers/dma/meson.build @@ -6,6 +6,7 @@ if is_windows endif drivers = [ + 'idxd', 'skeleton', ] std_deps = ['dmadev'] -- 2.30.2