From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A951AA0C54; Fri, 3 Sep 2021 13:17:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AC443410FF; Fri, 3 Sep 2021 13:17:43 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id C3315410F2 for ; Fri, 3 Sep 2021 13:17:41 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10095"; a="217533930" X-IronPort-AV: E=Sophos;i="5.85,265,1624345200"; d="scan'208";a="217533930" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Sep 2021 04:17:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,265,1624345200"; d="scan'208";a="500322802" Received: from silpixa00401160.ir.intel.com ([10.55.128.248]) by fmsmga008.fm.intel.com with ESMTP; 03 Sep 2021 04:17:39 -0700 From: Conor Walsh To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com, kevin.laatz@intel.com Cc: dev@dpdk.org, Conor Walsh Date: Fri, 3 Sep 2021 11:17:25 +0000 Message-Id: <20210903111734.2734545-2-conor.walsh@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210903111734.2734545-1-conor.walsh@intel.com> References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210903111734.2734545-1-conor.walsh@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 01/10] dma/ioat: add device probe and removal functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the basic device probe/remove skeleton code and initial documentation for new IOAT DMA driver. Maintainers update is also included in this patch. Signed-off-by: Conor Walsh --- MAINTAINERS | 6 +++ doc/guides/dmadevs/index.rst | 1 + doc/guides/dmadevs/ioat.rst | 64 ++++++++++++++++++++++++ doc/guides/rel_notes/release_21_11.rst | 7 +-- drivers/dma/ioat/ioat_dmadev.c | 69 ++++++++++++++++++++++++++ drivers/dma/ioat/ioat_hw_defs.h | 35 +++++++++++++ drivers/dma/ioat/ioat_internal.h | 20 ++++++++ drivers/dma/ioat/meson.build | 7 +++ drivers/dma/ioat/version.map | 3 ++ drivers/dma/meson.build | 1 + 10 files changed, 210 insertions(+), 3 deletions(-) create mode 100644 doc/guides/dmadevs/ioat.rst create mode 100644 drivers/dma/ioat/ioat_dmadev.c create mode 100644 drivers/dma/ioat/ioat_hw_defs.h create mode 100644 drivers/dma/ioat/ioat_internal.h create mode 100644 drivers/dma/ioat/meson.build create mode 100644 drivers/dma/ioat/version.map diff --git a/MAINTAINERS b/MAINTAINERS index b4c614a229..00b319c811 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1208,6 +1208,12 @@ M: Kevin Laatz F: drivers/dma/idxd/ F: doc/guides/dmadevs/idxd.rst +Intel IOAT - EXPERIMENTAL +M: Bruce Richardson +M: Conor Walsh +F: drivers/dma/ioat/ +F: doc/guides/dmadevs/ioat.rst + RegEx Drivers ------------- diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst index b30004fd65..0b01493c5a 100644 --- a/doc/guides/dmadevs/index.rst +++ b/doc/guides/dmadevs/index.rst @@ -12,3 +12,4 @@ application through DMAdev API. :numbered: idxd + ioat diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst new file mode 100644 index 0000000000..45a2e65d70 --- /dev/null +++ b/doc/guides/dmadevs/ioat.rst @@ -0,0 +1,64 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2021 Intel Corporation. + +.. include:: + +IOAT DMA Device Driver +======================= + +The ``ioat`` dmadev driver provides a poll-mode driver (PMD) for Intel\ +|reg| QuickData Technology which is part of part of Intel\ |reg| I/O +Acceleration Technology (`Intel I/OAT +`_). +This PMD, when used on supported hardware, allows data copies, for example, +cloning packet data, to be accelerated by IOAT hardware rather than having to +be done by software, freeing up CPU cycles for other tasks. + +Hardware Requirements +---------------------- + +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` +will show all the DMA devices on the system, IOAT devices are included in this +list. For Intel\ |reg| IOAT devices, the hardware will often be listed as +"Crystal Beach DMA", or "CBDMA" or on some newer systems '0b00' due to the +absence of pci-id database entries for them at this point. + +Compilation +------------ + +For builds using ``meson`` and ``ninja``, the driver will be built when the +target platform is x86-based. No additional compilation steps are necessary. + +Device Setup +------------- + +Intel\ |reg| IOAT devices will need to be bound to a suitable DPDK-supported +user-space IO driver such as ``vfio-pci`` in order to be used by DPDK. + +The ``dpdk-devbind.py`` script can be used to view the state of the devices using:: + + $ dpdk-devbind.py --status-dev dma + +The ``dpdk-devbind.py`` script can also be used to bind devices to a suitable driver. +For example:: + + $ dpdk-devbind.py -b vfio-pci 00:01.0 00:01.1 + +Device Probing and Initialization +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For devices bound to a suitable DPDK-supported driver (``vfio-pci``), the HW +devices will be found as part of the device scan done at application +initialization time without the need to pass parameters to the application. + +If the application does not require all the devices available an allowlist can +be used in the same way that other DPDK devices use them. + +For example:: + + $ dpdk-test -a + +Once probed successfully, the device will appear as a ``dmadev``, that is a +"DMA device type" inside DPDK, and can be accessed using APIs from the +``rte_dmadev`` library. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index f7b0fcf3fd..6ce33e2787 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -60,10 +60,11 @@ New Features The dmadev library provides a DMA device framework for management and provision of hardware and software DMA devices. -* **Added IDXD dmadev driver implementation.** +* **Added Intel dmadev driver implementations.** - The IDXD dmadev driver provide device drivers for the Intel DSA devices. - This device driver can be used through the generic dmadev API. + The IDXD and IOAT dmadev drivers provide device drivers for Intel DSA + and IOAT devices. These device drivers can be used through the generic + dmadev API. Removed Items diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c new file mode 100644 index 0000000000..f3491d45b1 --- /dev/null +++ b/drivers/dma/ioat/ioat_dmadev.c @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include +#include + +#include "ioat_internal.h" + +static struct rte_pci_driver ioat_pmd_drv; + +RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); + +#define IOAT_PMD_NAME dmadev_ioat +#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) + +/* Probe DMA device. */ +static int +ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) +{ + char name[32]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); + + dev->device.driver = &drv->driver; + return 0; +} + +/* Remove DMA device. */ +static int +ioat_dmadev_remove(struct rte_pci_device *dev) +{ + char name[32]; + + rte_pci_device_name(&dev->addr, name, sizeof(name)); + + IOAT_PMD_INFO("Closing %s on NUMA node %d", + name, dev->device.numa_node); + + return 0; +} + +static const struct rte_pci_id pci_id_ioat_map[] = { + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_SKX) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX0) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX1) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX2) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX3) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX4) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX5) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX6) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX7) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXE) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXF) }, + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_ICX) }, + { .vendor_id = 0, /* sentinel */ }, +}; + +static struct rte_pci_driver ioat_pmd_drv = { + .id_table = pci_id_ioat_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, + .probe = ioat_dmadev_probe, + .remove = ioat_dmadev_remove, +}; + +RTE_PMD_REGISTER_PCI(IOAT_PMD_NAME, ioat_pmd_drv); +RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_NAME, pci_id_ioat_map); +RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci"); diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h new file mode 100644 index 0000000000..eeabba41ef --- /dev/null +++ b/drivers/dma/ioat/ioat_hw_defs.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#ifndef IOAT_HW_DEFS_H +#define IOAT_HW_DEFS_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#define IOAT_VER_3_0 0x30 +#define IOAT_VER_3_3 0x33 + +#define IOAT_VENDOR_ID 0x8086 +#define IOAT_DEVICE_ID_SKX 0x2021 +#define IOAT_DEVICE_ID_BDX0 0x6f20 +#define IOAT_DEVICE_ID_BDX1 0x6f21 +#define IOAT_DEVICE_ID_BDX2 0x6f22 +#define IOAT_DEVICE_ID_BDX3 0x6f23 +#define IOAT_DEVICE_ID_BDX4 0x6f24 +#define IOAT_DEVICE_ID_BDX5 0x6f25 +#define IOAT_DEVICE_ID_BDX6 0x6f26 +#define IOAT_DEVICE_ID_BDX7 0x6f27 +#define IOAT_DEVICE_ID_BDXE 0x6f2E +#define IOAT_DEVICE_ID_BDXF 0x6f2F +#define IOAT_DEVICE_ID_ICX 0x0b00 + +#ifdef __cplusplus +} +#endif + +#endif /* IOAT_HW_DEFS_H */ diff --git a/drivers/dma/ioat/ioat_internal.h b/drivers/dma/ioat/ioat_internal.h new file mode 100644 index 0000000000..f1ec12a919 --- /dev/null +++ b/drivers/dma/ioat/ioat_internal.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2021 Intel Corporation + */ + +#ifndef _IOAT_INTERNAL_H_ +#define _IOAT_INTERNAL_H_ + +#include "ioat_hw_defs.h" + +extern int ioat_pmd_logtype; + +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ + ioat_pmd_logtype, "IOAT: %s(): " fmt "\n", __func__, ##args) + +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) + +#endif /* _IOAT_INTERNAL_H_ */ diff --git a/drivers/dma/ioat/meson.build b/drivers/dma/ioat/meson.build new file mode 100644 index 0000000000..d67fac96fb --- /dev/null +++ b/drivers/dma/ioat/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2021 Intel Corporation + +build = dpdk_conf.has('RTE_ARCH_X86') +reason = 'only supported on x86' +sources = files('ioat_dmadev.c') +deps += ['bus_pci', 'dmadev'] diff --git a/drivers/dma/ioat/version.map b/drivers/dma/ioat/version.map new file mode 100644 index 0000000000..c2e0723b4c --- /dev/null +++ b/drivers/dma/ioat/version.map @@ -0,0 +1,3 @@ +DPDK_22 { + local: *; +}; diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build index 0b01b6a8ab..f8f5154a40 100644 --- a/drivers/dma/meson.build +++ b/drivers/dma/meson.build @@ -7,6 +7,7 @@ endif drivers = [ 'idxd', + 'ioat', 'skeleton', ] std_deps = ['dmadev'] -- 2.25.1