From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 2C3B7A0548;
Fri, 24 Sep 2021 16:33:51 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id BDDD841326;
Fri, 24 Sep 2021 16:33:45 +0200 (CEST)
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
by mails.dpdk.org (Postfix) with ESMTP id F0DEE411FA
for <dev@dpdk.org>; Fri, 24 Sep 2021 16:33:41 +0200 (CEST)
X-IronPort-AV: E=McAfee;i="6200,9189,10116"; a="309640124"
X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="309640124"
Received: from fmsmga006.fm.intel.com ([10.253.24.20])
by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
24 Sep 2021 07:33:41 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.85,320,1624345200"; d="scan'208";a="703871419"
Received: from silpixa00401160.ir.intel.com ([10.55.129.96])
by fmsmga006.fm.intel.com with ESMTP; 24 Sep 2021 07:33:39 -0700
From: Conor Walsh <conor.walsh@intel.com>
To: bruce.richardson@intel.com, fengchengwen@huawei.com, jerinj@marvell.com,
kevin.laatz@intel.com
Cc: dev@dpdk.org,
Conor Walsh <conor.walsh@intel.com>
Date: Fri, 24 Sep 2021 14:33:24 +0000
Message-Id: <20210924143335.1092300-2-conor.walsh@intel.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20210924143335.1092300-1-conor.walsh@intel.com>
References: <20210827172550.1522362-1-conor.walsh@intel.com>
<20210924143335.1092300-1-conor.walsh@intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Subject: [dpdk-dev] [PATCH v5 01/12] dma/ioat: add device probe and removal
functions
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Add the basic device probe/remove skeleton code and initial documentation
for new IOAT DMA driver. Maintainers update is also included in this
patch.
Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
---
MAINTAINERS | 6 +++
doc/guides/dmadevs/index.rst | 2 +
doc/guides/dmadevs/ioat.rst | 69 ++++++++++++++++++++++++++
doc/guides/rel_notes/release_21_11.rst | 6 +++
drivers/dma/ioat/ioat_dmadev.c | 69 ++++++++++++++++++++++++++
drivers/dma/ioat/ioat_hw_defs.h | 35 +++++++++++++
drivers/dma/ioat/ioat_internal.h | 20 ++++++++
drivers/dma/ioat/meson.build | 7 +++
drivers/dma/ioat/version.map | 3 ++
drivers/dma/meson.build | 1 +
10 files changed, 218 insertions(+)
create mode 100644 doc/guides/dmadevs/ioat.rst
create mode 100644 drivers/dma/ioat/ioat_dmadev.c
create mode 100644 drivers/dma/ioat/ioat_hw_defs.h
create mode 100644 drivers/dma/ioat/ioat_internal.h
create mode 100644 drivers/dma/ioat/meson.build
create mode 100644 drivers/dma/ioat/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 497219e948..ccabba9169 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1209,6 +1209,12 @@ M: Kevin Laatz <kevin.laatz@intel.com>
F: drivers/dma/idxd/
F: doc/guides/dmadevs/idxd.rst
+Intel IOAT
+M: Bruce Richardson <bruce.richardson@intel.com>
+M: Conor Walsh <conor.walsh@intel.com>
+F: drivers/dma/ioat/
+F: doc/guides/dmadevs/ioat.rst
+
RegEx Drivers
-------------
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
index 5d4abf880e..c59f4b5c92 100644
--- a/doc/guides/dmadevs/index.rst
+++ b/doc/guides/dmadevs/index.rst
@@ -12,3 +12,5 @@ an application through DMA API.
:numbered:
idxd
+ ioat
+
diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst
new file mode 100644
index 0000000000..9ae1d8a2ad
--- /dev/null
+++ b/doc/guides/dmadevs/ioat.rst
@@ -0,0 +1,69 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2021 Intel Corporation.
+
+.. include:: <isonum.txt>
+
+IOAT DMA Device Driver
+=======================
+
+The ``ioat`` dmadev driver provides a poll-mode driver (PMD) for Intel\
+|reg| QuickData Technology which is part of part of Intel\ |reg| I/O
+Acceleration Technology (`Intel I/OAT
+<https://www.intel.com/content/www/us/en/wireless-network/accel-technology.html>`_).
+This PMD, when used on supported hardware, allows data copies, for example,
+cloning packet data, to be accelerated by IOAT hardware rather than having to
+be done by software, freeing up CPU cycles for other tasks.
+
+Hardware Requirements
+----------------------
+
+The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the
+presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma``
+will show all the DMA devices on the system, IOAT devices are included in this
+list. For Intel\ |reg| IOAT devices, the hardware will often be listed as
+"Crystal Beach DMA", or "CBDMA" or on some newer systems '0b00' due to the
+absence of pci-id database entries for them at this point.
+
+.. note::
+ Error handling is not supported by this driver on hardware prior to
+ Intel Ice Lake. Unsupported systems include Broadwell, Skylake and
+ Cascade Lake.
+
+Compilation
+------------
+
+For builds using ``meson`` and ``ninja``, the driver will be built when the
+target platform is x86-based. No additional compilation steps are necessary.
+
+Device Setup
+-------------
+
+Intel\ |reg| IOAT devices will need to be bound to a suitable DPDK-supported
+user-space IO driver such as ``vfio-pci`` in order to be used by DPDK.
+
+The ``dpdk-devbind.py`` script can be used to view the state of the devices using::
+
+ $ dpdk-devbind.py --status-dev dma
+
+The ``dpdk-devbind.py`` script can also be used to bind devices to a suitable driver.
+For example::
+
+ $ dpdk-devbind.py -b vfio-pci 00:01.0 00:01.1
+
+Device Probing and Initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For devices bound to a suitable DPDK-supported driver (``vfio-pci``), the HW
+devices will be found as part of the device scan done at application
+initialization time without the need to pass parameters to the application.
+
+If the application does not require all the devices available an allowlist can
+be used in the same way that other DPDK devices use them.
+
+For example::
+
+ $ dpdk-test -a <b:d:f>
+
+Once probed successfully, the device will appear as a ``dmadev``, that is a
+"DMA device type" inside DPDK, and can be accessed using APIs from the
+``rte_dmadev`` library.
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index c980e729f8..e34957069f 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -103,6 +103,12 @@ New Features
The IDXD dmadev driver provide device drivers for the Intel DSA devices.
This device driver can be used through the generic dmadev API.
+* **Added IOAT dmadev driver implementation.**
+
+ The Intel I/O Acceleration Technology (IOAT) dmadev driver provides a device
+ driver for Intel IOAT devices such as Crystal Beach DMA (CBDMA) on Ice Lake,
+ Skylake and Broadwell. This device driver can be used through the generic dmadev API.
+
Removed Items
-------------
diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c
new file mode 100644
index 0000000000..f3491d45b1
--- /dev/null
+++ b/drivers/dma/ioat/ioat_dmadev.c
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#include <rte_bus_pci.h>
+#include <rte_dmadev_pmd.h>
+
+#include "ioat_internal.h"
+
+static struct rte_pci_driver ioat_pmd_drv;
+
+RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO);
+
+#define IOAT_PMD_NAME dmadev_ioat
+#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME)
+
+/* Probe DMA device. */
+static int
+ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev)
+{
+ char name[32];
+
+ rte_pci_device_name(&dev->addr, name, sizeof(name));
+ IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node);
+
+ dev->device.driver = &drv->driver;
+ return 0;
+}
+
+/* Remove DMA device. */
+static int
+ioat_dmadev_remove(struct rte_pci_device *dev)
+{
+ char name[32];
+
+ rte_pci_device_name(&dev->addr, name, sizeof(name));
+
+ IOAT_PMD_INFO("Closing %s on NUMA node %d",
+ name, dev->device.numa_node);
+
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_ioat_map[] = {
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_SKX) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX0) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX1) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX2) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX3) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX4) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX5) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX6) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX7) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXE) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXF) },
+ { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_ICX) },
+ { .vendor_id = 0, /* sentinel */ },
+};
+
+static struct rte_pci_driver ioat_pmd_drv = {
+ .id_table = pci_id_ioat_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC,
+ .probe = ioat_dmadev_probe,
+ .remove = ioat_dmadev_remove,
+};
+
+RTE_PMD_REGISTER_PCI(IOAT_PMD_NAME, ioat_pmd_drv);
+RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_NAME, pci_id_ioat_map);
+RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci");
diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h
new file mode 100644
index 0000000000..eeabba41ef
--- /dev/null
+++ b/drivers/dma/ioat/ioat_hw_defs.h
@@ -0,0 +1,35 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 Intel Corporation
+ */
+
+#ifndef IOAT_HW_DEFS_H
+#define IOAT_HW_DEFS_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <stdint.h>
+
+#define IOAT_VER_3_0 0x30
+#define IOAT_VER_3_3 0x33
+
+#define IOAT_VENDOR_ID 0x8086
+#define IOAT_DEVICE_ID_SKX 0x2021
+#define IOAT_DEVICE_ID_BDX0 0x6f20
+#define IOAT_DEVICE_ID_BDX1 0x6f21
+#define IOAT_DEVICE_ID_BDX2 0x6f22
+#define IOAT_DEVICE_ID_BDX3 0x6f23
+#define IOAT_DEVICE_ID_BDX4 0x6f24
+#define IOAT_DEVICE_ID_BDX5 0x6f25
+#define IOAT_DEVICE_ID_BDX6 0x6f26
+#define IOAT_DEVICE_ID_BDX7 0x6f27
+#define IOAT_DEVICE_ID_BDXE 0x6f2E
+#define IOAT_DEVICE_ID_BDXF 0x6f2F
+#define IOAT_DEVICE_ID_ICX 0x0b00
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* IOAT_HW_DEFS_H */
diff --git a/drivers/dma/ioat/ioat_internal.h b/drivers/dma/ioat/ioat_internal.h
new file mode 100644
index 0000000000..f1ec12a919
--- /dev/null
+++ b/drivers/dma/ioat/ioat_internal.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2021 Intel Corporation
+ */
+
+#ifndef _IOAT_INTERNAL_H_
+#define _IOAT_INTERNAL_H_
+
+#include "ioat_hw_defs.h"
+
+extern int ioat_pmd_logtype;
+
+#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \
+ ioat_pmd_logtype, "IOAT: %s(): " fmt "\n", __func__, ##args)
+
+#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args)
+#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args)
+#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args)
+#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args)
+
+#endif /* _IOAT_INTERNAL_H_ */
diff --git a/drivers/dma/ioat/meson.build b/drivers/dma/ioat/meson.build
new file mode 100644
index 0000000000..d67fac96fb
--- /dev/null
+++ b/drivers/dma/ioat/meson.build
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2021 Intel Corporation
+
+build = dpdk_conf.has('RTE_ARCH_X86')
+reason = 'only supported on x86'
+sources = files('ioat_dmadev.c')
+deps += ['bus_pci', 'dmadev']
diff --git a/drivers/dma/ioat/version.map b/drivers/dma/ioat/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/dma/ioat/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
index 411be7a240..a69418ce9b 100644
--- a/drivers/dma/meson.build
+++ b/drivers/dma/meson.build
@@ -3,6 +3,7 @@
drivers = [
'idxd',
+ 'ioat',
'skeleton',
]
std_deps = ['dmadev']
--
2.25.1