From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7AD12A0C45; Wed, 22 Sep 2021 05:59:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DAF1B4003F; Wed, 22 Sep 2021 05:59:44 +0200 (CEST) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id 5B2294003C for ; Wed, 22 Sep 2021 05:59:42 +0200 (CEST) Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4HDkvv5YZvz8x35; Wed, 22 Sep 2021 11:55:07 +0800 (CST) Received: from dggpeml500024.china.huawei.com (7.185.36.10) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 11:59:40 +0800 Received: from [10.40.190.165] (10.40.190.165) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 11:59:39 +0800 To: Conor Walsh , , , CC: References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210917154227.737554-1-conor.walsh@intel.com> <20210917154227.737554-2-conor.walsh@intel.com> From: fengchengwen Message-ID: Date: Wed, 22 Sep 2021 11:59:39 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <20210917154227.737554-2-conor.walsh@intel.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.40.190.165] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] [PATCH v4 01/11] dma/ioat: add device probe and removal functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Reviewed-by: Chengwen Feng On 2021/9/17 23:42, Conor Walsh wrote: > Add the basic device probe/remove skeleton code and initial documentation > for new IOAT DMA driver. Maintainers update is also included in this > patch. > > Signed-off-by: Conor Walsh > Reviewed-by: Kevin Laatz > --- > MAINTAINERS | 6 +++ > doc/guides/dmadevs/index.rst | 2 + > doc/guides/dmadevs/ioat.rst | 64 ++++++++++++++++++++++++ > doc/guides/rel_notes/release_21_11.rst | 7 +-- > drivers/dma/ioat/ioat_dmadev.c | 69 ++++++++++++++++++++++++++ > drivers/dma/ioat/ioat_hw_defs.h | 35 +++++++++++++ > drivers/dma/ioat/ioat_internal.h | 20 ++++++++ > drivers/dma/ioat/meson.build | 7 +++ > drivers/dma/ioat/version.map | 3 ++ > drivers/dma/meson.build | 1 + > 10 files changed, 211 insertions(+), 3 deletions(-) > create mode 100644 doc/guides/dmadevs/ioat.rst > create mode 100644 drivers/dma/ioat/ioat_dmadev.c > create mode 100644 drivers/dma/ioat/ioat_hw_defs.h > create mode 100644 drivers/dma/ioat/ioat_internal.h > create mode 100644 drivers/dma/ioat/meson.build > create mode 100644 drivers/dma/ioat/version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index 9cb59b831d..70993d23e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1209,6 +1209,12 @@ M: Kevin Laatz > F: drivers/dma/idxd/ > F: doc/guides/dmadevs/idxd.rst > > +Intel IOAT - EXPERIMENTAL > +M: Bruce Richardson > +M: Conor Walsh > +F: drivers/dma/ioat/ > +F: doc/guides/dmadevs/ioat.rst > + > > RegEx Drivers > ------------- > diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst > index 5d4abf880e..c59f4b5c92 100644 > --- a/doc/guides/dmadevs/index.rst > +++ b/doc/guides/dmadevs/index.rst > @@ -12,3 +12,5 @@ an application through DMA API. > :numbered: > > idxd > + ioat > + > diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst > new file mode 100644 > index 0000000000..45a2e65d70 > --- /dev/null > +++ b/doc/guides/dmadevs/ioat.rst > @@ -0,0 +1,64 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2021 Intel Corporation. > + > +.. include:: > + > +IOAT DMA Device Driver > +======================= > + > +The ``ioat`` dmadev driver provides a poll-mode driver (PMD) for Intel\ > +|reg| QuickData Technology which is part of part of Intel\ |reg| I/O > +Acceleration Technology (`Intel I/OAT > +`_). > +This PMD, when used on supported hardware, allows data copies, for example, > +cloning packet data, to be accelerated by IOAT hardware rather than having to > +be done by software, freeing up CPU cycles for other tasks. > + > +Hardware Requirements > +---------------------- > + > +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the > +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` > +will show all the DMA devices on the system, IOAT devices are included in this > +list. For Intel\ |reg| IOAT devices, the hardware will often be listed as > +"Crystal Beach DMA", or "CBDMA" or on some newer systems '0b00' due to the > +absence of pci-id database entries for them at this point. > + > +Compilation > +------------ > + > +For builds using ``meson`` and ``ninja``, the driver will be built when the > +target platform is x86-based. No additional compilation steps are necessary. > + > +Device Setup > +------------- > + > +Intel\ |reg| IOAT devices will need to be bound to a suitable DPDK-supported > +user-space IO driver such as ``vfio-pci`` in order to be used by DPDK. > + > +The ``dpdk-devbind.py`` script can be used to view the state of the devices using:: > + > + $ dpdk-devbind.py --status-dev dma > + > +The ``dpdk-devbind.py`` script can also be used to bind devices to a suitable driver. > +For example:: > + > + $ dpdk-devbind.py -b vfio-pci 00:01.0 00:01.1 > + > +Device Probing and Initialization > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +For devices bound to a suitable DPDK-supported driver (``vfio-pci``), the HW > +devices will be found as part of the device scan done at application > +initialization time without the need to pass parameters to the application. > + > +If the application does not require all the devices available an allowlist can > +be used in the same way that other DPDK devices use them. > + > +For example:: > + > + $ dpdk-test -a > + > +Once probed successfully, the device will appear as a ``dmadev``, that is a > +"DMA device type" inside DPDK, and can be accessed using APIs from the > +``rte_dmadev`` library. > diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst > index c0bfd9c1ba..4d2b7bde1b 100644 > --- a/doc/guides/rel_notes/release_21_11.rst > +++ b/doc/guides/rel_notes/release_21_11.rst > @@ -92,10 +92,11 @@ New Features > * Device allocation and it's multi-process support. > * Control and data plane functions. > > -* **Added IDXD dmadev driver implementation.** > +* **Added Intel dmadev driver implementations.** > > - The IDXD dmadev driver provide device drivers for the Intel DSA devices. > - This device driver can be used through the generic dmadev API. > + The IDXD and IOAT dmadev drivers provide device drivers for Intel DSA > + and IOAT devices. These device drivers can be used through the generic > + dmadev API. > > > Removed Items > diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c > new file mode 100644 > index 0000000000..f3491d45b1 > --- /dev/null > +++ b/drivers/dma/ioat/ioat_dmadev.c > @@ -0,0 +1,69 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2021 Intel Corporation > + */ > + > +#include > +#include > + > +#include "ioat_internal.h" > + > +static struct rte_pci_driver ioat_pmd_drv; > + > +RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); > + > +#define IOAT_PMD_NAME dmadev_ioat > +#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) > + > +/* Probe DMA device. */ > +static int > +ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) > +{ > + char name[32]; > + > + rte_pci_device_name(&dev->addr, name, sizeof(name)); > + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); > + > + dev->device.driver = &drv->driver; > + return 0; > +} > + > +/* Remove DMA device. */ > +static int > +ioat_dmadev_remove(struct rte_pci_device *dev) > +{ > + char name[32]; > + > + rte_pci_device_name(&dev->addr, name, sizeof(name)); > + > + IOAT_PMD_INFO("Closing %s on NUMA node %d", > + name, dev->device.numa_node); > + > + return 0; > +} > + > +static const struct rte_pci_id pci_id_ioat_map[] = { > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_SKX) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX0) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX1) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX2) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX3) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX4) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX5) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX6) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX7) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXE) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXF) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_ICX) }, > + { .vendor_id = 0, /* sentinel */ }, > +}; > + > +static struct rte_pci_driver ioat_pmd_drv = { > + .id_table = pci_id_ioat_map, > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, > + .probe = ioat_dmadev_probe, > + .remove = ioat_dmadev_remove, > +}; > + > +RTE_PMD_REGISTER_PCI(IOAT_PMD_NAME, ioat_pmd_drv); > +RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_NAME, pci_id_ioat_map); > +RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_NAME, "* igb_uio | uio_pci_generic | vfio-pci"); > diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h > new file mode 100644 > index 0000000000..eeabba41ef > --- /dev/null > +++ b/drivers/dma/ioat/ioat_hw_defs.h > @@ -0,0 +1,35 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2021 Intel Corporation > + */ > + > +#ifndef IOAT_HW_DEFS_H > +#define IOAT_HW_DEFS_H > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include > + > +#define IOAT_VER_3_0 0x30 > +#define IOAT_VER_3_3 0x33 > + > +#define IOAT_VENDOR_ID 0x8086 > +#define IOAT_DEVICE_ID_SKX 0x2021 > +#define IOAT_DEVICE_ID_BDX0 0x6f20 > +#define IOAT_DEVICE_ID_BDX1 0x6f21 > +#define IOAT_DEVICE_ID_BDX2 0x6f22 > +#define IOAT_DEVICE_ID_BDX3 0x6f23 > +#define IOAT_DEVICE_ID_BDX4 0x6f24 > +#define IOAT_DEVICE_ID_BDX5 0x6f25 > +#define IOAT_DEVICE_ID_BDX6 0x6f26 > +#define IOAT_DEVICE_ID_BDX7 0x6f27 > +#define IOAT_DEVICE_ID_BDXE 0x6f2E > +#define IOAT_DEVICE_ID_BDXF 0x6f2F > +#define IOAT_DEVICE_ID_ICX 0x0b00 > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* IOAT_HW_DEFS_H */ > diff --git a/drivers/dma/ioat/ioat_internal.h b/drivers/dma/ioat/ioat_internal.h > new file mode 100644 > index 0000000000..f1ec12a919 > --- /dev/null > +++ b/drivers/dma/ioat/ioat_internal.h > @@ -0,0 +1,20 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2021 Intel Corporation > + */ > + > +#ifndef _IOAT_INTERNAL_H_ > +#define _IOAT_INTERNAL_H_ > + > +#include "ioat_hw_defs.h" > + > +extern int ioat_pmd_logtype; > + > +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ > + ioat_pmd_logtype, "IOAT: %s(): " fmt "\n", __func__, ##args) > + > +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) > +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) > +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) > +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) > + > +#endif /* _IOAT_INTERNAL_H_ */ > diff --git a/drivers/dma/ioat/meson.build b/drivers/dma/ioat/meson.build > new file mode 100644 > index 0000000000..d67fac96fb > --- /dev/null > +++ b/drivers/dma/ioat/meson.build > @@ -0,0 +1,7 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2021 Intel Corporation > + > +build = dpdk_conf.has('RTE_ARCH_X86') > +reason = 'only supported on x86' > +sources = files('ioat_dmadev.c') > +deps += ['bus_pci', 'dmadev'] > diff --git a/drivers/dma/ioat/version.map b/drivers/dma/ioat/version.map > new file mode 100644 > index 0000000000..c2e0723b4c > --- /dev/null > +++ b/drivers/dma/ioat/version.map > @@ -0,0 +1,3 @@ > +DPDK_22 { > + local: *; > +}; > diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build > index 411be7a240..a69418ce9b 100644 > --- a/drivers/dma/meson.build > +++ b/drivers/dma/meson.build > @@ -3,6 +3,7 @@ > > drivers = [ > 'idxd', > + 'ioat', > 'skeleton', > ] > std_deps = ['dmadev'] >