From: Conor Walsh <conor.walsh@intel.com>
To: bruce.richardson@intel.com, thomas@monjalon.net,
fengchengwen@huawei.com, jerinj@marvell.com,
kevin.laatz@intel.com
Cc: dev@dpdk.org, Conor Walsh <conor.walsh@intel.com>
Subject: [dpdk-dev] [PATCH v8 06/12] dma/ioat: add data path job submission functions
Date: Mon, 18 Oct 2021 12:38:29 +0000 [thread overview]
Message-ID: <20211018123835.1080174-7-conor.walsh@intel.com> (raw)
In-Reply-To: <20211018123835.1080174-1-conor.walsh@intel.com>
Add data path functions for enqueuing and submitting operations to
IOAT devices.
Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/dmadevs/ioat.rst | 9 ++++
drivers/dma/ioat/ioat_dmadev.c | 92 ++++++++++++++++++++++++++++++++++
2 files changed, 101 insertions(+)
diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst
index df159f9957..9ee4e372a8 100644
--- a/doc/guides/dmadevs/ioat.rst
+++ b/doc/guides/dmadevs/ioat.rst
@@ -85,3 +85,12 @@ IOAT configuration requirements:
Once configured, the device can then be made ready for use by calling the
``rte_dma_start()`` API.
+
+Performing Data Copies
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Refer to the :ref:`Enqueue / Dequeue APIs <dmadev_enqueue_dequeue>` section of the dmadev library
+documentation for details on operation enqueue and submission API usage.
+
+It is expected that, for efficiency reasons, a burst of operations will be enqueued to the
+device via multiple enqueue calls between calls to the ``rte_dma_submit()`` function.
diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c
index cf28f4a7e6..4d00fec5c8 100644
--- a/drivers/dma/ioat/ioat_dmadev.c
+++ b/drivers/dma/ioat/ioat_dmadev.c
@@ -5,6 +5,7 @@
#include <rte_bus_pci.h>
#include <rte_dmadev_pmd.h>
#include <rte_malloc.h>
+#include <rte_prefetch.h>
#include "ioat_internal.h"
@@ -17,6 +18,12 @@ RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO);
#define IOAT_PMD_NAME dmadev_ioat
#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME)
+/* IOAT operations. */
+enum rte_ioat_ops {
+ ioat_op_copy = 0, /* Standard DMA Operation */
+ ioat_op_fill /* Block Fill */
+};
+
/* Configure a device. */
static int
ioat_dev_configure(struct rte_dma_dev *dev __rte_unused, const struct rte_dma_conf *dev_conf,
@@ -208,6 +215,87 @@ ioat_dev_close(struct rte_dma_dev *dev)
return 0;
}
+/* Trigger hardware to begin performing enqueued operations. */
+static inline void
+__submit(struct ioat_dmadev *ioat)
+{
+ *ioat->doorbell = ioat->next_write - ioat->offset;
+
+ ioat->last_write = ioat->next_write;
+}
+
+/* External submit function wrapper. */
+static int
+ioat_submit(void *dev_private, uint16_t qid __rte_unused)
+{
+ struct ioat_dmadev *ioat = dev_private;
+
+ __submit(ioat);
+
+ return 0;
+}
+
+/* Write descriptor for enqueue. */
+static inline int
+__write_desc(void *dev_private, uint32_t op, uint64_t src, phys_addr_t dst,
+ unsigned int length, uint64_t flags)
+{
+ struct ioat_dmadev *ioat = dev_private;
+ uint16_t ret;
+ const unsigned short mask = ioat->qcfg.nb_desc - 1;
+ const unsigned short read = ioat->next_read;
+ unsigned short write = ioat->next_write;
+ const unsigned short space = mask + read - write;
+ struct ioat_dma_hw_desc *desc;
+
+ if (space == 0)
+ return -ENOSPC;
+
+ ioat->next_write = write + 1;
+ write &= mask;
+
+ desc = &ioat->desc_ring[write];
+ desc->size = length;
+ desc->u.control_raw = (uint32_t)((op << IOAT_CMD_OP_SHIFT) |
+ (1 << IOAT_COMP_UPDATE_SHIFT));
+
+ /* In IOAT the fence ensures that all operations including the current one
+ * are completed before moving on, DMAdev assumes that the fence ensures
+ * all operations before the current one are completed before starting
+ * the current one, so in IOAT we set the fence for the previous descriptor.
+ */
+ if (flags & RTE_DMA_OP_FLAG_FENCE)
+ ioat->desc_ring[(write - 1) & mask].u.control.fence = 1;
+
+ desc->src_addr = src;
+ desc->dest_addr = dst;
+
+ rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]);
+
+ ret = (uint16_t)(ioat->next_write - 1);
+
+ if (flags & RTE_DMA_OP_FLAG_SUBMIT)
+ __submit(ioat);
+
+ return ret;
+}
+
+/* Enqueue a fill operation onto the ioat device. */
+static int
+ioat_enqueue_fill(void *dev_private, uint16_t qid __rte_unused, uint64_t pattern,
+ rte_iova_t dst, unsigned int length, uint64_t flags)
+{
+ return __write_desc(dev_private, ioat_op_fill, pattern, dst, length, flags);
+}
+
+/* Enqueue a copy operation onto the ioat device. */
+static int
+ioat_enqueue_copy(void *dev_private, uint16_t qid __rte_unused, rte_iova_t src,
+ rte_iova_t dst, unsigned int length, uint64_t flags)
+{
+ return __write_desc(dev_private, ioat_op_copy, src, dst, length, flags);
+}
+
/* Dump DMA device info. */
static int
__dev_dump(void *dev_private, FILE *f)
@@ -310,6 +398,10 @@ ioat_dmadev_create(const char *name, struct rte_pci_device *dev)
dmadev->dev_ops = &ioat_dmadev_ops;
+ dmadev->fp_obj->copy = ioat_enqueue_copy;
+ dmadev->fp_obj->fill = ioat_enqueue_fill;
+ dmadev->fp_obj->submit = ioat_submit;
+
ioat = dmadev->data->dev_private;
ioat->dmadev = dmadev;
ioat->regs = dev->mem_resource[0].addr;
--
2.25.1
next prev parent reply other threads:[~2021-10-18 12:39 UTC|newest]
Thread overview: 132+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-27 17:25 [dpdk-dev] [PATCH 0/8] dma: add dmadev driver for ioat devices Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 1/8] dma/ioat: add device probe and removal functions Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 2/8] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 3/8] dma/ioat: add datapath structures Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 4/8] dma/ioat: add configuration functions Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 5/8] dma/ioat: add start and stop functions Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 6/8] dma/ioat: add data path job submission functions Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 7/8] dma/ioat: add data path completion functions Conor Walsh
2021-08-27 17:25 ` [dpdk-dev] [PATCH 8/8] dma/ioat: add statistics Conor Walsh
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 00/10] dma: add dmadev driver for ioat devices Conor Walsh
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 01/10] dma/ioat: add device probe and removal functions Conor Walsh
2021-09-07 10:10 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 02/10] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-09-07 10:11 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 03/10] dma/ioat: add datapath structures Conor Walsh
2021-09-07 10:11 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 04/10] dma/ioat: add configuration functions Conor Walsh
2021-09-07 10:11 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 05/10] dma/ioat: add start and stop functions Conor Walsh
2021-09-07 10:11 ` Kevin Laatz
2021-09-07 10:35 ` Walsh, Conor
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 06/10] dma/ioat: add data path job submission functions Conor Walsh
2021-09-07 10:12 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 07/10] dma/ioat: add data path completion functions Conor Walsh
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 08/10] dma/ioat: add statistics Conor Walsh
2021-09-07 10:12 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 09/10] dma/ioat: add support for vchan idle function Conor Walsh
2021-09-07 10:12 ` Kevin Laatz
2021-09-03 11:17 ` [dpdk-dev] [PATCH v2 10/10] devbind: move ioat device ID for ICX to dmadev category Conor Walsh
2021-09-07 10:10 ` Kevin Laatz
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 00/11] dma: add dmadev driver for ioat devices Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 01/11] dma/ioat: add device probe and removal functions Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 02/11] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 03/11] dma/ioat: add datapath structures Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 04/11] dma/ioat: add configuration functions Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 05/11] dma/ioat: add start and stop functions Conor Walsh
2021-09-08 10:39 ` [dpdk-dev] [PATCH v3 06/11] dma/ioat: add data path job submission functions Conor Walsh
2021-09-08 10:40 ` [dpdk-dev] [PATCH v3 07/11] dma/ioat: add data path completion functions Conor Walsh
2021-09-08 10:40 ` [dpdk-dev] [PATCH v3 08/11] dma/ioat: add statistics Conor Walsh
2021-09-08 10:40 ` [dpdk-dev] [PATCH v3 09/11] dma/ioat: add support for vchan status function Conor Walsh
2021-09-08 10:40 ` [dpdk-dev] [PATCH v3 10/11] dma/ioat: add burst capacity function Conor Walsh
2021-09-08 10:40 ` [dpdk-dev] [PATCH v3 11/11] devbind: move ioat device ID for ICX to dmadev category Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 00/11] dma: add dmadev driver for ioat devices Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 01/11] dma/ioat: add device probe and removal functions Conor Walsh
2021-09-20 11:15 ` Bruce Richardson
2021-09-21 16:24 ` Conor Walsh
2021-09-22 3:59 ` fengchengwen
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 02/11] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-09-20 13:31 ` Bruce Richardson
2021-09-21 16:25 ` Conor Walsh
2021-09-22 8:04 ` fengchengwen
2021-09-22 16:40 ` Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 03/11] dma/ioat: add datapath structures Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 04/11] dma/ioat: add configuration functions Conor Walsh
2021-09-22 8:08 ` fengchengwen
2021-09-22 16:41 ` Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 05/11] dma/ioat: add start and stop functions Conor Walsh
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 06/11] dma/ioat: add data path job submission functions Conor Walsh
2021-09-20 13:36 ` Bruce Richardson
2021-09-21 16:25 ` Conor Walsh
2021-09-22 8:18 ` fengchengwen
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 07/11] dma/ioat: add data path completion functions Conor Walsh
2021-09-20 13:39 ` Bruce Richardson
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 08/11] dma/ioat: add statistics Conor Walsh
2021-09-20 13:48 ` Bruce Richardson
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 09/11] dma/ioat: add support for vchan status function Conor Walsh
2021-09-20 13:49 ` Bruce Richardson
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 10/11] dma/ioat: add burst capacity function Conor Walsh
2021-09-20 13:49 ` Bruce Richardson
2021-09-17 15:42 ` [dpdk-dev] [PATCH v4 11/11] devbind: move ioat device ID for ICX to dmadev category Conor Walsh
2021-09-20 13:53 ` Bruce Richardson
2021-09-21 16:26 ` Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 00/12] dma: add dmadev driver for ioat devices Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 01/12] dma/ioat: add device probe and removal functions Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 02/12] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 03/12] dma/ioat: add datapath structures Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 04/12] dma/ioat: add configuration functions Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 05/12] dma/ioat: add start and stop functions Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 06/12] dma/ioat: add data path job submission functions Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 07/12] dma/ioat: add data path completion functions Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 08/12] dma/ioat: add statistics Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 09/12] dma/ioat: add support for vchan status function Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 10/12] dma/ioat: add burst capacity function Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 11/12] devbind: move ioat device IDs to dmadev category Conor Walsh
2021-09-24 14:33 ` [dpdk-dev] [PATCH v5 12/12] raw/ioat: deprecate ioat rawdev driver Conor Walsh
2021-09-24 16:21 ` Kevin Laatz
2021-09-24 16:56 ` Bruce Richardson
2021-09-27 9:36 ` Walsh, Conor
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 00/12] dma: add dmadev driver for ioat devices Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 01/12] dma/ioat: add device probe and removal functions Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 02/12] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 03/12] dma/ioat: add datapath structures Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 04/12] dma/ioat: add configuration functions Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 05/12] dma/ioat: add start and stop functions Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 06/12] dma/ioat: add data path job submission functions Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 07/12] dma/ioat: add data path completion functions Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 08/12] dma/ioat: add statistics Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 09/12] dma/ioat: add support for vchan status function Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 10/12] dma/ioat: add burst capacity function Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 11/12] devbind: move ioat device IDs to dmadev category Conor Walsh
2021-09-27 10:21 ` [dpdk-dev] [PATCH v6 12/12] raw/ioat: deprecate ioat rawdev driver Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 00/12] dma: add dmadev driver for ioat devices Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 01/12] dma/ioat: add device probe and removal functions Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 02/12] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 03/12] dma/ioat: add datapath structures Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 04/12] dma/ioat: add configuration functions Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 05/12] dma/ioat: add start and stop functions Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 06/12] dma/ioat: add data path job submission functions Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 07/12] dma/ioat: add data path completion functions Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 08/12] dma/ioat: add statistics Conor Walsh
2021-10-14 9:48 ` [dpdk-dev] [PATCH v7 09/12] dma/ioat: add support for vchan status function Conor Walsh
2021-10-14 9:49 ` [dpdk-dev] [PATCH v7 10/12] dma/ioat: add burst capacity function Conor Walsh
2021-10-14 9:49 ` [dpdk-dev] [PATCH v7 11/12] devbind: move ioat device IDs to dmadev category Conor Walsh
2021-10-14 9:49 ` [dpdk-dev] [PATCH v7 12/12] raw/ioat: deprecate ioat rawdev driver Conor Walsh
2021-10-18 7:54 ` Thomas Monjalon
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 00/12] dma: add dmadev driver for ioat devices Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 01/12] dma/ioat: add device probe and removal functions Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 02/12] dma/ioat: create dmadev instances on PCI probe Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 03/12] dma/ioat: add datapath structures Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 04/12] dma/ioat: add configuration functions Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 05/12] dma/ioat: add start and stop functions Conor Walsh
2021-10-18 12:38 ` Conor Walsh [this message]
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 07/12] dma/ioat: add data path completion functions Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 08/12] dma/ioat: add statistics Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 09/12] dma/ioat: add support for vchan status function Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 10/12] dma/ioat: add burst capacity function Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 11/12] devbind: move ioat device IDs to dmadev category Conor Walsh
2021-10-18 12:38 ` [dpdk-dev] [PATCH v8 12/12] raw/ioat: deprecate ioat rawdev driver Conor Walsh
2021-10-19 11:28 ` [dpdk-dev] [PATCH v8 00/12] dma: add dmadev driver for ioat devices Walsh, Conor
2021-10-19 14:23 ` Walsh, Conor
2021-10-20 16:35 ` Walsh, Conor
2021-10-22 19:25 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211018123835.1080174-7-conor.walsh@intel.com \
--to=conor.walsh@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=fengchengwen@huawei.com \
--cc=jerinj@marvell.com \
--cc=kevin.laatz@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).