From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B185CA04C0; Fri, 25 Sep 2020 13:14:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BF41C1E993; Fri, 25 Sep 2020 13:10:59 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id AC7291E98A for ; Fri, 25 Sep 2020 13:10:56 +0200 (CEST) IronPort-SDR: ATp45IMs+zEswKL8mzfomrqht4OZKEAIKElBJ2PidgNG3IFPaEoSuwHjz24qUyIGodVZe/DCAd I1erKZjUtkpQ== X-IronPort-AV: E=McAfee;i="6000,8403,9754"; a="225651754" X-IronPort-AV: E=Sophos;i="5.77,301,1596524400"; d="scan'208";a="225651754" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 04:10:53 -0700 IronPort-SDR: pMfqRe4XF4YQbWQJzDNUsmTcaF9hH1jC8XmJdQOIES3tcGRz7UjjkCN8vd0GlrdthfpvpGUUN8 uCcLRawUlE+g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,301,1596524400"; d="scan'208";a="455788163" Received: from unknown (HELO silpixa00399126.ir.intel.com) ([10.237.222.4]) by orsmga004.jf.intel.com with ESMTP; 25 Sep 2020 04:10:51 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: patrick.fu@intel.com, Kevin Laatz , Bruce Richardson Date: Fri, 25 Sep 2020 12:09:10 +0100 Message-Id: <20200925110910.284098-26-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200925110910.284098-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> <20200925110910.284098-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v3 25/25] raw/ioat: add fill operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kevin Laatz Add fill operation enqueue support for IOAT and IDXD. The fill enqueue is similar to the copy enqueue, but takes a 'pattern' rather than a source address to transfer to the destination address. This patch also includes an additional test case for the new operation type. Signed-off-by: Kevin Laatz Reviewed-by: Bruce Richardson --- doc/guides/rawdevs/ioat.rst | 10 ++++ doc/guides/rel_notes/release_20_11.rst | 2 + drivers/raw/ioat/ioat_rawdev_test.c | 44 +++++++++++++++++ drivers/raw/ioat/rte_ioat_rawdev.h | 26 +++++++++++ drivers/raw/ioat/rte_ioat_rawdev_fns.h | 65 ++++++++++++++++++++++++-- 5 files changed, 142 insertions(+), 5 deletions(-) diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst index 7c2a2d457..250cfc48a 100644 --- a/doc/guides/rawdevs/ioat.rst +++ b/doc/guides/rawdevs/ioat.rst @@ -285,6 +285,16 @@ is correct before freeing the data buffers using the returned handles: } +Filling an Area of Memory +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The IOAT driver also has support for the ``fill`` operation, where an area +of memory is overwritten, or filled, with a short pattern of data. +Fill operations can be performed in much the same was as copy operations +described above, just using the ``rte_ioat_enqueue_fill()`` function rather +than the ``rte_ioat_enqueue_copy()`` function. + + Querying Device Statistics ~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index 4d8b78154..dd65b779d 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -84,6 +84,8 @@ New Features * Added support for Intel\ |reg| Data Streaming Accelerator hardware. For more information, see https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator + * Added support for the fill operation via the API ``rte_ioat_enqueue_fill()``, + where the hardware fills an area of memory with a repeating pattern. * Added a per-device configuration flag to disable management of user-provided completion handles * Renamed the ``rte_ioat_do_copies()`` API to ``rte_ioat_perform_ops()``, and renamed the ``rte_ioat_completed_copies()`` API to ``rte_ioat_completed_ops()`` diff --git a/drivers/raw/ioat/ioat_rawdev_test.c b/drivers/raw/ioat/ioat_rawdev_test.c index 7be6f2a2d..64269af55 100644 --- a/drivers/raw/ioat/ioat_rawdev_test.c +++ b/drivers/raw/ioat/ioat_rawdev_test.c @@ -152,6 +152,46 @@ test_enqueue_copies(int dev_id) return 0; } +static int +test_enqueue_fill(int dev_id) +{ + const unsigned int length[] = {8, 64, 1024, 50, 100, 89}; + struct rte_mbuf *dst = rte_pktmbuf_alloc(pool); + char *dst_data = rte_pktmbuf_mtod(dst, char *); + uint64_t pattern = 0xfedcba9876543210; + unsigned int i, j; + + for (i = 0; i < RTE_DIM(length); i++) { + /* reset dst_data */ + memset(dst_data, 0, length[i]); + + /* perform the fill operation */ + if (rte_ioat_enqueue_fill(dev_id, pattern, + dst->buf_iova + dst->data_off, length[i], + (uintptr_t)dst) != 1) { + PRINT_ERR("Error with rte_ioat_enqueue_fill\n"); + return -1; + } + + rte_ioat_perform_ops(dev_id); + usleep(100); + + /* check the result */ + for (j = 0; j < length[i]; j++) { + char pat_byte = ((char *)&pattern)[j % 8]; + if (dst_data[j] != pat_byte) { + PRINT_ERR("Error with fill operation (length = %u): got (%x), not (%x)\n", + length[i], dst_data[j], + pat_byte); + return -1; + } + } + } + + rte_pktmbuf_free(dst); + return 0; +} + int ioat_rawdev_test(uint16_t dev_id) { @@ -238,6 +278,10 @@ ioat_rawdev_test(uint16_t dev_id) } printf("\n"); + /* test enqueue fill operation */ + if (test_enqueue_fill(dev_id) != 0) + goto err; + rte_rawdev_stop(dev_id); if (rte_rawdev_xstats_reset(dev_id, NULL, 0) != 0) { PRINT_ERR("Error resetting xstat values\n"); diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h b/drivers/raw/ioat/rte_ioat_rawdev.h index 6b891cd44..b7632ebf3 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev.h +++ b/drivers/raw/ioat/rte_ioat_rawdev.h @@ -37,6 +37,32 @@ struct rte_ioat_rawdev_config { bool hdls_disable; /**< if set, ignore user-supplied handle params */ }; +/** + * Enqueue a fill operation onto the ioat device + * + * This queues up a fill operation to be performed by hardware, but does not + * trigger hardware to begin that operation. + * + * @param dev_id + * The rawdev device id of the ioat instance + * @param pattern + * The pattern to populate the destination buffer with + * @param dst + * The physical address of the destination buffer + * @param length + * The length of the destination buffer + * @param dst_hdl + * An opaque handle for the destination data, to be returned when this + * operation has been completed and the user polls for the completion details. + * NOTE: If hdls_disable configuration option for the device is set, this + * parameter is ignored. + * @return + * Number of operations enqueued, either 0 or 1 + */ +static inline int +rte_ioat_enqueue_fill(int dev_id, uint64_t pattern, phys_addr_t dst, + unsigned int length, uintptr_t dst_hdl); + /** * Enqueue a copy operation onto the ioat device * diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h b/drivers/raw/ioat/rte_ioat_rawdev_fns.h index d0045d8a4..c2c4601ca 100644 --- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h +++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h @@ -115,6 +115,13 @@ enum rte_idxd_ops { #define IDXD_FLAG_REQUEST_COMPLETION (1 << 3) #define IDXD_FLAG_CACHE_CONTROL (1 << 8) +#define IOAT_COMP_UPDATE_SHIFT 3 +#define IOAT_CMD_OP_SHIFT 24 +enum rte_ioat_ops { + ioat_op_copy = 0, /* Standard DMA Operation */ + ioat_op_fill /* Block Fill */ +}; + /** * Hardware descriptor used by DSA hardware, for both bursts and * for individual operations. @@ -203,11 +210,8 @@ struct rte_idxd_rawdev { struct rte_idxd_desc_batch *batch_ring; }; -/* - * Enqueue a copy operation onto the ioat device - */ static __rte_always_inline int -__ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, +__ioat_write_desc(int dev_id, uint32_t op, uint64_t src, phys_addr_t dst, unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl) { struct rte_ioat_rawdev *ioat = @@ -229,7 +233,8 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, desc = &ioat->desc_ring[write]; desc->size = length; /* set descriptor write-back every 16th descriptor */ - desc->u.control_raw = (uint32_t)((!(write & 0xF)) << 3); + desc->u.control_raw = (uint32_t)((op << IOAT_CMD_OP_SHIFT) | + (!(write & 0xF) << IOAT_COMP_UPDATE_SHIFT)); desc->src_addr = src; desc->dest_addr = dst; @@ -242,6 +247,27 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, return 1; } +static __rte_always_inline int +__ioat_enqueue_fill(int dev_id, uint64_t pattern, phys_addr_t dst, + unsigned int length, uintptr_t dst_hdl) +{ + static const uintptr_t null_hdl; + + return __ioat_write_desc(dev_id, ioat_op_fill, pattern, dst, length, + null_hdl, dst_hdl); +} + +/* + * Enqueue a copy operation onto the ioat device + */ +static __rte_always_inline int +__ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, + unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl) +{ + return __ioat_write_desc(dev_id, ioat_op_copy, src, dst, length, + src_hdl, dst_hdl); +} + /* add fence to last written descriptor */ static __rte_always_inline int __ioat_fence(int dev_id) @@ -380,6 +406,23 @@ __idxd_write_desc(int dev_id, const struct rte_idxd_hw_desc *desc, return 0; } +static __rte_always_inline int +__idxd_enqueue_fill(int dev_id, uint64_t pattern, rte_iova_t dst, + unsigned int length, uintptr_t dst_hdl) +{ + const struct rte_idxd_hw_desc desc = { + .op_flags = (idxd_op_fill << IDXD_CMD_OP_SHIFT) | + IDXD_FLAG_CACHE_CONTROL, + .src = pattern, + .dst = dst, + .size = length + }; + const struct rte_idxd_user_hdl hdl = { + .dst = dst_hdl + }; + return __idxd_write_desc(dev_id, &desc, &hdl); +} + static __rte_always_inline int __idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t dst, unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl) @@ -475,6 +518,18 @@ __idxd_completed_ops(int dev_id, uint8_t max_ops, return n; } +static inline int +rte_ioat_enqueue_fill(int dev_id, uint64_t pattern, phys_addr_t dst, + unsigned int len, uintptr_t dst_hdl) +{ + enum rte_ioat_dev_type *type = + (enum rte_ioat_dev_type *)rte_rawdevs[dev_id].dev_private; + if (*type == RTE_IDXD_DEV) + return __idxd_enqueue_fill(dev_id, pattern, dst, len, dst_hdl); + else + return __ioat_enqueue_fill(dev_id, pattern, dst, len, dst_hdl); +} + static inline int rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst, unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl) -- 2.25.1