From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 73348A0548; Mon, 20 Sep 2021 12:30:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EBDBC40DF7; Mon, 20 Sep 2021 12:30:15 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 7468840DF5 for ; Mon, 20 Sep 2021 12:30:13 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10112"; a="286778226" X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; d="scan'208";a="286778226" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Sep 2021 03:30:12 -0700 X-IronPort-AV: E=Sophos;i="5.85,308,1624345200"; d="scan'208";a="556054365" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.17.156]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 20 Sep 2021 03:30:10 -0700 Date: Mon, 20 Sep 2021 11:30:07 +0100 From: Bruce Richardson To: Kevin Laatz Cc: dev@dpdk.org, fengchengwen@huawei.com, jerinj@marvell.com, conor.walsh@intel.com Message-ID: References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> <20210917152437.3270330-10-kevin.laatz@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210917152437.3270330-10-kevin.laatz@intel.com> Subject: Re: [dpdk-dev] [PATCH v5 09/16] dma/idxd: add data-path job submission functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Sep 17, 2021 at 03:24:30PM +0000, Kevin Laatz wrote: > Add data path functions for enqueuing and submitting operations to DSA > devices. > > Signed-off-by: Bruce Richardson > Signed-off-by: Kevin Laatz > Reviewed-by: Conor Walsh > --- > doc/guides/dmadevs/idxd.rst | 64 +++++++++++++++ > drivers/dma/idxd/idxd_common.c | 136 +++++++++++++++++++++++++++++++ > drivers/dma/idxd/idxd_internal.h | 5 ++ > drivers/dma/idxd/meson.build | 1 + > 4 files changed, 206 insertions(+) > > diff --git a/doc/guides/dmadevs/idxd.rst b/doc/guides/dmadevs/idxd.rst > index a603c5dd22..7835461a22 100644 > --- a/doc/guides/dmadevs/idxd.rst > +++ b/doc/guides/dmadevs/idxd.rst > @@ -153,3 +153,67 @@ The following code shows how the device is configured in > > Once configured, the device can then be made ready for use by calling the > ``rte_dma_start()`` API. > + > +Performing Data Copies > +~~~~~~~~~~~~~~~~~~~~~~~ > + > +To perform data copies using IDXD dmadev devices, descriptors should be enqueued > +using the ``rte_dma_copy()`` API. The HW can be triggered to perform the copy > +in two ways, either via a ``RTE_DMA_OP_FLAG_SUBMIT`` flag or by calling > +``rte_dma_submit()``. Once copies have been completed, the completion will > +be reported back when the application calls ``rte_dma_completed()`` or > +``rte_dma_completed_status()``. The latter will also report the status of each > +completed operation. > + > +The ``rte_dma_copy()`` function enqueues a single copy to the device ring for > +copying at a later point. The parameters to that function include the IOVA addresses > +of both the source and destination buffers, as well as the length of the copy. > + > +The ``rte_dma_copy()`` function enqueues a copy operation on the device ring. > +If the ``RTE_DMA_OP_FLAG_SUBMIT`` flag is set when calling ``rte_dma_copy()``, > +the device hardware will be informed of the elements. Alternatively, if the flag > +is not set, the application needs to call the ``rte_dma_submit()`` function to > +notify the device hardware. Once the device hardware is informed of the elements > +enqueued on the ring, the device will begin to process them. It is expected > +that, for efficiency reasons, a burst of operations will be enqueued to the > +device via multiple enqueue calls between calls to the ``rte_dma_submit()`` > +function. > + > +The following code demonstrates how to enqueue a burst of copies to the > +device and start the hardware processing of them: > + > +.. code-block:: C > + > + struct rte_mbuf *srcs[COMP_BURST_SZ], *dsts[COMP_BURST_SZ]; > + unsigned int i; > + > + for (i = 0; i < RTE_DIM(srcs); i++) { > + uint64_t *src_data; > + > + srcs[i] = rte_pktmbuf_alloc(pool); > + dsts[i] = rte_pktmbuf_alloc(pool); > + src_data = rte_pktmbuf_mtod(srcs[i], uint64_t *); > + if (srcs[i] == NULL || dsts[i] == NULL) { > + PRINT_ERR("Error allocating buffers\n"); > + return -1; > + } > + > + for (j = 0; j < COPY_LEN/sizeof(uint64_t); j++) > + src_data[j] = rte_rand(); > + > + if (rte_dma_copy(dev_id, vchan, srcs[i]->buf_iova + srcs[i]->data_off, > + dsts[i]->buf_iova + dsts[i]->data_off, COPY_LEN, 0) < 0) { > + PRINT_ERR("Error with rte_dma_copy for buffer %u\n", i); > + return -1; > + } > + } > + rte_dma_submit(dev_id, vchan); > + I think this code block is larger than necessary, because it shows buffer allocation and initialization rather than just the basics of copy() and submit() APIs. Furthermore, rather than calling out the generic API use in the idxd-specific docs, can we just include a reference to the dmadev documentation? /Bruce