From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C4C38A0C49; Mon, 14 Jun 2021 20:18:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4834E4067E; Mon, 14 Jun 2021 20:18:20 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 27F574067A for ; Mon, 14 Jun 2021 20:18:18 +0200 (CEST) IronPort-SDR: 6JoTGlkUOaqV/dJiNeCtFHpzS7I0qFx95/SNyJ019KX4c7TjOhMfjMnQCKGNQJTG7Bey++DKoU 6IDNPq9+EE/w== X-IronPort-AV: E=McAfee;i="6200,9189,10015"; a="269706123" X-IronPort-AV: E=Sophos;i="5.83,273,1616482800"; d="scan'208";a="269706123" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2021 11:18:17 -0700 IronPort-SDR: D1fa5nu/P7tW3HGkPMN+m37WfqBznwtyaMTjKgBR5pZa4CfjFEh4fJCeHfCMh8WuXOYK/W0vk+ eg1pzbHyIQWw== X-IronPort-AV: E=Sophos;i="5.83,273,1616482800"; d="scan'208";a="484169229" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.30.190]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 14 Jun 2021 11:18:14 -0700 Date: Mon, 14 Jun 2021 19:18:09 +0100 From: Bruce Richardson To: Jerin Jacob Cc: Thomas Monjalon , fengchengwen , Ferruh Yigit , "dev@dpdk.org" , Nipun Gupta , Hemant Agrawal , Maxime Coquelin , Honnappa Nagarahalli , Jerin Jacob , David Marchand Message-ID: References: <27a77ca8-9406-1737-ff84-774b2ca561f7@huawei.com> <7807476.CTUdPGCxKm@thomas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] RFC: Kunpeng DMA driver API design decision X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Sat, Jun 12, 2021 at 02:11:10PM +0530, Jerin Jacob wrote: > On Sat, Jun 12, 2021 at 2:01 PM Thomas Monjalon wrote: > > > > 12/06/2021 09:01, fengchengwen: > > > Hi all, > > > > > > We prepare support Kunpeng DMA engine under rawdev framework, and observed that > > > there are two different implementations of the data plane API: > > > 1. rte_rawdev_enqueue/dequeue_buffers which was implemented by dpaa2_qdma and > > > octeontx2_dma driver. > > > 2. rte_ioat_enqueue_xxx/rte_ioat_completed_ops which was implemented by ioat > > > driver. > > > > > > Due to following consideration (mainly performance), we plan to implement API > > > like ioat (not the same, have some differences) in data plane: > > > 1. The rte_rawdev_enqueue_buffers use opaque buffer reference which is vendor's > > > specific, so it needs first to translate application parameters to opaque > > > pointer, and then driver writes the opaque data onto hardware, this may lead > > > to performance problem. > > > 2. rte_rawdev_xxx doesn't provide memory barrier API which may need to extend > > > by opaque data (e.g. add flag to every request), this may introduce some > > > complexity. > > > > > > Also the example/ioat was used to compare DMA and CPU-memcopy performance, > > > Could we generalized it so that it supports multiple-vendor ? > > > > > > I don't know if the community accepts this kind of implementation, so if you > > > have any comments, please provide feedback. > > > > I would love having a common generic API. > > I would prefer having drivers under drivers/dma/ directory, > > rather than rawdev. > > +1 for rte_dmadev. > > Now that we have multiple DMA drivers, it better to have a common > generic API for API. > > @fengchengwen If you would like to pursue generic DMA API the please > propose an RFC for dmadev PUBLIC API before implementing it, > We can help you review the proposal of API. > I'd like to volunteer to help with this effort also, having a large interest in it from my work on ioat driver (thanks for the positive words on the API :-)). Based on our experience with ioat driver, we are also looking into possible prototypes for a dmadev device type too, and hopefully will have some RFC to share soon. As might be expected this will be very similar to the existing ioat APIs, though with one change to the dataplane API I'll call out here initially. The use of explicit source and destination handles for each operation is a little inflexible, so we are looking at replacing that mechanism with one where the APIs return a (sequentially increasing) job id after each enqueue, and having the completion function return the id of the last completed job (or error info in case of an error). This would have the advantage of allowing each app or library using the dmadev to store as much or as little context information as desired in its own circular buffer or buffers, and not be limited to just two uint64_t's. It would also simplify the drivers, since they have less data to manage. I'd hope to have a more complete API description to send out very shortly to kick off reviews and discussion. Regards, /Bruce