From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 6450DA0548;
Thu, 15 Jul 2021 12:03:21 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 1936A4014D;
Thu, 15 Jul 2021 12:03:21 +0200 (CEST)
Received: from mga06.intel.com (mga06.intel.com [134.134.136.31])
by mails.dpdk.org (Postfix) with ESMTP id B62C240143
for <dev@dpdk.org>; Thu, 15 Jul 2021 12:03:18 +0200 (CEST)
X-IronPort-AV: E=McAfee;i="6200,9189,10045"; a="271629793"
X-IronPort-AV: E=Sophos;i="5.84,240,1620716400"; d="scan'208";a="271629793"
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
15 Jul 2021 03:03:17 -0700
X-IronPort-AV: E=Sophos;i="5.84,240,1620716400"; d="scan'208";a="489191643"
Received: from bricha3-mobl.ger.corp.intel.com ([10.252.5.22])
by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA;
15 Jul 2021 03:03:12 -0700
Date: Thu, 15 Jul 2021 11:03:08 +0100
From: Bruce Richardson <bruce.richardson@intel.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Chengwen Feng <fengchengwen@huawei.com>,
Thomas Monjalon <thomas@monjalon.net>,
Ferruh Yigit <ferruh.yigit@intel.com>, Jerin Jacob <jerinj@marvell.com>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>, dpdk-dev <dev@dpdk.org>,
Morten =?iso-8859-1?Q?Br=F8rup?= <mb@smartsharesystems.com>,
Nipun Gupta <nipun.gupta@nxp.com>, Hemant Agrawal <hemant.agrawal@nxp.com>,
Maxime Coquelin <maxime.coquelin@redhat.com>,
Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>,
David Marchand <david.marchand@redhat.com>,
Satananda Burla <sburla@marvell.com>, Prasun Kapoor <pkapoor@marvell.com>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Message-ID: <YPAH3EAY470CNd02@bricha3-MOBL.ger.corp.intel.com>
References: <1625231891-2963-1-git-send-email-fengchengwen@huawei.com>
<1626179263-14645-1-git-send-email-fengchengwen@huawei.com>
<CALBAE1MjEoYq5OLR=Lye5pCvn8eHj5S21jtHD-y=dyq_FDHA3A@mail.gmail.com>
<YO/5yYh+HZHLpBsV@bricha3-MOBL.ger.corp.intel.com>
<CALBAE1Nr1Q7QJFgotd3GcC7ZG_yFAGhW+Tgj7tyTHUa9VAv7-Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CALBAE1Nr1Q7QJFgotd3GcC7ZG_yFAGhW+Tgj7tyTHUa9VAv7-Q@mail.gmail.com>
Subject: Re: [dpdk-dev] [PATCH v3] dmadev: introduce DMA device library
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Thu, Jul 15, 2021 at 03:00:01PM +0530, Jerin Jacob wrote:
> On Thu, Jul 15, 2021 at 2:33 PM Bruce Richardson
> <bruce.richardson@intel.com> wrote:
> >
> > On Thu, Jul 15, 2021 at 12:40:01PM +0530, Jerin Jacob wrote:
> > > )
> > > a
> > >
> > > On Tue, Jul 13, 2021 at 6:01 PM Chengwen Feng <fengchengwen@huawei.com> wrote:
> > > >
> > > > This patch introduce 'dmadevice' which is a generic type of DMA
> > > > device.
> > > >
> > > > The APIs of dmadev library exposes some generic operations which can
> > > > enable configuration and I/O with the DMA devices.
> > > >
> > > > Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
> > >
> > > Thanks for v3. Seems like all major items as covered. Some more
> > > comments below inline.
> > >
> > > I would suggest v4 to split the patch like (so that we can review and
> > > ack each patch)
> > > 1) Only public header file with Doxygen inclusion, (There is a lot of
> > > Doxygen syntax issue in the patch)
> > > 2) 1 or more patches for implementation.
> > >
> >
> > One additional follow-up comment on flags below.
> >
> > /Bruce
> >
> > >
> > > > diff --git a/lib/dmadev/rte_dmadev.h b/lib/dmadev/rte_dmadev.h
> > > > new file mode 100644
> > > > index 0000000..f6cc4e5
> > <snip>
> > > > + enum rte_dmadev_port_type port_type;
> > > missing doxgen comment for this.
> > > > + union {
> > > > + /** For PCIE port
> > > > + *
> > > > + * The following model show SoC's PCIE module connects to
> > > > + * multiple PCIE hosts and multiple endpoints. The PCIE module
> > > > + * has an integrate DMA controller.
> > > > + * If the DMA wants to access the memory of host A, it can be
> > > > + * initiated by PF1 in core0, or by VF0 of PF0 in core0.
> > > > + *
> > <snip>
> > > + /** The pasid filed in TLP packet */
> > > > + uint64_t pasid : 20;
> > > > + /** The attributes filed in TLP packet */
> > > > + uint64_t attr : 3;
> > > > + /** The processing hint filed in TLP packet */
> > > > + uint64_t ph : 2;
> > > > + /** The steering tag filed in TLP packet */
> > > > + uint64_t st : 16;
> > >
> > > We don't support a few attributes like passid, ph, st. Do we need
> > > the capability of this? or ignore this. In either case, please update the doc.
> > >
> > > We also support additional flags for allocating LLC flag.
> > > This is a hint to DMA engine that the cache blocks should be allocated
> > > in the LLC (if they were not already).
> > > When the MEM pointer is a destination in DMA operation, the referenced
> > > cache blocks are allocated into the cache as part of completing the
> > > DMA (when not already present in the LLC)
> > > this is helpful if software has to access the data right after dma is completed.
> > >
> > > Could you add bit or flag for the same?
> > >
> >
> > I wonder if this is the best location for such a flag for LLC vs memory
> > writes. It would also apply to memory-to-memory transactions, not just for
> > those done to PCI devices.
>
> Ack. it can be used for MEM to MEM
>
> > As well as that, I think any flag should default
> > to "on" rather than "off" since writing to cache rather than DRAM is
> > generally the desired behaviour, I would think.
>
> I think, keeping it is "allocate in LLC" on all transfer will not be good.
> As large transters polute the LLC and dataplane may not touch the complete
> data only header. Also in device copy, Adding it LLC there is an
> additional cost unline MEM-MEM.
>
> So IMO, better to add the flag to allow to allocate to LLC as a HINT.
>
> > Should it be a per-operation flag, rather than per context?
>
> Yes. better it be per-operation as it is the hint.
>
Ok. Let's define a new per-op flag for LLC allocation, and keep default
(without flag) as no-alloc.