From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40726A0C43; Wed, 16 Jun 2021 20:08:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2EF7B40683; Wed, 16 Jun 2021 20:08:26 +0200 (CEST) Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) by mails.dpdk.org (Postfix) with ESMTP id E9AEA4067A for ; Wed, 16 Jun 2021 20:08:24 +0200 (CEST) Received: by mail-io1-f49.google.com with SMTP id f10so163064iok.6 for ; Wed, 16 Jun 2021 11:08:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zlAwUKsIewHqACaU7zrWL7bdrJxHyVJXnSJw0QlU7bA=; b=DsK02c6D6z1M77rZE6gL+y2EYxqPnbrQV7fi4ljEhk12fX4DnL+8xmEF9MztoqGXLc CDssbZ5chQwXjOCw9pMv77M1cKgInVwcdttzfumkQtv7Xx10C8cqGhMBM4zGQG2ajcyU CWHT3xE/l0AoqD5jMvIjLZlJWuxRm1XKdMZqz8BRPg05Ofl9dRBA2v9oCvUoEd2Q3O3F yHOde+nqhUVn+Q5ZAWrl01bxbpFaRWo4ATK5lepGK4grZFuTnJyiX5JtULhk5BL9sPk0 TttJwQ62XdVKwzvQBtmn6kFsCmKr4iEk/1qt2YC47+C8qLfoGwHJb7emBRHbkpfmUUCE 14xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zlAwUKsIewHqACaU7zrWL7bdrJxHyVJXnSJw0QlU7bA=; b=ZqrGS+b9uE84Hd5I779lk09GeDheQ0M1IHkQ3uz0e5nsTVjihjk+Shbp8r2ljmSu9/ U9ulKBDXsSVs+2HWLWqQux7so9qwgd5jzsvaiTaJhGalcYoJk/DgcYEyoJawtz24y6Mh wfxzcf548ECsttzS//MIs3tMxYFYlAucgtg4i3oEsbouIfZ4TwyVT5q0O5TSv2WcqscK akr7WHvR0YmcDmsvu0c2dibTdFX8Jz3q3e1ABQfL5bz84e+h6aAr/YDyqdcD9DANjQDZ mDXbpLEmQjGZUr1HElusWPbB3yE0aRCpfPMmA9hqN8/9QHK61eqz5HK5lMEpGpTo8m3P SpzA== X-Gm-Message-State: AOAM532MgX+9BqOwfEgwj3+hkOsp4gyTW2OKRx4NVnCb8YxbHOFjkYVt JDyTdd1eW94J7PTcD0iFM9zqROvQRkJGoNNTSq4= X-Google-Smtp-Source: ABdhPJwFirMtqLap70JvqDIdRAkeovCkQtNrcx2n2oKblQ/4WRRqSqcUQovfGexk3eU1EJytN455aIkvVRQl8w0Kvs4= X-Received: by 2002:a6b:3e88:: with SMTP id l130mr495665ioa.59.1623866904320; Wed, 16 Jun 2021 11:08:24 -0700 (PDT) MIME-Version: 1.0 References: <1623763327-30987-1-git-send-email-fengchengwen@huawei.com> In-Reply-To: From: Jerin Jacob Date: Wed, 16 Jun 2021 23:38:08 +0530 Message-ID: To: Bruce Richardson Cc: fengchengwen , Thomas Monjalon , Ferruh Yigit , dpdk-dev , Nipun Gupta , Hemant Agrawal , Maxime Coquelin , Honnappa Nagarahalli , Jerin Jacob , David Marchand Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 16, 2021 at 11:01 PM Bruce Richardson wrote: > > On Wed, Jun 16, 2021 at 05:41:45PM +0800, fengchengwen wrote: > > On 2021/6/16 0:38, Bruce Richardson wrote: > > > On Tue, Jun 15, 2021 at 09:22:07PM +0800, Chengwen Feng wrote: > > >> This patch introduces 'dmadevice' which is a generic type of DMA > > >> device. > > >> > > >> The APIs of dmadev library exposes some generic operations which can > > >> enable configuration and I/O with the DMA devices. > > >> > > >> Signed-off-by: Chengwen Feng > > >> --- > > > Thanks for sending this. > > > > > > Of most interest to me right now are the key data-plane APIs. While we are > > > still in the prototyping phase, below is a draft of what we are thinking > > > for the key enqueue/perform_ops/completed_ops APIs. > > > > > > Some key differences I note in below vs your original RFC: > > > * Use of void pointers rather than iova addresses. While using iova's makes > > > sense in the general case when using hardware, in that it can work with > > > both physical addresses and virtual addresses, if we change the APIs to use > > > void pointers instead it will still work for DPDK in VA mode, while at the > > > same time allow use of software fallbacks in error cases, and also a stub > > > driver than uses memcpy in the background. Finally, using iova's makes the > > > APIs a lot more awkward to use with anything but mbufs or similar buffers > > > where we already have a pre-computed physical address. > > > > The iova is an hint to application, and widely used in DPDK. > > If switch to void, how to pass the address (iova or just va ?) > > this may introduce implementation dependencies here. > > > > Or always pass the va, and the driver performs address translation, and this > > translation may cost too much cpu I think. > > > > On the latter point, about driver doing address translation I would agree. > However, we probably need more discussion about the use of iova vs just > virtual addresses. My thinking on this is that if we specify the API using > iovas it will severely hurt usability of the API, since it forces the user > to take more inefficient codepaths in a large number of cases. Given a > pointer to the middle of an mbuf, one cannot just pass that straight as an > iova but must instead do a translation into offset from mbuf pointer and > then readd the offset to the mbuf base address. > > My preference therefore is to require the use of an IOMMU when using a > dmadev, so that it can be a much closer analog of memcpy. Once an iommu is > present, DPDK will run in VA mode, allowing virtual addresses to our > hugepage memory to be sent directly to hardware. Also, when using > dmadevs on top of an in-kernel driver, that kernel driver may do all iommu > management for the app, removing further the restrictions on what memory > can be addressed by hardware. One issue of keeping void * is that memory can come from stack or heap . which HW can not really operate it on. Considering difficulty to expressing above constraints, IMO, iova is good. (So that contract is clear between driver and application) or have some other means to express that constrain.