From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 185A1A0C42; Wed, 16 Jun 2021 16:37:45 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 95E934069C; Wed, 16 Jun 2021 16:37:44 +0200 (CEST) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by mails.dpdk.org (Postfix) with ESMTP id 4B7F04067A for ; Wed, 16 Jun 2021 16:37:43 +0200 (CEST) Received: by mail-io1-f46.google.com with SMTP id q15so3270178ioi.4 for ; Wed, 16 Jun 2021 07:37:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=/43vI5uPd92vHnKqdLKIqYK/T5sQLhzX3e/zn6CHFDA=; b=ewyto2n+iok9487tLh2zJ8wfwHp4JieQQ8sICUwQryY1yu9W7QkDWCZAsqYaU26CDX oVNUSsBhqOZWGjVhI7jU5lS7tWw2IKzi05x1xvDmCBiQCkaHwzaaQWuW3QZNKaRRhKFe heQh91EJz7cuzhYlD2kPc1eZHM0CpfKyB+EPb4QYV8/qjtZdVWK+5JjDHLQ+WpbsIiFP bCl1W+HggtoXjGCplJWgWrh4AnzqaTGqfODzq9m6+eQvOX3DlEmeVjg4NSrVHfouIQ3p TGCD0EDDqYzRDQtjukHcRdnAPvinII+9ubqfCqgLVxEhhRX6Xw9+hmU0qflAkYUIComo 5adA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=/43vI5uPd92vHnKqdLKIqYK/T5sQLhzX3e/zn6CHFDA=; b=DZh597L44HeUL4r3S/V6ZhJURT0d5zVj5hOE23SmpVw1RBDXWoh/Fv4Z+7dLoYH7Ki C5GHE8KSHY8f04BHanbFc5n4xYNZdPZUvRksLg1XHcP2RHRRVNdfJWoTEj//9zN4xO12 yUvUqd+1jZKm434rXl+ISx5YhaLvbKLiPtjROwkwgf5U1Gghayp+XsmzATS5xAVv7jBe Zwsd/v+ProuDQfGKKG8kN0kUNEb8VxB1W2tcDbw+8RNdx6a8DfmyKesRNGrdUO7VT1P7 lH7ArSA9Zp7Z04duryEJK/kJyc2iF6C9cxgkCH9aoZIY2urKcsPWyhWSYTuwuQpgGfLg zqIg== X-Gm-Message-State: AOAM530XbJkXe1p9AcOrAwsbkq384P0ApK5gh8VSJgBqlu4/h/1vqr3s PVo5pItXi/2Owo+xhIlvptDV1i6ks0vQc617AxY= X-Google-Smtp-Source: ABdhPJyHa5Uw8EUQLvHWUQUxO0eFKA/NaBJHH+/KTMHkwNFJg//xnuYugBjeOHeOHpeYGaoqEMeLdXFir+9Sa7nIRRk= X-Received: by 2002:a6b:8f83:: with SMTP id r125mr229998iod.123.1623854262547; Wed, 16 Jun 2021 07:37:42 -0700 (PDT) MIME-Version: 1.0 References: <1623763327-30987-1-git-send-email-fengchengwen@huawei.com> <98CBD80474FA8B44BF855DF32C47DC35C61860@smartserver.smartshare.dk> In-Reply-To: From: Jerin Jacob Date: Wed, 16 Jun 2021 20:07:26 +0530 Message-ID: To: fengchengwen Cc: =?UTF-8?Q?Morten_Br=C3=B8rup?= , Bruce Richardson , Thomas Monjalon , Ferruh Yigit , dpdk-dev , Nipun Gupta , Hemant Agrawal , Maxime Coquelin , Honnappa Nagarahalli , Jerin Jacob , David Marchand , Satananda Burla , Prasun Kapoor Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 16, 2021 at 3:47 PM fengchengwen wrot= e: > > On 2021/6/16 15:09, Morten Br=C3=B8rup wrote: > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > >> Sent: Tuesday, 15 June 2021 18.39 > >> > >> On Tue, Jun 15, 2021 at 09:22:07PM +0800, Chengwen Feng wrote: > >>> This patch introduces 'dmadevice' which is a generic type of DMA > >>> device. > >>> > >>> The APIs of dmadev library exposes some generic operations which can > >>> enable configuration and I/O with the DMA devices. > >>> > >>> Signed-off-by: Chengwen Feng > >>> --- > >> Thanks for sending this. > >> > >> Of most interest to me right now are the key data-plane APIs. While we > >> are > >> still in the prototyping phase, below is a draft of what we are > >> thinking > >> for the key enqueue/perform_ops/completed_ops APIs. > >> > >> Some key differences I note in below vs your original RFC: > >> * Use of void pointers rather than iova addresses. While using iova's > >> makes > >> sense in the general case when using hardware, in that it can work > >> with > >> both physical addresses and virtual addresses, if we change the APIs > >> to use > >> void pointers instead it will still work for DPDK in VA mode, while > >> at the > >> same time allow use of software fallbacks in error cases, and also a > >> stub > >> driver than uses memcpy in the background. Finally, using iova's > >> makes the > >> APIs a lot more awkward to use with anything but mbufs or similar > >> buffers > >> where we already have a pre-computed physical address. > >> * Use of id values rather than user-provided handles. Allowing the > >> user/app > >> to manage the amount of data stored per operation is a better > >> solution, I > >> feel than proscribing a certain about of in-driver tracking. Some > >> apps may > >> not care about anything other than a job being completed, while othe= r > >> apps > >> may have significant metadata to be tracked. Taking the user-context > >> handles out of the API also makes the driver code simpler. > >> * I've kept a single combined API for completions, which differs from > >> the > >> separate error handling completion API you propose. I need to give > >> the > >> two function approach a bit of thought, but likely both could work. > >> If we > >> (likely) never expect failed ops, then the specifics of error > >> handling > >> should not matter that much. > >> > >> For the rest, the control / setup APIs are likely to be rather > >> uncontroversial, I suspect. However, I think that rather than xstats > >> APIs, > >> the library should first provide a set of standardized stats like > >> ethdev > >> does. If driver-specific stats are needed, we can add xstats later to > >> the > >> API. > >> > >> Appreciate your further thoughts on this, thanks. > >> > >> Regards, > >> /Bruce > > > > I generally agree with Bruce's points above. > > > > I would like to share a couple of ideas for further discussion: I believe some of the other requirements and comments for generic DMA will = be 1) Support for the _channel_, Each channel may have different capabilities and functionalities. Typical cases are, each channel have separate source and destination devices like DMA between PCIe EP to Host memory, Host memory to Host memory, PCIe EP to PCIe EP. So we need some notion of the channel in the specification. 2) I assume current data plane APIs are not thread-safe. Is it right? 3) Cookie scheme outlined earlier looks good to me. Instead of having generic dequeue() API 4) Can split the rte_dmadev_enqueue_copy(uint16_t dev_id, void * src, void * dst, unsigned int length); to two stage API like, Where one will be used in fastpath and other one will use used in slowpath. - slowpath API will for take channel and take other attributes for transfer Example syantx will be: struct rte_dmadev_desc { channel id; ops ; // copy, xor, fill etc other arguments specific to dma transfer // it can be set based on capability. }; rte_dmadev_desc_t rte_dmadev_preprare(uint16_t dev_id, struct rte_dmadev_desc *dec); - Fastpath takes arguments that need to change per transfer along with slow-path handle. rte_dmadev_enqueue(uint16_t dev_id, void * src, void * dst, unsigned int length, rte_dmadev_desc_t desc) This will help to driver to -Former API form the device-specific descriptors in slow path for a given channel and fixed attributes per transfer -Later API blend "variable" arguments such as src, dest address with slow-path created descriptors The above will give better performance and is the best trade-off between performance and per transfer variables.