From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C637EA0C41; Wed, 23 Jun 2021 09:21:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4F9034003F; Wed, 23 Jun 2021 09:21:35 +0200 (CEST) Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) by mails.dpdk.org (Postfix) with ESMTP id 5AD214003E for ; Wed, 23 Jun 2021 09:21:34 +0200 (CEST) Received: by mail-io1-f51.google.com with SMTP id g22so2164880iom.1 for ; Wed, 23 Jun 2021 00:21:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=iogy104Z1Io45KKcKsJqZPGjP4K039mdjheqnoU8Uz8=; b=eUsjj5F0gpJMlxoAvBW8wN1V9yg18mxVqXaWwryyR1LVtT6tn+sLLZ5TqpYbmPLKWG XUqQ9I39Aej/uDBPPzsHInULjnQJ28zQ+2VKUKhlVIsHUsV0KVt8oAkZ2DhQ9FeH/iqA eVKTdSfDkAMqM2YDDpBAU2Sea48dUCPdSPPnrsCOAS+SGx09v8hj0oaM1ScYdCg21o7j gDhPJO6+R6oMNKenochTawJsiOl1YyngwI2aGS5f6oTraZgBSnyWkiKO3Xg3WmuX+dim 73wPz0oCa1LPFgqDzOE7TQHoSggRF3KF0IWcqqMVDUl07J3ObeM03jHgJbMJnpRors8g X/dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iogy104Z1Io45KKcKsJqZPGjP4K039mdjheqnoU8Uz8=; b=Q+4SpHrsigmTWdypraUB28tbtsXlPDTYzt8Uy9NMzfqrH71Qj3B9LTqzTbAIOvr0gW wZWGdBmo86soAeV6WKLy0Q9tQ0OWpEBFEJUEZBGaM5ydthdE6MknKGLmvHYKM2xoJWqr TYCHXOkC7eJTPepOmA6nTHGvzr72Dt+C+lnJcCAtfs0k5iLPT42HjkongBz9ogKk1RXk eo4uE8A1qsjL0w7DFjnqBRvGkspC6jTfJJTJfnTMWNT0Vk245vrm6u1DiHFyQpNqa0Zp JXDxznlLxFCsHdve25V+vQCC43t7C1AinouFuymyK6LvnB3uw5UNaXAQ+GJH93w0Mo3/ Ykuw== X-Gm-Message-State: AOAM532/E4Gxs3hmfkIbl4e4RYtF1aJ+6AKN+Mcc0bNuFBXl2F2xpsRE OEIoHhE1J1fUK2LppEG+LLJJ45eCrXqGLKYymkk= X-Google-Smtp-Source: ABdhPJysn+9MkZDv3t3DD+gwajY7lOln51vCOhcc2j18nc9xNQJncvfLFu8MwuNk3TQ7Y3VhXdmrm05IqDBKl6UVhFw= X-Received: by 2002:a5e:c005:: with SMTP id u5mr5965571iol.1.1624432893584; Wed, 23 Jun 2021 00:21:33 -0700 (PDT) MIME-Version: 1.0 References: <1623763327-30987-1-git-send-email-fengchengwen@huawei.com> <98CBD80474FA8B44BF855DF32C47DC35C61860@smartserver.smartshare.dk> <3cb0bd01-2b0d-cf96-d173-920947466041@huawei.com> In-Reply-To: From: Jerin Jacob Date: Wed, 23 Jun 2021 12:51:07 +0530 Message-ID: To: fengchengwen Cc: Bruce Richardson , =?UTF-8?Q?Morten_Br=C3=B8rup?= , Thomas Monjalon , Ferruh Yigit , dpdk-dev , Nipun Gupta , Hemant Agrawal , Maxime Coquelin , Honnappa Nagarahalli , Jerin Jacob , David Marchand , Satananda Burla , Prasun Kapoor Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC PATCH] dmadev: introduce DMA device library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Jun 23, 2021 at 9:00 AM fengchengwen wrote: > > >>> > >>>> > >>>>> The above will give better performance and is the best trade-off c > >>>>> between performance and per transfer variables. > >>>> > >>>> We may need to have different APIs for context-aware and context-unaware > >>>> processing, with which to use determined by the capabilities discovery. > >>>> Given that for these DMA devices the offload cost is critical, more so than > >>>> any other dev class I've looked at before, I'd like to avoid having APIs > >>>> with extra parameters than need to be passed about since that just adds > >>>> extra CPU cycles to the offload. > >>> > >>> If driver does not support additional attributes and/or the > >>> application does not need it, rte_dmadev_desc_t can be NULL. > >>> So that it won't have any cost in the datapath. I think, we can go to > >>> different API > >>> cases if we can not abstract problems without performance impact. > >>> Otherwise, it will be too much > >>> pain for applications. > >> > >> Yes, currently we plan to use different API for different case, e.g. > >> rte_dmadev_memcpy() -- deal with local to local memcopy > >> rte_dmadev_memset() -- deal with fill with local memory with pattern > >> maybe: > >> rte_dmadev_imm_data() --deal with copy very little data > >> rte_dmadev_p2pcopy() --deal with peer-to-peer copy of diffenet PCIE addr > >> > >> These API capabilities will be reflected in the device capability set so that > >> application could know by standard API. > > > > > > There will be a lot of combination of that it will be like M x N cross > > base case, It won't scale. > > Currently, it is hard to define generic dma descriptor, I think the well-defined > APIs is feasible. I would like to understand why not feasible? if we move the preparation to the slow path. i.e struct rte_dmadev_desc defines all the "attributes" of all DMA devices available using capability. I believe with the scheme, we can scale and incorporate all features of all DMA HW without any performance impact. something like: struct rte_dmadev_desc { /* Attributes all DMA transfer available for all HW under capability. */ channel or port; ops ; // copy, fill etc.. /* impemention opqueue memory as zero length array, rte_dmadev_desc_prep() update this memory with HW specific information */ uint8_t impl_opq[]; } // allocate the memory for dma decriptor struct rte_dmadev_desc *rte_dmadev_desc_alloc(devid); // Convert DPDK specific descriptors to HW specific descriptors in slowpath */ rte_dmadev_desc_prep(devid, struct rte_dmadev_desc *desc); // Free dma descriptor memory rte_dmadev_desc_free(devid, struct rte_dmadev_desc *desc ) The above calls in slow path. Only below call in fastpath. // Here desc can be NULL(in case you don't need any specific attribute attached to transfer, if needed, it can be an object which is gone through rte_dmadev_desc_prep()) rte_dmadev_enq(devid, struct rte_dmadev_desc *desc, void *src, void *dest, unsigned int len, cookie) > > > > >> > >>> > >>> Just to understand, I think, we need to HW capabilities and how to > >>> have a common API. > >>> I assume HW will have some HW JOB descriptors which will be filled in > >>> SW and submitted to HW. > >>> In our HW, Job descriptor has the following main elements > >>> > >>> - Channel // We don't expect the application to change per transfer > >>> - Source address - It can be scatter-gather too - Will be changed per transfer > >>> - Destination address - It can be scatter-gather too - Will be changed > >>> per transfer > >>> - Transfer Length - - It can be scatter-gather too - Will be changed > >>> per transfer > >>> - IOVA address where HW post Job completion status PER Job descriptor > >>> - Will be changed per transfer > >>> - Another sideband information related to channel // We don't expect > >>> the application to change per transfer > >>> - As an option, Job completion can be posted as an event to > >>> rte_event_queue too // We don't expect the application to change per > >>> transfer > >> > >> The 'option' field looks like a software interface field, but not HW descriptor. > > > > It is in HW descriptor. > > The HW is interesting, something like: DMA could send completion direct to EventHWQueue, > the DMA and EventHWQueue are link in the hardware range, rather than by software. Yes. > > Could you provide public driver of this HW ? So we could know more about it's working > mechanism and software-hardware collaboration. http://code.dpdk.org/dpdk/v21.05/source/drivers/raw/octeontx2_dma/otx2_dpi_rawdev.h#L149 is the DMA instruction header.