From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8D6DCA0C45; Sat, 4 Sep 2021 05:05:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 115C340DDD; Sat, 4 Sep 2021 05:05:31 +0200 (CEST) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id B88054067E for ; Sat, 4 Sep 2021 05:05:28 +0200 (CEST) Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4H1fYw6j8nz8xxK; Sat, 4 Sep 2021 11:01:08 +0800 (CST) Received: from dggpeml500024.china.huawei.com (7.185.36.10) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Sat, 4 Sep 2021 11:05:25 +0800 Received: from [10.40.190.165] (10.40.190.165) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Sat, 4 Sep 2021 11:05:25 +0800 To: Bruce Richardson CC: , , , , , , , , , , , , , , , References: <1625231891-2963-1-git-send-email-fengchengwen@huawei.com> <1630588395-2804-1-git-send-email-fengchengwen@huawei.com> <1630588395-2804-2-git-send-email-fengchengwen@huawei.com> From: fengchengwen Message-ID: <560681cc-b1d2-5cfb-a873-8520827ed350@huawei.com> Date: Sat, 4 Sep 2021 11:05:25 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.40.190.165] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] [PATCH v19 1/7] dmadev: introduce DMA device library public APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 2021/9/3 21:03, Bruce Richardson wrote: > On Thu, Sep 02, 2021 at 09:13:09PM +0800, Chengwen Feng wrote: >> The 'dmadevice' is a generic type of DMA device. >> >> This patch introduce the 'dmadevice' public APIs which expose generic >> operations that can enable configuration and I/O with the DMA devices. >> >> Maintainers update is also included in this patch. >> >> Signed-off-by: Chengwen Feng >> Acked-by: Bruce Richardson >> Acked-by: Morten Brørup >> Acked-by: Jerin Jacob >> --- >> MAINTAINERS | 4 + >> doc/api/doxy-api-index.md | 1 + >> doc/api/doxy-api.conf.in | 1 + >> doc/guides/rel_notes/release_21_11.rst | 5 + >> lib/dmadev/meson.build | 4 + >> lib/dmadev/rte_dmadev.h | 949 +++++++++++++++++++++++++++++++++ >> lib/dmadev/version.map | 24 + >> lib/meson.build | 1 + >> 8 files changed, 989 insertions(+) >> create mode 100644 lib/dmadev/meson.build >> create mode 100644 lib/dmadev/rte_dmadev.h >> create mode 100644 lib/dmadev/version.map >> > > > >> +/** >> + * @warning >> + * @b EXPERIMENTAL: this API may change without prior notice. >> + * >> + * Trigger hardware to begin performing enqueued operations. >> + * >> + * This API is used to write the "doorbell" to the hardware to trigger it >> + * to begin the operations previously enqueued by rte_dmadev_copy/fill(). >> + * >> + * @param dev_id >> + * The identifier of the device. >> + * @param vchan >> + * The identifier of virtual DMA channel. >> + * >> + * @return >> + * 0 on success. Otherwise negative value is returned. >> + */ >> +__rte_experimental >> +int >> +rte_dmadev_submit(uint16_t dev_id, uint16_t vchan); >> + > > Putting this out here for discussion: > > Those developers here looking at how integration of dma acceleration into > vhost-virtio e.g. for OVS use, have come back with the request that we > provide a method for querying the amount of space in the descriptor ring, > or the size of the next burst, or similar. Basically, the reason for the > ask is to allow an app to determine if a set of jobs of size N can be > enqueued before the first one is, so that we don't get a half-offload of > copy of a multi-segment packet (for devices where scatter-gather is not > available). Agree > > In our "ioat" rawdev driver, we did this by providing a "burst_capacity" > API which returned the number of elements which could be enqueued in the > next burst without error (normally the available ring space). Looking at > the dmadev APIs, an alternative way to do this is to extend the "submit()" > function to allow a 3rd optional parameter to return this info. That is, > when submitting one burst of operations, you get info about how many more > you can enqueue in the next burst. [For submitting packets via the submit > flag, this info would not be available, as I feel ending all enqueue > operations would be excessive]. > > Therefore, I see a number of options for us to meet the ask for space > querying API: > 1. provide a capacity API as done with ioat driver > 2. provide (optional) capacity information from each submit() call > 3. provide both #1 and #2 above as they are compatible > 4. Maybe available ring space could be calculated based on enqueue/completed ring_idx, and ring_size, in this way, we only need to provide the following help function, e.g. uint16_t rte_dmadev_burst_capacity(uint16_t enqueue_idx, // ring_idx of the latest enqueue uint16_t completed_idx, // ring_idx of the latest completed uint16_t ring_size) { return ring_size - 1 - distance(enqueue_idx, completed_idx); } However, this does not apply to the scatter-gather scenario, in which one enqueue request may occupy multiple descriptors space. Alternatively, an sg_avg can be passed in: uint16_t rte_dmadev_burst_capacity(uint16_t enqueue_idx, // ring_idx of the latest enqueue uint16_t completed_idx, // ring_idx of the latest completed uint16_t ring_size, uint16_t sg_avg_descs) // average number of descriptors occupied by SG requests { return ring_size - 1 - (distance(enqueue_idx, completed_idx) * sg_avg_descs) } But it's just an estimate. It's probably too big or too small. > > For me, I think #3 is probably the most flexible approach. The benefit of > #2 is that the info can be provided to the application much more cheaply > than when the app has to call a separate API (which wouldn't be on the > fast-path). However, a way to provide the info apart from submitting a > burst would also be helpful, hence adding the extra function too (#1). > > What are other people's thoughts or ideas on this? In terms of the API definition, the two do not seem to be related, so I do not recommend extending them in submit. However, for data-plane APIs, certain compromises can be accepted I think. and it work well in burst enqueue mode: 1. application maintains a variable of available space, and init it with rte_dmadev_burst_capacity() which is new dataplacn API. 2. enqueue multiple copy request without submit flag. before enqueue, application could use step1's available space to check whether all requests can be accommodated. 3. submit doorbell and update available space info. 4. do other work. 5. call completed API: if this API return >=0 the driver actually available space will increase. it will bigger than application's available space. 6. do other work. 7. enqueue multiple copy request without submit flag. before enqueue, application could use step3's available space to check whether all requests can be accommodated. 8. submit doorbell and update available space again. ... As long as ring_size is set properly (ring_size equals at least 2*burst), the game can be played. > > Regards, > /Bruce > . >