From: fengchengwen <fengchengwen@huawei.com>
To: Stephen Hemminger <stephen@networkplumber.org>
Cc: <thomas@monjalon.net>, <ferruh.yigit@amd.com>, <dev@dpdk.org>
Subject: Re: [RFC 0/3] introduce coroutine library
Date: Tue, 25 Apr 2023 10:50:22 +0800 [thread overview]
Message-ID: <d8fe3ef3-393f-d032-35e3-d31ba5f17ff4@huawei.com> (raw)
In-Reply-To: <20230424191621.2075bf31@hermes.local>
On 2023/4/25 10:16, Stephen Hemminger wrote:
> On Tue, 25 Apr 2023 10:11:43 +0800
> fengchengwen <fengchengwen@huawei.com> wrote:
>
>> On 2023/4/25 0:08, Stephen Hemminger wrote:
>>> On Mon, 24 Apr 2023 13:02:05 +0000
>>> Chengwen Feng <fengchengwen@huawei.com> wrote:
>>>
>>>> This patchset introduces the coroutine library which will help refactor
>>>> the hns3 PMD's reset process.
>>>>
>>>> The hns3 single function reset process consists of the following steps:
>>>> 1.stop_service();
>>>> 2.prepare_reset();
>>>> 3.delay(100ms);
>>>> 4.notify_hw();
>>>> 5.wait_hw_reset_done(); // multiple sleep waits are involved.
>>>> 6.reinit();
>>>> 7.restore_conf();
>>>>
>>>> If the DPDK process take over multiple hns3 functions (e.g. 100),
>>>> it's impractical to reset and restore functions in sequence:
>>>> 1.proc_func(001); // will completed in 100+ms range.
>>>> 2.proc_func(002); // will completed in 100~200+ms range.
>>>> ...
>>>> x.proc_func(100); // will completed in 9900~10000+ms range.
>>>> The later functions will process fail because it's too late to deal with.
>>>>
>>>> One solution is that create a reset thread for each function, and it
>>>> will lead to large number of threads if the DPDK process take over
>>>> multiple hns3 functions.
>>>>
>>>> So the current hns3 driver uses asynchronous mechanism, for examples, it
>>>> use rte_eal_alarm_set() when process delay(100ms), it splits a serial
>>>> process into multiple asynchronous processes, and the code is complex
>>>> and difficult to understand.
>>>>
>>>> The coroutine is a good mechanism to provide programmers with the
>>>> simplicity of keeping serial processes within a limited number of
>>>> threads.
>>>>
>>>> This patchset use <ucontext.h> to build the coroutine framework, and it
>>>> just provides a demo. More APIs maybe added in the future.
>>>>
>>>> In addition, we would like to ask the community whether it it possible
>>>> to accept the library. If not, whether it is allowed to provide the
>>>> library in hns3 PMD.
>>>>
>>>> Chengwen Feng (3):
>>>> lib/coroutine: add coroutine library
>>>> examples/coroutine: support coroutine examples
>>>> net/hns3: refactor reset process with coroutine
>>>
>>> Interesting, but the DPDK really is not the right place for this.
>>> Also, why so much sleeping. Can't this device be handled with an event based
>>> model. Plus any complexity like this introduces more bugs into already fragile
>>> interaction of DPDK userspace applications and threads.
>>
>> A event base model will function as:
>> event-handler() {
>> for (...) {
>> event = get_next_event();
>> proc_event();
>> }
>> }
>> The root cause is that the proc_event() take too many time, and it will lead to other
>> function can't be processed timely.
>>
>> For which proc_event() may wait a lot of time, the coroutine could also used to optimize
>> it.
>>
>>>
>>> Not only that, coroutines add to the pre-existing problems with locking.
>>> If coroutine 1 acquires a lock, the coroutine 2 will deadlock itself.
>>> And someone will spend days figuring that out. And the existing analyzer
>>> tools will not know about the magic coroutine library.
>>
>> Analyzer tools like lock annotations maybe a problem.
>>
>> Locks in DPDK APIs are mostly no-blocking. We can add some restrictions(by reviewer), such
>> as once holding a lock, you can't invoke rte_co_yield() or rte_co_delay() API.
>
>> In addition, any technology has two sides, the greatest advantage of coroutine I think is
>> removes a large number of callbacks in asychronous programming. And also high-level languages
>> generally provide coroutines (e.g. C++/Python). With the development, the analyzer tools maybe
>> evolved to support detect.
>>
>>
>> And one more, if not acceptable as public library, whether it is allowed intergration of this
>> library in hns3 PMD ? Our internal evaluation solution (use coroutine refactor) is feasible,
>> but the code needs to be upstream, hope to listen to community's comments.
>
>
> The standard DPDK architecture is to have dedicated threads.
> Unless you convert the user application to some other model, there really is no other
Instead of adding a new running model, this coroutine library just adapts to the current
DPDK framework.
My visions:
1. DPDK launch a default thread run coroutine service, this thread just like interrupt thread,
it could provide server for PMD drivers.
2. Application could launch coroutine scheduler in lcore (just like 2/3 commit) if it want to
use this library.
> useful work that can be done while waiting for your driver.
>
> There was a previous DPDK library for lightweight threading, but it never got any
> usage and was abandoned and dropped. Why is this better?
DPDK lightweight threading ? I didn't know before, but will take a closer look at.
This patchset is caused by a problem in the our driver reset process. The reset process is
complex and difficult to understand. We think the coroutine implementation is practical to
solve this problem, so we try to send this RFC.
> .
>
next prev parent reply other threads:[~2023-04-25 2:50 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-24 13:02 Chengwen Feng
2023-04-24 13:02 ` [RFC 1/3] lib/coroutine: add " Chengwen Feng
2023-04-26 11:28 ` Ferruh Yigit
2023-04-24 13:02 ` [RFC 2/3] examples/coroutine: support coroutine examples Chengwen Feng
2023-04-24 13:02 ` [RFC 3/3] net/hns3: refactor reset process with coroutine Chengwen Feng
2023-04-24 16:08 ` [RFC 0/3] introduce coroutine library Stephen Hemminger
2023-04-25 2:11 ` fengchengwen
2023-04-25 2:16 ` Stephen Hemminger
2023-04-25 2:50 ` fengchengwen [this message]
2023-04-25 2:59 ` Garrett D'Amore
2023-04-25 21:06 ` Stephen Hemminger
2023-04-26 11:27 ` Ferruh Yigit
2023-04-28 7:20 ` fengchengwen
2023-04-25 9:27 ` Mattias Rönnblom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d8fe3ef3-393f-d032-35e3-d31ba5f17ff4@huawei.com \
--to=fengchengwen@huawei.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).