Re: [RFC 0/3] introduce coroutine library

["Mattias Rönnblom" <hofors@lysator.liu.se>]

On 2023-04-24 15:02, Chengwen Feng wrote:
> This patchset introduces the coroutine library which will help refactor
> the hns3 PMD's reset process.
> 
> The hns3 single function reset process consists of the following steps:
>      1.stop_service();
>      2.prepare_reset();
>      3.delay(100ms);
>      4.notify_hw();
>      5.wait_hw_reset_done(); // multiple sleep waits are involved.
>      6.reinit();
>      7.restore_conf();
> 
> If the DPDK process take over multiple hns3 functions (e.g. 100),
> it's impractical to reset and restore functions in sequence:
>      1.proc_func(001); // will completed in 100+ms range.
>      2.proc_func(002); // will completed in 100~200+ms range.
>      ...
>      x.proc_func(100); // will completed in 9900~10000+ms range.
> The later functions will process fail because it's too late to deal with.
> 
> One solution is that create a reset thread for each function, and it
> will lead to large number of threads if the DPDK process take over
> multiple hns3 functions.
> 
> So the current hns3 driver uses asynchronous mechanism, for examples, it
> use rte_eal_alarm_set() when process delay(100ms), it splits a serial
> process into multiple asynchronous processes, and the code is complex
> and difficult to understand.
> 
> The coroutine is a good mechanism to provide programmers with the
> simplicity of keeping serial processes within a limited number of
> threads.
> 

Coroutines (or anything with a stack, really) are generally too slow as 
a vehicle for concurrency in data plane applications, I would argue.

They might help you solve this particular slow path task, but that alone 
doesn't seem like anything close to a rationale why a new concurrency 
library should be accepted.

DPDK does need a deferred work mechanism, but I don't think coroutines 
are the answer. Currently, RTE timer is the closest thing you have in 
DPDK. To solve your issue (and many other, including things in the fast 
path), I would rather look in that direction, maybe extending to 
something Linux' tasklets.

> This patchset use <ucontext.h> to build the coroutine framework, and it
> just provides a demo. More APIs maybe added in the future.
> 
> In addition, we would like to ask the community whether it it possible
> to accept the library. If not, whether it is allowed to provide the
> library in hns3 PMD.
> 
> Chengwen Feng (3):
>    lib/coroutine: add coroutine library
>    examples/coroutine: support coroutine examples
>    net/hns3: refactor reset process with coroutine
> 
>   drivers/net/hns3/hns3_ethdev.c    | 217 ++++++++++++++++++++++++++++++
>   drivers/net/hns3/hns3_ethdev.h    |   3 +
>   drivers/net/hns3/hns3_intr.c      |  38 ++++++
>   drivers/net/hns3/meson.build      |   2 +-
>   examples/coroutine/main.c         | 153 +++++++++++++++++++++
>   examples/coroutine/meson.build    |  10 ++
>   examples/meson.build              |   1 +
>   lib/coroutine/meson.build         |   8 ++
>   lib/coroutine/rte_coroutine.c     | 190 ++++++++++++++++++++++++++
>   lib/coroutine/rte_coroutine.h     | 110 +++++++++++++++
>   lib/coroutine/rte_coroutine_imp.h |  46 +++++++
>   lib/coroutine/version.map         |  11 ++
>   lib/meson.build                   |   1 +
>   13 files changed, 789 insertions(+), 1 deletion(-)
>   create mode 100644 examples/coroutine/main.c
>   create mode 100644 examples/coroutine/meson.build
>   create mode 100644 lib/coroutine/meson.build
>   create mode 100644 lib/coroutine/rte_coroutine.c
>   create mode 100644 lib/coroutine/rte_coroutine.h
>   create mode 100644 lib/coroutine/rte_coroutine_imp.h
>   create mode 100644 lib/coroutine/version.map
>