Re: [PATCH v3 0/3] Add dispatcher library

["Mattias Rönnblom" <hofors@lysator.liu.se>]

On 2023-09-06 21:32, Stephen Hemminger wrote:
> On Mon, 4 Sep 2023 15:03:10 +0200
> Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote:
> 
>> The purpose of the dispatcher library is to decouple different parts
>> of an eventdev-based application (e.g., processing pipeline stages),
>> sharing the same underlying event device.
>>
>> The dispatcher replaces the conditional logic (often, a switch
>> statement) that typically follows an event device dequeue operation,
>> where events are dispatched to different parts of the application
>> based on event meta data, such as the queue id or scheduling type.
>>
>> The concept is similar to a UNIX file descriptor event loop library.
>> Instead of tying callback functions to fds as for example libevent
>> does, the dispatcher relies on application-supplied matching callback
>> functions to decide where to deliver events.
>>
>> A dispatcher is configured to dequeue events from a specific event
>> device, and ties into the service core framework, to do its (and the
>> application's) work.
>>
>> The dispatcher provides a convenient way for an eventdev-based
>> application to use service cores for application-level processing, and
>> thus for sharing those cores with other DPDK services.
>>
>> Although the dispatcher adds some overhead, experience suggests that
>> the net effect on the application (both synthetic benchmarks and more
>> real-world applications) may well be positive. This is primarily due
>> to clustering (see programming guide) reducing cache misses.
>>
>> Benchmarking indicates that the overhead is ~10 cc/event (on a
>> large core), with a handful of often-used handlers.
>>
>> The dispatcher does not support run-time reconfiguration.
>>
>> The use of the dispatcher library is optional, and an eventdev-based
>> application may still opt to access the event device using direct
>> eventdev API calls, or by some other means.
> 
> My experience with event libraries is mixed.
> There are already multiple choices libevent, libev, and libsystemd as
> well as rte_epoll().  Not sure if adding another one is helpful.
> 

This library *conceptually* provides the same kind of functionality as 
libevent, but has nothing to do with file descriptor events. These are 
eventdev events, and thus are tied to the arrival of a packet, a 
notification some kind of hardware offload, a timeout, or something else 
DPDK PMD-related.

> The main issue is dealing with external (non DPDK) events which usually
> are handled as file descriptors (signalfd, epoll, etc). The other issue
> is the thread safety. Most event libraries are not thread safe which
> makes handling one event waking up another difficult.
> 
This machinery is for use exclusively by EAL threads, for fast-path 
packet processing. No syscalls, no non-DPDK events.

> With DPDK, there is the additional questions about use from non-EAL threads.
> 

See above.

> For the test suite, you should look at what libsystemd does.
>