RE: RTE lock

["Morten Brørup" <mb@smartsharesystems.com>]

> From: Mattias Rönnblom [mailto:hofors@lysator.liu.se]
> Sent: Tuesday, 5 March 2024 21.18
> 
> Shouldn't we have a DPDK-native mutex API, rather than using direct
> POSIX mutex lock calls?
> 
> There are two reasons for this, as I see it
> 1) more cleanly support non-POSIX operating system (i.e., Microsoft
> Windows).
> 2) to discourage mechanical use of spinlocks in places where a regular
> mutex lock is more appropriate.
> 
> I think (and hope) DPDK developers will tend to pick DPDK-native rather
> than other APIs as their first choice.
> 
> For locks, they go for spinlocks, even in control (non-fast
> path/non-packet processing) code paths (e.g., calls made by the typical
> non-EAL thread).
> 
> Using spinlocks to synchronize threads that may be preempted aren't
> great idea.

The DPDK does interact with the control plane in ways that require locks, so adding preemption-friendly synchronization APIs seems appropriate.
And although I often rant about EAL bloat, I suppose this would belong into the EAL.

If you add any APIs with a timeout parameter, make sure to use struct timespec, like pthread_mutex_timedlock(). If a synchronization function is exclusively used for the control plane, millisecond resolution would probably suffice, but if it might be used by the data plane, millisecond level delays are intolerable. This was overlooked when adding rte_epoll_wait(), which was implemented using epoll_wait() instead of epoll_pwait2(), so you have to use the O/S specific timerfd_create/settime() functions for high-res timeout - and then the whole point of this EAL function goes out the window.

For reference, DPDK offers many types of locks [1]:
atomic, mcslock, pflock, rwlock, seqcount, seqlock, spinlock, ticketlock, RCU

[1] https://doc.dpdk.org/api/