Re: [dpdk-dev] [PATCH v16 0/9] eal: Add EAL API for threading

[Thomas Monjalon <thomas@monjalon.net>]

09/10/2021 09:41, Narcisa Ana Maria Vasile:
> From: Narcisa Vasile <navasile@microsoft.com>
> 
> EAL thread API
> 
> **Problem Statement**
> DPDK currently uses the pthread interface to create and manage threads.
> Windows does not support the POSIX thread programming model,
> so it currently
> relies on a header file that hides the Windows calls under
> pthread matched interfaces. Given that EAL should isolate the environment
> specifics from the applications and libraries and mediate
> all the communication with the operating systems, a new EAL interface
> is needed for thread management.
> 
> **Goals**
> * Introduce a generic EAL API for threading support that will remove
>   the current Windows pthread.h shim.
> * Replace references to pthread_* across the DPDK codebase with the new
>   RTE_THREAD_* API.
> * Allow users to choose between using the RTE_THREAD_* API or a
>   3rd party thread library through a configuration option.
> 
> **Design plan**
> New API main files:
> * rte_thread.h (librte_eal/include)
> * rte_thread.c (librte_eal/windows)
> * rte_thread.c (librte_eal/common)

Why this file is not in lib/eal/unix/ ?


> **A schematic example of the design**
> --------------------------------------------------
> lib/librte_eal/include/rte_thread.h
> int rte_thread_create();
> 
> lib/librte_eal/common/rte_thread.c
> int rte_thread_create() 
> {
> 	return pthread_create();
> }
> 
> lib/librte_eal/windows/rte_thread.c
> int rte_thread_create() 
> {
> 	return CreateThread();
> }
> -----------------------------------------------------

We must have the same error code, no matter the underlying implementation.
So you cannot return directly pthread or win32 error codes.


> **Thread attributes**
> 
> When or after a thread is created, specific characteristics of the thread
> can be adjusted. Given that the thread characteristics that are of interest
> for DPDK applications are affinity and priority, the following structure
> that represents thread attributes has been defined:
> 
> typedef struct
> {
> 	enum rte_thread_priority priority;
> 	rte_cpuset_t cpuset;
> } rte_thread_attr_t;
> 
> The *rte_thread_create()* function can optionally receive
> an rte_thread_attr_t
> object that will cause the thread to be created with the
> affinity and priority
> described by the attributes object. If no rte_thread_attr_t is passed
> (parameter is NULL), the default affinity and priority are used.
> An rte_thread_attr_t object can also be set to the default values
> by calling *rte_thread_attr_init()*.
> 
> *Priority* is represented through an enum that currently advertises
> two values for priority:
> 	- RTE_THREAD_PRIORITY_NORMAL
> 	- RTE_THREAD_PRIORITY_REALTIME_CRITICAL

The priority level realtime should never used.

I am not sure about handling the priority so precisely.
I think we can abstract the priority need through different functions.
We already have the function rte_ctrl_thread_create() where priority
should be fixed.
I think we have only 2 types of threads:
	- control thread (interrupt, timer, IPC)
	- datapath lcore (created in rte_eal_init, including service cores)
It means we need only one new function for datapath thread creation.

> The enum can be extended to allow for multiple priority levels.
> rte_thread_set_priority      - sets the priority of a thread
> rte_thread_get_priority      - retrieves the priority of a thread
>                                from the OS
> rte_thread_attr_set_priority - updates an rte_thread_attr_t object
>                                with a new value for priority
> 
> The user can choose thread priority through an EAL parameter,
> when starting an application.  If EAL parameter is not used,
> the per-platform default value for thread priority is used.
> Otherwise administrator has an option to set one of available options:
>  --thread-prio normal
>  --thread-prio realtime

I don't think we need such feature.
Anyway, it is a new feature, so it is beyond the initial replacement goal.


> Example:
> ./dpdk-l2fwd -l 0-3 -n 4 –thread-prio normal -- -q 8 -p ffff
> 
> *Affinity* is described by the already known “rte_cpuset_t” type.
> rte_thread_attr_set/get_affinity - sets/gets the affinity field in a
>                                    rte_thread_attr_t object
> rte_thread_set/get_affinity      – sets/gets the affinity of a thread
> 
> **Errors**
> A translation function that maps Windows error codes to errno-style
> error codes is provided. 
> 
> **Future work**
> The long term plan is for EAL to provide full threading support:
> * Add support for conditional variables
> * Additional functionality offered by pthread_*
>   (such as pthread_setname_np, etc.)