Re: [RFC] eal: per-thread constructors/destructors

["Mattias Rönnblom" <mattias.ronnblom@ericsson.com>]

On 2022-12-21 17:12, David Marchand wrote:
> On Sun, Dec 18, 2022 at 11:01 AM Morten Brørup <mb@smartsharesystems.com> wrote:
>>
>> This RFC introduces per-thread constructors/destructors:
>> * Per-thread constructors are functions called as the first thing from newly created threads. This can be used to initialize variables in thread-local storage, variables depending on the (tid_t) thread id, and to call setup functions that must be called from the thread itself.
>> * Per-thread destructors are functions called (from the thread) as the last thing when a thread ends.
>>
>> At this time, I am seeking feedback on the concept and the proposed limitations.
>>
>> Processes have __attribute__(constructor/destructor) to set up functions to be called before main(). Nothing similar exists for threads, so we have to design it ourselves.
>>
>> The proposed per-thread constructors/destructors should not apply to all threads - only to threads created through the DPDK threads API. Agree?
>>
>> DPDK has the RTE_INIT()/RTE_FINI() macros for adding process constructors/destructors at build time, so I propose a similar API, i.e. adding RTE_THREAD_INIT() and RTE_THREAD_FINI() macros to build a list of per-thread constructors and destructors at build time.
>>
>> Two functions that call the list of per-thread constructors respectively destructors will be introduced.
>>
>> The per-thread constructor function will be called from the newly created threads within DPDK:
>> 1. EAL/SERVICE threads: In eal_thread_loop() before the loop.
>> 2. CTRL threads: In ctrl_thread_init() before start_routine().
>> 3. Non-EAL threads: In rte_thread_register().
>>
>> Are any thread types missing in the list above?
> 
> I noticed a few drivers doing stuff behind EAL back.. you might want
> to grep pthread_create.
> They should probably be converted to control threads, or if there is a
> reason, it could give more inputs on this topic.
> 
> 
>>
>>
>> The per-thread destructor function will also be registered by the per-thread constructor, using the POSIX pthread_cleanup_push() function.
> 
> What about Windows?

I don't think there's a need to involve pthread_cleanup_push(). For 
DPDK-created threads, EAL could run the destructor in a wrapper 
function. The main lcore's fini callbacks could be run in rte_eal_cleanup().

> Cc: Tyler.
> 
>  >>
>>
>> Examples of where per-thread constructors are useful:
>> PRNG state initialization [1]: http://inbox.dpdk.org/dev/2a5121c7-369f-afde-0898-d45a5b368c3a@ericsson.com/
>> PMU event setup [2]: http://inbox.dpdk.org/dev/a059c403-8c6c-79c3-c3e9-a8c1815f5a14@ericsson.com/T/#m3d29fbc301059f007425ac148a4113e66d2ef983
>>
>>
>> Please also refer to the discussion [3] about lcore init callbacks for further thoughts. Especially the de-facto inability to establish constructors/destructors at runtime.
>>
>> [3]: http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35D875B6@smartserver.smartshare.dk/T/#m23b9a930a4050dc6b0305d3653754bd152c09ab7
> 
> 
>