* [dpdk-dev] Retrieving lcore worker thread id @ 2020-07-14 12:03 Mattias Rönnblom 2020-07-14 13:08 ` Van Haaren, Harry 0 siblings, 1 reply; 11+ messages in thread From: Mattias Rönnblom @ 2020-07-14 12:03 UTC (permalink / raw) To: dpdk-dev; +Cc: Tomasz Piatkowski Hi. In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made private, and with it the possibility to look up the thread id of the lcore worker threads disappears. One use case is an application with a monitoring function (on some control plane thread), which uses the thread ids to make sure the worker threads gets the CPU runtime they should, and thus is able to detect stalls. Is there some other way of finding out the thread_id of a lcore worker thread? All I can think of are hacks like using a temporary service function for service cores, in combination with requiring launched application threads also to store their thread id in some global structure (index by lcore_id). Is there some cleaner way? If not, would adding something like a rte_lcore_thread_id() function make sense? Regards, Mattias ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-14 12:03 [dpdk-dev] Retrieving lcore worker thread id Mattias Rönnblom @ 2020-07-14 13:08 ` Van Haaren, Harry 2020-07-14 18:59 ` Honnappa Nagarahalli 2020-07-15 10:21 ` [dpdk-dev] Retrieving lcore worker " Mattias Rönnblom 0 siblings, 2 replies; 11+ messages in thread From: Van Haaren, Harry @ 2020-07-14 13:08 UTC (permalink / raw) To: Mattias Rönnblom, dpdk-dev; +Cc: Tomasz Piatkowski > -----Original Message----- > From: dev <dev-bounces@dpdk.org> On Behalf Of Mattias Rönnblom > Sent: Tuesday, July 14, 2020 1:04 PM > To: dpdk-dev <dev@dpdk.org> > Cc: Tomasz Piatkowski <tomasz.piatkowski@ericsson.com> > Subject: [dpdk-dev] Retrieving lcore worker thread id > > Hi. Hey, > In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made private, > and with it the possibility to look up the thread id of the lcore worker > threads disappears. > > One use case is an application with a monitoring function (on some > control plane thread), which uses the thread ids to make sure the worker > threads gets the CPU runtime they should, and thus is able to detect > stalls. > > Is there some other way of finding out the thread_id of a lcore worker > thread? All I can think of are hacks like using a temporary service > function for service cores, in combination with requiring launched > application threads also to store their thread id in some global > structure (index by lcore_id). -1 for the service cores idea. I like the creative solution thinking, but not as a long-term solution. > Is there some cleaner way? If not, would adding something like a > rte_lcore_thread_id() function make sense? I think given your use-case description above, this makes sense. I guess it comes down to a discussion around if thread_id should be exposed at all - but given you have a use I'd argue yes it should. Doing so in a cross-platform way might be a bit tricky though. > Regards, > Mattias Regards, -Harry ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-14 13:08 ` Van Haaren, Harry @ 2020-07-14 18:59 ` Honnappa Nagarahalli 2020-07-14 20:51 ` Stephen Hemminger 2020-07-15 10:21 ` [dpdk-dev] Retrieving lcore worker " Mattias Rönnblom 1 sibling, 1 reply; 11+ messages in thread From: Honnappa Nagarahalli @ 2020-07-14 18:59 UTC (permalink / raw) To: Van Haaren, Harry, Mattias Rönnblom, dpdk-dev Cc: Tomasz Piatkowski, nd, Honnappa Nagarahalli, nd <snip> > > > > Hi. > > Hey, > > > In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made > > private, and with it the possibility to look up the thread id of the > > lcore worker threads disappears. > > > > One use case is an application with a monitoring function (on some > > control plane thread), which uses the thread ids to make sure the > > worker threads gets the CPU runtime they should, and thus is able to > > detect stalls. This sounds similar to 'keep alive' functionality. > > > > Is there some other way of finding out the thread_id of a lcore worker > > thread? All I can think of are hacks like using a temporary service > > function for service cores, in combination with requiring launched > > application threads also to store their thread id in some global > > structure (index by lcore_id). > > -1 for the service cores idea. I like the creative solution thinking, but not as a > long-term solution. > > > Is there some cleaner way? If not, would adding something like a > > rte_lcore_thread_id() function make sense? I guess here you mean the OS provided thread ID. Are there OS calls that provide the CPU runtime? > > I think given your use-case description above, this makes sense. > I guess it comes down to a discussion around if thread_id should be exposed > at all - but given you have a use I'd argue yes it should. > Doing so in a cross-platform way might be a bit tricky though. > > > Regards, > > Mattias > > Regards, -Harry ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-14 18:59 ` Honnappa Nagarahalli @ 2020-07-14 20:51 ` Stephen Hemminger 2020-07-15 10:17 ` Mattias Rönnblom 0 siblings, 1 reply; 11+ messages in thread From: Stephen Hemminger @ 2020-07-14 20:51 UTC (permalink / raw) To: Honnappa Nagarahalli Cc: Van Haaren, Harry, Mattias Rönnblom, dpdk-dev, Tomasz Piatkowski, nd On Tue, 14 Jul 2020 18:59:59 +0000 Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote: > <snip> > > > > > > > Hi. > > > > Hey, > > > > > In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made > > > private, and with it the possibility to look up the thread id of the > > > lcore worker threads disappears. > > > > > > One use case is an application with a monitoring function (on some > > > control plane thread), which uses the thread ids to make sure the > > > worker threads gets the CPU runtime they should, and thus is able to > > > detect stalls. > This sounds similar to 'keep alive' functionality. > > > > > > > Is there some other way of finding out the thread_id of a lcore worker > > > thread? All I can think of are hacks like using a temporary service > > > function for service cores, in combination with requiring launched > > > application threads also to store their thread id in some global > > > structure (index by lcore_id). > > > > -1 for the service cores idea. I like the creative solution thinking, but not as a > > long-term solution. > > > > > Is there some cleaner way? If not, would adding something like a > > > rte_lcore_thread_id() function make sense? > I guess here you mean the OS provided thread ID. Are there OS calls that provide the CPU runtime? This might be difficult sinc thread id in Linux/glibc is intentionally and opaque value. According to Posix the only valid way to look at it is to use return value from pthread_create() and pthread_self(). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-14 20:51 ` Stephen Hemminger @ 2020-07-15 10:17 ` Mattias Rönnblom 2020-07-15 16:21 ` Stephen Hemminger 0 siblings, 1 reply; 11+ messages in thread From: Mattias Rönnblom @ 2020-07-15 10:17 UTC (permalink / raw) To: Stephen Hemminger, Honnappa Nagarahalli Cc: Van Haaren, Harry, dpdk-dev, Tomasz Piatkowski, nd On 2020-07-14 22:51, Stephen Hemminger wrote: > On Tue, 14 Jul 2020 18:59:59 +0000 > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote: > >> <snip> >> >>>> Hi. >>> Hey, >>> >>>> In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made >>>> private, and with it the possibility to look up the thread id of the >>>> lcore worker threads disappears. >>>> >>>> One use case is an application with a monitoring function (on some >>>> control plane thread), which uses the thread ids to make sure the >>>> worker threads gets the CPU runtime they should, and thus is able to >>>> detect stalls. >> This sounds similar to 'keep alive' functionality. >> >>>> Is there some other way of finding out the thread_id of a lcore worker >>>> thread? All I can think of are hacks like using a temporary service >>>> function for service cores, in combination with requiring launched >>>> application threads also to store their thread id in some global >>>> structure (index by lcore_id). >>> -1 for the service cores idea. I like the creative solution thinking, but not as a >>> long-term solution. >>> >>>> Is there some cleaner way? If not, would adding something like a >>>> rte_lcore_thread_id() function make sense? >> I guess here you mean the OS provided thread ID. Are there OS calls that provide the CPU runtime? > This might be difficult sinc thread id in Linux/glibc is intentionally and opaque value. > According to Posix the only valid way to look at it is to use return value from > pthread_create() and pthread_self(). > The rte_lcore_thread_id() would return this value, which could subsequently be used in the application, calling pthread_getcpuclockid() and clock_gettime() to retrieve the run time for the lcore worker thread. No need to break the opacity in this case, although the Linux thread id (i.e. the result of a gettid()) would be useful in case you would want to dig around in /proc for other scheduler statistics. Regards, Mattias ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-15 10:17 ` Mattias Rönnblom @ 2020-07-15 16:21 ` Stephen Hemminger 2020-07-16 10:20 ` Mattias Rönnblom 0 siblings, 1 reply; 11+ messages in thread From: Stephen Hemminger @ 2020-07-15 16:21 UTC (permalink / raw) To: Mattias Rönnblom Cc: Honnappa Nagarahalli, Van Haaren, Harry, dpdk-dev, Tomasz Piatkowski, nd On Wed, 15 Jul 2020 10:17:09 +0000 Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote: > On 2020-07-14 22:51, Stephen Hemminger wrote: > > On Tue, 14 Jul 2020 18:59:59 +0000 > > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote: > > > >> <snip> > >> > >>>> Hi. > >>> Hey, > >>> > >>>> In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made > >>>> private, and with it the possibility to look up the thread id of the > >>>> lcore worker threads disappears. > >>>> > >>>> One use case is an application with a monitoring function (on some > >>>> control plane thread), which uses the thread ids to make sure the > >>>> worker threads gets the CPU runtime they should, and thus is able to > >>>> detect stalls. > >> This sounds similar to 'keep alive' functionality. > >> > >>>> Is there some other way of finding out the thread_id of a lcore worker > >>>> thread? All I can think of are hacks like using a temporary service > >>>> function for service cores, in combination with requiring launched > >>>> application threads also to store their thread id in some global > >>>> structure (index by lcore_id). > >>> -1 for the service cores idea. I like the creative solution thinking, but not as a > >>> long-term solution. > >>> > >>>> Is there some cleaner way? If not, would adding something like a > >>>> rte_lcore_thread_id() function make sense? > >> I guess here you mean the OS provided thread ID. Are there OS calls that provide the CPU runtime? > > This might be difficult sinc thread id in Linux/glibc is intentionally and opaque value. > > According to Posix the only valid way to look at it is to use return value from > > pthread_create() and pthread_self(). > > > > The rte_lcore_thread_id() would return this value, which could > subsequently be used in the application, calling pthread_getcpuclockid() > and clock_gettime() to retrieve the run time for the lcore worker > thread. No need to break the opacity in this case, although the Linux > thread id (i.e. the result of a gettid()) would be useful in case you > would want to dig around in /proc for other scheduler statistics. > > > Regards, > > Mattias > The issue is glibc doesn't want to allow gettid() there is no wrapper, the only way to get it is using syscall() ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-15 16:21 ` Stephen Hemminger @ 2020-07-16 10:20 ` Mattias Rönnblom 2020-08-10 9:36 ` [dpdk-dev] [RFC] eal: introduce function to get lcore " Mattias Rönnblom 0 siblings, 1 reply; 11+ messages in thread From: Mattias Rönnblom @ 2020-07-16 10:20 UTC (permalink / raw) To: Stephen Hemminger Cc: Honnappa Nagarahalli, Van Haaren, Harry, dpdk-dev, Tomasz Piatkowski, nd On 2020-07-15 18:21, Stephen Hemminger wrote: > On Wed, 15 Jul 2020 10:17:09 +0000 > Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote: > >> On 2020-07-14 22:51, Stephen Hemminger wrote: >>> On Tue, 14 Jul 2020 18:59:59 +0000 >>> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote: >>> >>>> <snip> >>>> >>>>>> Hi. >>>>> Hey, >>>>> >>>>>> In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made >>>>>> private, and with it the possibility to look up the thread id of the >>>>>> lcore worker threads disappears. >>>>>> >>>>>> One use case is an application with a monitoring function (on some >>>>>> control plane thread), which uses the thread ids to make sure the >>>>>> worker threads gets the CPU runtime they should, and thus is able to >>>>>> detect stalls. >>>> This sounds similar to 'keep alive' functionality. >>>> >>>>>> Is there some other way of finding out the thread_id of a lcore worker >>>>>> thread? All I can think of are hacks like using a temporary service >>>>>> function for service cores, in combination with requiring launched >>>>>> application threads also to store their thread id in some global >>>>>> structure (index by lcore_id). >>>>> -1 for the service cores idea. I like the creative solution thinking, but not as a >>>>> long-term solution. >>>>> >>>>>> Is there some cleaner way? If not, would adding something like a >>>>>> rte_lcore_thread_id() function make sense? >>>> I guess here you mean the OS provided thread ID. Are there OS calls that provide the CPU runtime? >>> This might be difficult sinc thread id in Linux/glibc is intentionally and opaque value. >>> According to Posix the only valid way to look at it is to use return value from >>> pthread_create() and pthread_self(). >>> >> The rte_lcore_thread_id() would return this value, which could >> subsequently be used in the application, calling pthread_getcpuclockid() >> and clock_gettime() to retrieve the run time for the lcore worker >> thread. No need to break the opacity in this case, although the Linux >> thread id (i.e. the result of a gettid()) would be useful in case you >> would want to dig around in /proc for other scheduler statistics. >> >> >> Regards, >> >> Mattias >> > The issue is glibc doesn't want to allow gettid() > there is no wrapper, the only way to get it is using syscall() There was certainly a lot of reluctance to add it, but since glibc 2.30, there is a syscall wrapper in place. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [dpdk-dev] [RFC] eal: introduce function to get lcore thread id 2020-07-16 10:20 ` Mattias Rönnblom @ 2020-08-10 9:36 ` Mattias Rönnblom 2020-10-20 13:38 ` David Marchand 0 siblings, 1 reply; 11+ messages in thread From: Mattias Rönnblom @ 2020-08-10 9:36 UTC (permalink / raw) To: Ray Kinsella, Neil Horman Cc: dev, tomasz.piatkowski, nd, stephen, Honnappa.Nagarahalli, harry.van.haaren, Mattias Rönnblom Introduce function in <rte_lcore.h> to retrieve a lcore worker thread's pthread id. This change allows applications to do things like changing the name of a lcore thread, or detect if it's getting interrupted (e.g. getting preempted by another thread). Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com> --- lib/librte_eal/common/eal_common_thread.c | 8 ++++++++ lib/librte_eal/include/rte_lcore.h | 14 ++++++++++++++ lib/librte_eal/rte_eal_version.map | 3 +++ 3 files changed, 25 insertions(+) diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c index 73a055902..6233aef11 100644 --- a/lib/librte_eal/common/eal_common_thread.c +++ b/lib/librte_eal/common/eal_common_thread.c @@ -294,3 +294,11 @@ rte_thread_unregister(void) RTE_LOG(DEBUG, EAL, "Unregistered non-EAL thread (was lcore %u).\n", lcore_id); } + +int rte_lcore_thread_id(unsigned int lcore_id, pthread_t *id) +{ + if (lcore_id >= RTE_MAX_LCORE) + return -1; + *id = lcore_config[lcore_id].thread_id; + return 0; +} diff --git a/lib/librte_eal/include/rte_lcore.h b/lib/librte_eal/include/rte_lcore.h index b8b64a625..dae8d11ca 100644 --- a/lib/librte_eal/include/rte_lcore.h +++ b/lib/librte_eal/include/rte_lcore.h @@ -363,6 +363,20 @@ int rte_thread_set_affinity(rte_cpuset_t *cpusetp); */ void rte_thread_get_affinity(rte_cpuset_t *cpusetp); +/** + * Get the thread id of a lcore worker thread. + * + * @param lcore_id + * The identifier of the lcore, which MUST be between 0 and RTE_MAX_LCORE-1. + * @param[out] id + * A pointer that will be filled in with the lcore worker thread's POSIX + * thread id, on success. + * @return + * On success, return 0; otherwise return a negative value. + */ +__rte_experimental +int rte_lcore_thread_id(unsigned int lcore_id, pthread_t *id); + /** * Set thread names. * diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index bf0c17c23..571aacd17 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -403,6 +403,9 @@ EXPERIMENTAL { rte_mp_disable; rte_thread_register; rte_thread_unregister; + + # added in 20.11 + rte_lcore_thread_id; }; INTERNAL { -- 2.25.1 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] [RFC] eal: introduce function to get lcore thread id 2020-08-10 9:36 ` [dpdk-dev] [RFC] eal: introduce function to get lcore " Mattias Rönnblom @ 2020-10-20 13:38 ` David Marchand 2020-10-20 18:40 ` Mattias Rönnblom 0 siblings, 1 reply; 11+ messages in thread From: David Marchand @ 2020-10-20 13:38 UTC (permalink / raw) To: Mattias Rönnblom Cc: Ray Kinsella, Neil Horman, dev, tomasz.piatkowski, nd, Stephen Hemminger, Honnappa Nagarahalli, Van Haaren Harry, Dmitry Kozlyuk On Mon, Aug 10, 2020 at 11:37 AM Mattias Rönnblom <mattias.ronnblom@ericsson.com> wrote: > > Introduce function in <rte_lcore.h> to retrieve a lcore worker > thread's pthread id. > > This change allows applications to do things like changing the name of > a lcore thread, or detect if it's getting interrupted (e.g. getting > preempted by another thread). > > Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com> I can see no feedback. I am a bit reluctant at giving access to this info which is not really abstracted across OS.. Is there no other way to achieve what you wanted? -- David Marchand ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] [RFC] eal: introduce function to get lcore thread id 2020-10-20 13:38 ` David Marchand @ 2020-10-20 18:40 ` Mattias Rönnblom 0 siblings, 0 replies; 11+ messages in thread From: Mattias Rönnblom @ 2020-10-20 18:40 UTC (permalink / raw) To: David Marchand Cc: Ray Kinsella, Neil Horman, dev, Tomasz Piatkowski, nd, Stephen Hemminger, Honnappa Nagarahalli, Van Haaren Harry, Dmitry Kozlyuk On 2020-10-20 15:38, David Marchand wrote: > On Mon, Aug 10, 2020 at 11:37 AM Mattias Rönnblom > <mattias.ronnblom@ericsson.com> wrote: >> Introduce function in <rte_lcore.h> to retrieve a lcore worker >> thread's pthread id. >> >> This change allows applications to do things like changing the name of >> a lcore thread, or detect if it's getting interrupted (e.g. getting >> preempted by another thread). >> >> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com> > I can see no feedback. > I am a bit reluctant at giving access to this info which is not really > abstracted across OS.. > Is there no other way to achieve what you wanted? > > The fact that POSIX threading, or at least such thread identifiers, are used internally is already visible in the lcore API. What we are using now is a little hack with a service started on each core, which calls pthead_self(), stores the result, and turns itself off. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] Retrieving lcore worker thread id 2020-07-14 13:08 ` Van Haaren, Harry 2020-07-14 18:59 ` Honnappa Nagarahalli @ 2020-07-15 10:21 ` Mattias Rönnblom 1 sibling, 0 replies; 11+ messages in thread From: Mattias Rönnblom @ 2020-07-15 10:21 UTC (permalink / raw) To: Van Haaren, Harry, dpdk-dev; +Cc: Tomasz Piatkowski On 2020-07-14 15:08, Van Haaren, Harry wrote: >> -----Original Message----- >> From: dev <dev-bounces@dpdk.org> On Behalf Of Mattias Rönnblom >> Sent: Tuesday, July 14, 2020 1:04 PM >> To: dpdk-dev <dev@dpdk.org> >> Cc: Tomasz Piatkowski <tomasz.piatkowski@ericsson.com> >> Subject: [dpdk-dev] Retrieving lcore worker thread id >> >> Hi. > Hey, > >> In DPDK 19.11, the lcore_config struct of <rte_lcore.h> is made private, >> and with it the possibility to look up the thread id of the lcore worker >> threads disappears. >> >> One use case is an application with a monitoring function (on some >> control plane thread), which uses the thread ids to make sure the worker >> threads gets the CPU runtime they should, and thus is able to detect >> stalls. >> >> Is there some other way of finding out the thread_id of a lcore worker >> thread? All I can think of are hacks like using a temporary service >> function for service cores, in combination with requiring launched >> application threads also to store their thread id in some global >> structure (index by lcore_id). > -1 for the service cores idea. I like the creative solution thinking, > but not as a long-term solution. > >> Is there some cleaner way? If not, would adding something like a >> rte_lcore_thread_id() function make sense? > I think given your use-case description above, this makes sense. > I guess it comes down to a discussion around if thread_id should > be exposed at all - but given you have a use I'd argue yes it should. > Doing so in a cross-platform way might be a bit tricky though. > With the Windows port ongoing, I guess that's a valid concern. <rte_lcore.h> already is a heavy user of pthread ids though. Maybe you would leave that change to a later patch, which would typedef the whole thing into something DPDK specific and portable. Regards, Mattias ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-10-20 18:40 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-07-14 12:03 [dpdk-dev] Retrieving lcore worker thread id Mattias Rönnblom 2020-07-14 13:08 ` Van Haaren, Harry 2020-07-14 18:59 ` Honnappa Nagarahalli 2020-07-14 20:51 ` Stephen Hemminger 2020-07-15 10:17 ` Mattias Rönnblom 2020-07-15 16:21 ` Stephen Hemminger 2020-07-16 10:20 ` Mattias Rönnblom 2020-08-10 9:36 ` [dpdk-dev] [RFC] eal: introduce function to get lcore " Mattias Rönnblom 2020-10-20 13:38 ` David Marchand 2020-10-20 18:40 ` Mattias Rönnblom 2020-07-15 10:21 ` [dpdk-dev] Retrieving lcore worker " Mattias Rönnblom
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).