* [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets
@ 2023-05-23 13:06 Anatoly Burakov
2023-05-23 13:55 ` Bruce Richardson
0 siblings, 1 reply; 3+ messages in thread
From: Anatoly Burakov @ 2023-05-23 13:06 UTC (permalink / raw)
To: dev; +Cc: david.marchand, kaisenx.you, thomas
Currently, it is allowed to specify a cpuset for lcores such that it
will include physical cores from different NUMA nodes. This has an
effect of setting `rte_socket_id()` for that lcore to SOCKET_ID_ANY,
so add a warning about it.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/eal/common/eal_common_thread.c | 20 +++++++++++++++++---
1 file changed, 17 insertions(+), 3 deletions(-)
diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
index 079a385630..46fd2aca1e 100644
--- a/lib/eal/common/eal_common_thread.c
+++ b/lib/eal/common/eal_common_thread.c
@@ -65,10 +65,13 @@ static void
thread_update_affinity(rte_cpuset_t *cpusetp)
{
unsigned int lcore_id = rte_lcore_id();
+ int socket_id;
+
+ /* find socket ID from cpuset */
+ socket_id = eal_cpuset_socket_id(cpusetp);
/* store socket_id in TLS for quick access */
- RTE_PER_LCORE(_socket_id) =
- eal_cpuset_socket_id(cpusetp);
+ RTE_PER_LCORE(_socket_id) = socket_id;
/* store cpuset in TLS for quick access */
memmove(&RTE_PER_LCORE(_cpuset), cpusetp,
@@ -76,9 +79,20 @@ thread_update_affinity(rte_cpuset_t *cpusetp)
if (lcore_id != (unsigned)LCORE_ID_ANY) {
/* EAL thread will update lcore_config */
- lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id);
+ lcore_config[lcore_id].socket_id = socket_id;
memmove(&lcore_config[lcore_id].cpuset, cpusetp,
sizeof(rte_cpuset_t));
+
+ /*
+ * lcore_id is not LCORE_ID_ANY, meaning this is a DPDK lcore,
+ * so having a valid NUMA affinity for this lcore is important.
+ * However, if cpuset includes cores from multiple NUMA nodes,
+ * the socket ID will be set to SOCKET_ID_ANY. Notify the user
+ * about it if that happens.
+ */
+ if (socket_id == SOCKET_ID_ANY)
+ RTE_LOG(INFO, EAL, "DPDK lcore %u has NUMA affinity set to SOCKET_ID_ANY\n",
+ lcore_id);
}
}
--
2.37.2
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets
2023-05-23 13:06 [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets Anatoly Burakov
@ 2023-05-23 13:55 ` Bruce Richardson
2023-05-23 14:29 ` Burakov, Anatoly
0 siblings, 1 reply; 3+ messages in thread
From: Bruce Richardson @ 2023-05-23 13:55 UTC (permalink / raw)
To: Anatoly Burakov; +Cc: dev, david.marchand, kaisenx.you, thomas
On Tue, May 23, 2023 at 01:06:16PM +0000, Anatoly Burakov wrote:
> Currently, it is allowed to specify a cpuset for lcores such that it
> will include physical cores from different NUMA nodes. This has an
> effect of setting `rte_socket_id()` for that lcore to SOCKET_ID_ANY,
> so add a warning about it.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> lib/eal/common/eal_common_thread.c | 20 +++++++++++++++++---
> 1 file changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
> index 079a385630..46fd2aca1e 100644
> --- a/lib/eal/common/eal_common_thread.c
> +++ b/lib/eal/common/eal_common_thread.c
> @@ -65,10 +65,13 @@ static void
> thread_update_affinity(rte_cpuset_t *cpusetp)
> {
> unsigned int lcore_id = rte_lcore_id();
> + int socket_id;
> +
> + /* find socket ID from cpuset */
> + socket_id = eal_cpuset_socket_id(cpusetp);
>
> /* store socket_id in TLS for quick access */
> - RTE_PER_LCORE(_socket_id) =
> - eal_cpuset_socket_id(cpusetp);
> + RTE_PER_LCORE(_socket_id) = socket_id;
>
> /* store cpuset in TLS for quick access */
> memmove(&RTE_PER_LCORE(_cpuset), cpusetp,
> @@ -76,9 +79,20 @@ thread_update_affinity(rte_cpuset_t *cpusetp)
>
> if (lcore_id != (unsigned)LCORE_ID_ANY) {
> /* EAL thread will update lcore_config */
> - lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id);
> + lcore_config[lcore_id].socket_id = socket_id;
> memmove(&lcore_config[lcore_id].cpuset, cpusetp,
> sizeof(rte_cpuset_t));
> +
> + /*
> + * lcore_id is not LCORE_ID_ANY, meaning this is a DPDK lcore,
> + * so having a valid NUMA affinity for this lcore is important.
> + * However, if cpuset includes cores from multiple NUMA nodes,
> + * the socket ID will be set to SOCKET_ID_ANY. Notify the user
> + * about it if that happens.
> + */
> + if (socket_id == SOCKET_ID_ANY)
> + RTE_LOG(INFO, EAL, "DPDK lcore %u has NUMA affinity set to SOCKET_ID_ANY\n",
> + lcore_id);
> }
> }
While having a warning comment in the code is good, should we not also have
a user visible warning to the user when the specific a corelist parameter
value which includes the cross-socket scenario?
/Bruce
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets
2023-05-23 13:55 ` Bruce Richardson
@ 2023-05-23 14:29 ` Burakov, Anatoly
0 siblings, 0 replies; 3+ messages in thread
From: Burakov, Anatoly @ 2023-05-23 14:29 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev, david.marchand, kaisenx.you, thomas
On 5/23/2023 2:55 PM, Bruce Richardson wrote:
> On Tue, May 23, 2023 at 01:06:16PM +0000, Anatoly Burakov wrote:
>> Currently, it is allowed to specify a cpuset for lcores such that it
>> will include physical cores from different NUMA nodes. This has an
>> effect of setting `rte_socket_id()` for that lcore to SOCKET_ID_ANY,
>> so add a warning about it.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>> lib/eal/common/eal_common_thread.c | 20 +++++++++++++++++---
>> 1 file changed, 17 insertions(+), 3 deletions(-)
>>
>> diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
>> index 079a385630..46fd2aca1e 100644
>> --- a/lib/eal/common/eal_common_thread.c
>> +++ b/lib/eal/common/eal_common_thread.c
>> @@ -65,10 +65,13 @@ static void
>> thread_update_affinity(rte_cpuset_t *cpusetp)
>> {
>> unsigned int lcore_id = rte_lcore_id();
>> + int socket_id;
>> +
>> + /* find socket ID from cpuset */
>> + socket_id = eal_cpuset_socket_id(cpusetp);
>>
>> /* store socket_id in TLS for quick access */
>> - RTE_PER_LCORE(_socket_id) =
>> - eal_cpuset_socket_id(cpusetp);
>> + RTE_PER_LCORE(_socket_id) = socket_id;
>>
>> /* store cpuset in TLS for quick access */
>> memmove(&RTE_PER_LCORE(_cpuset), cpusetp,
>> @@ -76,9 +79,20 @@ thread_update_affinity(rte_cpuset_t *cpusetp)
>>
>> if (lcore_id != (unsigned)LCORE_ID_ANY) {
>> /* EAL thread will update lcore_config */
>> - lcore_config[lcore_id].socket_id = RTE_PER_LCORE(_socket_id);
>> + lcore_config[lcore_id].socket_id = socket_id;
>> memmove(&lcore_config[lcore_id].cpuset, cpusetp,
>> sizeof(rte_cpuset_t));
>> +
>> + /*
>> + * lcore_id is not LCORE_ID_ANY, meaning this is a DPDK lcore,
>> + * so having a valid NUMA affinity for this lcore is important.
>> + * However, if cpuset includes cores from multiple NUMA nodes,
>> + * the socket ID will be set to SOCKET_ID_ANY. Notify the user
>> + * about it if that happens.
>> + */
>> + if (socket_id == SOCKET_ID_ANY)
>> + RTE_LOG(INFO, EAL, "DPDK lcore %u has NUMA affinity set to SOCKET_ID_ANY\n",
>> + lcore_id);
>> }
>> }
> While having a warning comment in the code is good, should we not also have
> a user visible warning to the user when the specific a corelist parameter
> value which includes the cross-socket scenario?
>
> /Bruce
We could, yes. We could walk through the cpuset for a given lcore, and
list their socket ID's, would that work?
--
Thanks,
Anatoly
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-05-23 14:29 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-23 13:06 [PATCH v1 1/1] eal: warn user when lcore cpuset includes multiple sockets Anatoly Burakov
2023-05-23 13:55 ` Bruce Richardson
2023-05-23 14:29 ` Burakov, Anatoly
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).