[dpdk-dev] Does memif support primary<->secondary process communication?

[dpdk-dev] Does memif support primary<->secondary process communication?

[Wu, Jianyue (NSB - CN/Hangzhou)]

Hello,

May I ask a question about memif?
When I'm trying to develop a communication mechanism for two processes, which also need to share memzones between the processes, so need primary<->secondary.
https://doc.dpdk.org/guides/nics/memif.html
But from the guideline, it seems always use primary<->primary. Does it support primary<->secondary? Thanks indeed in advance!

Best Regards,
Dave(Jianyue)

Re: [dpdk-dev] Does memif support primary<->secondary process communication?

[Wu, Jianyue (NSB - CN/Hangzhou)]

BTW, I use it for primary and secondary communication, so that will use memif to share data between primary and secondary. Thanks.

Best Regards,
Dave(Jianyue)

From: Wu, Jianyue (NSB - CN/Hangzhou)
Sent: 2021年6月9日 19:26
To: users@dpdk.org; 'dev@dpdk.org' <dev@dpdk.org>
Subject: Does memif support primary<->secondary process communication?

Hello,

May I ask a question about memif?
When I’m trying to develop a communication mechanism for two processes, which also need to share memzones between the processes, so need primary<->secondary.
https://doc.dpdk.org/guides/nics/memif.html
But from the guideline, it seems always use primary<->primary. Does it support primary<->secondary? Thanks indeed in advance!

Best Regards,
Dave(Jianyue)

[PATCH] eal/linux: enhanced error handling for affinity

[Jianyue Wu (NSB)]

[-- Attachment #1.1: Type: text/plain, Size: 360 bytes --]

Hello,

Good day~
I hope this message finds you well. I am writing to submit a patch for consideration, which primarily adds enhanced error handling for affinity sets within the eal/linux of DPDK. Unfortunately, my current environment does not support git send-email, so I am sending this patch attached to this email.

Thank you~
Best regards,
Dave


[-- Attachment #1.2: Type: text/html, Size: 3047 bytes --]

[-- Attachment #2: 0001-eal-linux-enhanced-error-handling-for-affinity.patch --]
[-- Type: application/octet-stream, Size: 3326 bytes --]

From 89d879f762dc5e6901fc6cde06b8c36b641424de Mon Sep 17 00:00:00 2001
From: Jianyue Wu <jianyue.wu@nokia-sbell.com>
Date: Fri, 19 Apr 2024 13:57:35 +0300
Subject: [PATCH] eal/linux: enhanced error handling for affinity

Improve the robustness of setting thread affinity in DPDK
by adding detailed error logging.

Changes:
1. Set `errno` to 0 before calling `pthread_setaffinity_np()` to ensure
clean error status.
2. Check the return value of `pthread_setaffinity_np()` and log an error
if the call fails.
3. Include the current thread name, the intended CPU set, and a detailed
error message in the log.

Sample prints:
EAL: Cannot set affinity for thread dpdk-test with cpus 0,
ret: 22, errno: 0, error description: Success
EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
ret: 22, errno: 0, error description: Success

Signed-off-by: Jianyue Wu <jianyue.wu@nokia-sbell.com>
---
 lib/eal/unix/rte_thread.c | 51 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)

diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 1b4c73f58e..56bc7995eb 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -5,6 +5,7 @@
 
 #include <errno.h>
 #include <pthread.h>
+#include <sched.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -114,6 +115,35 @@ thread_start_wrapper(void *arg)
 	return (void *)(uintptr_t)thread_func(thread_args);
 }
 
+/* Function to convert cpu_set_t to a string. */
+static void cpuset_to_string(const cpu_set_t *cpuset,
+			char *cpus_str, size_t cpus_str_size) {
+	int cpu;
+	// Track the current position in the string
+	size_t offset = 0;
+
+	// Clear the string buffer
+	memset(cpus_str, 0, cpus_str_size);
+	cpus_str_size = RTE_MAX_LCORE < cpus_str_size ?
+		RTE_MAX_LCORE : cpus_str_size;
+
+	// Iterate over each CPU core, and check if it is included in the set
+	for (cpu = 0; cpu < RTE_MAX_LCORE && offset < cpus_str_size - 1; ++cpu) {
+		if (CPU_ISSET(cpu, cpuset)) {
+			// Append the current CPU number to the string
+			int written = snprintf(cpus_str + offset, cpus_str_size - offset,
+				"%s%d", (offset > 0 ? "," : ""), cpu);
+			if (written > 0)
+				offset += written;
+			if (offset >= cpus_str_size - 1)
+				break;
+			}
+	}
+
+	// Ensure the string is properly terminated
+	cpus_str[cpus_str_size - 1] = '\0';
+}
+
 int
 rte_thread_create(rte_thread_t *thread_id,
 		const rte_thread_attr_t *thread_attr,
@@ -369,8 +399,25 @@ int
 rte_thread_set_affinity_by_id(rte_thread_t thread_id,
 		const rte_cpuset_t *cpuset)
 {
-	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
-		sizeof(*cpuset), cpuset);
+	int ret;
+	char cpus_str[RTE_MAX_LCORE + 1] = {'\0'};
+	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
+
+	errno = 0;
+	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
+				sizeof(*cpuset), cpuset);
+	if (ret != 0) {
+		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
+					thread_name, sizeof(thread_name)) != 0)
+			EAL_LOG(ERR, "pthread_getname_np failed!");
+		cpuset_to_string(cpuset, cpus_str, sizeof(cpus_str));
+		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
+			"ret: %d, errno: %d, error description: %s",
+			thread_name, cpus_str,
+			ret, errno, strerror(errno));
+	}
+
+	return ret;
 }
 
 int
-- 
2.42.0

[PATCH] eal/linux: enhanced error handling for affinity

[Ferruh Yigit]

From: Jianyue Wu <jianyue.wu@nokia-sbell.com>

Improve the robustness of setting thread affinity in DPDK
by adding detailed error logging.

Changes:
1. Set `errno` to 0 before calling `pthread_setaffinity_np()` to ensure
clean error status.
2. Check the return value of `pthread_setaffinity_np()` and log an error
if the call fails.
3. Include the current thread name, the intended CPU set, and a detailed
error message in the log.

Sample prints:
EAL: Cannot set affinity for thread dpdk-test with cpus 0,
ret: 22, errno: 0, error description: Success
EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
ret: 22, errno: 0, error description: Success

Signed-off-by: Jianyue Wu <jianyue.wu@nokia-sbell.com>
---
 .mailmap                  |  1 +
 lib/eal/unix/rte_thread.c | 51 +++++++++++++++++++++++++++++++++++++--
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/.mailmap b/.mailmap
index 3843868716bd..66cff91f30a9 100644
--- a/.mailmap
+++ b/.mailmap
@@ -639,6 +639,7 @@ Jianfeng Tan <jianfeng.tan@intel.com>
 Jiangu Zhao <zhaojg@arraynetworks.com.cn>
 Jianwei Ma <jianwei.ma@intel.com>
 Jianwei Mei <jianweix.mei@intel.com>
+Jianyue Wu <jianyue.wu@nokia-sbell.com>
 Jiaqi Min <jiaqix.min@intel.com>
 Jiawei Wang <jiaweiw@nvidia.com> <jiaweiw@mellanox.com>
 Jiawei Zhu <zhujiawei12@huawei.com>
diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 1b4c73f58e91..56bc7995eb5f 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -5,6 +5,7 @@
 
 #include <errno.h>
 #include <pthread.h>
+#include <sched.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <string.h>
@@ -114,6 +115,35 @@ thread_start_wrapper(void *arg)
 	return (void *)(uintptr_t)thread_func(thread_args);
 }
 
+/* Function to convert cpu_set_t to a string. */
+static void cpuset_to_string(const cpu_set_t *cpuset,
+			char *cpus_str, size_t cpus_str_size) {
+	int cpu;
+	// Track the current position in the string
+	size_t offset = 0;
+
+	// Clear the string buffer
+	memset(cpus_str, 0, cpus_str_size);
+	cpus_str_size = RTE_MAX_LCORE < cpus_str_size ?
+		RTE_MAX_LCORE : cpus_str_size;
+
+	// Iterate over each CPU core, and check if it is included in the set
+	for (cpu = 0; cpu < RTE_MAX_LCORE && offset < cpus_str_size - 1; ++cpu) {
+		if (CPU_ISSET(cpu, cpuset)) {
+			// Append the current CPU number to the string
+			int written = snprintf(cpus_str + offset, cpus_str_size - offset,
+				"%s%d", (offset > 0 ? "," : ""), cpu);
+			if (written > 0)
+				offset += written;
+			if (offset >= cpus_str_size - 1)
+				break;
+			}
+	}
+
+	// Ensure the string is properly terminated
+	cpus_str[cpus_str_size - 1] = '\0';
+}
+
 int
 rte_thread_create(rte_thread_t *thread_id,
 		const rte_thread_attr_t *thread_attr,
@@ -369,8 +399,25 @@ int
 rte_thread_set_affinity_by_id(rte_thread_t thread_id,
 		const rte_cpuset_t *cpuset)
 {
-	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
-		sizeof(*cpuset), cpuset);
+	int ret;
+	char cpus_str[RTE_MAX_LCORE + 1] = {'\0'};
+	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
+
+	errno = 0;
+	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
+				sizeof(*cpuset), cpuset);
+	if (ret != 0) {
+		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
+					thread_name, sizeof(thread_name)) != 0)
+			EAL_LOG(ERR, "pthread_getname_np failed!");
+		cpuset_to_string(cpuset, cpus_str, sizeof(cpus_str));
+		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
+			"ret: %d, errno: %d, error description: %s",
+			thread_name, cpus_str,
+			ret, errno, strerror(errno));
+	}
+
+	return ret;
 }
 
 int
-- 
2.34.1

Re: [PATCH] eal/linux: enhanced error handling for affinity

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 296 bytes --]

Recheck-request: iol-compile-amd64-testing

The DPDK Community Lab updated to the latest Alpine image yesterday, which
resulted in all Alpine builds failing. The failure is unrelated to your
patch, and this recheck should remove the fail on Patchwork, as we have
disabled Alpine testing for now.

[-- Attachment #2: Type: text/html, Size: 361 bytes --]

Re: [PATCH] eal/linux: enhanced error handling for affinity

[Ferruh Yigit]

On 4/22/2024 2:23 PM, Jianyue Wu (NSB) wrote:
> Hello,
> 
>  
> 
> Good day~
> 
> I hope this message finds you well. I am writing to submit a patch for
> consideration, which primarily adds enhanced error handling for affinity
> sets within the eal/linux of DPDK. Unfortunately, my current environment
> does not support git send-email, so I am sending this patch attached to
> this email.
> 

Thanks Dave, I sent the patch for you, you can follow the CI test
results from patchwork link:
https://patchwork.dpdk.org/project/dpdk/patch/20240422134917.3740545-1-ferruh.yigit@amd.com/

Re: [PATCH] eal/linux: enhanced error handling for affinity

[Stephen Hemminger]

On Mon, 22 Apr 2024 13:23:50 +0000
"Jianyue Wu (NSB)" <jianyue.wu@nokia-sbell.com> wrote:

> +/* Function to convert cpu_set_t to a string. */
> +static void cpuset_to_string(const cpu_set_t *cpuset,
> +			char *cpus_str, size_t cpus_str_size) {
> +	int cpu;
> +	// Track the current position in the string

DPDK does not use C++ style comments.

All patches should match the style of existing code.

Re: [PATCH] eal/linux: enhanced error handling for affinity

[Stephen Hemminger]

On Mon, 22 Apr 2024 13:23:50 +0000
"Jianyue Wu (NSB)" <jianyue.wu@nokia-sbell.com> wrote:

> +	char cpus_str[RTE_MAX_LCORE + 1] = {'\0'};
> +	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
> +

You don't need '\0' here, just doing { } will have same effect.

Re: [PATCH] eal/linux: enhanced error handling for affinity

[Stephen Hemminger]

On Mon, 22 Apr 2024 13:23:50 +0000
"Jianyue Wu (NSB)" <jianyue.wu@nokia-sbell.com> wrote:

> +		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
> +			"ret: %d, errno: %d, error description: %s",
> +			thread_name, cpus_str,
> +			ret, errno, strerror(errno));
> +	}

That error message is way too long, and error strings should not be broken across lines.

			"Set affinity thread %s to %s failed: %s"

And it is a lot effort to deduce the cpuset and thread name here.
Probably better to just move the error message back to the caller.

RE: [PATCH] eal/linux: enhanced error handling for affinity

[Jianyue Wu (NSB)]

[-- Attachment #1: Type: text/plain, Size: 800 bytes --]

Hello,

Please disregard this email; I will use a different mailbox instead.

Thank you~
Best regards,
Dave

From: Jianyue Wu (NSB)
Sent: 2024年4月22日 21:24
To: 'roretzla@linux.microsoft.com' <roretzla@linux.microsoft.com>; 'thomas@monjalon.net' <thomas@monjalon.net>; 'david.marchand@redhat.com' <david.marchand@redhat.com>
Cc: 'dev@dpdk.org' <dev@dpdk.org>
Subject: [PATCH] eal/linux: enhanced error handling for affinity

Hello,

Good day~
I hope this message finds you well. I am writing to submit a patch for consideration, which primarily adds enhanced error handling for affinity sets within the eal/linux of DPDK. Unfortunately, my current environment does not support git send-email, so I am sending this patch attached to this email.

Thank you~
Best regards,
Dave


[-- Attachment #2: Type: text/html, Size: 5029 bytes --]

RE: [PATCH v2] eal/linux: enhanced error handling for affinity

[Jianyue Wu (NSB)]

[-- Attachment #1.1: Type: text/plain, Size: 1079 bytes --]

Hello,

I've noticed some coding style issues eal/linux: enhanced error handling for affinity - Patchwork (dpdk.org)<https://patches.dpdk.org/project/dpdk/patch/20240422134917.3740545-1-ferruh.yigit@amd.com/> and have attempted to update the patch accordingly. Could you please help by updating it with version 2 of the patch? Thank you very much! 😊

Thank you~
Best regards,
Dave

From: Jianyue Wu (NSB)
Sent: 2024年4月22日 21:24
To: 'roretzla@linux.microsoft.com' <roretzla@linux.microsoft.com>; 'thomas@monjalon.net' <thomas@monjalon.net>; 'david.marchand@redhat.com' <david.marchand@redhat.com>
Cc: 'dev@dpdk.org' <dev@dpdk.org>
Subject: [PATCH] eal/linux: enhanced error handling for affinity

Hello,

Good day~
I hope this message finds you well. I am writing to submit a patch for consideration, which primarily adds enhanced error handling for affinity sets within the eal/linux of DPDK. Unfortunately, my current environment does not support git send-email, so I am sending this patch attached to this email.

Thank you~
Best regards,
Dave


[-- Attachment #1.2: Type: text/html, Size: 5539 bytes --]

[-- Attachment #2: 0001-eal-linux-enhanced-error-handling-for-affinity.patch --]
[-- Type: application/octet-stream, Size: 2035 bytes --]

From bc7ed0f08373b0159a5990511f15312815bf36bd Mon Sep 17 00:00:00 2001
From: Jianyue Wu <jianyue.wu@nokia-sbell.com>
Date: Fri, 19 Apr 2024 13:57:35 +0300
Subject: [PATCH v2] eal/linux: enhanced error handling for affinity

Improve the robustness of setting thread affinity in DPDK
by adding detailed error logging.

Changes:
1. Check the return value of pthread_setaffinity_np() and log an error
if the call fails.
2. Include the current thread name, the intended CPU set, and a detailed
error message in the log.

Sample prints:
EAL: Cannot set affinity for thread dpdk-test with cpus 0,
ret: 22, errno: 0, error description: Success
EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
ret: 22, errno: 0, error description: Success

Signed-off-by: Jianyue Wu <jianyue.wu@nokia-sbell.com>
---
 lib/eal/unix/rte_thread.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 1b4c73f58e..8f9eaf0dcf 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -369,8 +369,26 @@ int
 rte_thread_set_affinity_by_id(rte_thread_t thread_id,
 		const rte_cpuset_t *cpuset)
 {
-	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
-		sizeof(*cpuset), cpuset);
+	int ret;
+	char cpus_str[RTE_CPU_AFFINITY_STR_LEN] = {'\0'};
+	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
+
+	errno = 0;
+	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
+				sizeof(*cpuset), cpuset);
+	if (ret != 0) {
+		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
+					thread_name, sizeof(thread_name)) != 0)
+			EAL_LOG(ERR, "pthread_getname_np failed!");
+		if (eal_thread_dump_affinity(cpuset, cpus_str, RTE_CPU_AFFINITY_STR_LEN) != 0)
+			EAL_LOG(ERR, "eal_thread_dump_affinity failed!");
+		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
+			"ret: %d, errno: %d, error description: %s",
+			thread_name, cpus_str,
+			ret, errno, strerror(errno));
+	}
+
+	return ret;
 }
 
 int
-- 
2.34.1

[PATCH v2] eal/linux: enhanced error handling for affinity

[Jianyue Wu]

Improve the robustness of setting thread affinity in DPDK
by adding detailed error logging.

Changes:
1. Check the return value of pthread_setaffinity_np() and log an error
if the call fails.
2. Include the current thread name, the intended CPU set, and a detailed
error message in the log.

Sample prints:
EAL: Cannot set affinity for thread dpdk-test with cpus 0,
ret: 22, errno: 0, error description: Success
EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
ret: 22, errno: 0, error description: Success

Signed-off-by: Jianyue Wu <wujianyue000@163.com>
---
 lib/eal/common/eal_common_thread.c |  2 +-
 lib/eal/common/eal_thread.h        |  2 +-
 lib/eal/unix/rte_thread.c          | 22 ++++++++++++++++++++--
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
index a53bc639ae..31a2fab2a7 100644
--- a/lib/eal/common/eal_common_thread.c
+++ b/lib/eal/common/eal_common_thread.c
@@ -103,7 +103,7 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp)
 }
 
 int
-eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size)
+eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size)
 {
 	unsigned cpu;
 	int ret;
diff --git a/lib/eal/common/eal_thread.h b/lib/eal/common/eal_thread.h
index 1c3c3442d3..85ab84baa5 100644
--- a/lib/eal/common/eal_thread.h
+++ b/lib/eal/common/eal_thread.h
@@ -50,7 +50,7 @@ unsigned eal_cpu_socket_id(unsigned cpu_id);
  *   0 for success, -1 if truncation happens.
  */
 int
-eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size);
+eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size);
 
 /**
  * Dump the current thread cpuset.
diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 1b4c73f58e..8f9eaf0dcf 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -369,8 +369,26 @@ int
 rte_thread_set_affinity_by_id(rte_thread_t thread_id,
 		const rte_cpuset_t *cpuset)
 {
-	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
-		sizeof(*cpuset), cpuset);
+	int ret;
+	char cpus_str[RTE_CPU_AFFINITY_STR_LEN] = {'\0'};
+	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
+
+	errno = 0;
+	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
+				sizeof(*cpuset), cpuset);
+	if (ret != 0) {
+		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
+					thread_name, sizeof(thread_name)) != 0)
+			EAL_LOG(ERR, "pthread_getname_np failed!");
+		if (eal_thread_dump_affinity(cpuset, cpus_str, RTE_CPU_AFFINITY_STR_LEN) != 0)
+			EAL_LOG(ERR, "eal_thread_dump_affinity failed!");
+		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
+			"ret: %d, errno: %d, error description: %s",
+			thread_name, cpus_str,
+			ret, errno, strerror(errno));
+	}
+
+	return ret;
 }
 
 int
-- 
2.34.1

[PATCH v2] eal/linux: enhanced error handling for affinity

[Jianyue Wu]

From: Jianyue Wu <jianyue.wu@nokia-sbell.com>

Improve the robustness of setting thread affinity in DPDK
by adding detailed error logging.

Changes:
1. Check the return value of pthread_setaffinity_np() and log an error
if the call fails.
2. Include the current thread name, the intended CPU set, and a detailed
error message in the log.

Sample prints:
EAL: Cannot set affinity for thread dpdk-test with cpus 0,
ret: 22, errno: 0, error description: Success
EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
ret: 22, errno: 0, error description: Success

Signed-off-by: Jianyue Wu <jianyue.wu@nokia-sbell.com>
---
 lib/eal/common/eal_common_thread.c |  2 +-
 lib/eal/common/eal_thread.h        |  2 +-
 lib/eal/unix/rte_thread.c          | 27 +++++++++++++++++++++++++--
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
index a53bc639ae..31a2fab2a7 100644
--- a/lib/eal/common/eal_common_thread.c
+++ b/lib/eal/common/eal_common_thread.c
@@ -103,7 +103,7 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp)
 }
 
 int
-eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size)
+eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size)
 {
 	unsigned cpu;
 	int ret;
diff --git a/lib/eal/common/eal_thread.h b/lib/eal/common/eal_thread.h
index 1c3c3442d3..85ab84baa5 100644
--- a/lib/eal/common/eal_thread.h
+++ b/lib/eal/common/eal_thread.h
@@ -50,7 +50,7 @@ unsigned eal_cpu_socket_id(unsigned cpu_id);
  *   0 for success, -1 if truncation happens.
  */
 int
-eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size);
+eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size);
 
 /**
  * Dump the current thread cpuset.
diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
index 1b4c73f58e..34ac0eabbf 100644
--- a/lib/eal/unix/rte_thread.c
+++ b/lib/eal/unix/rte_thread.c
@@ -369,8 +369,31 @@ int
 rte_thread_set_affinity_by_id(rte_thread_t thread_id,
 		const rte_cpuset_t *cpuset)
 {
-	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
-		sizeof(*cpuset), cpuset);
+	int ret;
+#if defined(__linux__) && defined(_GNU_SOURCE)
+	char cpus_str[RTE_CPU_AFFINITY_STR_LEN] = {'\0'};
+	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
+	errno = 0;
+#endif
+
+	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
+				sizeof(*cpuset), cpuset);
+
+#if defined(__linux__) && defined(_GNU_SOURCE)
+	if (ret != 0) {
+		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
+					thread_name, sizeof(thread_name)) != 0)
+			EAL_LOG(ERR, "pthread_getname_np failed!");
+		if (eal_thread_dump_affinity(cpuset, cpus_str, RTE_CPU_AFFINITY_STR_LEN) != 0)
+			EAL_LOG(ERR, "eal_thread_dump_affinity failed!");
+		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
+			"ret: %d, errno: %d, error description: %s",
+			thread_name, cpus_str,
+			ret, errno, strerror(errno));
+	}
+#endif
+
+	return ret;
 }
 
 int
-- 
2.34.1

Re: [PATCH v2] eal/linux: enhanced error handling for affinity

[Tyler Retzlaff]

On Thu, Apr 25, 2024 at 07:11:30PM +0800, Jianyue Wu wrote:
> From: Jianyue Wu <jianyue.wu@nokia-sbell.com>
> 
> Improve the robustness of setting thread affinity in DPDK
> by adding detailed error logging.
> 
> Changes:
> 1. Check the return value of pthread_setaffinity_np() and log an error
> if the call fails.
> 2. Include the current thread name, the intended CPU set, and a detailed
> error message in the log.
> 
> Sample prints:
> EAL: Cannot set affinity for thread dpdk-test with cpus 0,
> ret: 22, errno: 0, error description: Success
> EAL: Cannot set affinity for thread dpdk-worker1 with cpus 1,
> ret: 22, errno: 0, error description: Success
> 
> Signed-off-by: Jianyue Wu <jianyue.wu@nokia-sbell.com>
> ---
>  lib/eal/common/eal_common_thread.c |  2 +-
>  lib/eal/common/eal_thread.h        |  2 +-
>  lib/eal/unix/rte_thread.c          | 27 +++++++++++++++++++++++++--
>  3 files changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/eal/common/eal_common_thread.c b/lib/eal/common/eal_common_thread.c
> index a53bc639ae..31a2fab2a7 100644
> --- a/lib/eal/common/eal_common_thread.c
> +++ b/lib/eal/common/eal_common_thread.c
> @@ -103,7 +103,7 @@ rte_thread_get_affinity(rte_cpuset_t *cpusetp)
>  }
>  
>  int
> -eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size)
> +eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size)
>  {
>  	unsigned cpu;
>  	int ret;
> diff --git a/lib/eal/common/eal_thread.h b/lib/eal/common/eal_thread.h
> index 1c3c3442d3..85ab84baa5 100644
> --- a/lib/eal/common/eal_thread.h
> +++ b/lib/eal/common/eal_thread.h
> @@ -50,7 +50,7 @@ unsigned eal_cpu_socket_id(unsigned cpu_id);
>   *   0 for success, -1 if truncation happens.
>   */
>  int
> -eal_thread_dump_affinity(rte_cpuset_t *cpuset, char *str, unsigned int size);
> +eal_thread_dump_affinity(const rte_cpuset_t *cpuset, char *str, unsigned int size);

no objection to adding const

>  
>  /**
>   * Dump the current thread cpuset.
> diff --git a/lib/eal/unix/rte_thread.c b/lib/eal/unix/rte_thread.c
> index 1b4c73f58e..34ac0eabbf 100644
> --- a/lib/eal/unix/rte_thread.c
> +++ b/lib/eal/unix/rte_thread.c
> @@ -369,8 +369,31 @@ int
>  rte_thread_set_affinity_by_id(rte_thread_t thread_id,
>  		const rte_cpuset_t *cpuset)
>  {
> -	return pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
> -		sizeof(*cpuset), cpuset);
> +	int ret;
> +#if defined(__linux__) && defined(_GNU_SOURCE)
> +	char cpus_str[RTE_CPU_AFFINITY_STR_LEN] = {'\0'};
> +	char thread_name[RTE_MAX_THREAD_NAME_LEN] = {'\0'};
> +	errno = 0;
> +#endif
> +
> +	ret = pthread_setaffinity_np((pthread_t)thread_id.opaque_id,
> +				sizeof(*cpuset), cpuset);
> +
> +#if defined(__linux__) && defined(_GNU_SOURCE)
> +	if (ret != 0) {
> +		if (pthread_getname_np((pthread_t)thread_id.opaque_id,
> +					thread_name, sizeof(thread_name)) != 0)
> +			EAL_LOG(ERR, "pthread_getname_np failed!");
> +		if (eal_thread_dump_affinity(cpuset, cpus_str, RTE_CPU_AFFINITY_STR_LEN) != 0)
> +			EAL_LOG(ERR, "eal_thread_dump_affinity failed!");
> +		EAL_LOG(ERR, "Cannot set affinity for thread %s with cpus %s, "
> +			"ret: %d, errno: %d, error description: %s",
> +			thread_name, cpus_str,
> +			ret, errno, strerror(errno));
> +	}
> +#endif
> +
> +	return ret;
>  }
>  
>  int
> -- 

i do not think introducing os specific behavior/logging to the EAL
is a good idea. logging although not formally part of the api surface
should present the same experience for all platforms. the EAL should
have a higher standard here.

> 2.34.1

Re: [PATCH v2] eal/linux: enhanced error handling for affinity

[Stephen Hemminger]

On Fri, 26 Apr 2024 08:47:37 -0700
Tyler Retzlaff <roretzla@linux.microsoft.com> wrote:

> >  int
> > --   
> 
> i do not think introducing os specific behavior/logging to the EAL
> is a good idea. logging although not formally part of the api surface
> should present the same experience for all platforms. the EAL should
> have a higher standard here.
> 
> > 2.34.1  

For this case, the error message would be better if there was some way
to look at the cgroups and mark the cores that are not available as if
they were offline lcores.

Re:Re: [PATCH v2] eal/linux: enhanced error handling for affinity

[Jianyue Wu]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

Yes, agree with that, there is also trace from kernel can see that. I'll ignore this patch.


At 2024-04-27 08:18:53, "Stephen Hemminger" <stephen@networkplumber.org> wrote:
>On Fri, 26 Apr 2024 08:47:37 -0700
>Tyler Retzlaff <roretzla@linux.microsoft.com> wrote:
>
>> >  int
>> > --   
>> 
>> i do not think introducing os specific behavior/logging to the EAL
>> is a good idea. logging although not formally part of the api surface
>> should present the same experience for all platforms. the EAL should
>> have a higher standard here.
>> 
>> > 2.34.1  
>
>For this case, the error message would be better if there was some way
>to look at the cgroups and mark the cores that are not available as if
>they were offline lcores.
>
>

[-- Attachment #2: Type: text/html, Size: 988 bytes --]

Re: [PATCH v2] eal/linux: enhanced error handling for affinity

[Stephen Hemminger]

On Sun, 28 Apr 2024 20:26:18 +0800 (CST)
"Jianyue Wu" <wujianyue000@163.com> wrote:

> Yes, agree with that, there is also trace from kernel can see that. I'll ignore this patch.
> 
> 
> At 2024-04-27 08:18:53, "Stephen Hemminger" <stephen@networkplumber.org> wrote:
> >On Fri, 26 Apr 2024 08:47:37 -0700
> >Tyler Retzlaff <roretzla@linux.microsoft.com> wrote:
> >  
> >> >  int
> >> > --     
> >> 
> >> i do not think introducing os specific behavior/logging to the EAL
> >> is a good idea. logging although not formally part of the api surface
> >> should present the same experience for all platforms. the EAL should
> >> have a higher standard here.
> >>   
> >> > 2.34.1    
> >
> >For this case, the error message would be better if there was some way
> >to look at the cgroups and mark the cores that are not available as if
> >they were offline lcores.
> >
> >  

The current behavior is good enough. If application has an issue with
affinity because of cgroups or something else, then the developer is going
to need to investigate and adding another message in EAL is not going
to help.

And introducing #ifdef is adding unnecessary special cases.