* [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() @ 2022-01-30 3:21 wangyunjian 2022-01-31 4:21 ` Honnappa Nagarahalli 0 siblings, 1 reply; 10+ messages in thread From: wangyunjian @ 2022-01-30 3:21 UTC (permalink / raw) To: dev, users; +Cc: feifei.wang2, ruifeng.wang, Huangshaozhang, dingxiaoxiong [-- Attachment #1: Type: text/plain, Size: 226 bytes --] Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 3239 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-01-30 3:21 [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() wangyunjian @ 2022-01-31 4:21 ` Honnappa Nagarahalli 2022-02-07 2:43 ` wangyunjian 2022-02-08 8:01 ` wangyunjian 0 siblings, 2 replies; 10+ messages in thread From: Honnappa Nagarahalli @ 2022-01-31 4:21 UTC (permalink / raw) To: wangyunjian, dev, users Cc: Feifei Wang, Ruifeng Wang, Huangshaozhang, dingxiaoxiong, Honnappa Nagarahalli, nd [-- Attachment #1: Type: text/plain, Size: 930 bytes --] Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 5006 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-01-31 4:21 ` Honnappa Nagarahalli @ 2022-02-07 2:43 ` wangyunjian 2022-02-08 8:01 ` wangyunjian 1 sibling, 0 replies; 10+ messages in thread From: wangyunjian @ 2022-02-07 2:43 UTC (permalink / raw) To: Honnappa Nagarahalli, dev, users Cc: Feifei Wang, Ruifeng Wang, Huangshaozhang, dingxiaoxiong, nd [-- Attachment #1: Type: text/plain, Size: 3054 bytes --] Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 12603 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-01-31 4:21 ` Honnappa Nagarahalli 2022-02-07 2:43 ` wangyunjian @ 2022-02-08 8:01 ` wangyunjian 2022-02-08 18:05 ` Honnappa Nagarahalli 1 sibling, 1 reply; 10+ messages in thread From: wangyunjian @ 2022-02-08 8:01 UTC (permalink / raw) To: Honnappa Nagarahalli, dev, users Cc: Feifei Wang, Ruifeng Wang, Huangshaozhang, dingxiaoxiong, nd [-- Attachment #1: Type: text/plain, Size: 3892 bytes --] There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 14814 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-08 8:01 ` wangyunjian @ 2022-02-08 18:05 ` Honnappa Nagarahalli 2022-02-10 12:11 ` wangyunjian 0 siblings, 1 reply; 10+ messages in thread From: Honnappa Nagarahalli @ 2022-02-08 18:05 UTC (permalink / raw) To: wangyunjian, dev, users Cc: Feifei Wang, Ruifeng Wang, Huangshaozhang, dingxiaoxiong, nd, nd [-- Attachment #1: Type: text/plain, Size: 4777 bytes --] Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 17445 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-08 18:05 ` Honnappa Nagarahalli @ 2022-02-10 12:11 ` wangyunjian 2022-02-22 9:17 ` Ruifeng Wang 0 siblings, 1 reply; 10+ messages in thread From: wangyunjian @ 2022-02-10 12:11 UTC (permalink / raw) To: Honnappa Nagarahalli, dev, users Cc: Burakov, Anatoly, Thomas Monjalon, sergio.gonzalez.monroy, Feifei Wang, Ruifeng Wang, Huangshaozhang, dingxiaoxiong, nd, nd [-- Attachment #1: Type: text/plain, Size: 6652 bytes --] Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 27512 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-10 12:11 ` wangyunjian @ 2022-02-22 9:17 ` Ruifeng Wang 2022-02-23 11:22 ` wangyunjian 0 siblings, 1 reply; 10+ messages in thread From: Ruifeng Wang @ 2022-02-22 9:17 UTC (permalink / raw) To: wangyunjian, Honnappa Nagarahalli, dev, users Cc: Burakov, Anatoly, thomas, sergio.gonzalez.monroy, Feifei Wang, Huangshaozhang, dingxiaoxiong, nd, nd, nd [-- Attachment #1: Type: text/plain, Size: 7651 bytes --] From: wangyunjian <wangyunjian@huawei.com> Sent: Thursday, February 10, 2022 8:11 PM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 25267 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-22 9:17 ` Ruifeng Wang @ 2022-02-23 11:22 ` wangyunjian 2022-02-23 15:38 ` Honnappa Nagarahalli 0 siblings, 1 reply; 10+ messages in thread From: wangyunjian @ 2022-02-23 11:22 UTC (permalink / raw) To: Ruifeng Wang, Honnappa Nagarahalli, dev, users Cc: Burakov, Anatoly, thomas, sergio.gonzalez.monroy, Feifei Wang, Huangshaozhang, dingxiaoxiong, nd, nd, nd [-- Attachment #1: Type: text/plain, Size: 8748 bytes --] From: Ruifeng Wang [mailto:Ruifeng.Wang@arm.com] Sent: Tuesday, February 22, 2022 5:17 PM To: wangyunjian <wangyunjian@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Thursday, February 10, 2022 8:11 PM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Burakov, Anatoly <anatoly.burakov@intel.com<mailto:anatoly.burakov@intel.com>>; thomas@monjalon.net<mailto:thomas@monjalon.net>; sergio.gonzalez.monroy@intel.com<mailto:sergio.gonzalez.monroy@intel.com>; Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform. Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 31694 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-23 11:22 ` wangyunjian @ 2022-02-23 15:38 ` Honnappa Nagarahalli 2023-04-12 17:35 ` Stephen Hemminger 0 siblings, 1 reply; 10+ messages in thread From: Honnappa Nagarahalli @ 2022-02-23 15:38 UTC (permalink / raw) To: wangyunjian, Ruifeng Wang, dev, users Cc: Burakov, Anatoly, thomas, sergio.gonzalez.monroy, Feifei Wang, Huangshaozhang, dingxiaoxiong, nd, nd [-- Attachment #1: Type: text/plain, Size: 7510 bytes --] <snip> Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform. [Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time). I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled? Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 32708 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() 2022-02-23 15:38 ` Honnappa Nagarahalli @ 2023-04-12 17:35 ` Stephen Hemminger 0 siblings, 0 replies; 10+ messages in thread From: Stephen Hemminger @ 2023-04-12 17:35 UTC (permalink / raw) To: Honnappa Nagarahalli Cc: wangyunjian, Ruifeng Wang, dev, users, Burakov, Anatoly, thomas, sergio.gonzalez.monroy, Feifei Wang, Huangshaozhang, dingxiaoxiong, nd On Wed, 23 Feb 2022 15:38:09 +0000 Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote: > I have a question, does the dpdk code implement to ensure that the memory initialization is 0? > [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " > Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". > http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation > [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform. > [Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time). > > I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled? > > > Thanks, > Yunjian Normally. - hugepage memory is zero'd by kernel when mapped in. DPDK assumes this because the overhead of zeroing large amounts of memory can impact application startup time. If kernel is not zeroing, then your kernel is buggy. - when memory is freed by rte_free() it is set to zero before returning to the pool. - when malloc gets memory it will be zero'd RTE_MALLOC_DEBUG changes this so that: - when memory is freed it gets overwritten by a poison value - when malloc gets memory it will zero it. ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-04-12 17:35 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-30 3:21 [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() wangyunjian 2022-01-31 4:21 ` Honnappa Nagarahalli 2022-02-07 2:43 ` wangyunjian 2022-02-08 8:01 ` wangyunjian 2022-02-08 18:05 ` Honnappa Nagarahalli 2022-02-10 12:11 ` wangyunjian 2022-02-22 9:17 ` Ruifeng Wang 2022-02-23 11:22 ` wangyunjian 2022-02-23 15:38 ` Honnappa Nagarahalli 2023-04-12 17:35 ` Stephen Hemminger
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).