[-- Attachment #1: Type: text/plain, Size: 226 bytes --] Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 3239 bytes --]
[-- Attachment #1: Type: text/plain, Size: 930 bytes --] Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 5006 bytes --]
[-- Attachment #1: Type: text/plain, Size: 3054 bytes --] Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 12603 bytes --]
[-- Attachment #1: Type: text/plain, Size: 3892 bytes --] There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 14814 bytes --]
[-- Attachment #1: Type: text/plain, Size: 4777 bytes --] Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 17445 bytes --]
[-- Attachment #1: Type: text/plain, Size: 6652 bytes --] Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 27512 bytes --]
[-- Attachment #1: Type: text/plain, Size: 7651 bytes --] From: wangyunjian <wangyunjian@huawei.com> Sent: Thursday, February 10, 2022 8:11 PM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 25267 bytes --]
[-- Attachment #1: Type: text/plain, Size: 8748 bytes --] From: Ruifeng Wang [mailto:Ruifeng.Wang@arm.com] Sent: Tuesday, February 22, 2022 5:17 PM To: wangyunjian <wangyunjian@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Thursday, February 10, 2022 8:11 PM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Burakov, Anatoly <anatoly.burakov@intel.com<mailto:anatoly.burakov@intel.com>>; thomas@monjalon.net<mailto:thomas@monjalon.net>; sergio.gonzalez.monroy@intel.com<mailto:sergio.gonzalez.monroy@intel.com>; Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform. Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 31694 bytes --]
[-- Attachment #1: Type: text/plain, Size: 7510 bytes --] <snip> Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory initialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform. [Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time). I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled? Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system? Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) { + for (int i = 0; i < 1024 * 1024; i++) { + if (a[i] != 0) { + printf("a[%d] = %d\n",i,a[i]); + } + a[i] = 255; // This assignment is important. It can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) != 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>> Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code. Thanks, Honnappa From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>> Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org> Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>> Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian [-- Attachment #2: Type: text/html, Size: 32708 bytes --]
On Wed, 23 Feb 2022 15:38:09 +0000
Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote:
> I have a question, does the dpdk code implement to ensure that the memory initialization is 0?
> [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: "
> Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage".
> http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
> [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform.
> [Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time).
>
> I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled?
>
>
> Thanks,
> Yunjian
Normally.
- hugepage memory is zero'd by kernel when mapped in. DPDK assumes this because the overhead
of zeroing large amounts of memory can impact application startup time.
If kernel is not zeroing, then your kernel is buggy.
- when memory is freed by rte_free() it is set to zero before returning to the pool.
- when malloc gets memory it will be zero'd
RTE_MALLOC_DEBUG changes this so that:
- when memory is freed it gets overwritten by a poison value
- when malloc gets memory it will zero it.