[dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[wangyunjian]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 3239 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[Honnappa Nagarahalli]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 5006 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[wangyunjian]

[-- Attachment #1: Type: text/plain, Size: 3054 bytes --]

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 12603 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[wangyunjian]

[-- Attachment #1: Type: text/plain, Size: 3892 bytes --]

There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 14814 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[Honnappa Nagarahalli]

[-- Attachment #1: Type: text/plain, Size: 4777 bytes --]

Hi Yunjian,
               This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system?

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()


There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 17445 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[wangyunjian]

[-- Attachment #1: Type: text/plain, Size: 6652 bytes --]

Hi, Honnappa

The CPU information is as follows:
Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              1
Core(s) per socket:              64
Socket(s):                       2
NUMA node(s):                    4
Stepping:                        0x1
L1d cache:                       8 MiB
L1i cache:                       8 MiB
L2 cache:                        64 MiB
L3 cache:                        256 MiB
NUMA node0 CPU(s):               0-31
NUMA node1 CPU(s):               32-63
NUMA node2 CPU(s):               64-95
NUMA node3 CPU(s):               96-127

I have a question, does the dpdk code implement to ensure that the memory initialization is 0?

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Wednesday, February 9, 2022 2:05 AM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
               This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system?

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()


There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 27512 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[Ruifeng Wang]

[-- Attachment #1: Type: text/plain, Size: 7651 bytes --]

From: wangyunjian <wangyunjian@huawei.com>
Sent: Thursday, February 10, 2022 8:11 PM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa

The CPU information is as follows:
Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              1
Core(s) per socket:              64
Socket(s):                       2
NUMA node(s):                    4
Stepping:                        0x1
L1d cache:                       8 MiB
L1i cache:                       8 MiB
L2 cache:                        64 MiB
L3 cache:                        256 MiB
NUMA node0 CPU(s):               0-31
NUMA node1 CPU(s):               32-63
NUMA node2 CPU(s):               64-95
NUMA node3 CPU(s):               96-127

I have a question, does the dpdk code implement to ensure that the memory initialization is 0?
[Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: "
Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage".
http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Wednesday, February 9, 2022 2:05 AM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
               This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system?

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()


There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 25267 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[wangyunjian]

[-- Attachment #1: Type: text/plain, Size: 8748 bytes --]



From: Ruifeng Wang [mailto:Ruifeng.Wang@arm.com]
Sent: Tuesday, February 22, 2022 5:17 PM
To: wangyunjian <wangyunjian@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Thursday, February 10, 2022 8:11 PM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Burakov, Anatoly <anatoly.burakov@intel.com<mailto:anatoly.burakov@intel.com>>; thomas@monjalon.net<mailto:thomas@monjalon.net>; sergio.gonzalez.monroy@intel.com<mailto:sergio.gonzalez.monroy@intel.com>; Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa

The CPU information is as follows:
Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              1
Core(s) per socket:              64
Socket(s):                       2
NUMA node(s):                    4
Stepping:                        0x1
L1d cache:                       8 MiB
L1i cache:                       8 MiB
L2 cache:                        64 MiB
L3 cache:                        256 MiB
NUMA node0 CPU(s):               0-31
NUMA node1 CPU(s):               32-63
NUMA node2 CPU(s):               64-95
NUMA node3 CPU(s):               96-127

I have a question, does the dpdk code implement to ensure that the memory initialization is 0?
[Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: "
Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage".
http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
[Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform.
Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Wednesday, February 9, 2022 2:05 AM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
               This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system?

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()


There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 31694 bytes --]

RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[Honnappa Nagarahalli]

[-- Attachment #1: Type: text/plain, Size: 7510 bytes --]

<snip>

Hi, Honnappa

The CPU information is as follows:
Architecture:                    aarch64
CPU op-mode(s):                  64-bit
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              1
Core(s) per socket:              64
Socket(s):                       2
NUMA node(s):                    4
Stepping:                        0x1
L1d cache:                       8 MiB
L1i cache:                       8 MiB
L2 cache:                        64 MiB
L3 cache:                        256 MiB
NUMA node0 CPU(s):               0-31
NUMA node1 CPU(s):               32-63
NUMA node2 CPU(s):               64-95
NUMA node3 CPU(s):               96-127

I have a question, does the dpdk code implement to ensure that the memory initialization is 0?
[Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: "
Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage".
http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
[Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform.
[Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time).

I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled?


Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Wednesday, February 9, 2022 2:05 AM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
               This is not a synchronization problem. The memory is getting allocated and used in the same thread. Are you using a single socket system?

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()


There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

Thanks,
Yunjian

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, Honnappa



This problem is probability. Test case need to be executed multiple times.



The test steps and code are as follows:



/home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192



app/test-pmd/testpmd.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)



diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623 100644

--- a/app/test-pmd/testpmd.c

+++ b/app/test-pmd/testpmd.c

@@ -4251,6 +4251,20 @@ main(int argc, char** argv)

                  rte_stats_bitrate_reg(bitrate_data);

        }

#endif

+

+       printf("start test rte_zmalloc_socket\n");

+       char *a;

+       while((a = rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY)) != NULL) {

+                for (int i = 0; i < 1024 * 1024; i++) {

+                          if (a[i] != 0) {

+                                   printf("a[%d] = %d\n",i,a[i]);

+                          }

+                          a[i] = 255; // This assignment is important. It can increase the probability.

+                }

+       }

+       printf("end test rte_zmalloc_socket\n");

+       return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE

        if (strlen(cmdline_filename) != 0)

                  cmdline_read_from_file(cmdline_filename);

Thanks,
Yunjian

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com<mailto:Honnappa.Nagarahalli@arm.com>>; nd <nd@arm.com<mailto:nd@arm.com>>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi Yunjian,
                That's interesting. Is it possible to elaborate the use case or possibly provide the code snippet?

It is possible that it is a synchronization problem due to relaxed memory model that Arm architecture uses. There could be a barrier missing in the code.

Thanks,
Honnappa

From: wangyunjian <wangyunjian@huawei.com<mailto:wangyunjian@huawei.com>>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
Cc: Feifei Wang <Feifei.Wang2@arm.com<mailto:Feifei.Wang2@arm.com>>; Ruifeng Wang <Ruifeng.Wang@arm.com<mailto:Ruifeng.Wang@arm.com>>; Huangshaozhang <huangshaozhang@huawei.com<mailto:huangshaozhang@huawei.com>>; dingxiaoxiong <dingxiaoxiong@huawei.com<mailto:dingxiaoxiong@huawei.com>>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

Hi, all

There's a problem that the memory are allocated by rte_zmalloc_socket()
may not be all-zero on the ARM platform.

However, the x86 platform does not have this problem.


Any ideas ?


Thanks,

Yunjian


[-- Attachment #2: Type: text/html, Size: 32708 bytes --]

Re: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket()

[Stephen Hemminger]

On Wed, 23 Feb 2022 15:38:09 +0000
Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote:

> I have a question, does the dpdk code implement to ensure that the memory initialization is 0?
> [Ruifeng] Clearing of the memory should be done by the kernel. In section 3.1.4.6 of Programmer's Guide, it says: "
> Hugepages are cleared by the kernel when a file in hugetlbfs or its part is mapped for the first time system-wide to prevent data leaks from previous users of the same hugepage".
> http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
> [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version 4.19.90) on the ARM platform.
> [Honnappa] I think that is besides the point we are discussing. rte_zmalloc_socket should be able to zero the memory every time it is called (not just the first time).
> 
> I see that rte_zmalloc_socket explicitly clears the memory using memset when the RTE_MALLOC_DEBUG is enabled. Have you tested with RTE_MALLOC_DEBUG enabled?
> 
> 
> Thanks,
> Yunjian

Normally.
  - hugepage memory is zero'd by kernel when mapped in.  DPDK assumes this because the overhead
    of zeroing large amounts of memory can impact application startup time.
    If kernel is not zeroing, then your kernel is buggy.
  - when memory is freed by rte_free() it is set to zero before returning to the pool.
  - when malloc gets memory it will be zero'd

RTE_MALLOC_DEBUG changes this so that:
   - when memory is freed it gets overwritten by a poison value
   - when malloc gets memory it will zero it.