From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16D9EA034C; Wed, 23 Feb 2022 12:22:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0293E40E5A; Wed, 23 Feb 2022 12:22:18 +0100 (CET) Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by mails.dpdk.org (Postfix) with ESMTP id C2F2740DF6; Wed, 23 Feb 2022 12:22:15 +0100 (CET) Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4K3YSk4HVPz9sGV; Wed, 23 Feb 2022 19:18:46 +0800 (CST) Received: from dggpemm100014.china.huawei.com (7.185.36.55) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 23 Feb 2022 19:22:13 +0800 Received: from dggpemm500008.china.huawei.com (7.185.36.136) by dggpemm100014.china.huawei.com (7.185.36.55) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Wed, 23 Feb 2022 19:22:13 +0800 Received: from dggpemm500008.china.huawei.com ([7.185.36.136]) by dggpemm500008.china.huawei.com ([7.185.36.136]) with mapi id 15.01.2308.021; Wed, 23 Feb 2022 19:22:13 +0800 From: wangyunjian To: Ruifeng Wang , Honnappa Nagarahalli , "dev@dpdk.org" , "users@dpdk.org" CC: "Burakov, Anatoly" , "thomas@monjalon.net" , "sergio.gonzalez.monroy@intel.com" , Feifei Wang , Huangshaozhang , dingxiaoxiong , nd , nd , nd Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Thread-Topic: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero allocated by rte_zmalloc_socket() Thread-Index: AdgViF0h9zoMAuSNQ6SwzCLA7pDXJwA0WT2gAVxGsfAAPUvqgAATMLywAFSYBYACWvBhcAA28xwQ Date: Wed, 23 Feb 2022 11:22:12 +0000 Message-ID: <09c3aeaa7bf647f380cce93668759dda@huawei.com> References: <61c1d85e0f8a42d4812830864fc59b0a@huawei.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.174.242.157] Content-Type: multipart/alternative; boundary="_000_09c3aeaa7bf647f380cce93668759ddahuaweicom_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_09c3aeaa7bf647f380cce93668759ddahuaweicom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable From: Ruifeng Wang [mailto:Ruifeng.Wang@arm.com] Sent: Tuesday, February 22, 2022 5:17 PM To: wangyunjian ; Honnappa Nagarahalli ; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly ; thomas@monjalon.net; serg= io.gonzalez.monroy@intel.com; Feifei Wang ; Huangshao= zhang ; dingxiaoxiong = ; nd ; nd ; nd Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() From: wangyunjian > Sent: Thursday, February 10, 2022 8:11 PM To: Honnappa Nagarahalli >; dev@dpdk.org; users@dpdk.org Cc: Burakov, Anatoly >; thomas@monjalon.net; sergio.gonzalez.m= onroy@intel.com; Feifei Wang >; Ruifeng Wang >; Huangshaozhang >; dingxiaoxiong >; nd = >; nd > Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() Hi, Honnappa The CPU information is as follows: Architecture: aarch64 CPU op-mode(s): 64-bit Byte Order: Little Endian CPU(s): 128 On-line CPU(s) list: 0-127 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 4 Stepping: 0x1 L1d cache: 8 MiB L1i cache: 8 MiB L2 cache: 64 MiB L3 cache: 256 MiB NUMA node0 CPU(s): 0-31 NUMA node1 CPU(s): 32-63 NUMA node2 CPU(s): 64-95 NUMA node3 CPU(s): 96-127 I have a question, does the dpdk code implement to ensure that the memory i= nitialization is 0? [Ruifeng] Clearing of the memory should be done by the kernel. In section 3= .1.4.6 of Programmer's Guide, it says: " Hugepages are cleared by the kernel when a file in hugetlbfs or its part is= mapped for the first time system-wide to prevent data leaks from previous = users of the same hugepage". http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-map= ping-discovery-and-memory-reservation [Yunjian] Thanks. However, hugepages are not cleared by the kernel(version = 4.19.90) on the ARM platform. Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Wednesday, February 9, 2022 2:05 AM To: wangyunjian >; de= v@dpdk.org; users@dpdk.org Cc: Feifei Wang >; Ruifen= g Wang >; Huangshaozhang = >; dingxiaoxion= g >; nd >; nd > Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() Hi Yunjian, This is not a synchronization problem. The memory is getting= allocated and used in the same thread. Are you using a single socket syste= m? Thanks, Honnappa From: wangyunjian > Sent: Tuesday, February 8, 2022 2:01 AM To: Honnappa Nagarahalli >; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang >; Ruifen= g Wang >; Huangshaozhang = >; dingxiaoxion= g >; nd > Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() There is also a condition that the hugepagesz is 1G. If the hugepagesz is 2M, this problem cannot be repeated. Thanks, Yunjian From: wangyunjian Sent: Monday, February 7, 2022 10:44 AM To: 'Honnappa Nagarahalli' >; dev@dpdk.org; users@dpdk.org Cc: Feifei Wang >; Ruifen= g Wang >; Huangshaozhang = >; dingxiaoxion= g >; nd > Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() Hi, Honnappa This problem is probability. Test case need to be executed multiple times. The test steps and code are as follows: /home/dpdk #./arm64-armv8a-linuxapp-gcc/app/dpdk-testpmd --legacy-mem -c 0= xC -m 8192 app/test-pmd/testpmd.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 55eb293cc0..3c127f9623 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -4251,6 +4251,20 @@ main(int argc, char** argv) rte_stats_bitrate_reg(bitrate_data); } #endif + + printf("start test rte_zmalloc_socket\n"); + char *a; + while((a =3D rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_ID_ANY= )) !=3D NULL) { + for (int i =3D 0; i < 1024 * 1024; i++) { + if (a[i] !=3D 0) { + printf("a[%d] =3D %d\n",i,a[i]); + } + a[i] =3D 255; // This assignment is important. I= t can increase the probability. + } + } + printf("end test rte_zmalloc_socket\n"); + return EXIT_SUCCESS; + #ifdef RTE_LIB_CMDLINE if (strlen(cmdline_filename) !=3D 0) cmdline_read_from_file(cmdline_filename); Thanks, Yunjian From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] Sent: Monday, January 31, 2022 12:22 PM To: wangyunjian >; de= v@dpdk.org; users@dpdk.org Cc: Feifei Wang >; Ruifen= g Wang >; Huangshaozhang = >; dingxiaoxion= g >; Honnappa Nag= arahalli = >; nd > Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not be all-z= ero allocated by rte_zmalloc_socket() Hi Yunjian, That's interesting. Is it possible to elaborate the use cas= e or possibly provide the code snippet? It is possible that it is a synchronization problem due to relaxed memory m= odel that Arm architecture uses. There could be a barrier missing in the co= de. Thanks, Honnappa From: wangyunjian > Sent: Saturday, January 29, 2022 9:21 PM To: dev@dpdk.org; users@dpdk.org Cc: Feifei Wang >; Ruifen= g Wang >; Huangshaozhang = >; dingxiaoxion= g > Subject: [dpdk-dev][dpdk-users] A problem about memory may not be all-zero = allocated by rte_zmalloc_socket() Hi, all There's a problem that the memory are allocated by rte_zmalloc_socket() may not be all-zero on the ARM platform. However, the x86 platform does not have this problem. Any ideas ? Thanks, Yunjian --_000_09c3aeaa7bf647f380cce93668759ddahuaweicom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

&n= bsp;

&n= bsp;

From: Ruifeng Wang [mailto:Ruifeng.Wang@arm.com]
Sent: Tuesday, February 22, 2022 5:17 PM
To: wangyunjian <wangyunjian@huawei.com>; Honnappa Nagarahalli= <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monja= lon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm= .com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong &= lt;dingxiaoxiong@huawei.com>; nd <nd@arm.com>; nd <nd@arm.com&g= t;; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

From: wangyunjian <wangyunjian@huawei.com>
Sent: Thursday, February 10, 2022 8:11 PM
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; thomas@monjalon.net; sergio.gonzalez.monroy@intel.com; Feifei Wang <Feifei.Wang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozha= ng <huangshaozhang@huawei.c= om>; dingxiaoxiong <dingxiaoxion= g@huawei.com>; nd <nd@arm.com&g= t;; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

Hi, = Honnappa

 

The = CPU information is as follows:

Arch= itecture:           =          aarch64<= /p>

CPU = op-mode(s):          &nbs= p;       64-bit

Byte= Order:           &n= bsp;          Little Endian

CPU(= s):            =             &nb= sp; 128

On-l= ine CPU(s) list:          = ;   0-127

Thre= ad(s) per core:          =     1

Core= (s) per socket:          =     64

Sock= et(s):           &nb= sp;           2

NUMA= node(s):           =          4

Step= ping:           &nbs= p;            0x1

L1d = cache:           &nb= sp;           8 MiB<= /o:p>

L1i = cache:           &nb= sp;           8 MiB<= /o:p>

L2 c= ache:           &nbs= p;            64 MiB=

L3 c= ache:           &nbs= p;            256 Mi= B

NUMA= node0 CPU(s):          &= nbsp;    0-31

NUMA= node1 CPU(s):          &= nbsp;    32-63

NUMA= node2 CPU(s):          &= nbsp;    64-95

NUMA= node3 CPU(s):          &= nbsp;    96-127

 

I ha= ve a question, does the dpdk code implement to ensure that the memory initi= alization is 0?

[Rui= feng] Clearing of the memory should be done by the kernel. In section 3.1.4= .6 of Programmer’s Guide, it says: “

Huge= pages are cleared by the kernel when a file in hugetlbfs or its part is map= ped for the first time system-wide to prevent data leaks from previous user= s of the same hugepage”.

http://doc.dpdk.org/guides/pr= og_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-res= ervation

[Yunjian] Thanks. However, hugepages are not cleared by the kerne= l(version 4.19.90) on the ARM platform.

Than= ks,

Yunj= ian

&n= bsp;

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Wednesday, February 9, 2022 2:05 AM
To: wangyunjian <wangyu= njian@huawei.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.W= ang2@arm.com>; Ruifeng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.com>; dingxiaoxiong <dingxiaoxion= g@huawei.com>; nd <nd@arm.com&g= t;; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

Hi Y= unjian,

&nbs= p;            &= nbsp; This is not a synchronization problem. The memory is getting allocate= d and used in the same thread. Are you using a single socket system?

 

Than= ks,

Honn= appa

 

From: wangyunjian <<= a href=3D"mailto:wangyunjian@huawei.com">w= angyunjian@huawei.com>
Sent: Tuesday, February 8, 2022 2:01 AM
To: Honnappa Nagarahalli <Honna= ppa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruife= ng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.co= m>; di= ngxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

There is also a condition that the hugepagesz is 1G.

If the hugepagesz is 2M, this problem cannot be repeated.

&n= bsp;

Than= ks,

Yunj= ian

&n= bsp;

From: wangyunjian
Sent: Monday, February 7, 2022 10:44 AM
To: 'Honnappa Nagarahalli' <H= onnappa.Nagarahalli@arm.com>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruife= ng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.co= m>; di= ngxiaoxiong <dingxiaoxiong@huawei.com>; nd <nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

Hi, Honnappa

 

This problem is probability.= Test case need to be executed multiple times.

 

The test steps and code are = as follows:

 

/home/dpdk #./arm64-armv8a-l= inuxapp-gcc/app/dpdk-testpmd --legacy-mem  -c 0xC  -m 8192

 

app/test-pmd/testpmd.c | 14 = ++++++++++++++=

1 file changed, 14 insertion= s(+)

 

diff --git a/app/test-pmd/te= stpmd.c b/app/test-pmd/testpmd.c

index 55eb293cc0..3c127f9623= 100644

--- a/app/test-pmd/testpmd.c=

+++ b/app/test-p= md/testpmd.c

@@ -4251,6 +4251,20 @@ m= ain(int argc, char** argv)

    &nbs= p;             = rte_stats_bitrate_reg(bitrate_data);

    &nbs= p;   }

#endif

+

+    = ;   printf("start test rte_zmalloc_socket\n");

+    = ;   char *a;

+    = ;   while((a =3D rte_zmalloc_socket(NULL, 1024 * 1024, 0, SOCKET_= ID_ANY)) !=3D NULL) {

+    = ;            for (in= t i =3D 0; i < 1024 * 1024; i++) {

+    = ;            &n= bsp;         if (a[i] !=3D 0) {

+    = ;            &n= bsp;            = ;      printf("a[%d] =3D %d\n",i,a[i]);<= o:p>

+    = ;            &n= bsp;         }

+    = ;            &n= bsp;         a[i] =3D 255; // This = assignment is important. It can increase the probability.=

+    = ;            }<= /o:p>

+    = ;   }

+    = ;   printf("end test rte_zmalloc_socket\n");=

+    = ;   return EXIT_SUCCESS;

+

#ifdef RTE_LIB_CMDLINE<= /o:p>

    &nbs= p;   if (strlen(cmdline_filename) !=3D 0)

    &nbs= p;             = cmdline_read_from_file(cmdline_filename);

&n= bsp;

Than= ks,

Yunj= ian

&n= bsp;

From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
Sent: Monday, January 31, 2022 12:22 PM
To: wangyunjian <wangyunjian@huawei.c= om>; dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruife= ng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.co= m>; di= ngxiaoxiong <dingxiaoxiong@huawei.com>; Honnappa Nagarahalli <Honnappa.Nagara= halli@arm.com>; nd <= nd@arm.com>
Subject: RE: [dpdk-dev][dpdk-users] A problem about memory may not b= e all-zero allocated by rte_zmalloc_socket()

 

Hi Y= unjian,

&nbs= p;            &= nbsp;  That’s interesting. Is it possible to elaborate the use c= ase or possibly provide the code snippet?

 

It i= s possible that it is a synchronization problem due to relaxed memory model= that Arm architecture uses. There could be a barrier missing in the code.<= o:p>

 

Than= ks,

Honn= appa

 

From: wangyunjian <<= a href=3D"mailto:wangyunjian@huawei.com">w= angyunjian@huawei.com>
Sent: Saturday, January 29, 2022 9:21 PM
To: dev@dpdk.org; users@dpdk.org
Cc: Feifei Wang <Feifei.Wang2@arm.com>; Ruife= ng Wang <Ruifeng.Wang@arm.com>; Huangshaozhang <huangshaozhang@huawei.co= m>; di= ngxiaoxiong <dingxiaoxiong@huawei.com>
Subject: [dpdk-dev][dpdk-users] A problem about memory may not be al= l-zero allocated by rte_zmalloc_socket()

 

Hi, all

 

There's a problem that the memo= ry are allocated by rte_zmalloc_socket()

may not be all-zero on the ARM = platform.

 

However, the x86 platform does = not have this problem.

 

Any ideas ?

 

Thanks,

Yunjian

 

--_000_09c3aeaa7bf647f380cce93668759ddahuaweicom_--