From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130042.outbound.protection.outlook.com [40.107.13.42]) by dpdk.org (Postfix) with ESMTP id 3F12E2C60; Tue, 19 Mar 2019 11:05:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vAZkMYlIFkHy7v5SxZjp/cwLy+S1PYoqrM4DAWC9Sc0=; b=cI852n7y+TTaHKS4N0O4TYz7i2WHX+Ke+OD+r4NDjixOiUWlgvguRYqo/jQ7bHEKYV/2tlMHQaLuwmu16SfU7f/aM7FdnCCsTUZcO2gWTLJr3q8+hwv/ybbzdHP2n+UJSi0MkHhX7M1bamR0YAlcLgyp3ttyS7ua5ggAS1+nV6I= Received: from VI1PR05MB4224.eurprd05.prod.outlook.com (52.133.12.13) by VI1PR05MB4480.eurprd05.prod.outlook.com (52.133.13.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.15; Tue, 19 Mar 2019 10:05:04 +0000 Received: from VI1PR05MB4224.eurprd05.prod.outlook.com ([fe80::bcb0:ed58:d76:cac]) by VI1PR05MB4224.eurprd05.prod.outlook.com ([fe80::bcb0:ed58:d76:cac%4]) with mapi id 15.20.1709.015; Tue, 19 Mar 2019 10:05:04 +0000 From: Dekel Peled To: Chao Zhu CC: Yongseok Koh , Shahaf Shuler , "dev@dpdk.org" , Ori Kam , Thomas Monjalon , "stable@dpdk.org" Thread-Topic: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER Thread-Index: AQHU3Yp0jK3S6l0vmk2P4CMRCfI9FaYSS1UAgABrb7A= Date: Tue, 19 Mar 2019 10:05:03 +0000 Message-ID: References: <1552913893-43407-1-git-send-email-dekelp@mellanox.com> <001d01d4de03$378f18a0$a6ad49e0$@linux.vnet.ibm.com> In-Reply-To: <001d01d4de03$378f18a0$a6ad49e0$@linux.vnet.ibm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=dekelp@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f0a493ef-a0d7-4297-6022-08d6ac525b6f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR05MB4480; x-ms-traffictypediagnostic: VI1PR05MB4480: x-ms-exchange-purlcount: 1 x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0981815F2F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(376002)(396003)(136003)(39860400002)(13464003)(189003)(199004)(229853002)(105586002)(3846002)(446003)(68736007)(76176011)(6116002)(5660300002)(106356001)(7696005)(8936002)(966005)(478600001)(33656002)(99286004)(54906003)(186003)(316002)(6916009)(14454004)(66066001)(6246003)(14444005)(7736002)(81166006)(81156014)(26005)(256004)(8676002)(6306002)(11346002)(476003)(52536014)(45080400002)(53546011)(74316002)(6436002)(2906002)(86362001)(4326008)(25786009)(6506007)(486006)(97736004)(71190400001)(71200400001)(55016002)(102836004)(53936002)(305945005)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR05MB4480; H:VI1PR05MB4224.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: i9jFQffJbCEgxQdOZRRlAmdBGfou1IqChXJ32c5oCcKaMzYzjbjf1MJss+6psvYq1NwE0nZwfrGJf9ZHX3cEHgP/+F2men/u+gqVVkg4y6CFt39VpOjiLAN9L8FGE8jVHqzjOtdOkScJ5g2chVlrwpo9+/RZFpYmv6FCWFT5aQVgIX7u4AvKCbHVTgcl9BV+44udzdHNMkwDuBPJ2MPWzUV/V4iwjDaPNRizaqKymAQhjkI6nNH8dDYA2DzMjDqS5fzAp8oLg9lgH1KiclFD3Gja+QNR6ijKaP581x4DwgN0itqW9RwwKAEKumfuFNcyO7N9I3bRyAohxlSzlm3uZOs5hXiD9SDWw5D3HY0L3e3eAkR46Vov0Hd9eDRAnvPkDuWn/D0QjlbKhUP43Tgfy2R0OxXsmUV2HDju9l1kvW8= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0a493ef-a0d7-4297-6022-08d6ac525b6f X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2019 10:05:03.9494 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB4480 Subject: Re: [dpdk-dev] [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2019 10:05:06 -0000 Hi, For ppc, rte_io_mb() is defined as rte_mb(), which is defined as asm sync. According to comments in arch/ppc_64/rte_atomic.h, rte_wmb() and rte_rmb() = are the same as rte_mb(), for store and load respectively. My patch propose to define rte_wmb() and rte_rmb() as asm sync, like rte_mb= (), since using lwsync is incorrect for them. Regards, Dekel > -----Original Message----- > From: Chao Zhu > Sent: Tuesday, March 19, 2019 5:24 AM > To: Dekel Peled > Cc: Yongseok Koh ; Shahaf Shuler > ; dev@dpdk.org; Ori Kam ; > Thomas Monjalon ; stable@dpdk.org > Subject: RE: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER >=20 > Dekel=A3=AC >=20 > To control the memory order for device memory, I think you should use > rte_io_mb() instead of rte_mb(). This will generate correct result. rte_w= mb() > is used for system memory. >=20 > > -----Original Message----- > > From: Dekel Peled > > Sent: Monday, March 18, 2019 8:58 PM > > To: chaozhu@linux.vnet.ibm.com > > Cc: yskoh@mellanox.com; shahafs@mellanox.com; dev@dpdk.org; > > orika@mellanox.com; thomas@monjalon.net; dekelp@mellanox.com; > > stable@dpdk.org > > Subject: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER > > > > From previous patch description: "to improve performance on PPC64, use > > light weight sync instruction instead of sync instruction." > > > > Excerpt from IBM doc [1], section "Memory barrier instructions": > > "The second form of the sync instruction is light-weight sync, or lwsyn= c. > > This form is used to control ordering for storage accesses to system > > memory only. It does not create a memory barrier for accesses to device > memory." > > > > This patch removes the use of lwsync, so calls to rte_wmb() and > > rte_rmb() will provide correct memory barrier to ensure order of > > accesses to system memory and device memory. > > > > [1] > > > https://eur03.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww > . > > > ibm.com%2Fdeveloperworks%2Fsystems%2Farticles%2Fpowerpc.html& > ;data=3D > > > 02%7C01%7Cdekelp%40mellanox.com%7C381426b6b9d042f776fa08d6ac1a5d > c5%7Ca > > > 652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636885626593364016&am > p;sdata > > > =3DwFYTcFX2A%2BMdtQMgtojTAtUOzqds7U5pypNS%2F2SoXUM%3D&re > served=3D0 > > > > Fixes: d23a6bd04d72 ("eal/ppc: fix memory barrier for IBM POWER") > > Cc: stable@dpdk.org > > > > Signed-off-by: Dekel Peled > > --- > > lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h | 8 -------- > > 1 file changed, 8 deletions(-) > > > > diff --git a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > index ce38350..797381c 100644 > > --- a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > +++ b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > @@ -63,11 +63,7 @@ > > * Guarantees that the STORE operations generated before the barrier > > * occur before the STORE operations generated after. > > */ > > -#ifdef RTE_ARCH_64 > > -#define rte_wmb() asm volatile("lwsync" : : : "memory") > > -#else > > #define rte_wmb() asm volatile("sync" : : : "memory") > > -#endif > > > > /** > > * Read memory barrier. > > @@ -75,11 +71,7 @@ > > * Guarantees that the LOAD operations generated before the barrier > > * occur before the LOAD operations generated after. > > */ > > -#ifdef RTE_ARCH_64 > > -#define rte_rmb() asm volatile("lwsync" : : : "memory") > > -#else > > #define rte_rmb() asm volatile("sync" : : : "memory") > > -#endif > > > > #define rte_smp_mb() rte_mb() > > > > -- > > 1.8.3.1 >=20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 92A8CA00E6 for ; Tue, 19 Mar 2019 11:05:07 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 61A554C88; Tue, 19 Mar 2019 11:05:07 +0100 (CET) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130042.outbound.protection.outlook.com [40.107.13.42]) by dpdk.org (Postfix) with ESMTP id 3F12E2C60; Tue, 19 Mar 2019 11:05:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vAZkMYlIFkHy7v5SxZjp/cwLy+S1PYoqrM4DAWC9Sc0=; b=cI852n7y+TTaHKS4N0O4TYz7i2WHX+Ke+OD+r4NDjixOiUWlgvguRYqo/jQ7bHEKYV/2tlMHQaLuwmu16SfU7f/aM7FdnCCsTUZcO2gWTLJr3q8+hwv/ybbzdHP2n+UJSi0MkHhX7M1bamR0YAlcLgyp3ttyS7ua5ggAS1+nV6I= Received: from VI1PR05MB4224.eurprd05.prod.outlook.com (52.133.12.13) by VI1PR05MB4480.eurprd05.prod.outlook.com (52.133.13.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.15; Tue, 19 Mar 2019 10:05:04 +0000 Received: from VI1PR05MB4224.eurprd05.prod.outlook.com ([fe80::bcb0:ed58:d76:cac]) by VI1PR05MB4224.eurprd05.prod.outlook.com ([fe80::bcb0:ed58:d76:cac%4]) with mapi id 15.20.1709.015; Tue, 19 Mar 2019 10:05:04 +0000 From: Dekel Peled To: Chao Zhu CC: Yongseok Koh , Shahaf Shuler , "dev@dpdk.org" , Ori Kam , Thomas Monjalon , "stable@dpdk.org" Thread-Topic: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER Thread-Index: AQHU3Yp0jK3S6l0vmk2P4CMRCfI9FaYSS1UAgABrb7A= Date: Tue, 19 Mar 2019 10:05:03 +0000 Message-ID: References: <1552913893-43407-1-git-send-email-dekelp@mellanox.com> <001d01d4de03$378f18a0$a6ad49e0$@linux.vnet.ibm.com> In-Reply-To: <001d01d4de03$378f18a0$a6ad49e0$@linux.vnet.ibm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=dekelp@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f0a493ef-a0d7-4297-6022-08d6ac525b6f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR05MB4480; x-ms-traffictypediagnostic: VI1PR05MB4480: x-ms-exchange-purlcount: 1 x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0981815F2F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(376002)(396003)(136003)(39860400002)(13464003)(189003)(199004)(229853002)(105586002)(3846002)(446003)(68736007)(76176011)(6116002)(5660300002)(106356001)(7696005)(8936002)(966005)(478600001)(33656002)(99286004)(54906003)(186003)(316002)(6916009)(14454004)(66066001)(6246003)(14444005)(7736002)(81166006)(81156014)(26005)(256004)(8676002)(6306002)(11346002)(476003)(52536014)(45080400002)(53546011)(74316002)(6436002)(2906002)(86362001)(4326008)(25786009)(6506007)(486006)(97736004)(71190400001)(71200400001)(55016002)(102836004)(53936002)(305945005)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR05MB4480; H:VI1PR05MB4224.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: i9jFQffJbCEgxQdOZRRlAmdBGfou1IqChXJ32c5oCcKaMzYzjbjf1MJss+6psvYq1NwE0nZwfrGJf9ZHX3cEHgP/+F2men/u+gqVVkg4y6CFt39VpOjiLAN9L8FGE8jVHqzjOtdOkScJ5g2chVlrwpo9+/RZFpYmv6FCWFT5aQVgIX7u4AvKCbHVTgcl9BV+44udzdHNMkwDuBPJ2MPWzUV/V4iwjDaPNRizaqKymAQhjkI6nNH8dDYA2DzMjDqS5fzAp8oLg9lgH1KiclFD3Gja+QNR6ijKaP581x4DwgN0itqW9RwwKAEKumfuFNcyO7N9I3bRyAohxlSzlm3uZOs5hXiD9SDWw5D3HY0L3e3eAkR46Vov0Hd9eDRAnvPkDuWn/D0QjlbKhUP43Tgfy2R0OxXsmUV2HDju9l1kvW8= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: f0a493ef-a0d7-4297-6022-08d6ac525b6f X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2019 10:05:03.9494 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB4480 Subject: Re: [dpdk-dev] [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190319100503.ybnyqaf0KVcZrCZJM5C0_ksojGvs5uIYz1LkyP7MiYU@z> Hi, For ppc, rte_io_mb() is defined as rte_mb(), which is defined as asm sync. According to comments in arch/ppc_64/rte_atomic.h, rte_wmb() and rte_rmb() = are the same as rte_mb(), for store and load respectively. My patch propose to define rte_wmb() and rte_rmb() as asm sync, like rte_mb= (), since using lwsync is incorrect for them. Regards, Dekel > -----Original Message----- > From: Chao Zhu > Sent: Tuesday, March 19, 2019 5:24 AM > To: Dekel Peled > Cc: Yongseok Koh ; Shahaf Shuler > ; dev@dpdk.org; Ori Kam ; > Thomas Monjalon ; stable@dpdk.org > Subject: RE: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER >=20 > Dekel=A3=AC >=20 > To control the memory order for device memory, I think you should use > rte_io_mb() instead of rte_mb(). This will generate correct result. rte_w= mb() > is used for system memory. >=20 > > -----Original Message----- > > From: Dekel Peled > > Sent: Monday, March 18, 2019 8:58 PM > > To: chaozhu@linux.vnet.ibm.com > > Cc: yskoh@mellanox.com; shahafs@mellanox.com; dev@dpdk.org; > > orika@mellanox.com; thomas@monjalon.net; dekelp@mellanox.com; > > stable@dpdk.org > > Subject: [PATCH] eal/ppc: remove fix of memory barrier for IBM POWER > > > > From previous patch description: "to improve performance on PPC64, use > > light weight sync instruction instead of sync instruction." > > > > Excerpt from IBM doc [1], section "Memory barrier instructions": > > "The second form of the sync instruction is light-weight sync, or lwsyn= c. > > This form is used to control ordering for storage accesses to system > > memory only. It does not create a memory barrier for accesses to device > memory." > > > > This patch removes the use of lwsync, so calls to rte_wmb() and > > rte_rmb() will provide correct memory barrier to ensure order of > > accesses to system memory and device memory. > > > > [1] > > > https://eur03.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwww > . > > > ibm.com%2Fdeveloperworks%2Fsystems%2Farticles%2Fpowerpc.html& > ;data=3D > > > 02%7C01%7Cdekelp%40mellanox.com%7C381426b6b9d042f776fa08d6ac1a5d > c5%7Ca > > > 652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636885626593364016&am > p;sdata > > > =3DwFYTcFX2A%2BMdtQMgtojTAtUOzqds7U5pypNS%2F2SoXUM%3D&re > served=3D0 > > > > Fixes: d23a6bd04d72 ("eal/ppc: fix memory barrier for IBM POWER") > > Cc: stable@dpdk.org > > > > Signed-off-by: Dekel Peled > > --- > > lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h | 8 -------- > > 1 file changed, 8 deletions(-) > > > > diff --git a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > index ce38350..797381c 100644 > > --- a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > +++ b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h > > @@ -63,11 +63,7 @@ > > * Guarantees that the STORE operations generated before the barrier > > * occur before the STORE operations generated after. > > */ > > -#ifdef RTE_ARCH_64 > > -#define rte_wmb() asm volatile("lwsync" : : : "memory") > > -#else > > #define rte_wmb() asm volatile("sync" : : : "memory") > > -#endif > > > > /** > > * Read memory barrier. > > @@ -75,11 +71,7 @@ > > * Guarantees that the LOAD operations generated before the barrier > > * occur before the LOAD operations generated after. > > */ > > -#ifdef RTE_ARCH_64 > > -#define rte_rmb() asm volatile("lwsync" : : : "memory") > > -#else > > #define rte_rmb() asm volatile("sync" : : : "memory") > > -#endif > > > > #define rte_smp_mb() rte_mb() > > > > -- > > 1.8.3.1 >=20 >=20