From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DB790A2EDB for ; Fri, 6 Sep 2019 09:20:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8FFDE1D175; Fri, 6 Sep 2019 09:20:44 +0200 (CEST) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30058.outbound.protection.outlook.com [40.107.3.58]) by dpdk.org (Postfix) with ESMTP id EE8241D175; Fri, 6 Sep 2019 09:20:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JhK+zEwf0fepCohBt49GU/fL5DEsTbP0l9CPJamW0TA=; b=bnZYmxeOoO8jcaHpWvYDxB9QV/7YGbC3kAhNi19opmny26ZMm4Gy1EGI7xTSUCJfG2WSfXQC/t6So+2h9HwYAJ7SLPe3KwzKrXObBqTC30/BpvylTfFul386PpSENtbwkOq6aHdlCgbrTMxwynxu/GboZO+KkbgJ2a5OsFeB4lI= Received: from VI1PR08CA0123.eurprd08.prod.outlook.com (2603:10a6:800:d4::25) by DB7PR08MB3052.eurprd08.prod.outlook.com (2603:10a6:5:28::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2220.20; Fri, 6 Sep 2019 07:20:40 +0000 Received: from VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::207) by VI1PR08CA0123.outlook.office365.com (2603:10a6:800:d4::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 07:20:40 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT007.mail.protection.outlook.com (10.152.18.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2241.14 via Frontend Transport; Fri, 6 Sep 2019 07:20:38 +0000 Received: ("Tessian outbound aa6cb5c8f945:v27"); Fri, 06 Sep 2019 07:20:36 +0000 X-CR-MTA-TID: 64aa7808 Received: from f8a6cd41c013.1 (cr-mta-lb-1.cr-mta-net [104.47.12.51]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id F94CABC6-D106-4221-8A59-087B61BC113A.1; Fri, 06 Sep 2019 07:20:31 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2051.outbound.protection.outlook.com [104.47.12.51]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f8a6cd41c013.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 06 Sep 2019 07:20:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nYpDhMDn6YTX+MQCG4kZq76Qbj6RTnON/pqGEPIT1t+GpI8OpN3c0IvdSqiQeHzaZNaaxj+Sjvxi8UtlpOJ1xS/Y1zeg0Xp2bjLlkSFHwA8gUcTuvu4xzJAqeRHKpEB/ZApQs9VBjCZcgS8C9WXLool/AdLhNIpdqbtgekOhjH5L6VTKCNhps+nHA6G/vII5lHQqTGVY92OTDc3r5Iscear2u0LeMRFVkEyVpU80K/EHo0x5RVQ0UyHlPJ2HHzBXqpvfNWVV3iO+QcOfeyT3HdRp9NCRDKGksrV7hkwKjxT+mNHgFZKkASLTwKSFhYyu+m1+RIupJXKzIJ7r1Fk7pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JhK+zEwf0fepCohBt49GU/fL5DEsTbP0l9CPJamW0TA=; b=oNmW67vXoVXm1OaDaoGWRojSwmfSO4kze9uTUJrAUU/dC6PIWD1oS+NtjyqmLGb5mbqXGbPgsdFxhZqKwiR78ID89w6m63pYrn/XiY0RXJTq4xEr4eVb+gENeHDOL8ylRPm+2y6vtqYGeMTBiOgqwz4xsvGmdKomnLyPJ9gjiumZTdZZR6NbqFNh/yPBwgEQ4ghoAk32kEuDHGkj+FsJZ0tS7cn/e8XkxGWZmiFwahhCbu5tO7tZgC30WqH6y5zieer/Sv0Eoa1LxOMwMIMY7oyLvpd2bujLLgSW7YWcADtHVfYNPTXtTOdGX60xb+jKyM+B28gk1nyCEV0Azb6KCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JhK+zEwf0fepCohBt49GU/fL5DEsTbP0l9CPJamW0TA=; b=bnZYmxeOoO8jcaHpWvYDxB9QV/7YGbC3kAhNi19opmny26ZMm4Gy1EGI7xTSUCJfG2WSfXQC/t6So+2h9HwYAJ7SLPe3KwzKrXObBqTC30/BpvylTfFul386PpSENtbwkOq6aHdlCgbrTMxwynxu/GboZO+KkbgJ2a5OsFeB4lI= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (10.255.27.75) by VE1PR08MB5054.eurprd08.prod.outlook.com (10.255.159.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2220.19; Fri, 6 Sep 2019 07:20:28 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::2d67:163d:b192:4b9]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::2d67:163d:b192:4b9%7]) with mapi id 15.20.2220.022; Fri, 6 Sep 2019 07:20:28 +0000 From: "Phil Yang (Arm Technology China)" To: Slava Ovsiienko , "yskoh@mellanox.com" , Matan Azrad , =?iso-8859-1?Q?N=E9lio_Laranjeiro?= , "dev@dpdk.org" CC: "thomas@monjalon.net" , "jerinj@marvell.com" , Honnappa Nagarahalli , "Gavin Hu (Arm Technology China)" , nd , "stable@dpdk.org" , nd Thread-Topic: [PATCH 2/2] net/mlx5: fix Tx CQ doorbell synchronization on aarch64 Thread-Index: AQHVY+MlefwZT1B+GUKGfvGmUb8pcqceJ2yQ Date: Fri, 6 Sep 2019 07:20:28 +0000 Message-ID: References: <1567680908-31210-1-git-send-email-phil.yang@arm.com> <1567680908-31210-2-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 0e088f97-2a4b-47d1-819d-e1aeba61a6c1.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 8543990f-6173-426d-0355-08d7329ab7dc X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VE1PR08MB5054; X-MS-TrafficTypeDiagnostic: VE1PR08MB5054:|VE1PR08MB5054:|DB7PR08MB3052: X-MS-Exchange-PUrlCount: 1 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 0152EBA40F X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(136003)(346002)(376002)(39860400002)(366004)(189003)(199004)(13464003)(25786009)(71190400001)(102836004)(64756008)(186003)(26005)(256004)(6506007)(14454004)(71200400001)(53546011)(86362001)(11346002)(14444005)(66066001)(110136005)(316002)(54906003)(446003)(966005)(2501003)(6116002)(4326008)(76176011)(66476007)(66556008)(478600001)(7696005)(66446008)(66946007)(3846002)(76116006)(7736002)(5660300002)(99286004)(6436002)(229853002)(6246003)(2906002)(52536014)(66574012)(9686003)(81166006)(81156014)(8676002)(486006)(6306002)(55236004)(476003)(8936002)(305945005)(74316002)(33656002)(53936002)(55016002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5054; H:VE1PR08MB4640.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: xt8SCMrYbWh6dJy5nesSrwu8N4NzcY3zA+SX5eybl7/01U8s90VkdNMURftRtD7wQFXntyd6R5rG5YloqaWbnAe9Tong04lSrYH1G5mGy+z5bESW7AXXnBGeOa95JiTvizCeEl/yQthA3K3RTd3nWsmtnY+we2ljt0ZLhBumz5PJu2i6PI/X0BdHwFpzOQfZD2sUpSI6jTbkKis+hRVLTRLruHoexnE8d3bdJxUcKY4VQqL1uaXI1QCqVdfEswRGq1gEO4jc6Q0L4R0BbkD+DHkOvDHroMfBp/7/1qdYAo/uK1zGjtl6piuR34Y4PMH8DD6kltCbkz6JN5hgw7KACKs0cDCSihlfzbnMDP1rsOeQAgV745VYZWh0EDJq9hREewz2qhl9dpKz11xK9pjXABkcaHndgzcb4zLM4rmDBoc= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5054 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT007.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(136003)(39860400002)(376002)(396003)(2980300002)(13464003)(199004)(189003)(33656002)(8676002)(446003)(6306002)(476003)(9686003)(11346002)(14444005)(63370400001)(126002)(486006)(63350400001)(76176011)(23756003)(55016002)(7696005)(2906002)(8936002)(36906005)(70206006)(316002)(70586007)(6506007)(26005)(53546011)(81166006)(99286004)(110136005)(8746002)(81156014)(2501003)(336012)(6116002)(3846002)(54906003)(102836004)(186003)(52536014)(478600001)(66574012)(50466002)(356004)(6246003)(5660300002)(26826003)(66066001)(966005)(450100002)(229853002)(25786009)(4326008)(86362001)(7736002)(47776003)(14454004)(305945005)(22756006)(74316002)(76130400001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3052; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: aafb9c2d-547b-48d3-5241-08d7329ab1a1 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600166)(710020)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DB7PR08MB3052; NoDisclaimer: True X-Forefront-PRVS: 0152EBA40F X-Microsoft-Antispam-Message-Info: A997452iiDLwf5gpL2FCC3sRXGnx15sri8em47Z6u5Uo9PEa8Ldvb1JkDp7WycWUVI0wrfioLhOFCEmHYfgUttzwUHTQc410u8urxiZ+dmFXbfMG/W+sVDOYvBmN9ym1Hqt3UZjnKHeebrtVRc6lPtGDt8W+nLVHHowPRvFIV8GruUiVMTPzyRvEOZiKQPvragM7KzfcPeHPdajOfnaKpmjUG4a8EOLBP2L2v9keVV55Z1GkilcUJahLyoBRpzdSfbupCIY8qnCgZc5iwYsuLNhrR9DNgvqp1IS0QPWIX6odJtwkoVULGzJ/XJZIgtuqyEoTzipgga+/c66NxpeTVuvpxpQMd0HqO13S9znqFwzDe+yfMumr9UuPUp6+zvyl8Kqd2BnkVFrVV6D+2XM9HS+eRY5MSVr9DLjVcM3SkbQ= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2019 07:20:38.5154 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8543990f-6173-426d-0355-08d7329ab7dc X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3052 Subject: Re: [dpdk-stable] [PATCH 2/2] net/mlx5: fix Tx CQ doorbell synchronization on aarch64 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, Slava Thanks for your comments.=20 > -----Original Message----- > From: Slava Ovsiienko > Sent: Thursday, September 5, 2019 8:12 PM > To: Phil Yang (Arm Technology China) ; > yskoh@mellanox.com; Matan Azrad ; N=E9lio > Laranjeiro ; dev@dpdk.org > Cc: thomas@monjalon.net; jerinj@marvell.com; Honnappa Nagarahalli > ; Gavin Hu (Arm Technology China) > ; nd ; stable@dpdk.org > Subject: RE: [PATCH 2/2] net/mlx5: fix Tx CQ doorbell synchronization on > aarch64 >=20 > Hi, Phil >=20 > This point is in datapath and performance is very critical. > The rte_cio_wmb() may take a lot of CPU cycles, waiting till all previous > writes become visible for all external (relating to core) agents.=20 > The Tx CQE doorbelling does not need any writes to other locations to be = completed,=20 In my understanding, the PMD needs to wait till all txq fields update is co= mpleted then ring the doorbell for HW. Before the Tx CQE doorbelling, it will update the producer index of work qu= eue in Tx queue descriptor (at line 2037).=20 The compiler barrier cannot guarantee the ordering of these operations. So = use the explicit HW fence to achieve that. As same as the HW Tx doorbell in vectorized Tx burst routine, it uses a wri= te memory barrier to enforce the register update visible to HW immediately. Section 32.5.2 in https://doc.dpdk.org/guides/nics/mlx5.html=20 > the only concern is not to reorder/merge the writes to the same doorbell = register of > the same sending queue in the tx_burst() internal sending loop/subsequent= calls. >=20 > As far as I know - the writes to the same location should not be reordere= d by any arch > (may be merged if memory settings allow this, it is not critical for CQE = doorbell), > could you, please, explain why we need explicit hardware fence before CQE= doorbell > update? Do you think doorbell write might be rearranged with previously > reads from the ring buffer? >=20 > WBR, > Slava >=20 > > -----Original Message----- > > From: Phil Yang > > Sent: Thursday, September 5, 2019 13:55 > > To: Yongseok Koh ; Slava Ovsiienko > > ; Matan Azrad ; > N=E9lio > > Laranjeiro ; dev@dpdk.org > > Cc: Thomas Monjalon ; jerinj@marvell.com; > > Honnappa.Nagarahalli@arm.com; gavin.hu@arm.com; nd@arm.com; > > stable@dpdk.org > > Subject: [PATCH 2/2] net/mlx5: fix Tx CQ doorbell synchronization on > > aarch64 > > > > For the weaker memory model processors, the compiler barrier is not > > sufficient to guarantee the coherent memory update be observed by I/O > > device. It needs the coherent I/O memory barrier to enforce the orderin= g > of > > Tx completion queue doorbell operation. > > > > Fixes: da1df1ccabad ("net/mlx5: fix completion queue drain loop") > > Cc: stable@dpdk.org > > > > Suggested-by: Gavin Hu > > Signed-off-by: Phil Yang > > Reviewed-by: Gavin Hu > > --- > > drivers/net/mlx5/mlx5_rxtx.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.= c > > index 4c01187..c11148b 100644 > > --- a/drivers/net/mlx5/mlx5_rxtx.c > > +++ b/drivers/net/mlx5/mlx5_rxtx.c > > @@ -2042,7 +2042,7 @@ mlx5_tx_comp_flush(struct mlx5_txq_data > > *restrict txq, > > } else { > > return; > > } > > - rte_compiler_barrier(); > > + rte_cio_wmb(); > > *txq->cq_db =3D rte_cpu_to_be_32(txq->cq_ci); > > if (likely(tail !=3D txq->elts_tail)) { > > mlx5_tx_free_elts(txq, tail, olx); > > -- > > 2.7.4