From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id F2079A0A0C; Thu, 1 Jul 2021 14:52:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A7B5E412CA; Thu, 1 Jul 2021 14:52:42 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2073.outbound.protection.outlook.com [40.107.223.73]) by mails.dpdk.org (Postfix) with ESMTP id C21BC40141 for ; Thu, 1 Jul 2021 14:52:40 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T/fo1AhA35kiIREa2SCthiQ7TdcBh3AEUXinWlMmAZPyx36DjOEO5yiZ5sZzIdd1xi/W2HEeFwFl3MnpGQDPzfzivZDxmpc9h7lPsGU31J4BqaELVzfOn5vY1HsjEd/7XipHNzrnElWim1cDQRl+ZVFTeYZrvs30SOo+Yp4UXUA5DsABIEMam8NuEQ4Nav9IcmAisVFvFTkVOH11WspdVPUJgZG3Ue5cKPKRFu/Uim74hcz3C51M9c212Nw8QGYjNA/EruPI+0FxxsZcAURr4hCZCMaL8iY2QtjeAzrVfWujCUV1gdDystU3ZQ2MUSwQcBODIvnHnj67JWJfqT88ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BWY7Qw04hxZ3yPClHde/PxS5e46PyMbY+qseBSRPq1w=; b=jHoezjGcHZOSRozKT7pFntHGlNUkHYWNjQZSPQc8q5Vxozn0fRBAZZ2Q2LI4gOhqbGhJNv85QH7EEaXukPhIobUNAhzgCViI/MyJhLFTl8WstBiDZhN1etTPWAHTugiNk/MNLK+pLgWLTp2JB/zBp77LG15D1tGFTyejjb6O3JIuA4BqPJpVIMsqpQfVQvn7wZJH6WzZltujfq066RGWiiNP40eEr2ZGccm0MhAmSxF3wgsQtwl0DBLskJSg+WiFf2Sy/0XuJITRVkCTGpKJXdE5Mwm2yUfgBMHAzKMc6MGFiVUuqmOaI9OlTmOlNg86tfHZR2B+LP71I9c2e0H11Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BWY7Qw04hxZ3yPClHde/PxS5e46PyMbY+qseBSRPq1w=; b=Rz4QAChicT5mCyVK8nI9QQ4mwgWknC7UD3WJC+KAW4IzuQHH7FI3EzR528lqr/BvEB4ntU7z/QyUh7Noi5Zc0u2TANr/fv5Aetw++d/mnpbVVRGHrznkq3xO07v+ayMmBjcvIlIcyq44Zn+mPg9jTWIk853hlJUP8VDOclarzjA7QwO1kgBagdQJPmnbW+iajgx5Ayy5oEpnkKHkcr//brltWUTJxqvqi10BAcdgg/uugM+CYzq7lcUGfh9y30nhXjCt5V8BpAxuIKqbX7+cLa3IiFXF6p+2ZGvcGj9/C2PO22rlmqheMob4+4/C4usRlGyQc2NuooMD866nZw8NVQ== Received: from DM6PR12MB3753.namprd12.prod.outlook.com (2603:10b6:5:1c7::18) by DM6PR12MB4386.namprd12.prod.outlook.com (2603:10b6:5:28f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.18; Thu, 1 Jul 2021 12:52:38 +0000 Received: from DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::2521:f25e:e8db:6e35]) by DM6PR12MB3753.namprd12.prod.outlook.com ([fe80::2521:f25e:e8db:6e35%6]) with mapi id 15.20.4287.024; Thu, 1 Jul 2021 12:52:38 +0000 From: Slava Ovsiienko To: Honnappa Nagarahalli , "dev@dpdk.org" , Matan Azrad , Shahaf Shuler CC: "ruifeng.wang@arm.com" , Matan Azrad , "stable@dpdk.org" Thread-Topic: [PATCH] net/mlx5: remove unwanted barrier Thread-Index: AQHXWvQGk/qeEMXKyEOAmwoZnR6qUKsuKwNQ Date: Thu, 1 Jul 2021 12:52:38 +0000 Message-ID: References: <20210606164948.35997-1-honnappa.nagarahalli@arm.com> In-Reply-To: <20210606164948.35997-1-honnappa.nagarahalli@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [188.163.75.31] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c776f006-76ee-4709-2347-08d93c8f1b65 x-ms-traffictypediagnostic: DM6PR12MB4386: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: yMbv3/DkUvoz+o15Z6f0XPxKHxMNoFUBYz+NHU6g4mVvNku81ufy+sJUDha1Am28Z4CTuFgsXKyItivphzqIRa3rrA8vpxZQIzJaa5XJ6dhmqeGSftLkJ0AkfyC/dL7bBaV1HTUE9oCCPq0PFmvgvtijLq5KEwbP0Ujrg4o2/6nVuzjkqXjQFFYVGUww+FZitjTNf+8HN3vk0qg/ORiLNq61ZGt9gOq4hXktgi9aLk7sYQA0JJuTtzw8JFAa8CJDC9JLDQ48rGnIhTbefoXzU4IbmRWZn6zDzl6h+mQfx7heRQpgVb/c5/IkuWt8KBeZOrn8HyLIGcNSYadMNOs6OdWt0qxZsQDIg6v8eNG7/31zIasLBI5moE/0PZepgApfpW1J2EpZYN3JAqVGtw6zF+gDJttzprk0T83a8dJ8Vk2r1AALT7bcBW25lQtz/NZ+egIvKdsot3gXq2bNaooukpatM4DyoIvm/lOkUc+JGPHxMigv25WVAKZNeuAieY/E+hyNlQrD+ZmbCVddrX60ZLuPVrZa/hqCd6m9e3/uG7BuNhK4KmAC6e7DiWwXY2b3kH4usJmB5f7ILDmJRkYMfIv64B3rDZZEamvF+bHhoFoJvx5Gs9scwcXfqNkJ6WHFWvNEaEkqj0fxjIhABCRZFuocMA7TOoDUqVtaVVKJcdWQGbAKTGSI/zGWyPN0CbPXn7FwD6M/h1+b6nDswFIOdg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR12MB3753.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(366004)(346002)(376002)(396003)(39860400002)(8676002)(33656002)(5660300002)(38100700002)(26005)(64756008)(186003)(54906003)(8936002)(122000001)(55236004)(66476007)(66946007)(71200400001)(316002)(76116006)(6506007)(53546011)(7696005)(66556008)(4326008)(86362001)(66446008)(110136005)(52536014)(83380400001)(9686003)(55016002)(6636002)(478600001)(2906002)(26123001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?rPiQEVdj+rokl5ImuTy6JutBXJXKJlEeTfemnFjudSBgFt7nCyd5df0BpBGQ?= =?us-ascii?Q?2KLUX9aCkc8bnRG5jaxAKFGlETct0DjHXLajVjLSrRl6KEExLyZj20n84rR5?= =?us-ascii?Q?9JOm6ZXpWUFoGJQ4ifkrOxoiJ1Mli+kJ8ypDWRtNbuvUer0D/fyksZ6Toowq?= =?us-ascii?Q?F2fRZRNNs02XLjY6JXvEZa5eVIs6o76/VSHn+HYcaNP3ZqJGuO9IHiNHKOOn?= =?us-ascii?Q?0sQldKzLBWmeqCC5F7rkTHCtTYOcqE8mRNSWxeB3RpczyXfCoiC5Y9d6k3mE?= =?us-ascii?Q?cYBn5s0XtTyrebGBF1wJRCcV/IxBlqqfunvpgCJUlS1w8oLr9E+9x8Ozls3k?= =?us-ascii?Q?+7OrfOUkIlSfCCgjPdRLGrbUwSZbWIxTF7CT/bNUrn2+hhsniD7+e9oXD+Xo?= =?us-ascii?Q?Dj41vdVbFo3+7g4Pu4yMstV1rpi9Scyl3fTlZy1/U/Zyrl9OqiX4WuM0ta+Z?= =?us-ascii?Q?SdYAA/SYCm7PRjsGDhT1y+wJJbbRLx/Oo28FVnaHPZep5CCuoTvsZV44ndlN?= =?us-ascii?Q?WYzqMcRhlYdzuQd0969QTD+6r1IRhNP2vU73OYUafmexiAUXTLmEFop5dqWZ?= =?us-ascii?Q?xesGx00AwXewfZ7SwjuYucr8JKI/Na2ZalwpiRM4yiO/tKwI3IeoFHak5vG6?= =?us-ascii?Q?HEfh+0KcYhoEf2Y20jErnhK5yeHG6a30RrL5w9BrmBgt0HqJ5WSFzHWn10Is?= =?us-ascii?Q?gwxne7cLud7hUi7bDQDofMiCL/1BFHRqb80vhL5U1hKxgk+ecAcsdjkuleZO?= =?us-ascii?Q?99GfkqUNzbGX9mtYNaAOuz5lgZrAvDhKciPhh76qEeHMxzH8K6rbcE+ixInr?= =?us-ascii?Q?uweHMG8dF6c49235UaRDKa7XvdtpDHa2C0s9SVw8zI4FdBq2O74QcScemI1C?= =?us-ascii?Q?RK/N5aqaevqJ19h4xzvq632boUzwvrtOMmzq5uyzalnObDHYvq/+fjT3VIJh?= =?us-ascii?Q?44092f6I40DqeJflXo0j5ghkCyO00XvkpLJUeO1Rbu+le7QoJ5HxXzC9riU2?= =?us-ascii?Q?XqEagTlmraopGeoD93ThKZ2e+rI+lEmicF9dMJhP+iu/4TP/DQVB+7ggR84D?= =?us-ascii?Q?GQK+Q0D8KEdwi0VckvFSEULTvbmFvGz4fjOvrYJz2T83NnLmH3jPLm4pxMAQ?= =?us-ascii?Q?jfj34Y1+eKCSGRFQgU1sYzm88Rq3aT7y3yjmvKeiUiYsslrBGEMfW/PkqL2q?= =?us-ascii?Q?ctzXCYGxbeIACzlAZr//1HNkH4OwIs7/unnyWS04igiAE4wtwZlHOufrYKb6?= =?us-ascii?Q?7RWZbgtOONq5RiU8zrkBiceYwAUHaBBiI5Kdk8uiy7wsLbHOWpbbaZ1VAIQJ?= =?us-ascii?Q?to21B27DkWXKXTPpd72d8yov?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3753.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c776f006-76ee-4709-2347-08d93c8f1b65 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2021 12:52:38.6559 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oLFvGWHeqCq6fjn7vIbvmDBSXBM5IKOK8SBTcNx6hILEWqGucSPhg28zRRaX2m7OXggd204LoL3WCG9Kc+ns0g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4386 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: remove unwanted barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, Honnappa The rte_io_rmb() was inserted not to prevent the extra access to cqe->op_ow= n (the volatile qualifier is quite enough, if we had some doubts, we would in= sert rte_compiler_barrier), but the real intention of io_rmw was to isolate cqe->op_own loads on hardwa= re level. cqe points to the Completion Queue Entry (CQE), that is mapped to the memor= y that is continuously being updated by the device (NIC). CQE is 64B size structure and op_own is = located at the structure end, and is updated by HW in last order, after the entire CQE is completely writ= ten to the host memory. After detecting by cqe_check() the CQE is owned by software (hardware compl= eted operation) the PMD starts touching other CQE fields, i.e. the next load transactions f= rom CQE are triggered. And we must make sure these loads happen in correct order, only if cqe->op_= own load was completed and valid ownership flags were seen, i.e. - do not allow speculative reads with= possible incorrect values fetched). Just hypothetical case (I agree in advance - it is very unlikely, but is no= t impossible :)): 1. owner =3D cqe->op_own - load A triggered 2. some code is being speculatively executed, no barrier 3. length =3D cqe->length - load B triggered 4. Let's suppose CPU reordered A and B, ie order of loads: B, A 5. In memory/CPU cache we have old CQE, owned by HW 6. B load gets the old length value (invalid) 7. Hardware writes the new CQE and CPU cache is invalidated 8. A load gets the CQE is owned by SW and the invalid results of load B wil= l be used by PMD Hence, I would consider the patch as risky, and as one that is extremely ha= rd to be covered completely with tests - we should test for race conditions on multiple architectures, on many CPU m= odels, PCIe root complexes, etc. With best regards, Slava > -----Original Message----- > From: Honnappa Nagarahalli > Sent: Sunday, June 6, 2021 19:50 > To: dev@dpdk.org; honnappa.nagarahalli@arm.com; Matan Azrad > ; Shahaf Shuler ; Slava Ovsiienko > > Cc: ruifeng.wang@arm.com; Matan Azrad ; > stable@dpdk.org > Subject: [PATCH] net/mlx5: remove unwanted barrier >=20 > The IO barrier is not required as cqe->op_own is read once. The checks do= ne on > the local variable and the memory is not read again. >=20 > Fixes: 88c0733535d6 ("net/mlx5: extend Rx completion with error handling"= ) > Cc: matan@mellanox.com > Cc: stable@dpdk.org >=20 > Signed-off-by: Honnappa Nagarahalli > Reviewed-by: Ruifeng Wang > --- > drivers/common/mlx5/mlx5_common.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) >=20 > diff --git a/drivers/common/mlx5/mlx5_common.h > b/drivers/common/mlx5/mlx5_common.h > index 5028a05b49..a4c29f51f1 100644 > --- a/drivers/common/mlx5/mlx5_common.h > +++ b/drivers/common/mlx5/mlx5_common.h > @@ -195,7 +195,7 @@ check_cqe(volatile struct mlx5_cqe *cqe, const > uint16_t cqes_n, >=20 > if (unlikely((op_owner !=3D (!!(idx))) || (op_code =3D=3D > MLX5_CQE_INVALID))) > return MLX5_CQE_STATUS_HW_OWN; > - rte_io_rmb(); > + > if (unlikely(op_code =3D=3D MLX5_CQE_RESP_ERR || > op_code =3D=3D MLX5_CQE_REQ_ERR)) > return MLX5_CQE_STATUS_ERR; > -- > 2.17.1