From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BEBC6A052A; Thu, 28 Jan 2021 00:00:52 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 46E51141078; Thu, 28 Jan 2021 00:00:52 +0100 (CET) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30076.outbound.protection.outlook.com [40.107.3.76]) by mails.dpdk.org (Postfix) with ESMTP id B6EF0141070 for ; Thu, 28 Jan 2021 00:00:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3DOVCrSqrz9/8Ut4Vh2K09sU5xC1uwyKhyQe4HGL0yI=; b=hIINH410vOma85MJEtPoKsZi7X20c+Kvrrid8/5w9nJ28O9UlhSfyFvrhv50nAUASry5YOMBVs+aFc7Il9qft08LPxrqxPIdmooDnF6MI6DHnleMiMoQq1qyeR0tsDmRr2BH24Bgjoaat07vkFMZay7MU5J75bIMqFXtKMopfyw= Received: from MR2P264CA0186.FRAP264.PROD.OUTLOOK.COM (2603:10a6:501::25) by VI1PR0802MB2157.eurprd08.prod.outlook.com (2603:10a6:800:9c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.14; Wed, 27 Jan 2021 23:00:49 +0000 Received: from VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com (2603:10a6:501:0:cafe::3a) by MR2P264CA0186.outlook.office365.com (2603:10a6:501::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.12 via Frontend Transport; Wed, 27 Jan 2021 23:00:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT008.mail.protection.outlook.com (10.152.18.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.11 via Frontend Transport; Wed, 27 Jan 2021 23:00:48 +0000 Received: ("Tessian outbound 2b57fdd78668:v71"); Wed, 27 Jan 2021 23:00:48 +0000 X-CR-MTA-TID: 64aa7808 Received: from d53fbdbbfd17.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 784F4F98-4A7C-4EB4-A219-3FCE66684535.1; Wed, 27 Jan 2021 23:00:42 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d53fbdbbfd17.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 27 Jan 2021 23:00:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hdDqOnCyXgoLmE0m/4bMuzO42+9nP8bjw+mgjvwgKO36LIat9e+QggVf53Yb8dB1H7kpwe+4YGR9XXzxo8lazv0Mu3DgaMbKX8jaHPQdERxqDvjtpUtdbtU3+MZJ+oBgqM6JEN4o6gKdyoY9rbl5MBQb0IrUYLxTPtF2lhF3Pan908N+JUu7RwL8JN/sb8CCvv6QeCVxLFN7R+WzM0ayjWsrJ52BfLmbAn6lW7L+UqFOUbf7PagYglg0Fxoxev9cHImI/EzwpPA0TBCFMCqME4Vk/Vl9oOlGAOZeJKvAV7uYut3xivwqivAfG16VrcEULLlfg95f50s2/nRh12Kxpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3DOVCrSqrz9/8Ut4Vh2K09sU5xC1uwyKhyQe4HGL0yI=; b=hlyOHyDfyvpmTt5/EKbYKPuYOS6XqwVYU+gEJsW0wSZAvw3HuCsQN9szGhDU9oQQgihbE3hIryY4yPymbyyppPPziaol6liUdk992Ndz6Mll4HQEeYNViopGS/VYuJVefFY7mJg3zvMQ41Rh+5lcf1NcLem481PYmmibC9JqIHTtjEny7DX9dDS4wxZgQoFU5BvahyehlbvDm2LkpTziFas4/YPq04q/UMa55n8pW5IU2k9SpJyacrZDGA0UwiXhIkQ7xw3Ne5KvBiwINzDlFddGTGPViR2GNypAnNHmK5mbizrX+Um/rorkZb814bovJpcEm1aK9Liy82SPhPBsuw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3DOVCrSqrz9/8Ut4Vh2K09sU5xC1uwyKhyQe4HGL0yI=; b=hIINH410vOma85MJEtPoKsZi7X20c+Kvrrid8/5w9nJ28O9UlhSfyFvrhv50nAUASry5YOMBVs+aFc7Il9qft08LPxrqxPIdmooDnF6MI6DHnleMiMoQq1qyeR0tsDmRr2BH24Bgjoaat07vkFMZay7MU5J75bIMqFXtKMopfyw= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DB7PR08MB3753.eurprd08.prod.outlook.com (2603:10a6:10:7e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.16; Wed, 27 Jan 2021 23:00:40 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::cbb:4373:1761:19b1]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::cbb:4373:1761:19b1%6]) with mapi id 15.20.3805.017; Wed, 27 Jan 2021 23:00:40 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , Feifei Wang CC: "dev@dpdk.org" , nd , Ruifeng Wang , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v1 1/2] app/test: remove unnecessary barriers for ring stress test Thread-Index: AQHW2CwVfBkkXDx33kGrB+/tCNr1jaoDD3UAgDk4PFA= Date: Wed, 27 Jan 2021 23:00:40 +0000 Message-ID: References: <20201222063054.44429-1-feifei.wang2@arm.com> <20201222063054.44429-2-feifei.wang2@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: CAA53E796DD05E479AE418A1E5C43843.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.113.13.105] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 157ac99b-7f06-48a6-bb71-08d8c317630c x-ms-traffictypediagnostic: DB7PR08MB3753:|VI1PR0802MB2157: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: r1EN/ddOd8BJEXjZTf62G7jJFDqVKgLy88eq6E30E4LqC/wyLKRI7ZJ1k055GLRoDOJmJCMttJ0Ry2H/p7NDUDTP7nvsHBA7YFY2zQ4g8dRLAtsxH01yvZSvOMaPWpylQNq5yHmOtWOCQWetGdkYEZBuKhLNwEJVcIyx4JmfahgfUQN+D1EUuAcd+W7DQ1HDFbrqx3NE6Sf/7lGYH6rKViprfwkbrzNn/nfyg6yfOw3xirUMUTYZZouXHpfYqnwD1sqtWFGbNGYiOwIOrwxmksp14uuWLhDUl4zVNAO4TWLRjWpyoCTjQMuQZKxpftlhxt9a918LmCnUeUqqyPirOAFEjBQRwDjwCAnCPbA0/RUpYG2Riu7laUeAxGmf5dQYsnEo8HXZ0ox+OwwSOzqgBQf+ZQvym8JKpg8dIm0fP6rDqKRcT2RLhVuTqd9YlEc/l8LTuf4oWi4Had2mpe8QJewM/k1nji6mPU9SRLI2KIM+DS84tEWxGVP+WZc66A3JVm/lb3xENqJTFkAoYthtK/COuNk7WzLVwM4JZAt+mI+KUg+Lw2EiOCT0ptZG1pl3zky6QqPpaaExSAbXe+eXdDOVOpP0B1ldvPymZ0lArwk= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(346002)(376002)(366004)(396003)(478600001)(71200400001)(33656002)(66556008)(86362001)(7696005)(2906002)(6506007)(76116006)(66476007)(64756008)(66446008)(83380400001)(26005)(4326008)(8676002)(186003)(66946007)(9686003)(110136005)(966005)(5660300002)(54906003)(6636002)(52536014)(316002)(55016002)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?5HjoOdwmbN/sKNhVLwjx7N0BJMgg3XWA3G7E5Auq6GosDVRDzIVs6FFyV+fU?= =?us-ascii?Q?dUAszgqAWsu58X9igTBnjRF3ypMilMIpyHTKxoIz9IUQGyAuD/TMJMPg2P6K?= =?us-ascii?Q?MIfyPZi0842DOYscKbt0uBiRQHy/mReFteBzv8+lKP41/DDIWC5db9e9RqJb?= =?us-ascii?Q?Vx32wlOd7hv+4w7QwsfvgMEdd02AJzDSsEU5UoZiHBIIBKV3uG2LONOw4X+z?= =?us-ascii?Q?nTNlBJ5MliPbbt5519o7kkGkAwUNEo5BhRlP4OXEGsmSqa53Frq4YQco2Ssf?= =?us-ascii?Q?HuhwL431m6XmP2raVtSQV8E2qcO8D3qKB16x+KFfts2BRMtmhIGrudG7XY+C?= =?us-ascii?Q?+awAvzDRuvFHXVdKcxVa8sunBUyikJ4xc2UtPjksxjW/YrhFVdqGrmO+cMaX?= =?us-ascii?Q?Qd9AjgxXc23LlFEykJjleNjAgdYVAP1lDBDvPCr43w6BZS1IRv3sbzRZfnks?= =?us-ascii?Q?ff0W5Ep0BGyYtr/9rmaCZ/Eho9Qu3PHQlLCC31ThSHs0/8HokbkQ9PgxjYV5?= =?us-ascii?Q?XL7hZFfCkeDuCTAFhTZk3esgrGTSbffJFpRUX0lC6APRz1rxE5gWhVyU7/+s?= =?us-ascii?Q?n2ozHXr+VdogjVi+p+chgZ+pwAloFqL5EtJ0W4o3VtkLoQHMAa0hL3kvBtEH?= =?us-ascii?Q?4XtfoT/vndBcQ3d1M/jouk/BvwQovifnRkZB97SypfTLUNJ9KVs+rAG4RoWW?= =?us-ascii?Q?0HOT633Ndq/OuK2iTKPYN/CGUN2HuVj3jK5/0b7mc3F3fMhcZsX87TWZLnoJ?= =?us-ascii?Q?5JZ1/F9ygjT9lmzkTdlD509qG2Dd7SZJgL+p5xkDGYflUgtaJwA5zLg35HLC?= =?us-ascii?Q?26SyeAUPV3IDhwryKkKzlQ0wW/WMrs0mD7TvkqVXFuUgAwQ414v9OUm8o6TT?= =?us-ascii?Q?YQVnOt02fQRvc++MYyp20vDJpUYF3U7LYvPuSCnGOvhajP5ypgIm4lP63AAa?= =?us-ascii?Q?7mGMizmwxVM8PPpi815pqsJk9C+yF402xwvgkuNCjKf0xbogche4Z6usH27m?= =?us-ascii?Q?dTPQEomRSZsENDxyVIncBNWJ2WwFYZBBzoNTaByaiWAbQetomwaMbWi4xZQ8?= =?us-ascii?Q?oTXB6w5Y?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3753 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: ab0f2c4a-1932-46d3-0153-08d8c3175e1f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4xFKyC/z8zMbvLsD9FhbPzK9EO2pxJ16+0ZUyCTtVMhyS/8qsy6U8inZsN25yFZw6eVJMwqFJ+U5Kt5phLVZlTPsWqQce2Q9+2VmdgtngcNQ/jh/DsVqF+0uXZJXG7blYDhPCHBIgCoBa6UQk/GdN4+6HIQe7RJGZFxx/t6HQYHwwFI7h/VHbD3NrxObd1Je2YvvoJUglwTXrhPQ1qNIL1SJmkhIBAw/5jbA58oIl7Fx2i25USY6PhQlgxsVGi/i8XMy/WCVdHlJVkgY/v3nS1VjdbpeTPL8ZkGx0XShuTByh2yjwkchTHx6QIvUrgud5pyu2Mmsh8eVUjnNnTh1XNO3bMrJOk5BVjvXH92ZpOZ5hiM2nfEMuTdpY75UxUSNfRHzqpElj425NL/NtSrTviiE6kjSwXUJR0txmu+gcIexEDk/aab8EWRGgMe/yuyz+0gEP/Zqc9I2AjV6j9q05PchLOHy+CbGDlIk/auNpltZOXY+WNJFp7adISK6KyLgqMUubdtvSMlUsQ6xx8qJLCyjUXr2cu2XizmMxAI6JWfk5/1V7bS/1xi8XmclallWcb9Hffs3H3hCDwRRX92Yy73AAi9dIldYkzJFrNo47cfvAa39xtM9oUkjdebLcGLgytyITjivT+jSRdm7npzuXed/8iJXaQTICGCzaQ3YqNw= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(396003)(346002)(376002)(136003)(39860400002)(46966006)(2906002)(186003)(86362001)(47076005)(6636002)(9686003)(54906003)(33656002)(55016002)(5660300002)(26005)(83380400001)(316002)(4326008)(478600001)(8676002)(70586007)(70206006)(82310400003)(8936002)(6506007)(7696005)(336012)(82740400003)(52536014)(81166007)(356005)(110136005)(966005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jan 2021 23:00:48.4608 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 157ac99b-7f06-48a6-bb71-08d8c317630c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT008.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2157 Subject: Re: [dpdk-dev] [PATCH v1 1/2] app/test: remove unnecessary barriers for ring stress test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Hi Feifei, >=20 > > > > The variable "wrk_cmd" is a signal to control threads from running and > > stopping. When worker lcores load "wrk_cmd =3D=3D WRK_CMD_RUN", they > start > > running and when worker lcores load "wrk_cmd =3D=3D WRK_CMD_STOP", > they > > stop. > > > > For the wmb in test_mt1, no storing operations must keep the order > > after storing "wrk_cmd". Thus the wmb is unnecessary. >=20 > I think there is a bug in my original code, we should do smp_wmb() *befor= e* > setting wrk_cmd, not after: >=20 > /* launch on all workers */ > RTE_LCORE_FOREACH_WORKER(lc) { > arg[lc].rng =3D r; > arg[lc].stats =3D init_stat; > rte_eal_remote_launch(test, &arg[lc], lc); > } >=20 > /* signal worker to start test */ > + rte_smp_wmb(); > wrk_cmd =3D WRK_CMD_RUN; > - rte_smp_wmb(); >=20 > usleep(run_time * US_PER_S); >=20 >=20 > I still think we'd better have some synchronisation here. > Otherwise what would prevent compiler and/or cpu to update wrk_cmd out > of order (before _init_ phase is completed)? > We probably can safely assume no reordering from the compiler here, as we > have function calls straight before and after 'wrk_cmd =3D WRK_CMD_RUN;' > But for consistency and easier maintenance, I still think it is better to= have > something here, after all it is not performance critical pass. Agree that this is not performance critical. This is more about correctness (as usually people refer to code to understa= nd the concepts). You can refer to video [1]. Essentially, the pthread_crea= te has 'happens-before' behavior. i.e. all the memory operations before the= pthread_create are visible to the new thread. The rte_smp_rmb() barrier in= the thread function is not required as it reads the data that was set befo= re the thread was launched. I do not know why rte_smp_wmb is required. The update to 'wrk_cmd' is seen = by the thread eventually. rte_smp_wmb does not result in update being seen = sooner/immediately.=20 [1] https://www.youtube.com/watch?t=3D4170&v=3DKeLBd2EJLOU&feature=3Dyoutu.= be >=20 > > For the rmb in test_worker, the parameters have been prepared when > > worker lcores call "test_worker". It is unnessary to wait wrk_cmd to > > be loaded, then the parameters can be loaded, So the rmb can be > removed. >=20 > It is not only about parameters loading, it is to prevent worker core to= start > too early. Because 'pthread_launch' provides the 'happens-before' behavior, the worker= core will see the updates that happened before the worker was launched. I suggest changing the commit log to provide the reasoning around pthread_c= reate. >=20 > As I understand, your goal is to get rid of rte_smp_*() calls. > Might be better to replace such places here with _atomic_ semantics. > Then, as I can see, we also can get rid of 'volatile' fo wrk_cmd. >=20 > > In the meanwhile, fix a typo. The note above storing "stop" into > > "wrk_cmd" should be "stop test" rather than "start test". > > > > Signed-off-by: Feifei Wang > > Reviewed-by: Honnappa Nagarahalli > > Reviewed-by: Ruifeng Wang > > --- > > app/test/test_ring_stress_impl.h | 5 +---- > > 1 file changed, 1 insertion(+), 4 deletions(-) > > > > diff --git a/app/test/test_ring_stress_impl.h > > b/app/test/test_ring_stress_impl.h > > index f9ca63b90..384555ef9 100644 > > --- a/app/test/test_ring_stress_impl.h > > +++ b/app/test/test_ring_stress_impl.h > > @@ -198,7 +198,6 @@ test_worker(void *arg, const char *fname, int32_t > prcs) > > fill_ring_elm(&loc_elm, lc); > > > > while (wrk_cmd !=3D WRK_CMD_RUN) { > > - rte_smp_rmb(); > > rte_pause(); > > } > > > > @@ -357,13 +356,11 @@ test_mt1(int (*test)(void *)) > > > > /* signal worker to start test */ > > wrk_cmd =3D WRK_CMD_RUN; > > - rte_smp_wmb(); > > > > usleep(run_time * US_PER_S); > > > > - /* signal worker to start test */ > > + /* signal worker to stop test */ > > wrk_cmd =3D WRK_CMD_STOP; > > - rte_smp_wmb(); > > > > /* wait for workers and collect stats. */ > > mc =3D rte_lcore_id(); > > -- > > 2.17.1