From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03676A0540; Tue, 14 Jul 2020 20:36:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BBB3A1C434; Tue, 14 Jul 2020 20:36:42 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140049.outbound.protection.outlook.com [40.107.14.49]) by dpdk.org (Postfix) with ESMTP id 37DF21C2F6 for ; Tue, 14 Jul 2020 20:36:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ApORmZZ5MTidEe8b4KJrqEcmZR0O54OEpyuesUYD/i4=; b=AKpfLhun44NKjM3NKLvFfX7ZpR6+UmTLer18g+Rnrw3xfUjgCZqkx/X5IA5GlMEwZQGmnJwj2Fis9ak4s9pd7EbmhcN4i4p9B0KbeEJgBz5MGLVgeliQZrF6Pq44RuG4CgjcW46B/j1cD0o4WIHBYcRo4dRH+gbhLSOABIZAQ3A= Received: from DB6PR0202CA0036.eurprd02.prod.outlook.com (2603:10a6:4:a5::22) by AM0PR08MB3362.eurprd08.prod.outlook.com (2603:10a6:208:dc::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.20; Tue, 14 Jul 2020 18:36:38 +0000 Received: from DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:a5:cafe::96) by DB6PR0202CA0036.outlook.office365.com (2603:10a6:4:a5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.17 via Frontend Transport; Tue, 14 Jul 2020 18:36:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT036.mail.protection.outlook.com (10.152.20.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21 via Frontend Transport; Tue, 14 Jul 2020 18:36:38 +0000 Received: ("Tessian outbound 1c27ecaec3d6:v62"); Tue, 14 Jul 2020 18:36:38 +0000 X-CR-MTA-TID: 64aa7808 Received: from c03c283d9d1a.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 961B3003-3335-4373-87D2-ECA04C03894F.1; Tue, 14 Jul 2020 18:36:33 +0000 Received: from EUR02-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c03c283d9d1a.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 14 Jul 2020 18:36:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lsONHoMmxD8uhmeGo33TrcleyJnJV08JKbdU3NjCXwsfjG8UOrgipy9nebaKEUFrF1UahPWyM8CCYBZpGgfNoRC/TKD2YL1s2YAbO0cdcU9fSCiE28yo7rQLZ5PBA1c+EjR3VKJ+x2oxVv5ThLzCeKiLxHOQnxpYC2QvJWbz/HoJ6pITgIkr3kYFDc7OBbVag/v34CKQ7rtm21cCXykrjIItxm9BVDb6TY1vyrB+4JEqF5vyJNn/6/B5ocHBfHqbpAQNm8lJV6WZr2fqwqbPz8E1efik0pKQmywjRuW/y3OvqYSbDjj9LEe6jngiAYHk8qPnQi7RTZaLSWB8LFNlRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ApORmZZ5MTidEe8b4KJrqEcmZR0O54OEpyuesUYD/i4=; b=V4qg+jkxfKOvW9LkXsBAEtaTmquoUEoIeSEMu//d45gq9604RD9L/kmVktt3G6IT8uz7+FihnCULRIWZgALV4L28ZTwdEZuCzWnUKM92D7rVSMnry0XYW6J3B3n301b71ERzIGIxfndPyZmEJzHdqZ/mHyK9KZLLhW5FU1+Btx5Udv8RvhC4RyKRrhWWxvY7Wim8/2ZYnJcAv/zUKY09kvbXgODwpnMoZxzbW3opHhkYiwzvF3LbcDfA9iOTlNwwoY/1ZgUESys9rEaxBh/qrc3cN0ujks0fR9bp0JmyRAZcxVtoXg1gw+glgSzDkiTNjS2Rtq54G0wfRM++D5NyNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ApORmZZ5MTidEe8b4KJrqEcmZR0O54OEpyuesUYD/i4=; b=AKpfLhun44NKjM3NKLvFfX7ZpR6+UmTLer18g+Rnrw3xfUjgCZqkx/X5IA5GlMEwZQGmnJwj2Fis9ak4s9pd7EbmhcN4i4p9B0KbeEJgBz5MGLVgeliQZrF6Pq44RuG4CgjcW46B/j1cD0o4WIHBYcRo4dRH+gbhLSOABIZAQ3A= Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com (2603:10a6:4:85::9) by DB7PR08MB3625.eurprd08.prod.outlook.com (2603:10a6:10:42::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.20; Tue, 14 Jul 2020 18:36:28 +0000 Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d]) by DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d%10]) with mapi id 15.20.3174.026; Tue, 14 Jul 2020 18:36:27 +0000 From: Honnappa Nagarahalli To: Phil Yang , "thomas@monjalon.net" , "john.mcnamara@intel.com" , "drc@linux.vnet.ibm.com" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "konstantin.ananyev@intel.com" , Ola Liljedahl , "bruce.richardson@intel.com" , Ruifeng Wang , nd , Marko Kovacevic , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v7 1/3] doc: add generic atomic deprecation section Thread-Index: AQHWWN4867o6GcNZZEOC4JbGjekIYKkGMURQ Date: Tue, 14 Jul 2020 18:36:27 +0000 Message-ID: References: <1594115449-13750-1-git-send-email-phil.yang@arm.com> <1594621423-14796-1-git-send-email-phil.yang@arm.com> <1594621423-14796-2-git-send-email-phil.yang@arm.com> In-Reply-To: <1594621423-14796-2-git-send-email-phil.yang@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 2b61950a-e772-43d4-9a2a-1eb8bb165717.0 x-checkrecipientchecked: true Authentication-Results-Original: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.112.90.121] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: c8e31895-6444-4a05-ef04-08d82824d812 x-ms-traffictypediagnostic: DB7PR08MB3625:|AM0PR08MB3362: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: n1ocZcnqEfoP3zsWN50WeRg1yKZoCJ/eD/jTNfCCATwysaJFF1DOmXdLGLZmF92l2XdjGuAl2JXskgTXVNrw49WorNdH6oPTQmS9eum4FbJSyDkCdapl9XtlX1R4Pqb5Qi1fXxmO4Fzvwr/b41E3CfUaoM8eW5UfjvfmzwwiO7X6ZcDOm1XkqqBkz08gc2Twz3u4bspwOLIGW9pyBWu8CSJxTOxM9FiUVGN97uMJ/mTAs4nNc20g2qtxz/7rqF2iYq0X/oI4Lrlx5B1h8hyM2APidUsw1RBK12NSnzuCvZTYO7n+nO6ibHHULKHHdsdMAUtatk459u3ikE6LEjA7IIFr82tsBH4pdfMqZ/ytaUC4P6xBR/qRjcNTQzfvxAERVsGhcJy/q1cB2aXVVvNw2Gm3NEHL1rTtglOSmqJAhkGZwm7feL9AtcIn7Z+mWOElt+r/kHuI8TwIdJ1aatNIlg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB6PR0802MB2216.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(346002)(39860400002)(396003)(366004)(376002)(4326008)(83380400001)(26005)(316002)(478600001)(9686003)(8676002)(54906003)(186003)(86362001)(110136005)(55016002)(2906002)(8936002)(7696005)(71200400001)(64756008)(66476007)(66446008)(76116006)(66946007)(52536014)(66556008)(6506007)(33656002)(5660300002)(41533002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 16erk1cste0BOn+qOZdJ4dej+AoLxjpnraDeHl0mkzRVX7SOSiE+5Cal661IYrQR++PZdfiAWBOzN2AuUJE9GnJCufoSsA4XFWC4MJWLTthazcxXZ46UiBmp6G/ntXdpQNIizoBYSNqYcwuYpFdumj61Vy4baMINsqXIc+qjpU4gelBulhxbqkFryZrijHqpIyr8tgkfNxXED9I4tkZnjgC1gbrkhjO9MOl0+6gfGU0FnFAWZjypXmcuoGYUKpCeFPpYexXhovYlGrMKhxrc9CgJ7zatcbUe9BJMUecK8fzEaqB3AXppyS6EDH9v2tx4e1kHaOGyV7grvU0mDxhtJ5DuOBSeQUpONlCmXoBc8qjs8d7z3Brv9eh/mfcc0B5yHCRxFAe4wmZBBm7aUaH6P1nScmKJcgm6K2NfrUb5BcGSKLU07AJClX6t6HdUHQNll6UTIBi91K1ANrGEGdDk+YJx7we3Y6j/Kr9FubUI8LkcGzsTgvEaxxEqYBt8ljl1 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3625 Original-Authentication-Results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(376002)(396003)(136003)(346002)(39860400002)(46966005)(52536014)(82740400003)(110136005)(82310400002)(70206006)(478600001)(70586007)(4326008)(8676002)(55016002)(86362001)(5660300002)(7696005)(316002)(81166007)(54906003)(47076004)(9686003)(8936002)(186003)(2906002)(33656002)(336012)(83380400001)(356005)(6506007)(26005)(41533002); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: e7caa3a7-edad-4c17-6633-08d82824d1fc X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1Xfpnzq3pTsmtxOrl7RyjmjlGD2Qeba3e8Z6ub58NEpgiq3nHgfRHnEBZ5BwZBeBwouIS6lHTULbU9Qbo0xpOolOOgaMUs9Dqm/EprjYplRjiq/6yrIL2j5159vvJBfT/nJmgqIqKi7ZfB3g9624Fnc61JIMP/+QArlTOxSWtRUgDqCt5ZpvfmGwrfho1IfbW60ROjgeyqK1gFJ3dp79m2uHRS3WCWPpYvMHPjMo3570LvAAhO0bwnYNKQO6QyM1L34dDrbovN+9Jvldwj1MJHOyq7ERAcn/9L4pgcsODG0decErNmdyCfFBp9pNkzo1RJRHLEuzGRCiqTxgojjLtDRWKHlSmYFV+4OXhzFnsnZ7RHsVYhKYiylChKyZN4l8vJbdmzvP7ec/TkhzL9gSdEpBxHMizPp3CUyysLYulFxFlTfSSqimO3Kmm+yXa5RQRwq23VAMf66y4gmDTDQ8uerS68j38OdL3DMbuq/m/JTy21/Z6hQJ3dYd0bXLYuij X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2020 18:36:38.1908 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c8e31895-6444-4a05-ef04-08d82824d812 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT036.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3362 Subject: Re: [dpdk-dev] [PATCH v7 1/3] doc: add generic atomic deprecation section X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Phil, Few comments. > Subject: [PATCH v7 1/3] doc: add generic atomic deprecation section I think we should change this as follows: "doc: add optimizations using C11 atomic built-ins" >=20 > Add deprecating the generic rte_atomic_xx APIs to C11 atomic built-ins gu= ide > and examples. I think same here, something like following would help: "Add information about possible optimizations using C11 atomic built-ins" >=20 > Signed-off-by: Phil Yang > Signed-off-by: Honnappa Nagarahalli > --- > doc/guides/prog_guide/writing_efficient_code.rst | 64 > +++++++++++++++++++++++- > 1 file changed, 63 insertions(+), 1 deletion(-) >=20 > diff --git a/doc/guides/prog_guide/writing_efficient_code.rst > b/doc/guides/prog_guide/writing_efficient_code.rst > index 849f63e..16d6188 100644 > --- a/doc/guides/prog_guide/writing_efficient_code.rst > +++ b/doc/guides/prog_guide/writing_efficient_code.rst > @@ -167,7 +167,13 @@ but with the added cost of lower throughput. > Locks and Atomic Operations > --------------------------- >=20 > -Atomic operations imply a lock prefix before the instruction, > +This section describes some key considerations when using locks and > +atomic operations in the DPDK environment. > + > +Locks > +~~~~~ > + > +On x86, atomic operations imply a lock prefix before the instruction, > causing the processor's LOCK# signal to be asserted during execution of = the > following instruction. > This has a big impact on performance in a multicore environment. >=20 > @@ -176,6 +182,62 @@ It can often be replaced by other solutions like per= - > lcore variables. > Also, some locking techniques are more efficient than others. > For instance, the Read-Copy-Update (RCU) algorithm can frequently replac= e > simple rwlocks. >=20 > +Atomic Operations: Use C11 Atomic Built-ins > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +DPDK generic rte_atomic operations are implemented by `__sync built-ins > +`_. We can skip the hyper-link. Google search on "__sync built-in functions" le= ads to this page easily. > +These __sync built-ins result in full barriers on aarch64, which are > +unnecessary in many use cases. They can be replaced by `__atomic > +built-ins > +`_ Same here, we can skip the hyper-link. > +that conform to the C11 memory model and provide finer memory order > control. > + > +So replacing the rte_atomic operations with __atomic built-ins might > +improve performance for aarch64 machines. > + > +Some typical optimization cases are listed below: > + > +Atomicity > +^^^^^^^^^ > + > +Some use cases require atomicity alone, the ordering of the memory > +operations does not matter. For example the packets statistics in > +``virtio_xmit()`` function of ``vhost`` example application. It just > +updates the number of transmitted packets, no subsequent logic depends Instead of referring to the code here, it is better to keep it generic. May= be something like below? "For example, the packet statistics counters need to be incremented atomica= lly but do not need any particular memory ordering. So, RELAXED memory orde= ring is sufficient". > +on these counters. So the RELAXED memory ordering is sufficient. > + > +One-way Barrier > +^^^^^^^^^^^^^^^ > + > +Some use cases allow for memory reordering in one way while requiring > +memory ordering in the other direction. > + Spinlock is a good example, making is little more generic would be helpful. > +For example, the memory operations before the ``rte_spinlock_lock()`` Instead of referring to "rte_spinlock_lock" can you just say spinlock lock > +can move to the critical section, but the memory operations in the ^^^ are allowed to > +critical section cannot move above the lock. In this case, the full ^^^^^^ are not allowed to > +memory barrier in the compare-and-swap operation can be replaced to = ^^ with > +ACQUIRE. On the other hand, the memory operations after the ^^^^^^^^ ACQUIRE memory order. > +``rte_spinlock_unlock()`` can move to the critical section, but the ^^^ are allowed to > +memory operations in the critical section cannot move below the unlock. = ^^^^^^ are not allowed to > +So the full barrier in the STORE operation can be replaced with RELEASE. I think this above statement can be replaced with something like below: "So the full barrier in the store operation can use RELEASE memory order." > + > +Reader-Writer Concurrency > +^^^^^^^^^^^^^^^^^^^^^^^^^ > + > +Lock-free reader-writer concurrency is one of the common use cases in DP= DK. > + > +The payload or the data that the writer wants to communicate to the > +reader, can be written with RELAXED memory order. However, the guard > +variable should be written with RELEASE memory order. This ensures that > +the store to guard variable is observable only after the store to payloa= d is > observable. > +Refer to ``rte_hash_cuckoo_insert_mw()`` for an example. We can remove just this line above. > + > +Correspondingly, on the reader side, the guard variable should be read > +with ACQUIRE memory order. The payload or the data the writer > +communicated, can be read with RELAXED memory order. This ensures that, > +if the store to guard variable is observable, the store to payload is al= so > observable. > +Refer to rte_hash ``search_one_bucket_lf()`` for an example. Same here, suggest removing just the above line. > + > Coding Considerations > --------------------- >=20 > -- > 2.7.4