From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2C30A0540; Wed, 15 Jul 2020 04:59:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 537281C1A8; Wed, 15 Jul 2020 04:59:04 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2045.outbound.protection.outlook.com [40.107.21.45]) by dpdk.org (Postfix) with ESMTP id 8FE0E1C1A6 for ; Wed, 15 Jul 2020 04:59:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b/cX7EtbXYujA25StgtUS9y5M4Kq5py9uwQe+H9h2Hk=; b=mNctloNUt5M+HYW4mWWn4Kjt4z6/KrRO65iVyVlFnyBjAfnlOiwcTGxSG21R7qmB1rpol6ZEEqrPErmpFoJ9Mqzyqkqq9H0EFiEHLfqbQz/UoQeYyst2LedfKuslZTLKz3Yun83Ao305Jqo+OseR9M5kij1dw3wUq71T+lD/GKI= Received: from AM6P194CA0049.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::26) by AM0PR08MB4257.eurprd08.prod.outlook.com (2603:10a6:208:13c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.23; Wed, 15 Jul 2020 02:59:02 +0000 Received: from AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:84:cafe::d9) by AM6P194CA0049.outlook.office365.com (2603:10a6:209:84::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18 via Frontend Transport; Wed, 15 Jul 2020 02:59:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT018.mail.protection.outlook.com (10.152.16.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21 via Frontend Transport; Wed, 15 Jul 2020 02:59:01 +0000 Received: ("Tessian outbound 7de93d801f24:v62"); Wed, 15 Jul 2020 02:59:01 +0000 X-CR-MTA-TID: 64aa7808 Received: from b190f282d2fd.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id F2C1CE94-091D-47CA-A67D-C347EBE931EE.1; Wed, 15 Jul 2020 02:58:56 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b190f282d2fd.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 15 Jul 2020 02:58:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PLvPjZGpBMouatWeHc5Bv5w/EslNqOcPZH74vwoD5PH5+5lTa5Al5KKVd5UMpbNBnZZPUZW66dTBNQjT81danfGSSlf6tRGwL4WGr4FM9ngEWdV7QMzlaodhTc8Cywsh+KHKJ43ncEWsk6fFq5JiXBqtF0FWIwW5fPTssbdpiXbY4bkny5hMDQICTI9tNrJmlP3Gt1CaI5Fhq6lthZI1uOA94115zVk3n8ONCK4smVtm8O3Eq8SXPPqD+llo8kscIOAGlIOKWIcrZhXBndift0SPHgLYxE+i8SSCijRg59coTWl/9AagpRwkxgJ7lHSX+y7n6cICdKzoIlPpuprA0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b/cX7EtbXYujA25StgtUS9y5M4Kq5py9uwQe+H9h2Hk=; b=am4p/lgVcC7BF1HHsxPvNVRMHPJbKiJxLPvE/Zp0di+u4ulaArIH1PTvQK5zIRKUOUEqwBlxkdtPoPwb5shasOjtMnKiiYMXS03UdUzcgGYTQAJc+rXXda2S8/+RETHBeb7FS69RD4VOrHAol4PSk7gTzOw+D6Og6fF47HrB4HDGc41Aga7hZR0QMjhYztM5T57+coE4S7vxmJZ50P9TLHOOk8pK/qutmu53DWuo+0IcTNsFQKHL1zfGAjOCoSfJ2yHPXi7Aqsciq1c0+TjxKiHgW+VbOmoTXWkuE5wwli5nfV3aspUGc3GT4mH/F+uL7HA2vkqQARbTSNxC5zrZWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b/cX7EtbXYujA25StgtUS9y5M4Kq5py9uwQe+H9h2Hk=; b=mNctloNUt5M+HYW4mWWn4Kjt4z6/KrRO65iVyVlFnyBjAfnlOiwcTGxSG21R7qmB1rpol6ZEEqrPErmpFoJ9Mqzyqkqq9H0EFiEHLfqbQz/UoQeYyst2LedfKuslZTLKz3Yun83Ao305Jqo+OseR9M5kij1dw3wUq71T+lD/GKI= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (2603:10a6:802:b2::11) by VI1PR08MB3229.eurprd08.prod.outlook.com (2603:10a6:803:47::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.20; Wed, 15 Jul 2020 02:58:54 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::c2e:9ccb:a690:6863]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::c2e:9ccb:a690:6863%6]) with mapi id 15.20.3174.025; Wed, 15 Jul 2020 02:58:54 +0000 From: Phil Yang To: "john.mcnamara@intel.com" , Honnappa Nagarahalli CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "konstantin.ananyev@intel.com" , Ola Liljedahl , "thomas@monjalon.net" , "bruce.richardson@intel.com" , Ruifeng Wang , nd , Marko Kovacevic , nd , "drc@linux.vnet.ibm.com" , "dev@dpdk.org" , nd Thread-Topic: [PATCH v7 1/3] doc: add generic atomic deprecation section Thread-Index: AQHWWg226mYwQvKmk0utlsCPs/tW9akH8I9g Date: Wed, 15 Jul 2020 02:58:54 +0000 Message-ID: References: <1594115449-13750-1-git-send-email-phil.yang@arm.com> <1594621423-14796-1-git-send-email-phil.yang@arm.com> <1594621423-14796-2-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 8548cf3a-a738-426a-b94d-0c67b329de04.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [203.126.0.112] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 55fa4849-3a56-4acb-fc38-08d8286b0727 x-ms-traffictypediagnostic: VI1PR08MB3229:|AM0PR08MB4257: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: lBvtoznDC4KPz2tgpXNXAtRsPZXFIPioICVZoTM4NyDmRPjUK46tm9m4wMAS+47MwBqsvC2ie20d97o+ObLMWgRk36RH72FzMODmBTYvCRVRQdP4l8DKZbGg+x+7ugMvbWGkT8LZomI9ozawyikav0cTHdpiOAFvpb/93LSf58Ntb7PI27GEQfn8dBu7zWoTGowDPNc/aq5HVt/NCjZil4NMGGMwRdVrtzDYGkoTCk1d4s8OhGxJ9ZlUbGCZgFHQzmvGnF96YBSK7u6obAz5EHz+zum43S3a5BQXIj9D6etqD7roRQiSA65fkP8NxT0xxPEabUWxfSkqUEo8cado1YhM5y2r5PUnvG9WcYDvqGNj+UAeL4vTa2s8jpn316oMSLn8XUbHk7tS08rcr29I4umYvJ1E1JKBCFCiQOBTDNLmez4ZY2TIBqX5n8XwDLKIrGWj0nrK9FWpY6lFO9YNJQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VE1PR08MB4640.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39850400004)(396003)(376002)(346002)(366004)(136003)(8936002)(186003)(83380400001)(26005)(8676002)(4326008)(55016002)(6506007)(9686003)(110136005)(54906003)(76116006)(86362001)(7696005)(316002)(478600001)(66946007)(66556008)(52536014)(6636002)(5660300002)(64756008)(71200400001)(33656002)(66446008)(66476007)(2906002)(41533002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: op3zKbqaPjG83OzxmzfBokx6uOb2in4xmWYaNyDvqzhjB8BqHibo19NRRBVBsCnCDRlsDj1hk/HQjd/3Q9okTt56as9lyD7/k2lJORH60XHcUTt0UtJhr+MVHd6HQHrDDQEFcoP+n6nao3JLIGjI2W1uF2QGgwfO9V++xlutPh/CLg8w61m986DVkaQKJ7T60fT/AuGu4Lehpe8gAroJtAxIK2dw94XKAtTxDyGZTy+b7xhASZFFG8dZc3T2Xcwe5751+lbjrR5s3V1Cak0vUbszTW3qvzvO4mEJe7llJ4aGuT+rt+rfze5dYTP/3py2iFNcNMdueamw52t3rzTtyAA/AaD7pdbnX5+QPCk5Ym7cmG7r0gpGOUhwvZMjtdCZ2o0sHVIoJ9oE23P/ObjUUJRCcxD0g2vBfkWX9vYSiNtoTeAcs6sd89nGDrQ8wrkkowuD0adY9B2TBwOLNP6vfctxFooTKx3Fm1Nef5Noh+s= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3229 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39850400004)(376002)(136003)(396003)(46966005)(316002)(8676002)(47076004)(82740400003)(54906003)(7696005)(4326008)(6506007)(8936002)(110136005)(33656002)(186003)(83380400001)(336012)(6636002)(26005)(52536014)(55016002)(82310400002)(86362001)(356005)(478600001)(5660300002)(9686003)(70586007)(36906005)(2906002)(81166007)(70206006)(41533002); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: e858a003-87bd-4f0e-cf5c-08d8286b02c6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TLB+iCm0ZGNya1Ty5mhaXpduPtfITDClkbqKOibDyRC35/HznuTKzgJfVo0dr8/p5NKUhVIBiEYc6YcMHEGL/cZKgtLb7dt/sCUBO9hXKU0XtKP9UP7uLeBdqY+hbS50Uqer5c3kUjotDB27D/BEdW10K+2DK3PEA4OQmwwci1W/56jcC0frmrafNQg/q2kBaNFRGKpYIkvYEEihd8mSlb2m8OLZbsYOecUG4dJjqxWqivCPGhzefEyhSrNm/CR3cD4xcVQSOwNcbZPiqoCAPfePvgTc1gvfWsn2TAvf9/Zmqp66G95PET/0L1FYguYrq9GJocexe8fdE9fcG0eXVVRxCIA06YobvCF/yd5aNDAM1mYKywzfcsT4OKRa1smQ6i3btt8nz+PMeV00gNCuFSm5G9G6SFNLmeiSyyjVuslSpkgaNxURG17inFs51KKSTTt8ghDfwoJheRWBPCwxs/QqxXeZzEPBca2kO5qv8GXviFIo5nP1YU/FftBoWPh6 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2020 02:59:01.8908 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 55fa4849-3a56-4acb-fc38-08d8286b0727 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4257 Subject: Re: [dpdk-dev] [PATCH v7 1/3] doc: add generic atomic deprecation section X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Honnappa, Thanks for your comments and suggestions. I Will update it in the next version. Hi John, I would greatly appreciate it if you kindly give me some feedback on this p= roposal. Thanks, Phil > Hi Phil, > Few comments. >=20 > >=20 > > Subject: [PATCH v7 1/3] doc: add generic atomic deprecation section > I think we should change this as follows: > "doc: add optimizations using C11 atomic built-ins" >=20 > > > > Add deprecating the generic rte_atomic_xx APIs to C11 atomic built-ins > guide > > and examples. > I think same here, something like following would help: > "Add information about possible optimizations using C11 atomic built-ins" >=20 > > > > Signed-off-by: Phil Yang > > Signed-off-by: Honnappa Nagarahalli > > --- > > doc/guides/prog_guide/writing_efficient_code.rst | 64 > > +++++++++++++++++++++++- > > 1 file changed, 63 insertions(+), 1 deletion(-) > > > > diff --git a/doc/guides/prog_guide/writing_efficient_code.rst > > b/doc/guides/prog_guide/writing_efficient_code.rst > > index 849f63e..16d6188 100644 > > --- a/doc/guides/prog_guide/writing_efficient_code.rst > > +++ b/doc/guides/prog_guide/writing_efficient_code.rst > > @@ -167,7 +167,13 @@ but with the added cost of lower throughput. > > Locks and Atomic Operations > > --------------------------- > > > > -Atomic operations imply a lock prefix before the instruction, > > +This section describes some key considerations when using locks and > > +atomic operations in the DPDK environment. > > + > > +Locks > > +~~~~~ > > + > > +On x86, atomic operations imply a lock prefix before the instruction, > > causing the processor's LOCK# signal to be asserted during execution o= f > the > > following instruction. > > This has a big impact on performance in a multicore environment. > > > > @@ -176,6 +182,62 @@ It can often be replaced by other solutions like p= er- > > lcore variables. > > Also, some locking techniques are more efficient than others. > > For instance, the Read-Copy-Update (RCU) algorithm can frequently > replace > > simple rwlocks. > > > > +Atomic Operations: Use C11 Atomic Built-ins > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +DPDK generic rte_atomic operations are implemented by `__sync built-in= s > > +`_. > We can skip the hyper-link. Google search on "__sync built-in functions" > leads to this page easily. >=20 > > +These __sync built-ins result in full barriers on aarch64, which are > > +unnecessary in many use cases. They can be replaced by `__atomic > > +built-ins >=20 > > +`_ > Same here, we can skip the hyper-link. >=20 > > +that conform to the C11 memory model and provide finer memory order > > control. > > + > > +So replacing the rte_atomic operations with __atomic built-ins might > > +improve performance for aarch64 machines. > > + > > +Some typical optimization cases are listed below: > > + > > +Atomicity > > +^^^^^^^^^ > > + > > +Some use cases require atomicity alone, the ordering of the memory > > +operations does not matter. For example the packets statistics in > > +``virtio_xmit()`` function of ``vhost`` example application. It just > > +updates the number of transmitted packets, no subsequent logic > depends > Instead of referring to the code here, it is better to keep it generic. M= ay be > something like below? > "For example, the packet statistics counters need to be incremented > atomically but do not need any particular memory ordering. So, RELAXED > memory ordering is sufficient". >=20 > > +on these counters. So the RELAXED memory ordering is sufficient. > > + > > +One-way Barrier > > +^^^^^^^^^^^^^^^ > > + > > +Some use cases allow for memory reordering in one way while requiring > > +memory ordering in the other direction. > > + > Spinlock is a good example, making is little more generic would be helpfu= l. >=20 > > +For example, the memory operations before the ``rte_spinlock_lock()`` > Instead of referring to "rte_spinlock_lock" can you just say spinlock loc= k > > +can move to the critical section, but the memory operations in the > ^^^ are allowed to > > +critical section cannot move above the lock. In this case, the full > ^^^^^^ are not allowed to > > +memory barrier in the compare-and-swap operation can be replaced to > = ^^ with > > +ACQUIRE. On the other hand, the memory operations after the > ^^^^^^^^ ACQUIRE memory order. > > +``rte_spinlock_unlock()`` can move to the critical section, but the > ^^^ are allowed to > > +memory operations in the critical section cannot move below the unlock= . > = ^^^^^^ are not allowed to > > +So the full barrier in the STORE operation can be replaced with RELEAS= E. > I think this above statement can be replaced with something like below: >=20 > "So the full barrier in the store operation can use RELEASE memory order.= " >=20 > > + > > +Reader-Writer Concurrency > > +^^^^^^^^^^^^^^^^^^^^^^^^^ > > + > > +Lock-free reader-writer concurrency is one of the common use cases in > DPDK. > > + > > +The payload or the data that the writer wants to communicate to the > > +reader, can be written with RELAXED memory order. However, the guard > > +variable should be written with RELEASE memory order. This ensures tha= t > > +the store to guard variable is observable only after the store to payl= oad is > > observable. > > +Refer to ``rte_hash_cuckoo_insert_mw()`` for an example. > We can remove just this line above. >=20 > > + > > +Correspondingly, on the reader side, the guard variable should be read > > +with ACQUIRE memory order. The payload or the data the writer > > +communicated, can be read with RELAXED memory order. This ensures > that, > > +if the store to guard variable is observable, the store to payload is = also > > observable. > > +Refer to rte_hash ``search_one_bucket_lf()`` for an example. > Same here, suggest removing just the above line. >=20 > > + > > Coding Considerations > > --------------------- > > > > -- > > 2.7.4