From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98D44A0583; Fri, 20 Mar 2020 19:32:27 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7AA792BB9; Fri, 20 Mar 2020 19:32:27 +0100 (CET) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80079.outbound.protection.outlook.com [40.107.8.79]) by dpdk.org (Postfix) with ESMTP id 39BF4F94 for ; Fri, 20 Mar 2020 19:32:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BHw8O26RZMf4e5ki/JxNxLZgwekbmv6+HQ+AfaRdU/A=; b=eP+e7p6wf2F6vZdRP7lp/MctC69eh/WEZbh4Moy/9ndI2qM7gO0dsK2btRm5a7C+f+iXSs3mQWX+UqNDUIx7db3LzXgwgRTiKXERh6terNM4vifXEyIJzzfsxItMQ6J7kl0mkrtGYHUV8yH194rTqsoHiLny9vcJxJGDrV6rUZw= Received: from AM5PR0602CA0010.eurprd06.prod.outlook.com (2603:10a6:203:a3::20) by VE1PR08MB5231.eurprd08.prod.outlook.com (2603:10a6:802:a1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18; Fri, 20 Mar 2020 18:32:23 +0000 Received: from VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:a3:cafe::46) by AM5PR0602CA0010.outlook.office365.com (2603:10a6:203:a3::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18 via Frontend Transport; Fri, 20 Mar 2020 18:32:23 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT036.mail.protection.outlook.com (10.152.19.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.13 via Frontend Transport; Fri, 20 Mar 2020 18:32:23 +0000 Received: ("Tessian outbound f88013e987c7:v48"); Fri, 20 Mar 2020 18:32:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from e53aafeb5274.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 4C553443-8B42-421F-B7CD-12E79C20F7D7.1; Fri, 20 Mar 2020 18:32:16 +0000 Received: from EUR01-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e53aafeb5274.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 20 Mar 2020 18:32:16 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PJ0EHY6wA7yOYe8Z2AtFMzGhiKmKpQFQxbz8hhfRBxPMoHqbBkUhsYWH59GNITXGPg052MBYum7XTGuOJ4hJJggFJak4ORj4TVOxPO6FZblk2RIzpARY3KSJE8RGvfLg7sLW3NCpqzsXCwAYjIoZ81SQZpdTEXh7WwoY0nyirwlyPkxjeQ5yO7fRStu7kKV/sP23ScrXTDQBEyYS9hfHbX7/B19XlSpv0HvQhrUTvhxGuiwu6GIQvyHEjjSbN1o/GHTcSZ4crPjc2Iji6CbvPaGOaEUn4FSM1DmQian8V7bbEjrleVehx40nVv4qJZ1d1fc0neie5bwLVm1hpcnCaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BHw8O26RZMf4e5ki/JxNxLZgwekbmv6+HQ+AfaRdU/A=; b=gqFO8RxZ6FX3Y2onwsUHlsZFjJ96FrdIHp2twUmw/jRiaKhniC5tRE48EiOcJKUuXwIfkCAE/ap21+qfkdqV99IpKz7WhoDaUaAzPRi3ili1bZVdL2JDM4mMFZZE3VFVbrBrUVOgzTGB1eX4rANj6e0s/3w1Qk3kj0POky84sIcM+y8iJbYd5C4KTvkrxJd/Yi60bUIbjbe6bDl9YtG4j10vJOPoIG2yonkwGSXYRaJKz+fAsiW6kzvfEOPz37BNEZJ11uEEu5gk97ZP3VuD1mxq794kzjXwf6g7YC11oxfYB6Idu8fE1tdAPW6TcfI55sfz7zWzBnPVIg7L+DJ8zg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BHw8O26RZMf4e5ki/JxNxLZgwekbmv6+HQ+AfaRdU/A=; b=eP+e7p6wf2F6vZdRP7lp/MctC69eh/WEZbh4Moy/9ndI2qM7gO0dsK2btRm5a7C+f+iXSs3mQWX+UqNDUIx7db3LzXgwgRTiKXERh6terNM4vifXEyIJzzfsxItMQ6J7kl0mkrtGYHUV8yH194rTqsoHiLny9vcJxJGDrV6rUZw= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB5120.eurprd08.prod.outlook.com (20.179.30.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18; Fri, 20 Mar 2020 18:32:13 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd%6]) with mapi id 15.20.2835.017; Fri, 20 Mar 2020 18:32:13 +0000 From: Honnappa Nagarahalli To: "Van Haaren, Harry" , Phil Yang , "thomas@monjalon.net" , "Ananyev, Konstantin" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , "Carrillo, Erik G" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal Thread-Index: AQHV+/n7P6oepoZnDEWF5L0GAIIkk6hOY32AgAJ57aCAAPI5MA== Date: Fri, 20 Mar 2020 18:32:13 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: d86cf31c-0d54-4286-9597-7d255016a97f.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: fdd3e4aa-f6b2-49f7-23c1-08d7ccfd0838 x-ms-traffictypediagnostic: VE1PR08MB5120:|VE1PR08MB5120:|VE1PR08MB5231: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 03484C0ABF X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(366004)(376002)(346002)(39860400002)(136003)(396003)(199004)(52536014)(66946007)(55016002)(4326008)(2940100002)(33656002)(5660300002)(186003)(26005)(71200400001)(966005)(478600001)(110136005)(54906003)(76116006)(64756008)(7696005)(66476007)(9686003)(66446008)(86362001)(316002)(6506007)(7416002)(81156014)(66556008)(81166006)(8676002)(8936002)(2906002)(41533002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5120; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: wkoRjk3MRxbSTGkacsnpOLDG3j4t4FmenhDknQZBOSxgMtLvMAU7rnYLS9hLiFMfMgTpBRoGW0i0sIo2L8yzdEVK1sYk6Ckdwz7m/Pa11E4BfGJQ68R1lFGIhY+Uz899TQsYYwVQWfRyUMe7YxBfqn7cs9TeXVEuL1gGrNc4teiWe4QQreiqGNi+J3iAXeSN9ynvg4JA/D0kwSzX3rpRrp+2bQiKnJSFA4VSFHvolMJIBJjZecgSho7RWE93yZ76JQ5uxZjfIZM4hs6+hFZA32NMaVinLk+jmz38PUQspv8z3Zz6kO4lzOINs+Pvh/5saJd1b9WiK9+15lhJUYpQYq+iF6qni3UYNEpblZ8VnXSl/dmNWXVDLrpw/Zrz7u6koBP9XjmOx0GUuJFTy2E9KcPasDBqxNiEz+f+/FXLyB2ADSglDPoVhSmc266tlNp2+uKLfLoP3odwRu0h9OFNI6FzN7iHxNSCbMqcMxXnWq+rBn2KPkhwNvzr7NPFGExW0u4EiX4L4n741U1FSvkHASZt5NqLSltyYnt/Y7KS4XJ+LGT4lZRe1CGQ6Hfvhlst x-ms-exchange-antispam-messagedata: yOA0sU6ocq/tNnuWpX7hmbq2wMHrYy6Diha8qXTG5zTwyuNp4kzbExa2N5koJFP1I3iKPpk8PKKYUtUR4uHefE/L5rLrbE1q0VB8Uw66Z9J0WhCk0J7BZfk+VxIZzuvrflPPbuS2vKWlCY5O0AV/mA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5120 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT036.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(346002)(39860400002)(376002)(396003)(199004)(46966005)(6506007)(2906002)(8676002)(356004)(52536014)(316002)(54906003)(36906005)(8936002)(4326008)(47076004)(86362001)(26005)(336012)(186003)(966005)(55016002)(81166006)(7696005)(478600001)(70206006)(70586007)(110136005)(33656002)(2940100002)(26826003)(5660300002)(81156014)(9686003)(41533002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5231; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 7cdbad60-41a4-4f2a-d4f7-08d7ccfd022d X-Forefront-PRVS: 03484C0ABF X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KjDJF0usqnd552dN0LG7AYzftmdxlJgOGAZEN/OTqB6ujRdrLADH6iAGOfo9W/Ry6ea54xbnPcEaRJZDkLgp6bn1bYb1qnSsUQ7MhdVwe7U4GAzdtBvTLttZbe1/hhtbmOo01BfjCTboVZ+2UE45erGaJWQhI36XkbXfuw4U4RE+DCycp9qAce3rPcnYVLM0QCsFSmtdw5EesmeEWHYmw24TT1Jq8k8aEDQyDUHEcLLlHxPoQOE9qhTafsSGfJKMSjNRgxIuojb+ipKeZvff2VJcUnhIuli/poIoGeSjG9UB61CFbS17XBwaq3ldYNyTl752eRUASSQ4r0jhrtgU8bYtQMH7x0eezAWywjV9LuB+Fj4rvVXc7NgyaWH78P8pL0NyT2+pz9FOhIxdN4B1jM0LPVABLbAV/sMLNHd/ctee6vJphyi9KyPLhk1YpzxPYl0yWBIjODZ9tjhVPLEaKHYjQX5SPiIycpuRBPJSJTgEC0K19Y2/xTNRoyUMdpaVnEm9b9VSmzT+4QLxs6i0B4KdWXUcJPZ42sgHExAmqZOjk7224Vqv7ybgPbfLQCHZPT6kDeo7D7rapQA9OILdA8+KwFzHXxm27L/c5TFoOffzfGlWcv0Y2c+A5P+jtF6S X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2020 18:32:23.1232 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fdd3e4aa-f6b2-49f7-23c1-08d7ccfd0838 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5231 Subject: Re: [dpdk-dev] [PATCH v3 00/12] generic rte atomic APIs deprecate proposal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" + Erik as there are similar changes to timer library >=20 > > > Subject: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal > > > > > > DPDK provides generic rte_atomic APIs to do several atomic operations= . > > > These APIs are using the deprecated __sync built-ins and enforce > > > full memory barriers on aarch64. However, full barriers are not > > > necessary in many use cases. In order to address such use cases, C > > > language offers > > > C11 atomic APIs. The C11 atomic APIs provide finer memory barrier > > > control by making use of the memory ordering parameter provided by > > > the > > user. > > > Various patches submitted in the past [2] and the patches in this > > > series indicate significant performance gains on multiple aarch64 > > > CPUs and no performance loss on x86. > > > > > > But the existing rte_atomic API implementations cannot be changed as > > > the APIs do not take the memory ordering parameter. The only choice > > > available is replacing the usage of the rte_atomic APIs with C11 > > > atomic APIs. In order to make this change, the following steps are > proposed: > > > > > > [1] deprecate rte_atomic APIs so that future patches do not use > > > rte_atomic APIs (a script is added to flag the usages). > > > [2] refactor the code that uses rte_atomic APIs to use c11 atomic API= s. > > > > On [1] above, I feel deprecating DPDKs atomic functions and failing > > checkpatch is a bit sudden. Perhaps noting that in a future release > > (20.11?) DPDK will move to a > > C11 based atomics model is a more gradual step to achieving the goal, > > and at that point add a checkpatch warning for additions of rte_atomic*= ? > We have been working on changing existing usages of rte_atomic APIs in DP= DK > to use C11 atomics. Usually, the x.11 releases have significant amount of > changes (not sure how many would use rte_atomic APIs). I would prefer tha= t > in 20.11 no additional code is added using rte_atomics APIs. However, I a= m > open to suggestions on the exact time frame. > Once we decide on the release, I think it makes sense to add a 'warning' = in the > checkpatch to indicate the deprecation timeline and add an 'error' after = the > release. >=20 > > > > More on [2] in context below. > > > > The above is my point-of-view, of course I'd like more people from the > > DPDK community to provide their input too. > > > > > > > This patchset contains: > > > 1) the checkpatch script changes to flag rte_atomic API usage in patc= hes. > > > 2) changes to programmer guide describing writing efficient code for > > aarch64. > > > 3) changes to various libraries to make use of c11 atomic APIs. > > > > > > We are planning to replicate this idea across all the other > > > libraries, drivers, examples, test applications. In the next phase, > > > we will add changes to the mbuf, the EAL interrupts and the event > > > timer adapter > > libraries. > > > > About ~6/12 patches of this C11 set are targeting the Service Cores > > area of DPDK. I have some concerns over increased complexity of C11 > > implementation vs the (already complex) rte_atomic implementation today= . > I agree that it C11 changes are complex, especially if one is starting ou= t to > understand what these APIs provide. From my experience, once few > underlying concepts are understood, reviewing or making changes do not ta= ke > too much time. >=20 > > I see other patchsets enabling C11 across other DPDK components, so > > maybe we should also discuss C11 enabling in a wider context that just > service cores? > Yes, agree. We are in the process of making changes to other areas as wel= l. >=20 > > > > I don't think it fair to expect all developers to be well versed in > > C11 atomic semantics like understanding the complex interactions > > between the various > > C11 RELEASE, AQUIRE barriers requires. > C11 has been around from sometime now. To my surprise, OVS already uses > C11 APIs extensively. VPP has been accepting C11 related changes from pas= t > couple of years. Having said that, I agree in general that not everyone i= s well > versed. >=20 > > > > As maintainer of Service Cores I'm hesitant to accept the large-scale > > refactor > Right now, the patches are split into multiple commits. If required I can= host a > call to go over simple C11 API usages (sufficient to cover the usage in s= ervice > core) and the changes in this patch. If you find that particular areas ne= ed more > understanding I can work on providing additional information such as memo= ry > order ladder diagrams. Please let me know what you think. When I started working with C11 APIs, I had referred to the following blogs= . https://preshing.com/20120913/acquire-and-release-semantics/ https://preshing.com/20130702/the-happens-before-relation/ https://preshing.com/20130823/the-synchronizes-with-relation/ These will be helpful to understand the changes. >=20 > > of atomic-implementation, as it could lead to racey bugs that are > > likely extremely difficult to track down. (The recent race-on-exit has > > proven the difficulty in reproducing, and that's with an atomics model > > I'm quite familiar with). > > > > Let me be very clear: I don't wish to block a C11 atomic > > implementation, but I'd like to discuss how we (DPDK community) can > > best port-to and maintain a complex multi-threaded service library > > with best-in-class performance for the workload. > > > > To put some discussions/solutions on the table: > > - Shared Maintainership of a component? > > Split in functionality and C11 atomics implementation > > Obviously there would be collaboration required in such a case. > > - Maybe shared maintainership is too much? > > A gentlemans/womans agreement of "helping out" with C11 atomics > > debug is enough? > I think shared maintainer ship could be too much as there are many change= s. > But, I and other engineers from Arm (I would include Arm ecosystem as wel= l) > can definitely help out on debug and reviews involving C11 APIs. (I see > Thomas's reply on this topic). >=20 > > > > > > Hope my concerns are understandable, and of course input/feedback > > welcomed! -Harry > No worries at all. We are here to help and make this as easy as possible. >=20 > > > > > > PS: Apologies for the delay in reply - was OOO on Irish national holida= y. > Same here, was on vacation for 3 days. >=20 > > > > > > > v3: > > > add libatomic dependency for 32-bit clang > > > > > > v2: > > > 1. fix Clang '-Wincompatible-pointer-types' WARNING. > > > 2. fix typos. > > > > > > Honnappa Nagarahalli (2): > > > service: avoid race condition for MT unsafe service > > > service: identify service running on another core correctly > > > > > > Phil Yang (10): > > > doc: add generic atomic deprecation section > > > devtools: prevent use of rte atomic APIs in future patches > > > eal/build: add libatomic dependency for 32-bit clang > > > build: remove redundant code > > > vhost: optimize broadcast rarp sync with c11 atomic > > > ipsec: optimize with c11 atomic for sa outbound sqn update > > > service: remove rte prefix from static functions > > > service: remove redundant code > > > service: optimize with c11 one-way barrier > > > service: relax barriers with C11 atomic operations > > > > > > devtools/checkpatches.sh | 9 ++ > > > doc/guides/prog_guide/writing_efficient_code.rst | 60 +++++++- > > > drivers/event/octeontx/meson.build | 5 - > > > drivers/event/octeontx2/meson.build | 5 - > > > drivers/event/opdl/meson.build | 5 - > > > lib/librte_eal/common/rte_service.c | 175 ++++++++++++-= --------- > > > - > > > lib/librte_eal/meson.build | 6 + > > > lib/librte_ipsec/ipsec_sqn.h | 3 +- > > > lib/librte_ipsec/sa.h | 2 +- > > > lib/librte_rcu/meson.build | 5 - > > > lib/librte_vhost/vhost.h | 2 +- > > > lib/librte_vhost/vhost_user.c | 7 +- > > > lib/librte_vhost/virtio_net.c | 16 ++- > > > 13 files changed, 181 insertions(+), 119 deletions(-) > > > > > > -- > > > 2.7.4 >=20