From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4B4D6A0583; Fri, 20 Mar 2020 05:51:36 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A41EA2BB9; Fri, 20 Mar 2020 05:51:35 +0100 (CET) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2047.outbound.protection.outlook.com [40.107.20.47]) by dpdk.org (Postfix) with ESMTP id 412033B5 for ; Fri, 20 Mar 2020 05:51:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mU8XPjV3jxIxUYxYj0ifl6/YvO5Qq3LMOTrHIw8j2Aw=; b=kVBOg6sn7Z8xhPYyypyYObPPjg9pzt8lXu1CSbpmEKMRFu32jBPCmxpI6SHxwN5uqkxCkDCeKpZ7VhXONG+abDAhu1Kd+oLX8gAvWzTxSSwYnHXPmQzKQdm4jru7BGSSFM3Kn/7BPjgMEU9x+IEBU8nqQ0Mhsc/kiKc9NSsSRoc= Received: from AM4PR0902CA0019.eurprd09.prod.outlook.com (2603:10a6:200:9b::29) by VI1PR0802MB2222.eurprd08.prod.outlook.com (2603:10a6:800:9b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.18; Fri, 20 Mar 2020 04:51:31 +0000 Received: from VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com (2603:10a6:200:9b:cafe::45) by AM4PR0902CA0019.outlook.office365.com (2603:10a6:200:9b::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18 via Frontend Transport; Fri, 20 Mar 2020 04:51:31 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT044.mail.protection.outlook.com (10.152.19.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.13 via Frontend Transport; Fri, 20 Mar 2020 04:51:30 +0000 Received: ("Tessian outbound aed43bac6b97:v48"); Fri, 20 Mar 2020 04:51:30 +0000 X-CR-MTA-TID: 64aa7808 Received: from 6166ba6b7b5f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 376DCF88-D934-4940-B8F1-48074C79CE29.1; Fri, 20 Mar 2020 04:51:30 +0000 Received: from EUR03-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6166ba6b7b5f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 20 Mar 2020 04:51:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m63LrZN3uytLaKklAxWs+xH9TssheKlDMtT33dU08ujtYngovfyKkq/gY9jBsxyhx5l/wuBd15q3iv7oMSDEXI57bVPRJX4y7kGgmAa1/9EwVF2HuHdr0+a1gb9lip9b/SIStFH7+oPUpdlG/qFGZ1yaHNVQOz5eNzXaOBcFNrbhA825bWSm6Rud1DCHwFgJoJVay/TMPYU1QoIA4H6o1E+C3k4316oXKsvy4lY7WU97ooDDM7bTIjoto/9b+BX8ieB0+usPwMSRuV2hwo95sWuijg+PSmuz4JohKAmeOsw6IHPANzFx9YxtfIUn7nUsi6MrtEECbgeJ8E7xWkXBJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mU8XPjV3jxIxUYxYj0ifl6/YvO5Qq3LMOTrHIw8j2Aw=; b=PacNwBg5oRu4DVYnX3IXsN/m5EjJzb4g7Xrx6z8ibGCuzblU68Awl5BJIrCXFTmGVjRkuAidlJ51XgBxWdrLyiDzKAxQe1QKL692sbgztEkTABKciEzTnJ1T68ig+RIdcszQL5Ds1TtacsEpeLgbNtle5zbBDZ514c3CRIvTOfdyMfS2dkZMBpcbpBAYDcba1ucq2f6f/Nn3UvYSbE7P9SdL3MYUS7JCNURlh4QZaUSEfWUpE7POe28rzEhX23Hh9WLpr78EEMghCz03JXlOOfMTJ9cncGN0+hTa7qcXganFEReGwMjufCAs3TbjjLhc7CtHUllgQ60hfx/y4XMppQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mU8XPjV3jxIxUYxYj0ifl6/YvO5Qq3LMOTrHIw8j2Aw=; b=kVBOg6sn7Z8xhPYyypyYObPPjg9pzt8lXu1CSbpmEKMRFu32jBPCmxpI6SHxwN5uqkxCkDCeKpZ7VhXONG+abDAhu1Kd+oLX8gAvWzTxSSwYnHXPmQzKQdm4jru7BGSSFM3Kn/7BPjgMEU9x+IEBU8nqQ0Mhsc/kiKc9NSsSRoc= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB5181.eurprd08.prod.outlook.com (20.179.31.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.20; Fri, 20 Mar 2020 04:51:22 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd%6]) with mapi id 15.20.2835.017; Fri, 20 Mar 2020 04:51:22 +0000 From: Honnappa Nagarahalli To: "Van Haaren, Harry" , Phil Yang , "thomas@monjalon.net" , "Ananyev, Konstantin" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal Thread-Index: AQHV+/n7P6oepoZnDEWF5L0GAIIkk6hOY32AgAJ57aA= Date: Fri, 20 Mar 2020 04:51:21 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: ce318472-d5bb-49b4-9f1d-637975e31eaa.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b9cb16a3-b07d-4ae2-637b-08d7cc8a5ba1 x-ms-traffictypediagnostic: VE1PR08MB5181:|VE1PR08MB5181:|VI1PR0802MB2222: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 03484C0ABF X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(366004)(346002)(376002)(39860400002)(136003)(199004)(478600001)(186003)(316002)(86362001)(2906002)(33656002)(54906003)(110136005)(8676002)(76116006)(8936002)(64756008)(66556008)(66946007)(66476007)(6506007)(4326008)(71200400001)(66446008)(52536014)(26005)(5660300002)(55016002)(81166006)(9686003)(7696005)(81156014)(41533002); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5181; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 6HP82nBRwrQybwACt8XBD4TXvR6iGky5uDKV55XGcnSBa8MhqxyLlPcQBUq7UvoPxACqcQfZPujBQSqFTSxdMD3KZWCyE9tFY8fVryPqZFszNQFk6A5BJJRi60IQkEDXOpmIvec+A9j0/0Rp+LrnVHnCo2lLajPwywaI/m+/7vdwnS1kcBCUgRkzm5wTn5GURFJXCS6RP+cZr2NLXSf9GTMwQPzw8fd/Qf8jINRb2meaebr8KgE9XIZFlTsF9AlRl4tEYsHB1s1TbIkIh7jl7ukyrC7PXdxY9Xogt9f6PZxBBexInZaEoeC0FPONIoiFCaLvFpA7dxCpb3qCnIllxfsArn/lr6xiXuwGR9OE7UbtA8wAmyjVbYkjOJMW9FOeq9nSKTiwWsYfkOGA9Qe9DQzLjbZuSLmDhFdSamZhpL2x2e09/TmuQGXn0gZpuSFm5M5Vttd3qze2/Pcjr02YNtdPiu52FmZrz/RaMXSKvWrHcHmKSQoD7g+KUgiBDwzL x-ms-exchange-antispam-messagedata: 03xmwIzMToJrxV51bOTSOZzSoiUyr+MxNGfminwPLicCzUSBmY6X9jy40IT3dBKEn+Um1Ak/szol8MRV0+K4Hsbj0OhTWp6H2u4sl8miCl8rsRdH4WabZEkDg3jQbf8PhyFbXH82Bwm+3UYj1SNiFQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5181 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT044.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(346002)(396003)(136003)(376002)(199004)(46966005)(9686003)(52536014)(8936002)(70586007)(54906003)(2906002)(110136005)(8676002)(70206006)(81166006)(356004)(7696005)(47076004)(55016002)(5660300002)(81156014)(6506007)(36906005)(26826003)(4326008)(316002)(33656002)(478600001)(26005)(336012)(186003)(86362001)(41533002); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0802MB2222; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: c41aed1f-551d-45b0-1d03-08d7cc8a564e X-Forefront-PRVS: 03484C0ABF X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xyjVAfi5smpZzkXAv/LXXEmHSaKaQYePfPjCiEt8ZMet5+2Mgyve4ICkwRcCNCkNi6CxBjVKeLs3IiqPWuAMfgoc5bS3SgtLQeVesECyy99pffuhv0V87wIZ0/5WkvLkhdWQ2K4rZoa9J/HteaXyH/buwRX0C/MPRqNs2z8QWxvvm2xHkDSx72r398bLhOztz+Ejm6FaEgHVbJ3y887WWCO1XjnBervq8LWqZva+gnHlrGvXLGjVcltgEngFbzNxOtOfmPqPPV0YI5FHMck2VieDnClSHvjkb5M0oi6RQrAPLtA6KXFrpqIG52fC/tbOD+hvuRczdc9jm8u/wuQOVXPEknviA9nL8186F/uypi0ScqHOYd2SZl2sSiarcH5kciuV3OS3cTAXAr7jyQXYAxN5DHOMNQ/EMDgORgAJxMIMwXTRAHqXNgQda5JoTwt9EPF8APosX4OOJ2WvUapQx7VX56YRud/PsT58J2cdaNGi/m7KdsV5WpLfjzHzzXQeAVzUtLvqxMECW4GkCVPV0zVSAAMlzBfhXYYiAyuJonA= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2020 04:51:30.9659 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b9cb16a3-b07d-4ae2-637b-08d7cc8a5ba1 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2222 Subject: Re: [dpdk-dev] [PATCH v3 00/12] generic rte atomic APIs deprecate proposal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > Subject: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal > > > > DPDK provides generic rte_atomic APIs to do several atomic operations. > > These APIs are using the deprecated __sync built-ins and enforce full > > memory barriers on aarch64. However, full barriers are not necessary > > in many use cases. In order to address such use cases, C language > > offers > > C11 atomic APIs. The C11 atomic APIs provide finer memory barrier > > control by making use of the memory ordering parameter provided by the > user. > > Various patches submitted in the past [2] and the patches in this > > series indicate significant performance gains on multiple aarch64 CPUs > > and no performance loss on x86. > > > > But the existing rte_atomic API implementations cannot be changed as > > the APIs do not take the memory ordering parameter. The only choice > > available is replacing the usage of the rte_atomic APIs with C11 > > atomic APIs. In order to make this change, the following steps are prop= osed: > > > > [1] deprecate rte_atomic APIs so that future patches do not use > > rte_atomic APIs (a script is added to flag the usages). > > [2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs. >=20 > On [1] above, I feel deprecating DPDKs atomic functions and failing check= patch > is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK wi= ll > move to a > C11 based atomics model is a more gradual step to achieving the goal, and= at > that point add a checkpatch warning for additions of rte_atomic*? We have been working on changing existing usages of rte_atomic APIs in DPDK= to use C11 atomics. Usually, the x.11 releases have significant amount of = changes (not sure how many would use rte_atomic APIs). I would prefer that = in 20.11 no additional code is added using rte_atomics APIs. However, I am = open to suggestions on the exact time frame. Once we decide on the release, I think it makes sense to add a 'warning' in= the checkpatch to indicate the deprecation timeline and add an 'error' aft= er the release. >=20 > More on [2] in context below. >=20 > The above is my point-of-view, of course I'd like more people from the DP= DK > community to provide their input too. >=20 >=20 > > This patchset contains: > > 1) the checkpatch script changes to flag rte_atomic API usage in patche= s. > > 2) changes to programmer guide describing writing efficient code for > aarch64. > > 3) changes to various libraries to make use of c11 atomic APIs. > > > > We are planning to replicate this idea across all the other libraries, > > drivers, examples, test applications. In the next phase, we will add > > changes to the mbuf, the EAL interrupts and the event timer adapter > libraries. >=20 > About ~6/12 patches of this C11 set are targeting the Service Cores area = of > DPDK. I have some concerns over increased complexity of C11 implementatio= n > vs the (already complex) rte_atomic implementation today. I agree that it C11 changes are complex, especially if one is starting out = to understand what these APIs provide. From my experience, once few underly= ing concepts are understood, reviewing or making changes do not take too mu= ch time. > I see other patchsets enabling C11 across other DPDK components, so maybe > we should also discuss C11 enabling in a wider context that just service = cores? Yes, agree. We are in the process of making changes to other areas as well. >=20 > I don't think it fair to expect all developers to be well versed in C11 a= tomic > semantics like understanding the complex interactions between the various > C11 RELEASE, AQUIRE barriers requires. C11 has been around from sometime now. To my surprise, OVS already uses C11= APIs extensively. VPP has been accepting C11 related changes from past cou= ple of years. Having said that, I agree in general that not everyone is wel= l versed. >=20 > As maintainer of Service Cores I'm hesitant to accept the large-scale ref= actor Right now, the patches are split into multiple commits. If required I can h= ost a call to go over simple C11 API usages (sufficient to cover the usage = in service core) and the changes in this patch. If you find that particular= areas need more understanding I can work on providing additional informati= on such as memory order ladder diagrams. Please let me know what you think. > of atomic-implementation, as it could lead to racey bugs that are likely > extremely difficult to track down. (The recent race-on-exit has proven th= e > difficulty in reproducing, and that's with an atomics model I'm quite fam= iliar > with). >=20 > Let me be very clear: I don't wish to block a C11 atomic implementation, = but > I'd like to discuss how we (DPDK community) can best port-to and maintain= a > complex multi-threaded service library with best-in-class performance for= the > workload. >=20 > To put some discussions/solutions on the table: > - Shared Maintainership of a component? > Split in functionality and C11 atomics implementation > Obviously there would be collaboration required in such a case. > - Maybe shared maintainership is too much? > A gentlemans/womans agreement of "helping out" with C11 atomics debu= g > is enough? I think shared maintainer ship could be too much as there are many changes.= But, I and other engineers from Arm (I would include Arm ecosystem as well= ) can definitely help out on debug and reviews involving C11 APIs. (I see T= homas's reply on this topic). >=20 >=20 > Hope my concerns are understandable, and of course input/feedback > welcomed! -Harry No worries at all. We are here to help and make this as easy as possible. >=20 >=20 > PS: Apologies for the delay in reply - was OOO on Irish national holiday. Same here, was on vacation for 3 days. >=20 >=20 > > v3: > > add libatomic dependency for 32-bit clang > > > > v2: > > 1. fix Clang '-Wincompatible-pointer-types' WARNING. > > 2. fix typos. > > > > Honnappa Nagarahalli (2): > > service: avoid race condition for MT unsafe service > > service: identify service running on another core correctly > > > > Phil Yang (10): > > doc: add generic atomic deprecation section > > devtools: prevent use of rte atomic APIs in future patches > > eal/build: add libatomic dependency for 32-bit clang > > build: remove redundant code > > vhost: optimize broadcast rarp sync with c11 atomic > > ipsec: optimize with c11 atomic for sa outbound sqn update > > service: remove rte prefix from static functions > > service: remove redundant code > > service: optimize with c11 one-way barrier > > service: relax barriers with C11 atomic operations > > > > devtools/checkpatches.sh | 9 ++ > > doc/guides/prog_guide/writing_efficient_code.rst | 60 +++++++- > > drivers/event/octeontx/meson.build | 5 - > > drivers/event/octeontx2/meson.build | 5 - > > drivers/event/opdl/meson.build | 5 - > > lib/librte_eal/common/rte_service.c | 175 ++++++++++++---= ------- > > - > > lib/librte_eal/meson.build | 6 + > > lib/librte_ipsec/ipsec_sqn.h | 3 +- > > lib/librte_ipsec/sa.h | 2 +- > > lib/librte_rcu/meson.build | 5 - > > lib/librte_vhost/vhost.h | 2 +- > > lib/librte_vhost/vhost_user.c | 7 +- > > lib/librte_vhost/virtio_net.c | 16 ++- > > 13 files changed, 181 insertions(+), 119 deletions(-) > > > > -- > > 2.7.4