From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9BC4FA058A; Wed, 25 Mar 2020 10:38:28 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 63CE7378E; Wed, 25 Mar 2020 10:38:27 +0100 (CET) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00078.outbound.protection.outlook.com [40.107.0.78]) by dpdk.org (Postfix) with ESMTP id 78FDA374C for ; Wed, 25 Mar 2020 10:38:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=caE0DeoB53rFqieZkDQVT74R1TbyoO6lfIrSwi0q7dw=; b=r1Dh1XhXjpUOAVGV76rZJXo1TjtufdEBVg3g50Idfh5DP3LCPhKUuZDhpkWUasnHEMY/qIDpJBKXEGLaP81svYVLA5lhJlfD/qN9HFnC81pJpNdGt9wHihHhOrUG6meNnnDhzpu3r8JEPecwoFj66Z82HQxWmJsahL/4qAGbU+k= Received: from AM6PR0502CA0060.eurprd05.prod.outlook.com (2603:10a6:20b:56::37) by AM6PR08MB3976.eurprd08.prod.outlook.com (2603:10a6:20b:ae::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.22; Wed, 25 Mar 2020 09:38:23 +0000 Received: from AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:56:cafe::74) by AM6PR0502CA0060.outlook.office365.com (2603:10a6:20b:56::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18 via Frontend Transport; Wed, 25 Mar 2020 09:38:23 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT018.mail.protection.outlook.com (10.152.16.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.13 via Frontend Transport; Wed, 25 Mar 2020 09:38:23 +0000 Received: ("Tessian outbound e13acb17570e:v48"); Wed, 25 Mar 2020 09:38:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from 59a2e5da7f13.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id CEC4EAB7-5F18-4284-94A5-A1D790A3174A.1; Wed, 25 Mar 2020 09:38:17 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 59a2e5da7f13.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 25 Mar 2020 09:38:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OxFyz9OJjaGlfvxbIZoTlTCJPd/P8eNnNGy3hcuu935Fum/Tgqwvnl+GwOXi+J4QBpDFdQS6rDJKAlNMr/K0gfT2sA9wyPe1yk/gGvh+xNiegnv2ptrHBVLMdmanrDP+/AiRTzhPAx4aHRJU3C2tD13wP7wkhiftBh4NpPYSffkCwlpg1pnyTslVpkbcd7JArRY9RFQIr/pWOTKf70AUSgVXDnaiyysGqeOi4FD5gP0f2hhFFT4agzKfwu0w/yWglOVqcSUIKGH/1bjzwMNU8pLyuerPAMxGBGXtVlHHldDP5Qw8q56a16kzxD2f4G2mj+qGu2AcCIaHubgmmS/OLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=caE0DeoB53rFqieZkDQVT74R1TbyoO6lfIrSwi0q7dw=; b=SDjtGwhzrlk+LQfUfv8xvX/JnNuDWu5jG32tZApwJFerfj1lYdtDgZxdOjoXzReu/GdUX9ZrqHbmLmNcuARR8J/1ZtHaCvm/dsnzm2UINuSTzEFw1MJiGmrTyUzbtRCKb1OKCYzCPKkfme87ytn12JSh9Ykfl6mppSVhsyLgP0xfrsbC9PnGfyjWjYGTEnC0GyzzEvew5fEdqNfdYKC+ma0IDDXKxlN78wBTWIvTTit1VNHXPysZuB3fs9pzrM/OSOnAV2KKs+I1m+mY0vjdsj2MKea/XzPtGqYB02uQAxxEAyW1T+IJRuPRGy8Yec5vPoo8CMWaZqpufmCn/9uiaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=caE0DeoB53rFqieZkDQVT74R1TbyoO6lfIrSwi0q7dw=; b=r1Dh1XhXjpUOAVGV76rZJXo1TjtufdEBVg3g50Idfh5DP3LCPhKUuZDhpkWUasnHEMY/qIDpJBKXEGLaP81svYVLA5lhJlfD/qN9HFnC81pJpNdGt9wHihHhOrUG6meNnnDhzpu3r8JEPecwoFj66Z82HQxWmJsahL/4qAGbU+k= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (10.255.27.75) by VE1PR08MB4927.eurprd08.prod.outlook.com (10.255.27.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.20; Wed, 25 Mar 2020 09:38:15 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::7df3:e3c8:54f7:57c2]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::7df3:e3c8:54f7:57c2%6]) with mapi id 15.20.2835.023; Wed, 25 Mar 2020 09:38:14 +0000 From: Phil Yang To: "Ananyev, Konstantin" , Honnappa Nagarahalli , "thomas@monjalon.net" , "Van Haaren, Harry" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" , "Richardson, Bruce" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , nd , nd , nd , nd Thread-Topic: [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update Thread-Index: AQHWAUOu42jzlX7hWEmw28niYBSJr6hWiikAgAADAgCAAO2asIAAGmiAgAF4+JA= Date: Wed, 25 Mar 2020 09:38:14 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> <1584407863-774-7-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 09caf0e1-7cd7-4a05-a3d4-84efad542a9f.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: c942fa01-b9e0-428e-f21e-08d7d0a042eb x-ms-traffictypediagnostic: VE1PR08MB4927:|VE1PR08MB4927:|AM6PR08MB3976: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 0353563E2B X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(136003)(366004)(39860400002)(376002)(346002)(7696005)(52536014)(55236004)(86362001)(64756008)(54906003)(8676002)(9686003)(66476007)(66446008)(53546011)(66946007)(33656002)(6506007)(76116006)(71200400001)(2906002)(66556008)(316002)(110136005)(7416002)(186003)(5660300002)(15650500001)(4326008)(8936002)(55016002)(81166006)(478600001)(81156014)(26005)(921003)(1121003); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4927; H:VE1PR08MB4640.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 758Q+KDwrosd/3esaXsoJraMnnHLKdRLac+8VOgA42taolQPev5Qpy0le8NiVTjxjCk8ApNy4hoyjseBHC2lTlaq10K9u0NKQYquMYYVX9+8P2WzrF0yFV5NbSvG0u9IYdNbcGoHgIGNHSN/oFVGmgGSih94BknGgOpwO3fkdix+G/ZbnO7/RhuVDpTsKL+ox4pUfmRCw85eFeIslcT7u2MwJiJoFM+pqjeDpIvwq3L8krglkH+z0gl6L14u/7BrQ12AsuzY+A9vGynF/Ruu94gxT8WLPhVJMrUPqYLZmoGvIAEbBnhMEKY8et2EcpaxU0AluKZ/jId0UF48Tzrsx/uLnmrEIhY8jzoxuzkK42XMy3x9t1KpsUg8DucoSeYves6HzFDZfZHYO8m1n3uU3kmrP/dlNDpzhD5sJHWxYCfc0/qow8wyaS1ZaRUiQ7QClUAtCthDTV4zh1Ifo5NnB3Jk0mWYrMk/EUNdUmf6nDfU7JmtMx+s+oG2ulD+omZE x-ms-exchange-antispam-messagedata: kxcSrhhKag1Q4UKUZp3qz76/BlSc62cf2WZEA9uj131IRa26paZC6/hsSgVvpubWS3Ccu3cfw0A1jhVjADARfbBb6vgLcA69aSF+B0Nb5GosQFRju5fjvcuJ1uPn+DeN8PaCzv4IP8zkADjPLE7eKw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4927 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(396003)(39860400002)(346002)(136003)(376002)(46966005)(186003)(2906002)(26826003)(336012)(82740400003)(478600001)(26005)(52536014)(70206006)(53546011)(8676002)(6506007)(55016002)(81156014)(15650500001)(9686003)(81166006)(33656002)(86362001)(316002)(4326008)(54906003)(110136005)(7696005)(356004)(70586007)(5660300002)(36906005)(47076004)(8936002)(921003)(1121003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3976; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; X-MS-Office365-Filtering-Correlation-Id-Prvs: 5e1c2146-bb67-494a-fdb2-08d7d0a03e03 X-Forefront-PRVS: 0353563E2B X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dQrmGFa3hk8YN7xq8KwYd5ycgi8Pjm4g6XKrbTtVjvh2zg/1vC7+qysZmCKH4xb4Tu75a5ugow+wxtKjMH0El+jGecfsv/F3GPIWvVMTfl3B4OtSdX8A1PFeS6KEQsXAxIsFjKG5BQdyRIjz7Bywn9nQDbX63HuaPUZ4TqEQbN6LKo0QkqUyRonMBFjAeqNhmaRLEtah8ej8EFqmMXZyzdRaTtUYnwOaNXe3zWLO5RIeWG1O2pR8RLKv6SxknkY098No6j0WzRtEJpx7DabjricAh6Cgm0RQx/RcAfdniTypdQvUNf/pICBCYZi1u3wvtpapFBwvy3UeQmgN2hcHJ6bISkKy705qjXfup0gJJEmenGlLWL9r0CQWCcjmaoeBNBXPo0YuXko93voM9Nl2htF0S8+rrc9jIQ+2RKRsnRPXmg4b1l26/D44STDd2TScO11mPG7JxqMow9HSIUbj4iYqy+2iWxXaf07B8UaF0B0rNrMBS/2IUIc5PyrXW3dgrMHCKUEoA3xT4A7wgoPVz4OOI1VSZijdqg+ANfhdHz+jKkvVHLqB3loMwrzDGqoh X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2020 09:38:23.1667 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c942fa01-b9e0-428e-f21e-08d7d0a042eb X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3976 Subject: Re: [dpdk-dev] [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ananyev, Konstantin > Sent: Tuesday, March 24, 2020 7:04 PM > To: Phil Yang ; Honnappa Nagarahalli > ; thomas@monjalon.net; Van Haaren, > Harry ; stephen@networkplumber.org; > maxime.coquelin@redhat.com; dev@dpdk.org; Richardson, Bruce > > Cc: david.marchand@redhat.com; jerinj@marvell.com; > hemant.agrawal@nxp.com; Gavin Hu ; Ruifeng Wang > ; Joyce Kong ; nd > ; nd ; nd > Subject: RE: [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa > outbound sqn update >=20 >=20 > Hi Phil, >=20 > > > > > > > > > > > Subject: RE: [PATCH v3 06/12] ipsec: optimize with c11 atomic for= sa > > > outbound > > > > > sqn update > > > > > > > > > > Hi Phil, > > > > > > > > > > > > > > > > > For SA outbound packets, rte_atomic64_add_return is used to > > > generate > > > > > > SQN atomically. This introduced an unnecessary full barrier by = calling > > > > > > the '__sync' builtin implemented rte_atomic_XX API on aarch64. > This > > > > > > patch optimized it with c11 atomic and eliminated the expensive > > > > > > barrier for aarch64. > > > > > > > > > > > > Signed-off-by: Phil Yang > > > > > > Reviewed-by: Ruifeng Wang > > > > > > Reviewed-by: Gavin Hu > > > > > > --- > > > > > > lib/librte_ipsec/ipsec_sqn.h | 3 ++- > > > > > > lib/librte_ipsec/sa.h | 2 +- > > > > > > 2 files changed, 3 insertions(+), 2 deletions(-) > > > > > > > > > > > > diff --git a/lib/librte_ipsec/ipsec_sqn.h > > > > > > b/lib/librte_ipsec/ipsec_sqn.h index 0c2f76a..e884af7 100644 > > > > > > --- a/lib/librte_ipsec/ipsec_sqn.h > > > > > > +++ b/lib/librte_ipsec/ipsec_sqn.h > > > > > > @@ -128,7 +128,8 @@ esn_outb_update_sqn(struct rte_ipsec_sa > *sa, > > > > > > uint32_t *num) > > > > > > > > > > > > n =3D *num; > > > > > > if (SQN_ATOMIC(sa)) > > > > > > - sqn =3D (uint64_t)rte_atomic64_add_return(&sa- > > > > > >sqn.outb.atom, n); > > > > > > + sqn =3D __atomic_add_fetch(&sa->sqn.outb.atom, n, > > > > > > + __ATOMIC_RELAXED); > > > > > > > > > > One generic thing to note: > > > > > clang for i686 in some cases will generate a proper function call= for 64- > bit > > > > > __atomic builtins (gcc seems to always generate cmpxchng8b for su= ch > > > cases). > > > > > Does anyone consider it as a potential problem? > > > > > It probably not a big deal, but would like to know broader opinio= n. > > > > I had looked at this some time back for GCC. The function call is > generated > > > only if the underlying platform does not support the atomic > > > > instructions for the operand size. Otherwise, gcc generates the > instructions > > > directly. > > > > I would think the behavior would be the same for clang. > > > > > > From what I see not really. > > > As an example: > > > > > > $ cat tatm11.c > > > #include > > > > > > struct x { > > > uint64_t v __attribute__((aligned(8))); > > > }; > > > > > > uint64_t > > > ffxadd1(struct x *x, uint32_t n, uint32_t m) > > > { > > > return __atomic_add_fetch(&x->v, n, __ATOMIC_RELAXED); > > > } > > > > > > uint64_t > > > ffxadd11(uint64_t *v, uint32_t n, uint32_t m) > > > { > > > return __atomic_add_fetch(v, n, __ATOMIC_RELAXED); > > > } > > > > > > gcc for i686 will generate code with cmpxchng8b for both cases. > > > clang will generate cmpxchng8b for ffxadd1() - when data is explicitl= y 8B > > > aligned, > > > but will emit a function call for ffxadd11(). > > > > I guess your testbed is an i386 platform. However, what I see here is > different. > > > > Testbed i686: Ubuntu 18.04.4 LTS/GCC 8.3/ Clang 9.0.0-2 > > Both Clang and GCC for i686 generate code with xadd for these two cases= . >=20 > I suppose you meant x86_64 here (-m64), right? Yes. It is x86_64 here. >=20 >=20 > > > > Testbed i386: Ubuntu 16.04 LTS (Installed libatomic)/GCC 5.4.0/ Clang = 4.0.0 > > GCC will generate code with cmpxchng8b for both cases. > > Clang generated code emits a function call for both cases. >=20 > That's exactly what I am talking about above. > X86_64 (64 bit binary) - no function calls for both gcc and clang > i686 (32 bit binary) - no function calls with gcc, functions calls with c= lang > when explicit alignment is not specified. >=20 > As I said in my initial email, that's probably not a big deal - > from what I was told so far we don't officially support clang for IA-32 > and I don't know does anyone uses it at all right now. > Though if someone thinks it is a potential problem here - > it is better to flag it at early stage. > So once again my questions to the community: > 1/ Does anyone builds/uses DPDK with i686-clang? > 2/ If there are anyone, can these persons try to evaluate > how big perf drop it would cause for them? > 3/ Is there an option to switch to i686-gcc (supported one)? > Konstantin >=20 > > > > > > > > > > > > > > > > > > else { > > > > > > sqn =3D sa->sqn.outb.raw + n; > > > > > > sa->sqn.outb.raw =3D sqn; > > > > > > diff --git a/lib/librte_ipsec/sa.h b/lib/librte_ipsec/sa.h inde= x > > > > > > d22451b..cab9a2e 100644 > > > > > > --- a/lib/librte_ipsec/sa.h > > > > > > +++ b/lib/librte_ipsec/sa.h > > > > > > @@ -120,7 +120,7 @@ struct rte_ipsec_sa { > > > > > > */ > > > > > > union { > > > > > > union { > > > > > > - rte_atomic64_t atom; > > > > > > + uint64_t atom; > > > > > > uint64_t raw; > > > > > > } outb; > > > > > > > > > > If we don't need rte_atomic64 here anymore, then I think we can > > > collapse the > > > > > union to just: > > > > > uint64_t outb; > > > > > > > > > > > struct { > > > > > > -- > > > > > > 2.7.4