From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BCE8BA057B; Tue, 24 Mar 2020 12:03:38 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3BB221C0CD; Tue, 24 Mar 2020 12:03:37 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 7BFAE1C0C2 for ; Tue, 24 Mar 2020 12:03:35 +0100 (CET) IronPort-SDR: 5zytCW9q8kEnYpLw4Ch5NJIm0cB9v1qvd8v77lVRpKDnuDH7rwLqaIW1Y9Mkruo9pdTmGYz8sq QRyvNqvlVYnw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 04:03:34 -0700 IronPort-SDR: JDHGNsm2zg6Gjhl8b+tcFcICO0R0K1UHf9OJXVuzTP8+XIlVwCMTFFMQbDfauKUpwhcWltWNC1 h6R6nnqGXOfA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,300,1580803200"; d="scan'208";a="240237284" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga008.jf.intel.com with ESMTP; 24 Mar 2020 04:03:33 -0700 Received: from fmsmsx154.amr.corp.intel.com (10.18.116.70) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Mar 2020 04:03:33 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by FMSMSX154.amr.corp.intel.com (10.18.116.70) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Mar 2020 04:03:33 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Mar 2020 04:03:32 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jeGtl0WDZhP8tAGE10DANqtVEGbg3lyy4IwrvsnHplTVF0EpnBKwKL+lYsfeM8LEb+CT5Qa8ZEit1dr1zFwbKd3tkWWM9+IKB6AV01jPLKk9AM3ixqQ/jDP4LPsqGT/EO+QKvvft6WyvFT+899mKXxNw/3h76X4LdYIGJOP62vfX67Skm7vgc13Hk9PWTbSDv4PS5ADFQ2/ox74unAQE04NawALpbY0PO4+FxXG09AaehHqFPpj052rWVPlzNB8DBwyaPZN3/s2ArD3s6huEaWUzu2YRq3Zo65N9MGrSUP4Xmob6rzwgGRtEKFDjAmdSwkimtwOBBN6zfd46OCeC5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QsEjYvm56uXut1r+JWm09gAxTmfTqq8nSq+QYGZF0Ns=; b=DEWg33vio3D29PVf/WfiyRshjgYMp06VGKhdTBvKwQVvrqaIkDqyoL/hl0vEU6B0bJALukxOy0wsvV+B8YBSO7s7jyQeD63UurRxoN7fKr+didHVIsfDBKyfkPXB9d6MCgW81cMF5J2cJk+rZgFwwTi4amaaOvV2osQ39lmciDAnXYNrEme1Cua3Wrb4lZeBmL8dVL0Mp8rTMJQzUtLkN/RqcKDuRoqlYWX+qNsfiISVAkhZOTqqjMPKqSVSzMufC6+HEgspPjbrbwuCKdQP1TyfNK6gvfTe4N9TbI/BgFme2CR9uGtOopmuHJx4aXFHiKsWD8VyowuUqLxS7djE/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QsEjYvm56uXut1r+JWm09gAxTmfTqq8nSq+QYGZF0Ns=; b=ynGYQFaIClEJqS8F9isRv6Jk4mEOSW/dxSzY/NdcHbeLMpyFMpAOrqLwWiET5KZdq3QALMn9jcM8STQqMlmHfFVaK33UcNsYIQm4IgBaTYkmvB7/jP+XSV2aE+/DrP+hn29K+SjJPFufLaSI1H+veDFg0LpfWbKXq/AvdcJ+zvs= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (2603:10b6:805:5d::19) by SN6PR11MB3232.namprd11.prod.outlook.com (2603:10b6:805:be::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.22; Tue, 24 Mar 2020 11:03:31 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::5df7:d515:ec1d:8db1]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::5df7:d515:ec1d:8db1%7]) with mapi id 15.20.2835.021; Tue, 24 Mar 2020 11:03:31 +0000 From: "Ananyev, Konstantin" To: Phil Yang , Honnappa Nagarahalli , "thomas@monjalon.net" , "Van Haaren, Harry" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" , "Richardson, Bruce" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , nd , nd , nd Thread-Topic: [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update Thread-Index: AQHV+/oXyqmxuvMqMkaN9n+g1LYtyqhWh04ggAANbwCAAAGBEIABAi8AgAADylA= Date: Tue, 24 Mar 2020 11:03:31 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> <1584407863-774-7-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.178] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2a121c8c-ac52-4740-2727-08d7cfe2fd64 x-ms-traffictypediagnostic: SN6PR11MB3232: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 03524FBD26 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(376002)(366004)(396003)(136003)(39860400002)(186003)(8676002)(110136005)(478600001)(8936002)(316002)(76116006)(54906003)(6506007)(66476007)(81166006)(64756008)(66556008)(6636002)(66946007)(66446008)(26005)(71200400001)(52536014)(5660300002)(81156014)(4326008)(2906002)(9686003)(55016002)(15650500001)(7696005)(33656002)(7416002)(86362001)(921003)(1121003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB3232; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: +rkc7jFXMBlSepOPShFvDj2tuRewyoKiR975a95pajTcx3arE5Mwc8Ejgm+It5CcTuHKtxa28i0ao2xOGi6LsK22ztDxUKn/Ykl0p2DBMNJnwd64RpuaLCC6ymPmg7RxbUU+hVpzLT3szDxUaqBY9vQUOz9WVg+eooHu0wUi9oIg+TlPdx6foKtCmcZFxzdOZz4idZuj3oLFjkmmE52pLdVdhhVJXMYXX5v9b9eziWlvmmOrl7Zfq3dk9ErM0E8Xsp5TC/IBj4WChumEsYwxTAKx4tIJvCfhr/E03wARnX+kqTrxVHuFbeSa9u/QU6alosBSQoWx58oa1e5uw7YT2Z49Yh+hEyhrVShlywdOLUZmnwKFQIeb8ovAROBBIc+Nb/LdsID/Hc89TQUSeSSXN8paz5f9HmCyDpWl+VHm0foTv/1fFgooYou0S9GOQ5G9cd4TrkYi0faL3zy+tDTQp+DgM9Jq0kf/rAZiJLgtMdYtkAtJCGEWXqAXgGUyj0U5 x-ms-exchange-antispam-messagedata: p0chO+WYHL930MDlCKDmyB1jsGSiHOTqQ0FO2r4gsy4OYokzqU3FKcOw9QFEsAM8WRC0qhYSGQaY3jQQh9R/vdPoN8ZEsTEaZABSwEq5pjfMcSs/1a8AnN2YrjKjyy96U7/LwDkoz09GMueufZYBXQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 2a121c8c-ac52-4740-2727-08d7cfe2fd64 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2020 11:03:31.5213 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tkVKR/qDduNJ2GvmTTS0aNNbFrnZ3sdpFdmmnnu7qbyf/D5G4VoS/xQGGMTFFyOmj2IzW/NCT5XvbvJZoyftAMmimp8gpQa0TuRRPr4UYZ8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB3232 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Phil, > > > > > > > > Subject: RE: [PATCH v3 06/12] ipsec: optimize with c11 atomic for s= a > > outbound > > > > sqn update > > > > > > > > Hi Phil, > > > > > > > > > > > > > > For SA outbound packets, rte_atomic64_add_return is used to > > generate > > > > > SQN atomically. This introduced an unnecessary full barrier by ca= lling > > > > > the '__sync' builtin implemented rte_atomic_XX API on aarch64. Th= is > > > > > patch optimized it with c11 atomic and eliminated the expensive > > > > > barrier for aarch64. > > > > > > > > > > Signed-off-by: Phil Yang > > > > > Reviewed-by: Ruifeng Wang > > > > > Reviewed-by: Gavin Hu > > > > > --- > > > > > lib/librte_ipsec/ipsec_sqn.h | 3 ++- > > > > > lib/librte_ipsec/sa.h | 2 +- > > > > > 2 files changed, 3 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/lib/librte_ipsec/ipsec_sqn.h > > > > > b/lib/librte_ipsec/ipsec_sqn.h index 0c2f76a..e884af7 100644 > > > > > --- a/lib/librte_ipsec/ipsec_sqn.h > > > > > +++ b/lib/librte_ipsec/ipsec_sqn.h > > > > > @@ -128,7 +128,8 @@ esn_outb_update_sqn(struct rte_ipsec_sa *sa, > > > > > uint32_t *num) > > > > > > > > > > n =3D *num; > > > > > if (SQN_ATOMIC(sa)) > > > > > - sqn =3D (uint64_t)rte_atomic64_add_return(&sa- > > > > >sqn.outb.atom, n); > > > > > + sqn =3D __atomic_add_fetch(&sa->sqn.outb.atom, n, > > > > > + __ATOMIC_RELAXED); > > > > > > > > One generic thing to note: > > > > clang for i686 in some cases will generate a proper function call f= or 64-bit > > > > __atomic builtins (gcc seems to always generate cmpxchng8b for such > > cases). > > > > Does anyone consider it as a potential problem? > > > > It probably not a big deal, but would like to know broader opinion. > > > I had looked at this some time back for GCC. The function call is gen= erated > > only if the underlying platform does not support the atomic > > > instructions for the operand size. Otherwise, gcc generates the instr= uctions > > directly. > > > I would think the behavior would be the same for clang. > > > > From what I see not really. > > As an example: > > > > $ cat tatm11.c > > #include > > > > struct x { > > uint64_t v __attribute__((aligned(8))); > > }; > > > > uint64_t > > ffxadd1(struct x *x, uint32_t n, uint32_t m) > > { > > return __atomic_add_fetch(&x->v, n, __ATOMIC_RELAXED); > > } > > > > uint64_t > > ffxadd11(uint64_t *v, uint32_t n, uint32_t m) > > { > > return __atomic_add_fetch(v, n, __ATOMIC_RELAXED); > > } > > > > gcc for i686 will generate code with cmpxchng8b for both cases. > > clang will generate cmpxchng8b for ffxadd1() - when data is explicitly = 8B > > aligned, > > but will emit a function call for ffxadd11(). >=20 > I guess your testbed is an i386 platform. However, what I see here is di= fferent. >=20 > Testbed i686: Ubuntu 18.04.4 LTS/GCC 8.3/ Clang 9.0.0-2 > Both Clang and GCC for i686 generate code with xadd for these two cases. I suppose you meant x86_64 here (-m64), right? >=20 > Testbed i386: Ubuntu 16.04 LTS (Installed libatomic)/GCC 5.4.0/ Clang 4.= 0.0 > GCC will generate code with cmpxchng8b for both cases. > Clang generated code emits a function call for both cases. That's exactly what I am talking about above. X86_64 (64 bit binary) - no function calls for both gcc and clang i686 (32 bit binary) - no function calls with gcc, functions calls with cla= ng when explicit alignment is not specified. As I said in my initial email, that's probably not a big deal - from what I was told so far we don't officially support clang for IA-32 and I don't know does anyone uses it at all right now. Though if someone thinks it is a potential problem here - it is better to flag it at early stage. So once again my questions to the community: 1/ Does anyone builds/uses DPDK with i686-clang?=20 2/ If there are anyone, can these persons try to evaluate how big perf drop it would cause for them? =20 3/ Is there an option to switch to i686-gcc (supported one)? Konstantin > > > > > > > > > > > > > > else { > > > > > sqn =3D sa->sqn.outb.raw + n; > > > > > sa->sqn.outb.raw =3D sqn; > > > > > diff --git a/lib/librte_ipsec/sa.h b/lib/librte_ipsec/sa.h index > > > > > d22451b..cab9a2e 100644 > > > > > --- a/lib/librte_ipsec/sa.h > > > > > +++ b/lib/librte_ipsec/sa.h > > > > > @@ -120,7 +120,7 @@ struct rte_ipsec_sa { > > > > > */ > > > > > union { > > > > > union { > > > > > - rte_atomic64_t atom; > > > > > + uint64_t atom; > > > > > uint64_t raw; > > > > > } outb; > > > > > > > > If we don't need rte_atomic64 here anymore, then I think we can > > collapse the > > > > union to just: > > > > uint64_t outb; > > > > > > > > > struct { > > > > > -- > > > > > 2.7.4