From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F25C0A057B; Tue, 24 Mar 2020 14:10:24 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A52E91C0D9; Tue, 24 Mar 2020 14:10:23 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 3CC291C0D6 for ; Tue, 24 Mar 2020 14:10:21 +0100 (CET) IronPort-SDR: dVR9WVtibEfY2eN6k4bvsDfz9n8F88zp5tuCKivBgp/8sF3U8mjDdI8W+VRkXR2aVxJV4m4bGc uMIR+j5GcFSQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Mar 2020 06:10:20 -0700 IronPort-SDR: rjcaryLdDKuiH595PO845ytHxrwugz4m2xO1RT7RRH0Oqmt12hFRPMtNte1iHIIFECtlerQ3LX em/r97oF17dA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,300,1580803200"; d="scan'208";a="357420274" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by fmsmga001.fm.intel.com with ESMTP; 24 Mar 2020 06:10:19 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Mar 2020 06:10:19 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.109) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 24 Mar 2020 06:10:19 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SMdqU1YO9BlXjmFZuKQflh36I2MsH3WClEk1dtq3ZtAtN7pzaO/KrcU0PJtjh2RJkY/4lplczAKCQdmkG/21X08lflhq15EH3e/cAdRxkkc6Ik1Eobr2ID/PeJYtM10d8UYwREtL/eaB3VuL2jJ8yuicfbqUzCflL0XoGLdksutUjpvoM/F5Dx0FWINybJz0DG3qbQ1QzEbvuK0Wn3t+n6ogfoqws4uKiIEdFyxK7xUtC9zWmMWa2EBIXFAicXDjW9u1JHAOe6jsUpic6+eORMzF2NMeqe51ZNXt5vB73LMl+78Zbxchy0GblVUF1nZBX4FuVIXMXX1bywz3CYmTkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S52tKMEZ6m7ZgaUCTlQXltYuhgGpdK33aWckq12Ezjs=; b=Z6eSFuLNB5TbuCgTQnn9s5poRN2vAKd6maOYF8DrEDYFJ5N4TqX5TUiNjqbJqN7upR/K+YtbG69CIWdG3cMosOfJ9dgUE6myAuwvc5PFe87dqmDfZ6T0maOvv8TJAIGT/2xS/iAlxnvJcG3pco6WTkputEI9WrKaTPHXd5yZditbioe+BWKtuA17wf3reauJQkfzSwiwQ6U/lnSCHPZXEz8qYyMD61qoF34uTx0DswPOSB+h6zNqH6GkBnJDYiswdzfp6TWIjPwxir2lDbeANT049925NPNZmb9Yjlt5wNXfu4iH01dIZlQH2ju8wUDJdIP9CFSt+0T3up1Ji6/gVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S52tKMEZ6m7ZgaUCTlQXltYuhgGpdK33aWckq12Ezjs=; b=ahXrbXHnVZwP+lHskHJtwSuqPlb0TjRfWFqvIEwHS5QLkuWG2tIZ7eLqrfwuxJYRcrWh5VRpE9JKv7ZZkt8mkXcbPqEn+sUN3uODgx5VtnQrOlczwRtJegdUjmds/lg4y8TC74IdxIVY0c1AiPwvNx5MFGrm2ocQezjuwlkkdRY= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (2603:10b6:805:5d::19) by SN6PR11MB2750.namprd11.prod.outlook.com (2603:10b6:805:54::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.22; Tue, 24 Mar 2020 13:10:17 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::5df7:d515:ec1d:8db1]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::5df7:d515:ec1d:8db1%7]) with mapi id 15.20.2835.021; Tue, 24 Mar 2020 13:10:17 +0000 From: "Ananyev, Konstantin" To: Honnappa Nagarahalli , Phil Yang , "thomas@monjalon.net" , "Van Haaren, Harry" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" , "Richardson, Bruce" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , nd , nd Thread-Topic: [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update Thread-Index: AQHV+/oXyqmxuvMqMkaN9n+g1LYtyqhWh04ggAANbwCAAAGBEIAAEtYAgAEJS8A= Date: Tue, 24 Mar 2020 13:10:17 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> <1584407863-774-7-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.178] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5be17bb2-448f-4c14-ac42-08d7cff4b30d x-ms-traffictypediagnostic: SN6PR11MB2750: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 03524FBD26 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(376002)(346002)(136003)(396003)(366004)(39860400002)(55016002)(9686003)(2906002)(4326008)(5660300002)(33656002)(66446008)(66476007)(186003)(52536014)(76116006)(64756008)(66946007)(66556008)(26005)(6506007)(54906003)(8676002)(110136005)(81166006)(316002)(71200400001)(966005)(478600001)(86362001)(7416002)(6636002)(7696005)(45080400002)(8936002)(81156014)(921003)(1121003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2750; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: m3xcBQ+oN/6lRPTneK0HH7g0294lY/rZQQbMqlPgQMlBlo7XZbER/LUmbhLlfnMRWqWYTkExoXC/x8ya/A8GBnERpUlu7rxDpa3tnKh0gcQEySVll1Er9kEm3dvimZzbAv9Z1RVtcOQpi8kaaRCD96SzWkv3yzeqosGTeI5EYX3pxriL628hMFLYWUy9mh1JxWrkzpMSQuITW2K9IcJcVdPFmSzyjo1gPsj+UhGVmxl4lIs68zeRstqpGNUwJ6HTx4LQw/cXpzhTabLwbeJIsIWj1A/Z2t9y9LDJ72Kfbia531nM+CbVN28vlPuEWi7SUSrx9/hTyy4LK6CtuqMSeg+gnAwb6zdZzRCGP1SsoJRaoIDjGqHkXythDfGjN6SaQ6gGBm9+a9oHAQ/fCzMjbNQJJSel3DC8blpz+qTSfHGxez7wrupW/ojXTrlSsVWmYIpoySWfLOp+CZvd5NznRvby2VNuPEQTCPMnGe2adYvSZhLjJimyUbTYaCgiHGr8FE8gdcJ8M1PunuE+XhD8L2EDJk2xLGVoHF/sSjIeEefQG7kl9sBCB9l+eC15R1+eM7ZT396ZOuvk9Xw5A0/Y5w== x-ms-exchange-antispam-messagedata: LSy8ehrPYfEh8uVA2V/y4Hpma2mb5pW5Y+OhBapzljjZf8uTdNiiq7CdsIArSidjK0IXB0OrIBro69q50kooYcDl2kCO/zRBPMnmwsG94U+TQsS2//nUOwv3VPa1YZqaxlvdOUnGUu7KBdbSqWdGnw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 5be17bb2-448f-4c14-ac42-08d7cff4b30d X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Mar 2020 13:10:17.5004 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: eohPpVboX6uOCDZrylFOAROcVNSjXRv7J9zmZf1s+bVgkxXvySf+8C2cn5MBYCOSgmfzKqasQcWiZeMhZghj1r+B8f3DeOTOjAR6wNArAHA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2750 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 06/12] ipsec: optimize with c11 atomic for sa outbound sqn update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > For SA outbound packets, rte_atomic64_add_return is used to > > > > > generate SQN atomically. This introduced an unnecessary full > > > > > barrier by calling the '__sync' builtin implemented rte_atomic_XX > > > > > API on aarch64. This patch optimized it with c11 atomic and > > > > > eliminated the expensive barrier for aarch64. > > > > > > > > > > Signed-off-by: Phil Yang > > > > > Reviewed-by: Ruifeng Wang > > > > > Reviewed-by: Gavin Hu > > > > > --- > > > > > lib/librte_ipsec/ipsec_sqn.h | 3 ++- > > > > > lib/librte_ipsec/sa.h | 2 +- > > > > > 2 files changed, 3 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/lib/librte_ipsec/ipsec_sqn.h > > > > > b/lib/librte_ipsec/ipsec_sqn.h index 0c2f76a..e884af7 100644 > > > > > --- a/lib/librte_ipsec/ipsec_sqn.h > > > > > +++ b/lib/librte_ipsec/ipsec_sqn.h > > > > > @@ -128,7 +128,8 @@ esn_outb_update_sqn(struct rte_ipsec_sa *sa, > > > > > uint32_t *num) > > > > > > > > > > n =3D *num; > > > > > if (SQN_ATOMIC(sa)) > > > > > - sqn =3D (uint64_t)rte_atomic64_add_return(&sa- > > > > >sqn.outb.atom, n); > > > > > + sqn =3D __atomic_add_fetch(&sa->sqn.outb.atom, n, > > > > > + __ATOMIC_RELAXED); > > > > > > > > One generic thing to note: > > > > clang for i686 in some cases will generate a proper function call > > > > for 64-bit __atomic builtins (gcc seems to always generate cmpxchng= 8b for > > such cases). > > > > Does anyone consider it as a potential problem? > > > > It probably not a big deal, but would like to know broader opinion. > > > I had looked at this some time back for GCC. The function call is > > > generated only if the underlying platform does not support the atomic > > instructions for the operand size. Otherwise, gcc generates the instruc= tions > > directly. > > > I would think the behavior would be the same for clang. > > > > From what I see not really. > > As an example: > > > > $ cat tatm11.c > > #include > > > > struct x { > > uint64_t v __attribute__((aligned(8))); }; > > > > uint64_t > > ffxadd1(struct x *x, uint32_t n, uint32_t m) { > > return __atomic_add_fetch(&x->v, n, __ATOMIC_RELAXED); } > > > > uint64_t > > ffxadd11(uint64_t *v, uint32_t n, uint32_t m) { > > return __atomic_add_fetch(v, n, __ATOMIC_RELAXED); } > > > > gcc for i686 will generate code with cmpxchng8b for both cases. > > clang will generate cmpxchng8b for ffxadd1() - when data is explicitly = 8B > > aligned, but will emit a function call for ffxadd11(). > Does it require libatomic to be linked in this case?=20 Yes, it does. In fact same story even with current dpdk.org master. To make i686-native-linuxapp-clang successfully, I have to=20 explicitly add EXTRA_LDFLAGS=3D"-latomic". To be more specific: $ for i in i686-native-linuxapp-clang/lib/*.a; do x=3D`nm $i | grep __atomi= c_`; if [[ -n "${x}" ]]; then echo $i; echo $x; fi; done i686-native-linuxapp-clang/lib/librte_distributor.a U __atomic_load_8 U __atomic_store_8 i686-native-linuxapp-clang/lib/librte_pmd_opdl_event.a U __atomic_load_8 U __atomic_store_8 i686-native-linuxapp-clang/lib/librte_rcu.a U __atomic_compare_exchange_8 U __atomic_load_8 As there were no complains so far, it makes me think that probably no-one using clang for IA-32 builds. > Clang documentation calls out unaligned case where it would generate the = function call > [1]. Seems so, and it treats uin64_t as 4B aligned for IA. =20 > On aarch64, the atomic instructions need the address to be aligned. For that particular case (cmpxchng8b) there is no such restrictions for IA-= 32. Again, as I said before, gcc manages to emit code without function calls for exactly the same source. >=20 > [1] https://clang.llvm.org/docs/Toolchain.html#atomics-library >=20 > > > > > > > > > > > > > > else { > > > > > sqn =3D sa->sqn.outb.raw + n; > > > > > sa->sqn.outb.raw =3D sqn; > > > > > diff --git a/lib/librte_ipsec/sa.h b/lib/librte_ipsec/sa.h index > > > > > d22451b..cab9a2e 100644 > > > > > --- a/lib/librte_ipsec/sa.h > > > > > +++ b/lib/librte_ipsec/sa.h > > > > > @@ -120,7 +120,7 @@ struct rte_ipsec_sa { > > > > > */ > > > > > union { > > > > > union { > > > > > - rte_atomic64_t atom; > > > > > + uint64_t atom; > > > > > uint64_t raw; > > > > > } outb; > > > > > > > > If we don't need rte_atomic64 here anymore, then I think we can > > > > collapse the union to just: > > > > uint64_t outb; > > > > > > > > > struct { > > > > > -- > > > > > 2.7.4