From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87429A0350; Tue, 23 Jun 2020 23:01:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 895161D6D4; Tue, 23 Jun 2020 23:01:52 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id C946C1D6CD for ; Tue, 23 Jun 2020 23:01:50 +0200 (CEST) IronPort-SDR: OIBwJP1jWF3RJgqv2Da+IYMeoKBY4lkOrH2olltRVtseg8bHtCzJtybb05miXrXaATPEY6zt/e RLlxdj5PYGkQ== X-IronPort-AV: E=McAfee;i="6000,8403,9661"; a="142450304" X-IronPort-AV: E=Sophos;i="5.75,272,1589266800"; d="scan'208";a="142450304" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2020 14:01:46 -0700 IronPort-SDR: WrWqWi6z/QQ2d5GEbMegcAnv8D61YLKMehQ+pf91EyuuFSUksVrLmjM1tBUQ8dfyR1yrbXUwoW P/bZXwJsCfPg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,272,1589266800"; d="scan'208";a="353924894" Received: from orsmsx110.amr.corp.intel.com ([10.22.240.8]) by orsmga001.jf.intel.com with ESMTP; 23 Jun 2020 14:01:46 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX110.amr.corp.intel.com (10.22.240.8) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jun 2020 14:01:45 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 23 Jun 2020 14:01:45 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 23 Jun 2020 14:01:45 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.101) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jun 2020 14:01:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CkHsuIUczqnc0/3mhSjwd6AkqDczmaLrZSUIZfI0PjroeXaFh1z8A5CIyag25MmfP/n6j/KO3R3fNralpbBHfZ/zfRMqkwGuMz9LVKklzG1wp+LqHEIBnUQ0K8eHmo6CkeXU/foFOjPuefoyoUldZwPuDcnDJ53XRceg0tkVyv7rPe02wKoJlitMASyq+OsTz9Axip92Gpe4sLjvUbQYY0UYMDIQoZxiE26DMy0Ke6ubC2WiEOZ9QedoTh1cnR/+rfuXudNfmo8xbolUSOmhwWQY3BC2d0nEojBLqpZ63B1C6sc/nZhE6ZYfU3f42fIHDv9bakV7FWf9eDnkgn5Fgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HqJ5oJfoN+vUTblDqRF32EBpCcXTd2IMXgJcjQGlNKE=; b=DY6MZbk/6M43omBQiAYScXRapY4qAmIt/6Oi4YUxPEaanYJAPTw5piI0GgJ4JjyrWPO2tlnxWMvk9atAX2BETOj1t1WMEvbMmh3S8yRWiE0kokkRcuGRQjBjwAPDBZOpkIZ18hokZJPcRhN1/dk8Ww967IypenQzkPkNqkZGJXEeEm4U0nzxGVhV8hibx3Fqgx3uUE+MjcoOOGCTI6sxjU+XSWPHB+JjdTveOOw7WMoFlbxdNy6Lw08c+C1O3Nb4o1BVtIIlMZRWNyJyvknCj66QYJYsNppRoQ0URFgrkqQpOrtiWLXpzN9htd2i9PinQaZDCZLQiPkr+Fwuo1Wy6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HqJ5oJfoN+vUTblDqRF32EBpCcXTd2IMXgJcjQGlNKE=; b=QGZObv60/zOPajrYbRvUq6BKdCXp+uo0iqVoV+YiIDwjRAo0xpHPy12TKWFLeuuoAcb+g7AIF2pQwSqtuS+EHrRdnxw5FQln31LNZ4jCiGDHpFIOSQrLjVSrujN45xTZmY0s0Aj+Mg2rGtsHRYci4vBAdEIrjv5Y5CN2jet5+cI= Received: from CY4PR1101MB2118.namprd11.prod.outlook.com (2603:10b6:910:1f::10) by CY4PR11MB1510.namprd11.prod.outlook.com (2603:10b6:910:c::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.23; Tue, 23 Jun 2020 21:01:42 +0000 Received: from CY4PR1101MB2118.namprd11.prod.outlook.com ([fe80::410b:7807:181:1620]) by CY4PR1101MB2118.namprd11.prod.outlook.com ([fe80::410b:7807:181:1620%4]) with mapi id 15.20.3109.027; Tue, 23 Jun 2020 21:01:42 +0000 From: "Carrillo, Erik G" To: Phil Yang , "dev@dpdk.org" CC: "drc@linux.vnet.ibm.com" , "honnappa.nagarahalli@arm.com" , "ruifeng.wang@arm.com" , "dharmik.thakkar@arm.com" , "nd@arm.com" Thread-Topic: [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag Thread-Index: AQHWQKuE7f35LvPXW06J888vxz8FdKjmv1Tg Date: Tue, 23 Jun 2020 21:01:42 +0000 Message-ID: References: <1591960798-24024-1-git-send-email-phil.yang@arm.com> <1591960798-24024-2-git-send-email-phil.yang@arm.com> In-Reply-To: <1591960798-24024-2-git-send-email-phil.yang@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [136.49.135.17] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 08183c0c-ad3f-4ab4-5c59-08d817b8a1b5 x-ms-traffictypediagnostic: CY4PR11MB1510: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2958; x-forefront-prvs: 04433051BF x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XlnFhUJX6SeVgN2hLeCPsHWE8r7Ea7ooqIzMAgu/YdIALnSx1XaHLDnZ0NTCOHumLPcaM8XRX3uz8ndr+X7EDOUSuaMoLKkdtKGucp8MyZ7Qj6ntLkggTnCYkewDz2DpzRRUXJ1sedRc5bYf4mPEZnzuozGwAasybz6zCVSVYET+nrFwd3isvmOVGW22A8XEgBdo2py0T1qvogD6RVEnTqQyJVJKPv7gU+1YN5NbyskNWjHzj6UHkZ0cJOyQ8kWxdzZ3RLcQMfnt/0E5Xxw63u8VgTvQFSsuGrmQFS8ITPVZiT8UNzDZPuP0DkBggQIVSNS3CrW4rXobWWeLcPuQ9Q== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR1101MB2118.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(396003)(366004)(346002)(136003)(376002)(55016002)(5660300002)(76116006)(54906003)(9686003)(86362001)(66446008)(478600001)(66556008)(66476007)(71200400001)(316002)(52536014)(64756008)(2906002)(110136005)(186003)(66946007)(26005)(83380400001)(7696005)(6506007)(8936002)(33656002)(53546011)(8676002)(4326008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: 9CIWcDTqSobf6L3udN/HFbpwK/iP7LcDWCLPwRm/KL+w28ZF+fjSJoJooXh9pvCRFnmYeY4uQ77OiTkHsLkwi+Fpz0z5W7fLkr/jKtrqnuraZzCi5DzQlmlBrKK2w8/cXwqE001euavTtuInkjiau56GhAN6XEumnHJe8m6hRxcjcZvvwgaF5Q7PlVnBFve+/f+fVdGpi0Pz1vZF4JQUsveSe7DQbeIWbyYx4VHy2V20X+mC5S7tqUHMD5DizJef0S2Qrj/EF2x4wblwlXY6kZMhCT//DPEOkY7GK+KRLZBvkeufTVCfompvRcpHDUrO4iN2A5mAsQD43EI8YZldjMnsVJVjhdgCvZqGbVUsdjSm25TFRBMRajbP29l4ZxE14rD885pUOw6vb871XcO5f/wxAzno2wftOL8hTfeAs/ZMyQPQVr51NmwpctUhBINSpSvHaSLFavmJdXeOgAZMz37t7ckVPa8lVaeGinVBaPo= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 08183c0c-ad3f-4ab4-5c59-08d817b8a1b5 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jun 2020 21:01:42.6544 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1VrqK//8yWLWoMA/ebHP2yZ7mYqz5raTJ7OA863yWW7MR2bzwbM5Skk5ovIBXyrl2irIvuOBDXEAiLvBgNBAAV4OKic06X67OdTu3V0iSJI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1510 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Phil, Comment in-line: > -----Original Message----- > From: Phil Yang > Sent: Friday, June 12, 2020 6:20 AM > To: dev@dpdk.org; Carrillo, Erik G > Cc: drc@linux.vnet.ibm.com; honnappa.nagarahalli@arm.com; > ruifeng.wang@arm.com; dharmik.thakkar@arm.com; nd@arm.com > Subject: [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag >=20 > The in_use flag is a per core variable which is not shared between lcores= in > the normal case and the access of this variable should be ordered on the > same core. However, if non-EAL thread pick the highest lcore to insert ti= mers > into, there is the possibility of conflicts on this flag between threads.= Then > the atomic CAS operation is needed. >=20 > Use the c11 atomic CAS instead of the generic rte_atomic operations to av= oid > the unnecessary barrier on aarch64. >=20 > Signed-off-by: Phil Yang > Reviewed-by: Dharmik Thakkar > Reviewed-by: Ruifeng Wang > --- > lib/librte_eventdev/rte_event_timer_adapter.c | 15 +++++++++++---- > 1 file changed, 11 insertions(+), 4 deletions(-) >=20 > diff --git a/lib/librte_eventdev/rte_event_timer_adapter.c > b/lib/librte_eventdev/rte_event_timer_adapter.c > index 6a0e283..6947efb 100644 > --- a/lib/librte_eventdev/rte_event_timer_adapter.c > +++ b/lib/librte_eventdev/rte_event_timer_adapter.c > @@ -554,7 +554,7 @@ struct swtim { > uint32_t timer_data_id; > /* Track which cores have actually armed a timer */ > struct { > - rte_atomic16_t v; > + int16_t v; > } __rte_cache_aligned in_use[RTE_MAX_LCORE]; > /* Track which cores' timer lists should be polled */ > unsigned int poll_lcores[RTE_MAX_LCORE]; @@ -606,7 +606,8 @@ > swtim_callback(struct rte_timer *tim) > "with immediate expiry value"); > } >=20 > - if (unlikely(rte_atomic16_test_and_set(&sw- > >in_use[lcore].v))) { > + if (unlikely(sw->in_use[lcore].v =3D=3D 0)) { > + sw->in_use[lcore].v =3D 1; > n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, > 1, > __ATOMIC_RELAXED); > __atomic_store_n(&sw->poll_lcores[n_lcores], > lcore, @@ -834,7 +835,7 @@ swtim_init(struct rte_event_timer_adapter > *adapter) >=20 > /* Initialize the variables that track in-use timer lists */ > for (i =3D 0; i < RTE_MAX_LCORE; i++) > - rte_atomic16_init(&sw->in_use[i].v); > + sw->in_use[i].v =3D 0; >=20 > /* Initialize the timer subsystem and allocate timer data instance */ > ret =3D rte_timer_subsystem_init(); > @@ -1017,6 +1018,8 @@ __swtim_arm_burst(const struct > rte_event_timer_adapter *adapter, > struct rte_timer *tim, *tims[nb_evtims]; > uint64_t cycles; > int n_lcores; > + /* Timer is not armed state */ A more accurate comment would be something like "Timer list for this lcore = is not in use". With that change, it looks good to me: Acked-by: Erik Gabriel Carrillo > + int16_t exp_state =3D 0; >=20 > #ifdef RTE_LIBRTE_EVENTDEV_DEBUG > /* Check that the service is running. */ @@ -1035,8 +1038,12 @@ > __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, > /* If this is the first time we're arming an event timer on this lcore, > * mark this lcore as "in use"; this will cause the service > * function to process the timer list that corresponds to this lcore. > + * The atomic CAS operation can prevent the race condition on > in_use > + * flag between multiple non-EAL threads. > */ > - if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { > + if (unlikely(__atomic_compare_exchange_n(&sw- > >in_use[lcore_id].v, > + &exp_state, 1, 0, > + __ATOMIC_RELAXED, __ATOMIC_RELAXED))) { > EVTIM_LOG_DBG("Adding lcore id =3D %u to list of lcores to > poll", > lcore_id); > n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, 1, > -- > 2.7.4