From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4EF34A04F1; Thu, 18 Jun 2020 17:17:51 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 14A701B952; Thu, 18 Jun 2020 17:17:50 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 9C3FB1B951; Thu, 18 Jun 2020 17:17:47 +0200 (CEST) IronPort-SDR: YJQBpaXlJRf+FVWeRQGkzaihEwTRrMvsjKV1H74m49XQtCeqsnqOpTLxZv/QY/5JpKAg73ogNd Yb28YNPjpmPA== X-IronPort-AV: E=McAfee;i="6000,8403,9655"; a="204104117" X-IronPort-AV: E=Sophos;i="5.75,251,1589266800"; d="scan'208";a="204104117" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jun 2020 08:17:46 -0700 IronPort-SDR: Xkw2GzgCreIGujFwqUfSn1VoIGKoXuBi78Vl9SKzY6cZf8zvJdAq3ITyTeKQ+xPtvrq1DvxSD7 SmnQ45MjLcYw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,251,1589266800"; d="scan'208";a="273887383" Received: from orsmsx107.amr.corp.intel.com ([10.22.240.5]) by orsmga003.jf.intel.com with ESMTP; 18 Jun 2020 08:17:46 -0700 Received: from orsmsx157.amr.corp.intel.com (10.22.240.23) by ORSMSX107.amr.corp.intel.com (10.22.240.5) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 08:17:46 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX157.amr.corp.intel.com (10.22.240.23) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 08:17:45 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 08:17:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aEnUpswmJuf95aY9MtDDQ3nidy8fDkaWkbll4gv8VoI/MAxIMcBwTAoSIuaXv6DSJLIURX54HI+V/R7IsFZ894wsCS0XdbaETMjsTUhE58hur34fB7Ocd4cw27nIcmoiMc9r7IQjLi9k0BgBBMdLZ+Et3bMrmd0i+i8nEAdVmIrv0vSkZTa4+ugwlLIkEoMOxRAgX5945pkoZZnC2wRxEEmrsp5OwyqfR38PZKpuCZwOA5oxNryyyb7LuahN8QhCtQolEEHq/5DqHNo2Z2VGFetdLcJxuApVHH42/fxBBHRGnTAYJnA8e7Ts4qzBrwgkpwnB6pMVJFwP/dSDNuUC2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RZX0iksFcTVmS0X0yEOBDaYbp3KpPR0VTLo1JlTJCvM=; b=TvE2CuHiGReqHWsXG8vwNPH2e/yQ18A00DxUXv/rD/b00sBJ9KJZd2YyFCu8SP/+oX7w8uyR9FJHJ68kUGY7vWwbTKyLcr23H8pWFc/Iks/sqL0wr0B1Xh8LG0mPPRRML5vuIVaEvUortgV1X9Wv7XNmCg6giFWL/F7l8GBfs9kWi3nD/3bBA5TnwDJ0KqPr4GK5wYdfWV7xxQdhZmfdstPyOLW0TlOks7S+PfR0N8AVqytowNSdlxDWp6T6YHQZvaXDtbsqWDgi1Gyp0YL2jTWymEsqrPi6wmPbJa+HrysKiwo5ZJt+RXvZ2Ht37L3KrgwgbTdp8Bpk8fzKEBfI+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RZX0iksFcTVmS0X0yEOBDaYbp3KpPR0VTLo1JlTJCvM=; b=b/A5cUF8x+8EK1mqiWzz61qRZNLs4jk4TxD9reQokGtrdw1CnS/fDhEkV3d0A6+MUbk1VaJu9qTie5WGHiiVChf0qj9T7NrKUKEaSzGfe8BrUVZj75s007V2/rZruH9BfUiDXKeb+2meIHK16lnJbssUgD5Wvo5SLQWN2OBmXY8= Received: from CY4PR1101MB2118.namprd11.prod.outlook.com (2603:10b6:910:1f::10) by CY4PR11MB1990.namprd11.prod.outlook.com (2603:10b6:903:25::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Thu, 18 Jun 2020 15:17:44 +0000 Received: from CY4PR1101MB2118.namprd11.prod.outlook.com ([fe80::410b:7807:181:1620]) by CY4PR1101MB2118.namprd11.prod.outlook.com ([fe80::410b:7807:181:1620%4]) with mapi id 15.20.3088.029; Thu, 18 Jun 2020 15:17:44 +0000 From: "Carrillo, Erik G" To: Phil Yang , "dev@dpdk.org" CC: "drc@linux.vnet.ibm.com" , "honnappa.nagarahalli@arm.com" , "ruifeng.wang@arm.com" , "dharmik.thakkar@arm.com" , "nd@arm.com" , "stable@dpdk.org" Thread-Topic: [PATCH 1/3] eventdev: fix race condition on timer list counter Thread-Index: AQHWQKt/5wmus6+dYkqGj9dsFfyoo6jehGaw Date: Thu, 18 Jun 2020 15:17:44 +0000 Message-ID: References: <1591960798-24024-1-git-send-email-phil.yang@arm.com> In-Reply-To: <1591960798-24024-1-git-send-email-phil.yang@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.201] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6c046bb0-15c4-4f2b-71d5-08d8139ac031 x-ms-traffictypediagnostic: CY4PR11MB1990: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3513; x-forefront-prvs: 0438F90F17 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YrjfTYJv3U5TCnk26H80MqNo7QiotgQ8AqwJDkvv8v99OzkTw6kfF59TtuAZJ42sfESg+7sNDitNjlb6pPb+Q5RMaV9yA9oomIwvlj3wuAcXfdaUaYpWg9EGR3Rc6esvmIPZQSbd+57acT216rpv/X/KVrpedg/tcXjHtpydfok+i3aPF4IP2tU2dfM//52Bb5ZxESKW9f/rwXTx7uXcuIa1BaSX7+Cyod4gis1BA6uosiykPWOqM6ZL0fxmf8//GbIvxOIGX1Ry283iEsviVmleo0nQAekRUJQbZ7VpCOq2xxFeuq88YMt/Hz97/v03Q5UjIk5rkNqz15PHqk7Jww== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR1101MB2118.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(376002)(39860400002)(396003)(136003)(366004)(346002)(76116006)(66946007)(66476007)(66556008)(64756008)(66446008)(7696005)(26005)(186003)(33656002)(54906003)(83380400001)(316002)(86362001)(6506007)(53546011)(8936002)(52536014)(8676002)(110136005)(5660300002)(2906002)(4326008)(55016002)(9686003)(478600001)(71200400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: wC5U1L/cnU45eVcir5ilSwK22MP+Vy3UPUFNWPi7ed+Y16OGdqT15N6+rE+UF8IdB7v20UQKZMohS6mwILsT8aw2rs3WuPRru+U23CJTDpxumiz/RxfcWjrwgKwZxnxOYYBHWf7PK/3kiJVO/yb/wA3aEuwYKxnoPXHvOd23K9JHVFk6PFco9geaWBACrwLo2QkgjtEQeAzqHUi54icBiTunC7BlLQ73s9S0zT6r09uXDKK44pKiCbMORoGgqnkvzT3FqrY0NnijgOltN3GwNGvUzez61K3QY8CswWC1EbUb/0ByWbpN3Aaub2P2x7yRykh331DZvUgZDKJw0UGCjmqBFechSzTEa3rQZjX57W4i1zQZtGT9g2oMWdzzF6sSV0fg+Mg9NgssSgRkzpOa1en4eK0qx2dBQ1HMsWQ1zcHHaXDovEBZh67K6h98Y401hydhABHbtDQ2GenWcq8R2rwatoAYU5CzvGToKT8OjzM= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 6c046bb0-15c4-4f2b-71d5-08d8139ac031 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jun 2020 15:17:44.1893 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Pr3QjAnoue+ygO6kmNRfngLlozELoaJ6K+52ujfeD0APQowkNwfPWxAyhtQuxC5ePrkllJ/QfSiW/OqqgZdwoU6z1z0IzqPSsmrZwWRhvlE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1990 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Phil, Good catch - thanks for the fix. I've commented in-line: > -----Original Message----- > From: Phil Yang > Sent: Friday, June 12, 2020 6:20 AM > To: dev@dpdk.org; Carrillo, Erik G > Cc: drc@linux.vnet.ibm.com; honnappa.nagarahalli@arm.com; > ruifeng.wang@arm.com; dharmik.thakkar@arm.com; nd@arm.com; > stable@dpdk.org > Subject: [PATCH 1/3] eventdev: fix race condition on timer list counter >=20 > The n_poll_lcores counter and poll_lcore array are shared between lcores > and the update of these variables are out of the protection of spinlock o= n > each lcore timer list. The read-modify-write operations of the counter ar= e > not atomic, so it has the potential of race condition between lcores. >=20 > Use c11 atomics with RELAXED ordering to prevent confliction. >=20 > Fixes: cc7b73ea9e3b ("eventdev: add new software timer adapter") > Cc: erik.g.carrillo@intel.com > Cc: stable@dpdk.org >=20 > Signed-off-by: Phil Yang > Reviewed-by: Dharmik Thakkar > Reviewed-by: Ruifeng Wang > --- > lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- > 1 file changed, 12 insertions(+), 4 deletions(-) >=20 > diff --git a/lib/librte_eventdev/rte_event_timer_adapter.c > b/lib/librte_eventdev/rte_event_timer_adapter.c > index 005459f..6a0e283 100644 > --- a/lib/librte_eventdev/rte_event_timer_adapter.c > +++ b/lib/librte_eventdev/rte_event_timer_adapter.c > @@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim) > uint16_t nb_evs_invalid =3D 0; > uint64_t opaque; > int ret; > + int n_lcores; >=20 > opaque =3D evtim->impl_opaque[1]; > adapter =3D (struct rte_event_timer_adapter *)(uintptr_t)opaque; > @@ -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) > "with immediate expiry value"); > } >=20 > - if (unlikely(rte_atomic16_test_and_set(&sw- > >in_use[lcore].v))) > - sw->poll_lcores[sw->n_poll_lcores++] =3D lcore; > + if (unlikely(rte_atomic16_test_and_set(&sw- > >in_use[lcore].v))) { > + n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, > 1, > + __ATOMIC_RELAXED); Just a nit, but let's align the continued line with the opening parentheses= in this location and below. With these changes: Acked-by: Erik Gabriel Carrillo > + __atomic_store_n(&sw->poll_lcores[n_lcores], > lcore, > + __ATOMIC_RELAXED); > + } > } else { > EVTIM_BUF_LOG_DBG("buffered an event timer expiry > event"); >=20 > @@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct > rte_event_timer_adapter *adapter, > uint32_t lcore_id =3D rte_lcore_id(); > struct rte_timer *tim, *tims[nb_evtims]; > uint64_t cycles; > + int n_lcores; >=20 > #ifdef RTE_LIBRTE_EVENTDEV_DEBUG > /* Check that the service is running. */ @@ -1033,8 +1039,10 @@ > __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, > if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { > EVTIM_LOG_DBG("Adding lcore id =3D %u to list of lcores to > poll", > lcore_id); > - sw->poll_lcores[sw->n_poll_lcores] =3D lcore_id; > - ++sw->n_poll_lcores; > + n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, 1, > + __ATOMIC_RELAXED); > + __atomic_store_n(&sw->poll_lcores[n_lcores], lcore_id, > + __ATOMIC_RELAXED); > } >=20 > ret =3D rte_mempool_get_bulk(sw->tim_pool, (void **)tims, > -- > 2.7.4