From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A72B9A04F1; Thu, 18 Jun 2020 20:26:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 895F61BFEE; Thu, 18 Jun 2020 20:25:58 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2050.outbound.protection.outlook.com [40.107.22.50]) by dpdk.org (Postfix) with ESMTP id 9DE411BFD1; Thu, 18 Jun 2020 20:25:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mfn1msKplMnxrPR4yb8IaAWJHkoWVC4a2g2PsWX+E3I=; b=iRWKdJ592Dk8Z8fUaPFwor7mIDXzfi/dAtTz8lbfEQvSlFgnphenx4v+8+kODV3JcC+AloFxspzhkWV13iszonxwSuspGoWZAiQyJVjFf+R8k3+OU/jud+uz5A22wSvvZEEo0nh1oPNtj5YlkFr/ulDnKynRQbHXAMrIzM74ujM= Received: from AM6P193CA0093.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:88::34) by DB7PR08MB3868.eurprd08.prod.outlook.com (2603:10a6:10:32::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Thu, 18 Jun 2020 18:25:55 +0000 Received: from AM5EUR03FT053.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:88:cafe::cb) by AM6P193CA0093.outlook.office365.com (2603:10a6:209:88::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Thu, 18 Jun 2020 18:25:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT053.mail.protection.outlook.com (10.152.16.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Thu, 18 Jun 2020 18:25:54 +0000 Received: ("Tessian outbound cdcc6148d2f4:v59"); Thu, 18 Jun 2020 18:25:54 +0000 X-CR-MTA-TID: 64aa7808 Received: from decf331abd3c.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 1FF8F1C2-D1CB-4D09-A15D-8AD13F03FB83.1; Thu, 18 Jun 2020 18:25:49 +0000 Received: from EUR02-AM5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id decf331abd3c.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 18 Jun 2020 18:25:49 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HOKKXABEsq4KOP6/Ih6FyKLpZX2ti+xQr8itiydX0Ex03mlOa99MY6cV0Lov6lFcKoEHWqGGf7BS0uNV4mnKzA4cyxLpbnC37hddS8bBrubuTmEr/NB7I+ixjNQ71Ok5OQCwPWyh36nsfLc2XtmSekP6V2aJWVUF1jsPFBHgqoUxmtLyN7u9SV8+mnmJS8Z9qCgv7pe3PHHmQHfS8ms1lv0Nyt24Ff6P8k5T7NGuyqdYwGvgy32j8nuRz5yRNNBL0/JJHXgvLTPN8nyYZVS+J75QgrFOtdnDJ/rxupZ5vfMqxRCZ++fBgi0g+amF/QUx/CcozYrAJKB48PoZXmA3qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mfn1msKplMnxrPR4yb8IaAWJHkoWVC4a2g2PsWX+E3I=; b=YPgcs88fLdsY9HbbTWvwNnPZW948nLrFiDXPVlt3xerrnuGosF87AMbvLPGxXHf1JqxGw5LGGB5X1VdwHXK1FueMySetka3dQZPkaiw0eZeYQF/IcF61Q9wRYOqqKAqQOPNEhA/51/18ygCf2qH0wXVNjUnqcj6rW2/Af2nlGmGeUZeRH68UXAgS3TdauA08+bICgDwsukXEGAqLZXn0EN1GA24Z/69kQiHAuDC8V4+yW5MSrZIWMykWDTu0wwzKe0MxOkjso0lo+bM89EQv6EN3W9/DHJmXGVmbqACmSTtLvyrpW+KqwHLIimYAUoHYOoScu9ISVeenQgBUpjaxxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Mfn1msKplMnxrPR4yb8IaAWJHkoWVC4a2g2PsWX+E3I=; b=iRWKdJ592Dk8Z8fUaPFwor7mIDXzfi/dAtTz8lbfEQvSlFgnphenx4v+8+kODV3JcC+AloFxspzhkWV13iszonxwSuspGoWZAiQyJVjFf+R8k3+OU/jud+uz5A22wSvvZEEo0nh1oPNtj5YlkFr/ulDnKynRQbHXAMrIzM74ujM= Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com (2603:10a6:4:85::9) by DB6PR0802MB2565.eurprd08.prod.outlook.com (2603:10a6:4:a1::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Thu, 18 Jun 2020 18:25:48 +0000 Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::1128:b7e7:e832:310f]) by DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::1128:b7e7:e832:310f%9]) with mapi id 15.20.3109.021; Thu, 18 Jun 2020 18:25:48 +0000 From: Honnappa Nagarahalli To: "Carrillo, Erik G" , Phil Yang , "dev@dpdk.org" CC: "drc@linux.vnet.ibm.com" , Ruifeng Wang , Dharmik Thakkar , nd , "stable@dpdk.org" , Honnappa Nagarahalli , nd Thread-Topic: [PATCH 1/3] eventdev: fix race condition on timer list counter Thread-Index: AQHWQKuA3yJ7DVsy8kmXELnLFbhrvKjeheQAgAABwBA= Date: Thu, 18 Jun 2020 18:25:47 +0000 Message-ID: References: <1591960798-24024-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 8bd133b3-a796-4c06-8cff-51e5a7941047.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 7c4a6057-cbac-4222-72bd-08d813b509f8 x-ms-traffictypediagnostic: DB6PR0802MB2565:|DB7PR08MB3868: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:431;OLM:431; x-forefront-prvs: 0438F90F17 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: sIKSuTLnuypWEMoGKYcIaKLLlrPaE49bMcmMxInk1cL1O1Snyn2auPvIMhY+2yhKkAVTSDhrBq/WFl45AigCR7W5bEgnb01iu+AuZ8ePD9f1zW6jPDCClqx3O7u4vuqiSK4bNvB1NQ8PbC/Qz7G8Cpf0keQeFO1nqZhBzpTudqjTdh2bJqjYXN5OhYuWELXszuQYmrfa3dggEOf5dLrmqiMki/422kGtedQJBh+Fg4Yp2jM2LbDpjUBhb4eMhsN6xxuQJu6mJwOPqBDiqTA2pjDAnA/c+MqEr/6JkJTggRos86tE6BBxeiKQDuTc9h2yR6B0nuo/3eXPPqKcIvjDoA== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB6PR0802MB2216.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(376002)(366004)(39860400002)(346002)(136003)(110136005)(4326008)(83380400001)(66476007)(66556008)(64756008)(66946007)(76116006)(86362001)(66446008)(9686003)(8936002)(55016002)(54906003)(478600001)(2906002)(8676002)(316002)(26005)(186003)(6506007)(53546011)(7696005)(5660300002)(52536014)(71200400001)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: P6GcjjA0t4YrRzGwaTbLFiAr1qNaWUZMO2rjIxy6GnGnhNu+umDT40t9rmQq3KbmpXKdxsy7kyEoMM+Q7NCDx6IAKN8loZ0dg9EFZwSk+KxbihASrwpuuql/0EPloFTm0xpFu0GdwXkyqq70kLL5ytswKeylEb9hmBYkUgGF9uL2FhhISllylqTkqgwfToo8jp8Shl4gxKOp4jWKo5WmJ0zcGiXs3Cvv3aIbdLimVX2OYiTiGBkfuW11KykMxnJwM1UpuW+NRK6o4/we5F8V22jNb4PqSBxJTT4T6Ym6MccWhdTRFkeUB8fZMyjo+7I3JY+3pY9N+g4wjOvSUnrzjnuQb4LHUFOnRc2j4uTz79AFi/5RpnFGcCqWBWO0cJHSQaOSlK7AKHd7aaLN6bx2zUNOquVN0QSS85SsXPGIED/1Svva0rwl98dzH1rNsXZv4metjawgTrG8Q4g1jXec+zs2x3JvUMQ+Ur93pRHYnIg= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2565 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT053.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(396003)(376002)(346002)(46966005)(336012)(33656002)(4326008)(2906002)(8676002)(52536014)(5660300002)(450100002)(478600001)(8936002)(9686003)(55016002)(186003)(26005)(83380400001)(70586007)(70206006)(36906005)(7696005)(6506007)(53546011)(54906003)(86362001)(316002)(110136005)(82740400003)(47076004)(356005)(81166007)(82310400002); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 59388e84-658b-449e-cb95-08d813b505c9 X-Forefront-PRVS: 0438F90F17 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J9YtfcecMVJApLsWNJmaKGA4j8Bf/HS7CDviW0GMK+MB6S7ly2xrdiTGUGpcZ4F7drGWT3r0g21WgVuIo9rSSEdQFJJRS9EE1GLZF64ui51DQXUg+BttpTc/56WO7znb+/2dKpE+CFqHIozK//SYriBYIg8c5ROaInO//4aUenZNIavD0Jwu2i360bYIQoZAaTfYo9yfgKqJaycTnliTUaCeE9LbRMpBa4+UP1u+kZrCiWAD4nqooelHQyU6QC2lTFePdssxpp0n0L6dmS6YZbEg53VN3njPkru2LdBwIu554tlBR82XBRXW1B7UH5fEmO/0vaWEv+Aex1jUhrib9Geh/0yF+feXrhRp0arph0AEfowNJX6/rXOWbRuRFHmXYkE207bsXEYopoBnpgHo8A== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jun 2020 18:25:54.9381 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c4a6057-cbac-4222-72bd-08d813b509f8 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3868 Subject: Re: [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > Hi Phil, >=20 > Good catch - thanks for the fix. I've commented in-line: >=20 > > -----Original Message----- > > From: Phil Yang > > Sent: Friday, June 12, 2020 6:20 AM > > To: dev@dpdk.org; Carrillo, Erik G > > Cc: drc@linux.vnet.ibm.com; honnappa.nagarahalli@arm.com; > > ruifeng.wang@arm.com; dharmik.thakkar@arm.com; nd@arm.com; > > stable@dpdk.org > > Subject: [PATCH 1/3] eventdev: fix race condition on timer list > > counter > > > > The n_poll_lcores counter and poll_lcore array are shared between > > lcores and the update of these variables are out of the protection of > > spinlock on each lcore timer list. The read-modify-write operations of > > the counter are not atomic, so it has the potential of race condition > between lcores. > > > > Use c11 atomics with RELAXED ordering to prevent confliction. > > > > Fixes: cc7b73ea9e3b ("eventdev: add new software timer adapter") > > Cc: erik.g.carrillo@intel.com > > Cc: stable@dpdk.org > > > > Signed-off-by: Phil Yang > > Reviewed-by: Dharmik Thakkar > > Reviewed-by: Ruifeng Wang > > --- > > lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- > > 1 file changed, 12 insertions(+), 4 deletions(-) > > > > diff --git a/lib/librte_eventdev/rte_event_timer_adapter.c > > b/lib/librte_eventdev/rte_event_timer_adapter.c > > index 005459f..6a0e283 100644 > > --- a/lib/librte_eventdev/rte_event_timer_adapter.c > > +++ b/lib/librte_eventdev/rte_event_timer_adapter.c > > @@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim) > > uint16_t nb_evs_invalid =3D 0; > > uint64_t opaque; > > int ret; > > + int n_lcores; > > > > opaque =3D evtim->impl_opaque[1]; > > adapter =3D (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ > > -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) > > "with immediate expiry value"); > > } > > > > - if (unlikely(rte_atomic16_test_and_set(&sw- > > >in_use[lcore].v))) > > - sw->poll_lcores[sw->n_poll_lcores++] =3D lcore; > > + if (unlikely(rte_atomic16_test_and_set(&sw- > > >in_use[lcore].v))) { > > + n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, > > 1, > > + __ATOMIC_RELAXED); Since this commit will be back ported, we should prefer to use rte_atomic A= PIs for this commit. Otherwise, we will have a mix of rte_atomic and C11 AP= Is. My suggestion is to fix this bug using rte_atomic so that backported code w= ill have only rte_atomic APIs. Add another commit (if required) in this ser= ies to make the bug fix use C11 APIs (this commit will not be backported). >=20 > Just a nit, but let's align the continued line with the opening parenthes= es in > this location and below. With these changes: >=20 > Acked-by: Erik Gabriel Carrillo >=20 > > + __atomic_store_n(&sw->poll_lcores[n_lcores], > > lcore, > > + __ATOMIC_RELAXED); > > + } > > } else { > > EVTIM_BUF_LOG_DBG("buffered an event timer expiry event"); > > > > @@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct > > rte_event_timer_adapter *adapter, > > uint32_t lcore_id =3D rte_lcore_id(); > > struct rte_timer *tim, *tims[nb_evtims]; > > uint64_t cycles; > > + int n_lcores; > > > > #ifdef RTE_LIBRTE_EVENTDEV_DEBUG > > /* Check that the service is running. */ @@ -1033,8 +1039,10 @@ > > __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, > > if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { > > EVTIM_LOG_DBG("Adding lcore id =3D %u to list of lcores to > poll", > > lcore_id); > > - sw->poll_lcores[sw->n_poll_lcores] =3D lcore_id; > > - ++sw->n_poll_lcores; > > + n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, 1, > > + __ATOMIC_RELAXED); > > + __atomic_store_n(&sw->poll_lcores[n_lcores], lcore_id, > > + __ATOMIC_RELAXED); > > } > > > > ret =3D rte_mempool_get_bulk(sw->tim_pool, (void **)tims, > > -- > > 2.7.4