From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31769A0519; Mon, 22 Jun 2020 11:48:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 08F021D61B; Mon, 22 Jun 2020 11:48:21 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140075.outbound.protection.outlook.com [40.107.14.75]) by dpdk.org (Postfix) with ESMTP id 795311D5A4; Mon, 22 Jun 2020 11:48:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jqhp1C+QLd72Bnj2eXPyd/3RiGG4cbjC6YRifnfW14Y=; b=o80CGbPsmLmBXdykkXYV5aJLTyWJKEWi2pfhLVOQO/+RCwPZjHctuEd+sa+tAEW3oernHXAR4C5yhVEnln5tVxeHsrAgBXnwWxkKAGwiDFEInR8NPPIBM7DYCSuK2INx/ZGg//kZjK5oV7VJefF/dkf3+WtTCz8wpxXZmAw08ow= Received: from DB6PR0201CA0033.eurprd02.prod.outlook.com (2603:10a6:4:3f::43) by AM6PR08MB4868.eurprd08.prod.outlook.com (2603:10a6:20b:c5::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.23; Mon, 22 Jun 2020 09:48:17 +0000 Received: from DB5EUR03FT064.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:3f:cafe::2b) by DB6PR0201CA0033.outlook.office365.com (2603:10a6:4:3f::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21 via Frontend Transport; Mon, 22 Jun 2020 09:48:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT064.mail.protection.outlook.com (10.152.21.199) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Mon, 22 Jun 2020 09:48:17 +0000 Received: ("Tessian outbound 8fb20e43acb7:v59"); Mon, 22 Jun 2020 09:48:17 +0000 X-CR-MTA-TID: 64aa7808 Received: from 6ec13e9cb071.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C9367247-F8D6-4310-A51E-02E536DD1974.1; Mon, 22 Jun 2020 09:48:12 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6ec13e9cb071.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 22 Jun 2020 09:48:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KS3OvzOUmAxvjN2iTjKaRvA4pACrbI9YPhJTulFWeVAQZQmj7U3SIvSZ7gH1bYtgg1h3AOZuwU25qkeJQev2cM4Ju7JEERuF8Pdz9OeHhxoioFr5t3T/IMt1ILatNlNlwTm+I4UcuFitfA0jmyixsx2/lizdYFmPnC/r/kHiQjVyJ5JV5zNQI4WzdTJnsmhAA3J9J1I150nB3bheFazyf+XQ0X5kzkcnxq+TKMlCyZEdQZ16U+cq2IJtaUrvLLrnqAQJVwxczkW8e1KkLknPtXS67qKDX7bFpCYn0xjOX29blIdffJVr/m8l50kpt3h0OjrYMXs45fjgQYklx+L2rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jqhp1C+QLd72Bnj2eXPyd/3RiGG4cbjC6YRifnfW14Y=; b=j6N+uNxOqmr90LwOS0GohXqM+DicxhufDnP+xTeO0pKKJi7i6EcAAGubJt6QW0dHltjOXhXGwU4X9r59Xbb2yiScdsLdi+HrZzQE2R51saSQjBYaTPj9RyW78QHu1ZQpw1lyXQjcZL9nMcy09dZ/24yAH42+Zx/2VcsU58ys2hYUmppUjuojUeBvTqJ//4tuPWQXPT/+wuJ89IMKU8WFVPp2+sT84bitpxt76p+ja2CgD7Fmf6PiEr2o5b5dHLm1tVmU1YWvuT5WCxnZkIwgs+/SatFAUhoqAZiR6GiZVnMx2IkBpEQ3LqBBgJPIXwnAlCw+/I8+kY1T/eNlWLny6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jqhp1C+QLd72Bnj2eXPyd/3RiGG4cbjC6YRifnfW14Y=; b=o80CGbPsmLmBXdykkXYV5aJLTyWJKEWi2pfhLVOQO/+RCwPZjHctuEd+sa+tAEW3oernHXAR4C5yhVEnln5tVxeHsrAgBXnwWxkKAGwiDFEInR8NPPIBM7DYCSuK2INx/ZGg//kZjK5oV7VJefF/dkf3+WtTCz8wpxXZmAw08ow= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (2603:10a6:802:b2::11) by VE1PR08MB4637.eurprd08.prod.outlook.com (2603:10a6:802:a9::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Mon, 22 Jun 2020 09:48:10 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::c2e:9ccb:a690:6863]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::c2e:9ccb:a690:6863%6]) with mapi id 15.20.3109.027; Mon, 22 Jun 2020 09:48:10 +0000 From: Phil Yang To: Honnappa Nagarahalli , "Carrillo, Erik G" , "dev@dpdk.org" CC: "drc@linux.vnet.ibm.com" , Ruifeng Wang , Dharmik Thakkar , nd , "stable@dpdk.org" , nd Thread-Topic: [PATCH 1/3] eventdev: fix race condition on timer list counter Thread-Index: AQHWRYO0C4+OtcPfIEys7qAlWeZDrqjesL6AgAW3t8A= Date: Mon, 22 Jun 2020 09:48:10 +0000 Message-ID: References: <1591960798-24024-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 71b8db82-6495-433f-9b17-d3e1433807e7.0 x-checkrecipientchecked: true Authentication-Results-Original: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [203.126.0.112] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: d9e7fc80-c174-4a57-359d-08d8169163b6 x-ms-traffictypediagnostic: VE1PR08MB4637:|AM6PR08MB4868: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:632;OLM:632; x-forefront-prvs: 0442E569BC X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: p+vBWY4iO7aJjUHVxaVJftO2aeb+G40oAVu7aKxe/yUr1tUJwZlKdq9zN+BpNgpfZ5v3qXuGwkv32JtJthwptfbNGoUp/G8SUzGBbeV4YCFt7UB08Snb7R+DEkcTZ4qihlmPhwUma/h6p5JlviEUTIc56sfZM+NLereR0ZC9W1XYkGLNcK/TqyVet3tEf5JYlifkbim/2o0AzeOCjwHBi61e05gOg2MAwO2XJS/jKBhCwrPxU3dApIljoBYvyjFbtnUr4aFCi4g0NZcexzKr5Yoca8tgmxiGdB4OomBfzkWILBxJrz/dJp5t7TF84BCAAmKqv/VwnKfwWqFa60bAJA== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VE1PR08MB4640.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(366004)(346002)(376002)(396003)(136003)(64756008)(53546011)(66446008)(6506007)(4326008)(66476007)(71200400001)(33656002)(316002)(7696005)(110136005)(478600001)(54906003)(83380400001)(5660300002)(55016002)(9686003)(52536014)(8936002)(26005)(66556008)(66946007)(2906002)(186003)(8676002)(76116006)(86362001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: TDCDDPqycgsWx/5mhNJocxwW8oFByESHRgn6GdLKBy8Z5dc3j6vQcvBg4QZdGcvdfXw4uQrH/dltLXoFdOzpi+Mhz1JJFUKSxKoDUtM8ncV86dGqcpF7k5/JcpxhhBombq/JGPlR4cmQ5d1PziWtHVMbpHYJu9ePDunGWEaPnGQ2Am/Y64MF2i/BjAi76IbKzk90NeH7ZeXSUEEodWV2sLcQPK7F1FbT8HvzxTwJEFw2a4vEw1hUEAjJwkHkv0I2RSQRdsWIyJoza3uH90ioRsLkJ+Ilsf/qkGIqf6Urdy8RKLPM0iDkExLg9FpUrEdCwvpzDA9B4y9mPPZl14kiYdRJAkxpkkCdKobjV7+xewTgHFxQU9nXgMSvfQmKpu79Pgmvraws93Dz5EAIlUgO3yfGjU6Rd9KYK06rg4r+D2BNTR3y0kP+17tiKlEpPXO5k4jFEpTLRvMTUKGN5kZAtmov3sFWPW1q1IOGEL5+eD8= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4637 Original-Authentication-Results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT064.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(396003)(376002)(136003)(46966005)(53546011)(6506007)(7696005)(83380400001)(82310400002)(33656002)(26005)(86362001)(2906002)(81166007)(336012)(52536014)(8936002)(8676002)(82740400003)(356005)(478600001)(47076004)(5660300002)(70586007)(110136005)(70206006)(186003)(450100002)(316002)(4326008)(55016002)(9686003)(54906003); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 6563d170-1df0-45f3-9b8b-08d816915fb6 X-Forefront-PRVS: 0442E569BC X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZO1n2Q6LasqbgGm4Fui3mGmEel5M+kJ2GO/9oMNULcFk9BcxRuLk9XzVxHxiIozvUqouPbKWnS+giubJKSTdkFeV7veZ5TJqsjwqp5h/wfKYhCR1dPCP90gQGjzpNlVsAPvwXBcd6mgd5j7iOfetLqDPO7Or21oEgrGZfyyQ5K8Rued9c+tEziqN5lTN1fKKZTevk0r3Lu4kPWgwTP134ioplGkXiXSgPvzoDgcu6y1aB8YwRqa96G7ZBGpMfeNoup2RotemKsLxgfwzImLLQyXTNaYix5oYMuPYZE167TMHPTJK6uQTu0g6Y93WCKVN8bWYvbZocXAhU5jatP9MMBKuuSyb8hhPz5YYNlAZcL3cfQNrNNgOgN6YI2MX+jHmEp+YfsOhcc2HnmXWIbGWUQ== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2020 09:48:17.1789 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d9e7fc80-c174-4a57-359d-08d8169163b6 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4868 Subject: Re: [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Honnappa Nagarahalli > Sent: Friday, June 19, 2020 2:26 AM > To: Carrillo, Erik G ; Phil Yang > ; dev@dpdk.org > Cc: drc@linux.vnet.ibm.com; Ruifeng Wang ; > Dharmik Thakkar ; nd ; > stable@dpdk.org; Honnappa Nagarahalli ; > nd > Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list count= er >=20 > >=20 > > > > Hi Phil, > > > > Good catch - thanks for the fix. I've commented in-line: > > > > > -----Original Message----- > > > From: Phil Yang > > > Sent: Friday, June 12, 2020 6:20 AM > > > To: dev@dpdk.org; Carrillo, Erik G > > > Cc: drc@linux.vnet.ibm.com; honnappa.nagarahalli@arm.com; > > > ruifeng.wang@arm.com; dharmik.thakkar@arm.com; nd@arm.com; > > > stable@dpdk.org > > > Subject: [PATCH 1/3] eventdev: fix race condition on timer list > > > counter > > > > > > The n_poll_lcores counter and poll_lcore array are shared between > > > lcores and the update of these variables are out of the protection of > > > spinlock on each lcore timer list. The read-modify-write operations o= f > > > the counter are not atomic, so it has the potential of race condition > > between lcores. > > > > > > Use c11 atomics with RELAXED ordering to prevent confliction. > > > > > > Fixes: cc7b73ea9e3b ("eventdev: add new software timer adapter") > > > Cc: erik.g.carrillo@intel.com > > > Cc: stable@dpdk.org > > > > > > Signed-off-by: Phil Yang > > > Reviewed-by: Dharmik Thakkar > > > Reviewed-by: Ruifeng Wang > > > --- > > > lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++---- > > > 1 file changed, 12 insertions(+), 4 deletions(-) > > > > > > diff --git a/lib/librte_eventdev/rte_event_timer_adapter.c > > > b/lib/librte_eventdev/rte_event_timer_adapter.c > > > index 005459f..6a0e283 100644 > > > --- a/lib/librte_eventdev/rte_event_timer_adapter.c > > > +++ b/lib/librte_eventdev/rte_event_timer_adapter.c > > > @@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim) > > > uint16_t nb_evs_invalid =3D 0; > > > uint64_t opaque; > > > int ret; > > > +int n_lcores; > > > > > > opaque =3D evtim->impl_opaque[1]; > > > adapter =3D (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@ > > > -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim) > > > "with immediate expiry value"); > > > } > > > > > > -if (unlikely(rte_atomic16_test_and_set(&sw- > > > >in_use[lcore].v))) > > > -sw->poll_lcores[sw->n_poll_lcores++] =3D lcore; > > > +if (unlikely(rte_atomic16_test_and_set(&sw- > > > >in_use[lcore].v))) { > > > +n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, > > > 1, > > > +__ATOMIC_RELAXED); > Since this commit will be back ported, we should prefer to use rte_atomic > APIs for this commit. Otherwise, we will have a mix of rte_atomic and C11 > APIs. > My suggestion is to fix this bug using rte_atomic so that backported code= will > have only rte_atomic APIs. Add another commit (if required) in this serie= s to > make the bug fix use C11 APIs (this commit will not be backported). Agree.=20 I will change this patch to the rte_atomic version in the next version. Thanks, Phil >=20 > > > > Just a nit, but let's align the continued line with the opening parenth= eses in > > this location and below. With these changes: > > > > Acked-by: Erik Gabriel Carrillo > > > > > +__atomic_store_n(&sw->poll_lcores[n_lcores], > > > lcore, > > > +__ATOMIC_RELAXED); > > > +} > > > } else { > > > EVTIM_BUF_LOG_DBG("buffered an event timer expiry event"); > > > > > > @@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct > > > rte_event_timer_adapter *adapter, > > > uint32_t lcore_id =3D rte_lcore_id(); > > > struct rte_timer *tim, *tims[nb_evtims]; > > > uint64_t cycles; > > > +int n_lcores; > > > > > > #ifdef RTE_LIBRTE_EVENTDEV_DEBUG > > > /* Check that the service is running. */ @@ -1033,8 +1039,10 @@ > > > __swtim_arm_burst(const struct rte_event_timer_adapter *adapter, > > > if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) { > > > EVTIM_LOG_DBG("Adding lcore id =3D %u to list of lcores to > > poll", > > > lcore_id); > > > -sw->poll_lcores[sw->n_poll_lcores] =3D lcore_id; > > > -++sw->n_poll_lcores; > > > +n_lcores =3D __atomic_fetch_add(&sw->n_poll_lcores, 1, > > > +__ATOMIC_RELAXED); > > > +__atomic_store_n(&sw->poll_lcores[n_lcores], lcore_id, > > > +__ATOMIC_RELAXED); > > > } > > > > > > ret =3D rte_mempool_get_bulk(sw->tim_pool, (void **)tims, > > > -- > > > 2.7.4 >=20