From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B202CA04C3; Fri, 22 Nov 2019 17:07:44 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 338F72BF5; Fri, 22 Nov 2019 17:07:44 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 925EE2BAF; Fri, 22 Nov 2019 17:07:42 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xAMG6YdD001103; Fri, 22 Nov 2019 08:07:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=boGZcOtxplPJbB9aRrnRMoXWJnOD3A8WccneyE/NVHU=; b=M92j4FcvqkB9IzAe4uUApaO8QenIJSzj0zCAd7MJn+nIgeV+VvbNqKgugaBksLqascJN CxrgVZ7MQu8iEX5r7/68bW8M9ZrAadtVl3ykgTDEuxRzQdfFotrYJJ4QAgcPcAMd5OZi uQi2tZAa5hEgtVJ5/hL8tL+3rBDxjLId6iykxyixNTCCCa6O1IvZkgqyVhrnfKumUy6e 7MBOwREN96jadlzhuIRnR5ceGIWTlaxbHIehiybGI43sA2vkP0PgKQGxkjU1NcEAdi8P iTDKGVDX1QlTiEGelUV4Tu/VjJuhYFFrHGwBNrR4vhc4pIAFtIMnZuf7SFGumIxwO8Gk lw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2wearf1yvc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 22 Nov 2019 08:07:41 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 22 Nov 2019 08:07:40 -0800 Received: from NAM05-CO1-obe.outbound.protection.outlook.com (104.47.48.56) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Fri, 22 Nov 2019 08:07:40 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=el78Zu3k8+DcP/JAsJ1NtgKXNX+vFAv/OcucdwFBHMkjSlNQoO6vcNzyELyLkykym47/gVWpJaVdYOWkTZZrat6YFNzmbcdPARv0YjsieuPAUjb/72W/RhUrO/KU/DZP7hai3xC0/Wd4qQvMH8RplU6LcmHnSnoxbjIuNJjdZVJmh10110Ojg9MDI57rNLu3Nh3U/eEJXxNNx3wBQ23UCrLX1Y4u12yvQN2hiPLLWuMWo/Sv0wexVFfC33h9MwtiurtJyHw+Tbsc6Cg+th5nxNflLKN76+M8QMZdG5MPG0UU7BUvE7Antw4DMtX4VgW7oXHhxKsSNjDe2nkKS7hIeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=boGZcOtxplPJbB9aRrnRMoXWJnOD3A8WccneyE/NVHU=; b=jEfoGazapyr/WNrboX5inrWOi1ZJEA2Ae9mwypimTwzOtwTjxtMYMSnLR81zzUSr417klsquhCFhoWsmlfl9MeamlIGLGyDKRE3qdKq0tSGuJeosVqj61nq47SCI1nTImEixlDjmglbvu3GFrj16T7aiVzYHSdwVvqvuqYcYmK3AmF1PN1J+fsHCE6alyC5lOl82QHoLxWBQ0XN47KpxajuTfnp1oYESkxe0m9luldAogkI1OTreiOfUm71YOcQYrOOzokrgHOEQLiSCdcDJwImU8aH7n/qLwDQc073nyWdUQDPPb5eXrj8MgkpaBwP+4G74qRE1yqxjO0/larzOAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=boGZcOtxplPJbB9aRrnRMoXWJnOD3A8WccneyE/NVHU=; b=Wsq+Vjwyd/pn6nG0PFvjS+sKGZBIDlt4ZM4mRqJNj6+GptPTR8nu4MThY3jA5yM8Ql2JrgBqTonGmq7yL0qMQz6f/omN0jEwcfVoubbiwhWNoCay5EfHve3R3VG1KFrT0v058FiQRiAyxStfCtOcO82iNFuDRVTjjAC74iSjfOQ= Received: from CY4PR1801MB1863.namprd18.prod.outlook.com (10.171.255.160) by CY4PR1801MB1974.namprd18.prod.outlook.com (10.171.255.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.29; Fri, 22 Nov 2019 16:07:38 +0000 Received: from CY4PR1801MB1863.namprd18.prod.outlook.com ([fe80::d918:c54e:c3b1:3167]) by CY4PR1801MB1863.namprd18.prod.outlook.com ([fe80::d918:c54e:c3b1:3167%7]) with mapi id 15.20.2474.021; Fri, 22 Nov 2019 16:07:38 +0000 From: Pavan Nikhilesh Bhagavatula To: Pavan Nikhilesh Bhagavatula , "Jerin Jacob Kollanukkaran" CC: "dev@dpdk.org" , dpdk stable Thread-Topic: [dpdk-dev] [PATCH v3 1/5] event/octeontx2: fix TIM HW race condition Thread-Index: AQHVoUu/DjjHQ27MtUKMkMI+bnVvLqeXWxXQ Date: Fri, 22 Nov 2019 16:07:37 +0000 Message-ID: References: <20191122154431.17416-1-pbhagavatula@marvell.com> In-Reply-To: <20191122154431.17416-1-pbhagavatula@marvell.com> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [122.172.61.233] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4843fc4f-a803-4df5-c631-08d76f661854 x-ms-traffictypediagnostic: CY4PR1801MB1974: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1002; x-forefront-prvs: 02296943FF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(136003)(376002)(346002)(39860400002)(396003)(13464003)(189003)(199004)(74316002)(25786009)(7736002)(71190400001)(66446008)(14454004)(446003)(66946007)(66066001)(64756008)(66556008)(86362001)(81156014)(6436002)(81166006)(110136005)(9686003)(3846002)(7696005)(450100002)(8936002)(5660300002)(316002)(186003)(6116002)(55016002)(4326008)(52536014)(66476007)(102836004)(305945005)(54906003)(6246003)(6636002)(2906002)(76116006)(76176011)(478600001)(33656002)(229853002)(6506007)(99286004)(71200400001)(256004)(26005)(14444005)(8676002)(11346002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR1801MB1974; H:CY4PR1801MB1863.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qTx9BAM0Dkfd1MirXpYKMBMZlY9D589/OcZSqtXlcxx6i6aIyMmdx9IXd3wtiUnZ+ikCP35bXd5LPXQvmV6Qy7S/ZlV1j7+VxUOnJzJcUa3peTZGPjUjZhEgh19IpGjEfgpFlRvHnkkQWHCwaeGdkgKUK3NQWM5+LJMgnsfdn5u/3ZjsJdYAwRVYiUlWIXiWYLTvBjMy22gAkFV0nnsY4Jr0u6S5W++wEXQIATGDVESozWOaOnKhPhilDK8hqc6rXN8s1CTi967zThAHLHoC+EqG3wiTlL+7VJx4rZmy5Qofapwoi1mt4IlY3G1QdFS0rIrNrrMyPXNH9PbwhmaRSlpym2jrLd8EJElYyue9/4vZ36NgBgWVATuJRT8l5qMDO58TwoO6cj9fDL35LUjmVtXovNiI7ZHgj6ryIwAgYSczeWFDrjqxNq08ZFUxTcGL Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 4843fc4f-a803-4df5-c631-08d76f661854 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Nov 2019 16:07:38.0063 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ezudi+JEbJvFSewh06mZE6Zw/GxSBH4cEFiENvsCtgxSloQYlFps7FCAYRhlrIv12+ZmwbzXlnEDfFfPFtD0x813u9JqkxdthgbuGKEljD4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1801MB1974 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-11-22_03:2019-11-21,2019-11-22 signatures=0 Subject: Re: [dpdk-dev] [PATCH v3 1/5] event/octeontx2: fix TIM HW race condition X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" +Cc: stable@dpdk.org >-----Original Message----- >From: pbhagavatula@marvell.com >Sent: Friday, November 22, 2019 9:14 PM >To: Jerin Jacob Kollanukkaran ; Pavan Nikhilesh >Bhagavatula >Cc: dev@dpdk.org >Subject: [dpdk-dev] [PATCH v3 1/5] event/octeontx2: fix TIM HW race >condition > >From: Pavan Nikhilesh > >Fix HW race condition observed when timeout resolution is low (<5us). >When HW traverses a given TIM bucket it will clear chunk_remainder, >but since SW always decreases the chunk_remainder at the start of the >arm routine it might cause a race where SW updates chunk_remainder >after HW has cleared it that lead to nasty side effects. > >Fixes: 95e4e4ec7469 ("event/octeontx2: add timer arm timeout burst") > >Signed-off-by: Pavan Nikhilesh >--- > drivers/event/octeontx2/otx2_tim_worker.h | 141 >+++++++++++++++++++--- > 1 file changed, 124 insertions(+), 17 deletions(-) > >diff --git a/drivers/event/octeontx2/otx2_tim_worker.h >b/drivers/event/octeontx2/otx2_tim_worker.h >index 50db6543c..c896b5433 100644 >--- a/drivers/event/octeontx2/otx2_tim_worker.h >+++ b/drivers/event/octeontx2/otx2_tim_worker.h >@@ -7,6 +7,13 @@ > > #include "otx2_tim_evdev.h" > >+static inline uint8_t >+tim_bkt_fetch_lock(uint64_t w1) >+{ >+ return (w1 >> TIM_BUCKET_W1_S_LOCK) & >+ TIM_BUCKET_W1_M_LOCK; >+} >+ > static inline int16_t > tim_bkt_fetch_rem(uint64_t w1) > { >@@ -188,7 +195,6 @@ tim_insert_chunk(struct otx2_tim_bkt * const >bkt, > } else { > bkt->first_chunk =3D (uintptr_t)chunk; > } >- > return chunk; > } > >@@ -208,11 +214,38 @@ tim_add_entry_sp(struct otx2_tim_ring * >const tim_ring, > > __retry: > /* Get Bucket sema*/ >- lock_sema =3D tim_bkt_fetch_sema(bkt); >+ lock_sema =3D tim_bkt_fetch_sema_lock(bkt); > > /* Bucket related checks. */ >- if (unlikely(tim_bkt_get_hbt(lock_sema))) >- goto __retry; >+ if (unlikely(tim_bkt_get_hbt(lock_sema))) { >+ if (tim_bkt_get_nent(lock_sema) !=3D 0) { >+ uint64_t hbt_state; >+#ifdef RTE_ARCH_ARM64 >+ asm volatile( >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbz %[hbt], 33, dne%=3D > \n" >+ " sevl > \n" >+ "rty%=3D: wfe > \n" >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbnz %[hbt], 33, rty%=3D > \n" >+ "dne%=3D: > \n" >+ : [hbt] "=3D&r" (hbt_state) >+ : [w1] "r" ((&bkt->w1)) >+ : "memory" >+ ); >+#else >+ do { >+ hbt_state =3D __atomic_load_n(&bkt- >>w1, >+ __ATOMIC_ACQUIRE); >+ } while (hbt_state & BIT_ULL(33)); >+#endif >+ >+ if (!(hbt_state & BIT_ULL(34))) { >+ tim_bkt_dec_lock(bkt); >+ goto __retry; >+ } >+ } >+ } > > /* Insert the work. */ > rem =3D tim_bkt_fetch_rem(lock_sema); >@@ -224,14 +257,15 @@ tim_add_entry_sp(struct otx2_tim_ring * >const tim_ring, > chunk =3D tim_insert_chunk(bkt, tim_ring); > > if (unlikely(chunk =3D=3D NULL)) { >- tim_bkt_set_rem(bkt, 0); >+ bkt->chunk_remainder =3D 0; >+ tim_bkt_dec_lock(bkt); > tim->impl_opaque[0] =3D 0; > tim->impl_opaque[1] =3D 0; > tim->state =3D RTE_EVENT_TIMER_ERROR; > return -ENOMEM; > } > bkt->current_chunk =3D (uintptr_t)chunk; >- tim_bkt_set_rem(bkt, tim_ring->nb_chunk_slots - 1); >+ bkt->chunk_remainder =3D tim_ring->nb_chunk_slots - 1; > } else { > chunk =3D (struct otx2_tim_ent *)(uintptr_t)bkt- >>current_chunk; > chunk +=3D tim_ring->nb_chunk_slots - rem; >@@ -241,6 +275,7 @@ tim_add_entry_sp(struct otx2_tim_ring * const >tim_ring, > *chunk =3D *pent; > > tim_bkt_inc_nent(bkt); >+ tim_bkt_dec_lock(bkt); > > tim->impl_opaque[0] =3D (uintptr_t)chunk; > tim->impl_opaque[1] =3D (uintptr_t)bkt; >@@ -263,19 +298,60 @@ tim_add_entry_mp(struct otx2_tim_ring * >const tim_ring, > > __retry: > bkt =3D tim_get_target_bucket(tim_ring, rel_bkt, flags); >- > /* Get Bucket sema*/ > lock_sema =3D tim_bkt_fetch_sema_lock(bkt); > > /* Bucket related checks. */ > if (unlikely(tim_bkt_get_hbt(lock_sema))) { >- tim_bkt_dec_lock(bkt); >- goto __retry; >+ if (tim_bkt_get_nent(lock_sema) !=3D 0) { >+ uint64_t hbt_state; >+#ifdef RTE_ARCH_ARM64 >+ asm volatile( >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbz %[hbt], 33, dne%=3D > \n" >+ " sevl > \n" >+ "rty%=3D: wfe > \n" >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbnz %[hbt], 33, rty%=3D > \n" >+ "dne%=3D: > \n" >+ : [hbt] "=3D&r" (hbt_state) >+ : [w1] "r" ((&bkt->w1)) >+ : "memory" >+ ); >+#else >+ do { >+ hbt_state =3D __atomic_load_n(&bkt- >>w1, >+ __ATOMIC_ACQUIRE); >+ } while (hbt_state & BIT_ULL(33)); >+#endif >+ >+ if (!(hbt_state & BIT_ULL(34))) { >+ tim_bkt_dec_lock(bkt); >+ goto __retry; >+ } >+ } > } > > rem =3D tim_bkt_fetch_rem(lock_sema); >- > if (rem < 0) { >+#ifdef RTE_ARCH_ARM64 >+ asm volatile( >+ " ldaxrh %w[rem], [%[crem]] > \n" >+ " tbz %w[rem], 15, dne%=3D > \n" >+ " sevl > \n" >+ "rty%=3D: wfe > \n" >+ " ldaxrh %w[rem], [%[crem]] > \n" >+ " tbnz %w[rem], 15, rty%=3D > \n" >+ "dne%=3D: > \n" >+ : [rem] "=3D&r" (rem) >+ : [crem] "r" (&bkt->chunk_remainder) >+ : "memory" >+ ); >+#else >+ while (__atomic_load_n(&bkt->chunk_remainder, >+ __ATOMIC_ACQUIRE) < 0) >+ ; >+#endif > /* Goto diff bucket. */ > tim_bkt_dec_lock(bkt); > goto __retry; >@@ -294,17 +370,23 @@ tim_add_entry_mp(struct otx2_tim_ring * >const tim_ring, > tim->state =3D RTE_EVENT_TIMER_ERROR; > return -ENOMEM; > } >- bkt->current_chunk =3D (uintptr_t)chunk; >- tim_bkt_set_rem(bkt, tim_ring->nb_chunk_slots - 1); >+ *chunk =3D *pent; >+ while (tim_bkt_fetch_lock(lock_sema) !=3D >+ (-tim_bkt_fetch_rem(lock_sema))) >+ lock_sema =3D __atomic_load_n(&bkt->w1, >__ATOMIC_ACQUIRE); >+ >+ bkt->current_chunk =3D (uintptr_t)chunk; >+ __atomic_store_n(&bkt->chunk_remainder, >+ tim_ring->nb_chunk_slots - 1, >__ATOMIC_RELEASE); > } else { >- chunk =3D (struct otx2_tim_ent *)(uintptr_t)bkt- >>current_chunk; >+ chunk =3D (struct otx2_tim_ent *)bkt->current_chunk; > chunk +=3D tim_ring->nb_chunk_slots - rem; >+ *chunk =3D *pent; > } > > /* Copy work entry. */ >- *chunk =3D *pent; >- tim_bkt_dec_lock(bkt); > tim_bkt_inc_nent(bkt); >+ tim_bkt_dec_lock(bkt); > tim->impl_opaque[0] =3D (uintptr_t)chunk; > tim->impl_opaque[1] =3D (uintptr_t)bkt; > tim->state =3D RTE_EVENT_TIMER_ARMED; >@@ -360,8 +442,33 @@ tim_add_entry_brst(struct otx2_tim_ring * >const tim_ring, > > /* Bucket related checks. */ > if (unlikely(tim_bkt_get_hbt(lock_sema))) { >- tim_bkt_dec_lock(bkt); >- goto __retry; >+ if (tim_bkt_get_nent(lock_sema) !=3D 0) { >+ uint64_t hbt_state; >+#ifdef RTE_ARCH_ARM64 >+ asm volatile( >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbz %[hbt], 33, dne%=3D > \n" >+ " sevl > \n" >+ "rty%=3D: wfe > \n" >+ " ldaxr %[hbt], [%[w1]] > \n" >+ " tbnz %[hbt], 33, rty%=3D > \n" >+ "dne%=3D: > \n" >+ : [hbt] "=3D&r" (hbt_state) >+ : [w1] "r" ((&bkt->w1)) >+ : "memory" >+ ); >+#else >+ do { >+ hbt_state =3D __atomic_load_n(&bkt- >>w1, >+ __ATOMIC_ACQUIRE); >+ } while (hbt_state & BIT_ULL(33)); >+#endif >+ >+ if (!(hbt_state & BIT_ULL(34))) { >+ tim_bkt_dec_lock(bkt); >+ goto __retry; >+ } >+ } > } > > chunk_remainder =3D tim_bkt_fetch_rem(lock_sema); >-- >2.17.1