From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A7827A04DD; Mon, 21 Sep 2020 21:08:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DC8431D8DC; Mon, 21 Sep 2020 21:08:12 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2099.outbound.protection.outlook.com [40.107.92.99]) by dpdk.org (Postfix) with ESMTP id 8AA1B1D8D9 for ; Mon, 21 Sep 2020 21:08:11 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VF+9842I+8/Uy/ufq8/IeOxuWhl2+dElWr3Uq57i/jtFQYIZESNaDnhyFNihJ4rhj9e5tAEV0YU+566y0TlrzeB+vhOC0rJwJZ9HLYAzdPNoW9gPSLxdh5mcU5P+pcSEy/Teh4QWIHEY6/eqVtQEpCJC4JCDt7o8MIkGRiEbCWtPS4bpMr7+JtA2ByW8OrUSPZ5t/NLaVKCgT3Z5KH7/DBHx22CG4NR0aOojc3yca4Ns/3hmHPzphiZejExyXY606ILaqlVgd4hbEKAE9BfPJfkKP/hZNjS0f7UzMb23sD20361N9gzq26Bxb/O+SiwQYA+zEw/fzfaPLawXcZycSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F8w0c5MyH/kg8i5zhObtxyCi2/Ks+EhTj23OwNlwSZY=; b=OSPzxS4sAq8UgmoZAojGaf8S7KgpCuhFFxr80zD3A99oVSnRBoDO0/1r0+ANGkB3yTl4LCattYjzKishiRHBa9PJAAFsbKiLIHntB6HzC1fEnb99ke1ozzInf5AT8HNiRdQKNRFDYTFB0C+iAoxQQun0RzJcp+Glw98k4gBdSzc+HKZ34tske2R7KhlxV/mauLjXcD88e08Rvrs+/PQrW1+N5n4AVETzenW7H0kItpR8rb3P6e5RTFzmPnAb4IWO2jrwK+1amWgrITO7Bv5tbDLnxzhj/4mMNuWSvmecIIx8avo5vmHIVYRHlgogEefbFxTkLy1BLtHZhljziOsQIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F8w0c5MyH/kg8i5zhObtxyCi2/Ks+EhTj23OwNlwSZY=; b=NmRnpX/u5ZRj6QKrlU0iiHKAZU5hYunCF6vZhBHcbSjfBkvdScqzXmSpyowH4BwZFSYwEC777w3sR5bmrLlQep/qbI66uKOCq8amMdM4NYiE5IYXFgEimEb6hrgnjBV9KqKy+DvEMBdeZYJwoUlQlpL/7z7nKSH5Dw8DYFpBKyM= Received: from BY5PR21MB1380.namprd21.prod.outlook.com (2603:10b6:a03:23b::19) by BYAPR21MB1350.namprd21.prod.outlook.com (2603:10b6:a03:115::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.4; Mon, 21 Sep 2020 19:08:09 +0000 Received: from BY5PR21MB1380.namprd21.prod.outlook.com ([fe80::8836:b7ce:beb3:91ae]) by BY5PR21MB1380.namprd21.prod.outlook.com ([fe80::8836:b7ce:beb3:91ae%4]) with mapi id 15.20.3433.007; Mon, 21 Sep 2020 19:08:09 +0000 From: Khoa To To: Dmitry Kozlyuk , "dev@dpdk.org" CC: Narcisa Ana Maria Vasile , "Dmitry Malloy (MESHCHANINOV)" , Pallavi Kadam Thread-Topic: [EXTERNAL] [dpdk-dev] [PATCH 2/2] eal/windows: implement alarm API Thread-Index: AQHWh9Gm5/SuSJDZDU2M36+ME6jDN6lzdBGw Date: Mon, 21 Sep 2020 19:08:09 +0000 Message-ID: References: <20200911002207.31813-1-dmitry.kozliuk@gmail.com> <20200911002207.31813-3-dmitry.kozliuk@gmail.com> In-Reply-To: <20200911002207.31813-3-dmitry.kozliuk@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=a4eec463-6315-47b2-83f5-ebca5ff7f3d7; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2020-09-21T18:05:57Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=microsoft.com; x-originating-ip: [2001:4898:80e8:8:f153:5d5c:9f6:cc4] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: a413d18f-3fd8-47b9-09cf-08d85e61ae01 x-ms-traffictypediagnostic: BYAPR21MB1350: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: x9oin1cuyNdhAEjTLJpugHlZStlFXnToSZI8dEIA0i/huZVORa/2owdbuoSMgKcHHlWLDh5tu7ybaRo4vq8EMlTiN1q38vAjOfKm5QhqCVIkzjk3J01le8FnEjzOSqh/l26chPI+3wnSo/Wtx1gmMicqS/A9GLjR7Uhh8DZjAj1LvYt5HMWKT9uKKOpQeqeemviMf1oG3OA40q2a9X/FtpN79l8J6ce1cJoIK35l+Eo3v+0arXDuO1VkthzsgNP8G0cjHWcoH/AaZs3LIp/IVSI2APPOfl73n8LKbTeULeS4J83GCKbI7cZsDV56b2zzNr+rX7fKIQXKxboPJ9AyO9wLwOEEeoD+859lt6VWeRegYARO3VyUrvyo7+XUsbKf x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BY5PR21MB1380.namprd21.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(396003)(39860400002)(376002)(346002)(316002)(6506007)(478600001)(71200400001)(66946007)(8676002)(76116006)(55016002)(186003)(53546011)(8936002)(10290500003)(4326008)(86362001)(9686003)(82950400001)(82960400001)(5660300002)(110136005)(2906002)(7696005)(33656002)(8990500004)(66446008)(66556008)(52536014)(83380400001)(54906003)(64756008)(66476007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: LwyV3zvTA8yvxd+U+8bbmlG0qmv7DwvKUiJRSVG9PWtOvd7HQ21lyYrVgpMKUYO9G/DPbudKDiBwQAFCbTZjoV+NJQnH+i/ZtL9NSGOC5Rh6z1Sz7eb3OZsKfuF2+zqFDJ4PgZBhWiXbkLZ/5TbUdWERSvdJUROPpQyXJPOxebOCPaugDVNkf4+SDjswfysRnsxbcRrPH6yLspH3ZJvOwYZMI9lJqjqfeZgt859KMKqh4LLljoyZJ6bWRM2THiMRnxdBm2SiBZRzLkkhURu9+gQ/1nu5TXBLS2PYl4MuU98oN3ANoJQXxzPM3z23KB/Lde7AvkxyH4FiWyG76IXGp+69wH99Dx2ILw6lC8umtT4F+K4P9B9cdE1nm3GZshymcEGIkn+rlhDw1IJ2OJdJSpJzi48BUZkExHlrbC2dz4BmGFStXKq7hPBucP/LdsDfYcCBCkcbUAvf9CuAg/Udt5OkSLgytNefKnG8mR2u+eot7adKk2MdOl39uiZQz8DiZt81HCFHH8xiyhpFa/neH3jwuP4i4UFggwsmtArcJKMN6DkPYA5Dcj4GujvzvnmWeHNgJJZkcImOAwADtinkUBqVLYBZv228vTn2rJ9giaRUMdMErDipmIO6yX/QKcqSr2VHbwq6S1xVuGWvZJWcOObN7YJsOQSyQJmc3XuYT79h5mdQv/wu6z+2SBl5bgxiLqDpCRXOT+GLiPt+kqZ6Cw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BY5PR21MB1380.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a413d18f-3fd8-47b9-09cf-08d85e61ae01 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2020 19:08:09.6310 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XjTLUyogHK+EnWQ1nXIqheQnmWJBWbtgPDqEB4lzMqjfFOTS0Ao7w5z+EID5CfYHawNvpvfr2z/jiiclrK1l/BIVL9y2dbCS9vP0BWJBRBw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR21MB1350 Subject: Re: [dpdk-dev] [EXTERNAL] [PATCH 2/2] eal/windows: implement alarm API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Dmitry, Since all alarm callbacks are scheduled on the interrupt thread, looks like= this implementation of rte_eal_alarm_set() would create a head-of-line blo= cking issue, as the callbacks are serialized one after the other. Is that correct? If so, while this does not violate the API contract, it s= eems undesirable. I looked at the Linux implementation of alarm, and although I am not very f= amiliar with Linux system, it seems to have similar implementation, with th= e same head-of-line blocking issue. I'm wondering why in both Linux and this Windows implementation, we don't s= pawn a new thread to service each alarm independently. And, regardless of the Linux implementation, should we consider another imp= lementation that avoids the head-of-line blocking issue? Khoa. > -----Original Message----- > From: dev On Behalf Of Dmitry Kozlyuk > Sent: Thursday, September 10, 2020 5:22 PM > To: dev@dpdk.org > Cc: Dmitry Kozlyuk ; Narcisa Ana Maria Vasile <= navasile@linux.microsoft.com>; Dmitry Malloy > (MESHCHANINOV) ; Pallavi Kadam > Subject: [EXTERNAL] [dpdk-dev] [PATCH 2/2] eal/windows: implement alarm A= PI >=20 > Implementation is based on waitable timers Win32 API. When timer is set, > a callback and its argument are supplied to the OS, while timer handle > is stored in EAL alarm list. When timer expires, OS wakes up the > interrupt thread and runs the callback. Upon completion it removes the > alarm. >=20 > Waitable timers must be set from the thread their callback will run in, > eal_intr_thread_schedule() provides a way to schedule asyncronuous code > execution in the interrupt thread. Alarm module builds synchronous timer > setup on top of it. >=20 > Windows alarms are not a type of DPDK interrupt handle and do not > interact with interrupt module beyond executing in the same thread. >=20 > Signed-off-by: Dmitry Kozlyuk > --- > lib/librte_eal/rte_eal_exports.def | 2 + > lib/librte_eal/windows/eal_alarm.c | 219 +++++++++++++++++++++++++++++ > lib/librte_eal/windows/meson.build | 1 + > 3 files changed, 222 insertions(+) > create mode 100644 lib/librte_eal/windows/eal_alarm.c >=20 > diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_= exports.def > index 9baca0110..b6abb5ae1 100644 > --- a/lib/librte_eal/rte_eal_exports.def > +++ b/lib/librte_eal/rte_eal_exports.def > @@ -14,6 +14,8 @@ EXPORTS > rte_devargs_insert > rte_devargs_next > rte_devargs_remove > + rte_eal_alarm_set > + rte_eal_alarm_cancel > rte_eal_get_configuration > rte_eal_has_hugepages > rte_eal_has_pci > diff --git a/lib/librte_eal/windows/eal_alarm.c b/lib/librte_eal/windows/= eal_alarm.c > new file mode 100644 > index 000000000..3b262793a > --- /dev/null > +++ b/lib/librte_eal/windows/eal_alarm.c > @@ -0,0 +1,219 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright (c) 2020 Dmitry Kozlyuk > + */ > + > +#include > +#include > + > +#include > +#include > + > +#include > + > +#include "eal_windows.h" > + > +enum alarm_state { > + ALARM_ARMED, > + ALARM_TRIGGERED, > + ALARM_CANCELLED > +}; > + > +struct alarm_entry { > + LIST_ENTRY(alarm_entry) next; > + rte_eal_alarm_callback cb_fn; > + void *cb_arg; > + HANDLE timer; > + atomic_uint state; > +}; > + > +static LIST_HEAD(alarm_list, alarm_entry) alarm_list =3D LIST_HEAD_INITI= ALIZER(); > + > +static rte_spinlock_t alarm_lock =3D RTE_SPINLOCK_INITIALIZER; > + > +static int intr_thread_exec(void (*func)(void *arg), void *arg); > + > +static void > +alarm_remove_unsafe(struct alarm_entry *ap) > +{ > + LIST_REMOVE(ap, next); > + CloseHandle(ap->timer); > + free(ap); > +} > + > +static void > +alarm_callback(void *arg, DWORD low __rte_unused, DWORD high __rte_unuse= d) > +{ > + struct alarm_entry *ap =3D arg; > + unsigned int state =3D ALARM_ARMED; > + > + if (!atomic_compare_exchange_strong( > + &ap->state, &state, ALARM_TRIGGERED)) > + return; > + > + ap->cb_fn(ap->cb_arg); > + > + rte_spinlock_lock(&alarm_lock); > + alarm_remove_unsafe(ap); > + rte_spinlock_unlock(&alarm_lock); > +} > + > +struct alarm_task { > + struct alarm_entry *entry; > + LARGE_INTEGER deadline; > + int ret; > +}; > + > +static void > +alarm_set(void *arg) > +{ > + struct alarm_task *task =3D arg; > + > + BOOL ret =3D SetWaitableTimer( > + task->entry->timer, &task->deadline, > + 0, alarm_callback, task->entry, FALSE); > + task->ret =3D ret ? 0 : (-1); > +} > + > +int > +rte_eal_alarm_set(uint64_t us, rte_eal_alarm_callback cb_fn, void *cb_ar= g) > +{ > + struct alarm_entry *ap; > + HANDLE timer; > + FILETIME ft; > + struct alarm_task task; > + int ret; > + > + /* Calculate deadline ASAP, unit of measure =3D 100ns. */ > + GetSystemTimePreciseAsFileTime(&ft); > + task.deadline.LowPart =3D ft.dwLowDateTime; > + task.deadline.HighPart =3D ft.dwHighDateTime; > + task.deadline.QuadPart +=3D 10 * us; > + > + ap =3D calloc(1, sizeof(*ap)); > + if (ap =3D=3D NULL) { > + RTE_LOG(ERR, EAL, "Cannot allocate alarm entry\n"); > + ret =3D -ENOMEM; > + goto exit; > + } > + > + timer =3D CreateWaitableTimer(NULL, FALSE, NULL); > + if (timer =3D=3D NULL) { > + RTE_LOG_WIN32_ERR("CreateWaitableTimer()"); > + ret =3D -EINVAL; > + goto fail; > + } > + > + ap->timer =3D timer; > + ap->cb_fn =3D cb_fn; > + ap->cb_arg =3D cb_arg; > + task.entry =3D ap; > + > + /* Waitable timer must be set in the same thread that will > + * do an alertable wait for the alarm to trigger, that is, > + * in the interrupt thread. Setting can fail, so do it synchronously. > + */ > + ret =3D intr_thread_exec(alarm_set, &task); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "Cannot setup alarm in interrupt thread\n"); > + goto fail; > + } > + > + ret =3D task.ret; > + if (ret < 0) > + goto fail; > + > + rte_spinlock_lock(&alarm_lock); > + LIST_INSERT_HEAD(&alarm_list, ap, next); > + rte_spinlock_unlock(&alarm_lock); > + > + goto exit; > + > +fail: > + if (timer !=3D NULL) > + CloseHandle(timer); > + if (ap !=3D NULL) > + free(ap); > + > +exit: > + rte_eal_trace_alarm_set(us, cb_fn, cb_arg, ret); > + return ret; > +} > + > +static bool > +alarm_matches(const struct alarm_entry *ap, > + rte_eal_alarm_callback cb_fn, void *cb_arg) > +{ > + bool any_arg =3D cb_arg =3D=3D (void *)(-1); > + return (ap->cb_fn =3D=3D cb_fn) && (any_arg || ap->cb_arg =3D=3D cb_arg= ); > +} > + > +int > +rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg) > +{ > + struct alarm_entry *ap; > + unsigned int state; > + int removed; > + bool executing; > + > + removed =3D 0; > + do { > + executing =3D false; > + > + rte_spinlock_lock(&alarm_lock); > + > + LIST_FOREACH(ap, &alarm_list, next) { > + if (!alarm_matches(ap, cb_fn, cb_arg)) > + continue; > + > + state =3D ALARM_ARMED; > + if (atomic_compare_exchange_strong( > + &ap->state, &state, ALARM_CANCELLED)) { > + alarm_remove_unsafe(ap); > + removed++; > + } else if (state =3D=3D ALARM_TRIGGERED) > + executing =3D true; > + } > + > + rte_spinlock_unlock(&alarm_lock); > + } while (executing); > + > + rte_eal_trace_alarm_cancel(cb_fn, cb_arg, removed); > + return removed; > +} > + > +struct intr_task { > + void (*func)(void *arg); > + void *arg; > + rte_spinlock_t lock; /* unlocked at task completion */ > +}; > + > +static void > +intr_thread_entry(void *arg) > +{ > + struct intr_task *task =3D arg; > + task->func(task->arg); > + rte_spinlock_unlock(&task->lock); > +} > + > +static int > +intr_thread_exec(void (*func)(void *arg), void *arg) > +{ > + struct intr_task task; > + int ret; > + > + task.func =3D func; > + task.arg =3D arg; > + rte_spinlock_init(&task.lock); > + > + /* Make timers more precise by synchronizing in userspace. */ > + rte_spinlock_lock(&task.lock); > + ret =3D eal_intr_thread_schedule(intr_thread_entry, &task); > + if (ret < 0) { > + RTE_LOG(ERR, EAL, "Cannot schedule task to interrupt thread\n"); > + return -EINVAL; > + } > + > + /* Wait for the task to complete. */ > + rte_spinlock_lock(&task.lock); > + return 0; > +} > diff --git a/lib/librte_eal/windows/meson.build b/lib/librte_eal/windows/= meson.build > index b690bc6b0..3b2faf29e 100644 > --- a/lib/librte_eal/windows/meson.build > +++ b/lib/librte_eal/windows/meson.build > @@ -5,6 +5,7 @@ subdir('include') >=20 > sources +=3D files( > 'eal.c', > + 'eal_alarm.c', > 'eal_debug.c', > 'eal_file.c', > 'eal_hugepages.c', > -- > 2.25.4