From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C9A3D41D9B; Tue, 28 Feb 2023 10:45:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63B4040EE4; Tue, 28 Feb 2023 10:45:04 +0100 (CET) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2059.outbound.protection.outlook.com [40.107.22.59]) by mails.dpdk.org (Postfix) with ESMTP id 2134C4021F for ; Tue, 28 Feb 2023 10:45:02 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AfOnlIeD5D9zWxjibUYJLghmA8LcVxUm98A7O1SSSm5v7VOVrAKLZnvZpJCc3eijWmcDxTQeERRyr+5iCI8Va0u4zUDcch1m0W37wgjArxTKZh+HuhbPtqCRXslJbYdKYdqn5nu0rgL/qeEvh+saL3aS02ERb6afKnKyvesoCgRRHO4OzZBtyfYpmLhzH2VMF5viEqaguwuRvpKIccWL3IDuC0JuUxYRIGtnEi4Vx63p39XqoNla/yicGLuNRpNdkeK9GsrwKBDne8RqFQhcsa/iVz9Gx6wPhPDGeUfII9dmXjS/8fJ50dsMA28o5+5wgNinOXh50p7+UF96fUvG/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pO1v0jtpNOkieU/aKPT+aHC5KfIKpr08CAiEXwkJaWA=; b=cPWH7G+9MJAfoVvE/r/xOHZUSOoUfCYDcacGaajF8dJY33BI2jNNO4TLJPiakfHnE8W+Jj+CEhiFSxE4XBf9GguG9HbpGjgJRfa4IgXtXlLOnPW3cvrRgwOJwSUT9Us1eWgO9DhBF8reauveF/32cjMhggnS7Z2wuS6txBDDZM4dDYYzfvkzEwhENY3a2KLefHRzDQHGPWk4OfOOI0dRjGPPPolU534LP6KMJBq17I/4c3IYYHYjyRbyuuqo/JyKF2P76+UTz/tLTXGknoupZ7+CncWiY4DjbDaZBErR8b1CL/TNQI3CcPb7/8DcMnR9vyPmJXxsdpMn+vcH1p03gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none Received: from AS8PR04CA0175.eurprd04.prod.outlook.com (2603:10a6:20b:331::30) by GV1PR07MB8367.eurprd07.prod.outlook.com (2603:10a6:150:22::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30; Tue, 28 Feb 2023 09:44:59 +0000 Received: from AM0EUR02FT005.eop-EUR02.prod.protection.outlook.com (2603:10a6:20b:331:cafe::e6) by AS8PR04CA0175.outlook.office365.com (2603:10a6:20b:331::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30 via Frontend Transport; Tue, 28 Feb 2023 09:44:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM0EUR02FT005.mail.protection.outlook.com (10.13.54.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.6156.12 via Frontend Transport; Tue, 28 Feb 2023 09:44:59 +0000 Received: from ESESBMB501.ericsson.se (153.88.183.168) by ESESBMB503.ericsson.se (153.88.183.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.17; Tue, 28 Feb 2023 10:44:59 +0100 Received: from seliicinfr00050.seli.gic.ericsson.se (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.184) with Microsoft SMTP Server id 15.1.2507.17 via Frontend Transport; Tue, 28 Feb 2023 10:44:58 +0100 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00050.seli.gic.ericsson.se (Postfix) with ESMTP id F002F1C006A; Tue, 28 Feb 2023 10:44:58 +0100 (CET) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: Erik Gabriel Carrillo , David Marchand , , Stefan Sundkvist , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [RFC 0/2] Add high-performance timer facility Date: Tue, 28 Feb 2023 10:39:14 +0100 Message-ID: <20230228093916.87206-1-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0EUR02FT005:EE_|GV1PR07MB8367:EE_ X-MS-Office365-Filtering-Correlation-Id: 7d402c54-b966-4053-b3b4-08db1970751f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q1rHsNcrh7+N6H10VuUo9ZNhRVXOqfjvxo5YDZD89vSUc6c6gQjTe3U7Aw+4loe93cK+bODrjPN0D0/dRjwBFEGcY5OCG/EjdK4dek6fMB1ZWkFeWofX3KNc9opf5ZnjMS1xFqEAe+0VLJfQyRdADL3GAmli+4U4kG0D2Vw2/Eo4mtAWCY/3GA2KZIOgjIQD+JaJdtpyIyX+3tnUa4WkfjI9s5gCiKfeIVScPmraH6V0lmJJvMqpyEq37CTrydk32kYse/ar6WX264jOmyOhhL91BsRa52RujfLY0eT8P66BxXEQ7NRN1t4jrg6fEljT/Ac/bWVZ7B4rn3UrmNpiPahjQI+aA+PEaXMMx/oPzdL5Ega0Tgkb9lYy7PyPYXKWAaMmR+tg1jmSZgDFAkmpHNLEPkoizaYhYGEzncUJVk37Y8EqUszv6qLDDSvl2aGhSuU8DyWlmAZ0cW2kntePph3wi+i4W8toAynTqcUOE8cmAzID9MPRMAC1149uLbQi9W6dxdMEzBZhr1ihoRz1CywvpWGjSd0dmP8avMSW924ZSLoFylSKc1sbqK40J1HuMFfSLPjZH/fNt3OBMVfhakU/R5FJXjCfydIYh+9IRHxZBf3aLhEyFvNr5IBZNwAzNCi7b1ClTKpQmIRL7okGL9B3wtG5+Kmalyv++E3rD10msIWTvyvCUeO14hTb7fDdJjDiDM4k18YHg0qtCHdmyA== X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230025)(4636009)(346002)(39860400002)(136003)(376002)(396003)(451199018)(40470700004)(36840700001)(46966006)(66574015)(83380400001)(47076005)(54906003)(316002)(336012)(36756003)(4326008)(82960400001)(82740400003)(6916009)(7636003)(36860700001)(40480700001)(8676002)(82310400005)(2616005)(478600001)(40460700003)(186003)(26005)(6266002)(6666004)(107886003)(1076003)(5660300002)(70586007)(2906002)(70206006)(8936002)(86362001)(356005)(41300700001); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2023 09:44:59.4475 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7d402c54-b966-4053-b3b4-08db1970751f X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM0EUR02FT005.eop-EUR02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR07MB8367 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patchset is an attempt to introduce a high-performance, highly scalable timer facility into DPDK. More specifically, the goals for the htimer library are: * Efficient handling of a handful up to hundreds of thousands of concurrent timers. * Reduced overhead of adding and canceling timers. * Provide a service functionally equivalent to that of . API/ABI backward compatibility is secondary. In the author's opinion, there are two main shortcomings with the current DPDK timer library (i.e., rte_timer.[ch]). One is the synchronization overhead, where heavy-weight full-barrier type synchronization is used. rte_timer.c uses per-EAL/lcore skip lists, but any thread may add or cancel (or otherwise access) timers managed by another lcore (and thus resides in its timer skip list). The other is an algorithmic shortcoming, with rte_timer.c's reliance on a skip list, which, seemingly, is less efficient than certain alternatives. This patchset implements a hierarchical timer wheel (HWT, in rte_htw.c), as per the Varghese and Lauck paper "Hashed and Hierarchical Timing Wheels: Data Structures for the Efficient Implementation of a Timer Facility". A HWT is a data structure purposely design for this task, and used by many operating system kernel timer facilities. To further improve the solution described by Varghese and Lauck, a bitset is placed in front of each of the timer wheel in the HWT, reducing overhead of rte_htimer_mgr_manage() (i.e., progressing time and expiry processing). Cycle-efficient scanning and manipulation of these bitsets are crucial for the HWT's performance. The htimer module keeps a per-lcore (or per-registered EAL thread) HWT instance, much like rte_timer.c keeps a per-lcore skip list. To avoid expensive synchronization overhead for thread-local timer management, the HWTs are accessed only from the "owning" thread. Any interaction any other thread has with a particular lcore's timer wheel goes over a set of DPDK rings. A side-effect of this design is that all operations working toward a "remote" HWT must be asynchronous. The API is available only to EAL threads and registered non-EAL threads. The htimer API allows the application to supply the current time, useful in case it already has retrieved this for other purposes, saving the cost of a rdtsc instruction (or its equivalent). Relative htimer does not retrieve a new time, but reuse the current time (as known via/at-the-time of the manage-call), again to shave off some cycles of overhead. A semantic improvement compared to the API is that the htimer library can give a definite answer on the question if the timer expiry callback was called, after a timer has been canceled. Below is a performance data from DPDK's 'app/test' micro benchmarks, using 10k concurrent timers. The benchmarks (test_timer_perf.c and test_htimer_mgr_perf.c) aren't identical in their structure, but the numbers give some indication of the difference. Use case htimer timer ------------------------------------ Add timer 28 253 Cancel timer 10 412 Async add (source lcore) 64 Async add (target lcore) 13 (AMD 5900X CPU. Time in TSC.) Prototype integration of the htimer library into real, timer-heavy, applications indicates that htimer may result in significant application-level performance gains. The bitset implementation which the HWT implementation depends upon seemed generic-enough and potentially useful outside the world of HWTs, to justify being located in the EAL. This patchset is very much an RFC, and the author is yet to form an opinion on many important issues. * If deemed a suitable replacement, should the htimer replace the current DPDK timer library in some particular (ABI-breaking) release, or should it live side-by-side with the then-legacy API? A lot of things in and outside DPDK depend on , so coexistence may be required to facilitate a smooth transition. * Should the htimer and htw-related files be colocated with rte_timer.c in the timer library? * Would it be useful for applications using asynchronous cancel to have the option of having the timer callback run not only in case of timer expiration, but also cancellation (on the target lcore)? The timer cb signature would need to include an additional parameter in that case. * Should the rte_htimer be a nested struct, so the htw parts be separated from the htimer parts? * is kept separate from , so that may avoid a depedency to . Should it be so? * rte_htimer struct is only supposed to be used by the application to give an indication of how much memory it needs to allocate, and is its member are not supposed to be directly accessed (w/ the possible exception of the owner_lcore_id field). Should there be a dummy struct, or a #define RTE_HTIMER_MEMSIZE or a rte_htimer_get_memsize() function instead, serving the same purpose? Better encapsulation, but more inconvenient for applications. Run-time dynamic sizing would force application-level dynamic allocations. * Asynchronous cancellation is a little tricky to use for the application (primarily due to timer memory reclamation/race issues). Should this functionality be removed? * Should rte_htimer_mgr_init() also retrieve the current time? If so, there should to be a variant which allows the user to specify the time (to match rte_htimer_mgr_manage_time()). One pitfall with the current proposed API is an application calling rte_htimer_mgr_init() and then immediately adding a timer with a relative timeout, in which case the current absolute time used is 0, which might be a surprise. * Should libdivide (optionally) be used to avoid the div in the TSC -> tick conversion? (Doesn't improve performance on Zen 3, but may do on other CPUs.) Consider as well. * Should the TSC-per-tick be rounded up to a power of 2, so shifts can be used for conversion? Very minor performance gains to be found there, at least on Zen 3 cores. * Should it be possible to supply the time in rte_htimer_mgr_add() and/or rte_htimer_mgr_manage_time() functions as ticks, rather than as TSC? Should it be possible to also use nanoseconds? rte_htimer_mgr_manage_time() would need a flags parameter in that case. * Would the event timer adapter be best off using directly, or ? In the latter case, there needs to be a way to instantiate more HWTs (similar to the "alt" functions of )? * Should the PERIODICAL flag (and the complexity it brings) be removed? And leave the application with only single-shot timers, and the option to re-add them in the timer callback. * Should the async result codes and the sync cancel error codes be merged into one set of result codes? * Should the rte_htimer_mgr_async_add() have a flag which allow buffering add request messages until rte_htimer_mgr_process() is called? Or any manage function. Would reduce ring signaling overhead (i.e., burst enqueue operations instead of single-element enqueue). Could also be a rte_htimer_mgr_async_add_burst() function, solving the same "problem" a different way. (The signature of such a function would not be pretty.) * Does the functionality provided by the rte_htimer_mgr_process() function match its the use cases? Should there me a more clear separation between expiry processing and asynchronous operation processing? * Should the patchset be split into more commits? If so, how? Thanks to Erik Carrillo for his assistance. Mattias Rönnblom (2): eal: add bitset type eal: add high-performance timer facility app/test/meson.build | 10 +- app/test/test_bitset.c | 646 +++++++++++++++++++++++ app/test/test_htimer_mgr.c | 674 ++++++++++++++++++++++++ app/test/test_htimer_mgr_perf.c | 324 ++++++++++++ app/test/test_htw.c | 478 +++++++++++++++++ app/test/test_htw_perf.c | 181 +++++++ doc/api/doxy-api-index.md | 5 +- doc/api/doxy-api.conf.in | 1 + lib/eal/common/meson.build | 1 + lib/eal/common/rte_bitset.c | 29 + lib/eal/include/meson.build | 1 + lib/eal/include/rte_bitset.h | 878 +++++++++++++++++++++++++++++++ lib/eal/version.map | 3 + lib/htimer/meson.build | 7 + lib/htimer/rte_htimer.h | 65 +++ lib/htimer/rte_htimer_mgr.c | 488 +++++++++++++++++ lib/htimer/rte_htimer_mgr.h | 497 +++++++++++++++++ lib/htimer/rte_htimer_msg.h | 44 ++ lib/htimer/rte_htimer_msg_ring.c | 18 + lib/htimer/rte_htimer_msg_ring.h | 49 ++ lib/htimer/rte_htw.c | 437 +++++++++++++++ lib/htimer/rte_htw.h | 49 ++ lib/htimer/version.map | 17 + lib/meson.build | 1 + 24 files changed, 4901 insertions(+), 2 deletions(-) create mode 100644 app/test/test_bitset.c create mode 100644 app/test/test_htimer_mgr.c create mode 100644 app/test/test_htimer_mgr_perf.c create mode 100644 app/test/test_htw.c create mode 100644 app/test/test_htw_perf.c create mode 100644 lib/eal/common/rte_bitset.c create mode 100644 lib/eal/include/rte_bitset.h create mode 100644 lib/htimer/meson.build create mode 100644 lib/htimer/rte_htimer.h create mode 100644 lib/htimer/rte_htimer_mgr.c create mode 100644 lib/htimer/rte_htimer_mgr.h create mode 100644 lib/htimer/rte_htimer_msg.h create mode 100644 lib/htimer/rte_htimer_msg_ring.c create mode 100644 lib/htimer/rte_htimer_msg_ring.h create mode 100644 lib/htimer/rte_htw.c create mode 100644 lib/htimer/rte_htw.h create mode 100644 lib/htimer/version.map -- 2.34.1