From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D82BC45AC7; Sun, 6 Oct 2024 15:02:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6F3CB4025F; Sun, 6 Oct 2024 15:02:53 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 8D84C4025D for ; Sun, 6 Oct 2024 15:02:52 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 19E8CD224 for ; Sun, 6 Oct 2024 15:02:52 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 0C221D167; Sun, 6 Oct 2024 15:02:52 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,AWL, T_SCC_BODY_TEXT_LINE autolearn=disabled version=4.0.0 X-Spam-Score: -1.2 Received: from [192.168.1.85] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id B38B1D0E4; Sun, 6 Oct 2024 15:02:49 +0200 (CEST) Message-ID: <3229837d-fd8e-417e-8eb3-1a5c621ff0ee@lysator.liu.se> Date: Sun, 6 Oct 2024 15:02:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC v2 0/2] Add high-performance timer facility To: =?UTF-8?Q?Morten_Br=C3=B8rup?= , Stephen Hemminger , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= Cc: dev@dpdk.org, Erik Gabriel Carrillo , David Marchand , maria.lingemark@ericsson.com, Stefan Sundkvist , Tyler Retzlaff References: <20230228093916.87206-1-mattias.ronnblom@ericsson.com> <20230315170342.214127-1-mattias.ronnblom@ericsson.com> <20241003113632.2be4c2b7@hermes.local> <98CBD80474FA8B44BF855DF32C47DC35E9F771@smartserver.smartshare.dk> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F771@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-10-03 23:32, Morten Brørup wrote: >> From: Stephen Hemminger [mailto:stephen@networkplumber.org] >> Sent: Thursday, 3 October 2024 20.37 >> >> On Wed, 15 Mar 2023 18:03:40 +0100 >> Mattias Rönnblom wrote: >> >>> This patchset is an attempt to introduce a high-performance, highly >>> scalable timer facility into DPDK. >>> >>> More specifically, the goals for the htimer library are: >>> >>> * Efficient handling of a handful up to hundreds of thousands of >>> concurrent timers. >>> * Make adding and canceling timers low-overhead, constant-time >>> operations. >>> * Provide a service functionally equivalent to that of >>> . API/ABI backward compatibility is secondary. >> >> Worthwhile goals, and the problem needs to be addressed. >> But this patch never got accepted. > > I think work on it was put on hold due to the requested changes requiring a significant development effort. > I too look forward to work on this being resumed. ;-) > >> >> Please fix/improve/extend existing rte_timer instead. > > The rte_timer API is too "fat" for use in the fast path with millions of timers, e.g. TCP flow timers. > > Shoehorning a fast path feature into a slow path API is not going to cut it. I support having a separate htimer library with its own API for high volume, high-performance fast path timers. > > When striving for low latency across the internet, timing is everything. Packet pacing is the "new" hot thing in congestion control algorithms, and a simple software implementation would require a timer firing once per packet. > I think DPDK should have two public APIs in the timer area. One is a just a bare-bones hierarchical timer wheel API, without callbacks, auto-created per-lcore instances, MT safety or any other of the bells and whistles. It also doesn't make any assumptions about the time source (other it being monotonic) or resolution. The other is a new variant of , using the core HTW library for its implementation (and being public, it may also expose this library in its header files, which may be required for efficient operation). The new would provide the same kind of functionality as the old API, but with some quirks and bugs fixed, plus potentially some new functionality added. For example, it would be useful to allow non-preemption safe threads to add and remove timers (something rte_timer and its spinlocks doesn't allow). I would consider both "fast path APIs". In addition, there should probably also be a time source API. Considering the lead time of relatively small contributions like the bitops extensions and the new bitset API (which still aren't in), I can't imagine how long time it would take to get in a semi-backward compatible rte_timer with a new implementation, plus a new timer wheel library, into DPDK.