From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4149B431FC; Wed, 25 Oct 2023 19:54:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF8A94067B; Wed, 25 Oct 2023 19:54:12 +0200 (CEST) Received: from dkmailrelay1.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id BC38640042; Wed, 25 Oct 2023 19:54:10 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesys.local [192.168.4.10]) by dkmailrelay1.smartsharesystems.com (Postfix) with ESMTP id 8EEC520A85; Wed, 25 Oct 2023 19:54:08 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: [PATCH v2] eal/unix: allow creating thread with real-time priority X-MimeOLE: Produced By Microsoft Exchange V6.5 Date: Wed, 25 Oct 2023 19:54:06 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35E9EF8B@smartserver.smartshare.dk> In-Reply-To: <23265462.6Emhk5qWAg@thomas> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [PATCH v2] eal/unix: allow creating thread with real-time priority Thread-Index: AdoHYtJwmo0TaKd3SBeTl9/xdqo7kgAAfswg References: <20231024125416.798897-1-thomas@monjalon.net> <20231025151352.995318-1-thomas@monjalon.net> <20231025083700.4e3e274c@hermes.local> <23265462.6Emhk5qWAg@thomas> From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Thomas Monjalon" , "Stephen Hemminger" Cc: , "David Marchand" , , "Anatoly Burakov" , "Dmitry Kozlyuk" , "Narcisa Ana Maria Vasile" , "Dmitry Malloy" , "Pallavi Kadam" , "Tyler Retzlaff" , "Andrew Rybchenko" , "Konstantin Ananyev" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > From: Thomas Monjalon [mailto:thomas@monjalon.net] > Sent: Wednesday, 25 October 2023 18.46 >=20 > 25/10/2023 17:37, Stephen Hemminger: > > On Wed, 25 Oct 2023 17:13:14 +0200 > > Thomas Monjalon wrote: > > > > > case RTE_THREAD_PRIORITY_REALTIME_CRITICAL: > > > + /* > > > + * WARNING: Real-time busy loop takes priority on kernel > threads, > > > + * making the system unstable. > > > + * There is also a known issue when using > rte_ring. > > > + */ > > > > I was thinking something like: > > > > static bool warned; > > if (!warned) { > > RTE_LOG(NOTICE, EAL, "Real time priority is unstable when > thread is polling without sleep\n"); > > warned =3D true; > > } >=20 > I'm not sure about bothering users. > They can fear something is wrong even if the developer took care of = it. > I think doc warnings for developers are more appropriate. > I've added notes in the API. I agree with Thomas on this. If you want the log message, please degrade it to INFO or DEBUG level. = It is only relevant when chasing problems, not for normal production - = and thus NOTICE is too high. Someone might build a kernel with options to keep non-dataplane threads = off some dedicated CPU cores, so they can be used for guaranteed = low-latency dataplane threads. We do. We don't use real-time priority, = though. For reference, we did some experiments (using this custom built kernel) = with a dedicated thread doing nothing but a loop calling = rte_rdtsc_precise() and registering the delta. Although the overwhelming = majority is ca. CPU 80 cycles, there are some big outliers at ca. 9,000 = CPU cycles. (Order of magnitude: ca. 45 of these big outliers per = minute.) Apparently some kernel threads steal some cycles from this = thread, regardless of our customizations. We haven't bothered analyzing = and optimizing it further. I think our experiment supports the need to allow kernel threads to run, = e.g. by calling sleep() or similar, when an EAL thread has real-time = priority.