From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB49D4320A; Thu, 26 Oct 2023 18:07:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 447A540A8A; Thu, 26 Oct 2023 18:07:14 +0200 (CEST) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mails.dpdk.org (Postfix) with ESMTP id 12702402D4 for <dev@dpdk.org>; Thu, 26 Oct 2023 18:07:13 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 8D1815C015B; Thu, 26 Oct 2023 12:07:10 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 26 Oct 2023 12:07:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1698336430; x=1698422830; bh=bq1WfJaKUpjQrQjC/7EwlU+qu8rch9kQzxi zx2RdahY=; b=lOBSyRUGHSSndGSjkMmvp+z0C+e5noUMZETmh6bHJ/kbd/wYpaz neKPzh4uL5k+LgVrLfpTxKbdAUw4WLcOVwXUq/j2rDDyKuteMFm86X1kEsfUZyb+ NxNwJTcw5urDZ+bVUtP3Vx7dWm+HZMwBN/NGL2ChVS8MlyLppf1PEuj8OAjExis4 B7Va+dKw++M7upfMFmpLjm8WSMeJvbFoWkjC9ViLNmqr4ljUoYoN2WRDcrbxfXy1 kHSbXk7VrfU5gE2R66JW9g80VKolXn++rWQxXoHqU6Spx05SukcgaljAL5SOyEDN O9gj26Zc8SN5Pde+Uefws276qDQbUqni2tQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1698336430; x=1698422830; bh=bq1WfJaKUpjQrQjC/7EwlU+qu8rch9kQzxi zx2RdahY=; b=cI5KNHL9YWqOIJWJdv5xbhN+kD155QzjqzsQlX3qZO7xmLHQany BKXmZE9zQ+j25kgsXoAze+FjbPUDuVsYbUfnNXagewaRntW3Y/LJyXGcDnsx6Zf6 E/L15YqFN2Nh+PvbbIBWFgKtm7zVFPodKJUIt2ifeCLjWVCrUpXh/fqWEd+unBf4 ZmbgQBZOobyUOZnHngMhoFYDNq0vrrCeBbsvogrT0koADSu4iBbFWV9+c/If+Fkw rRir1BjWYxwN7kjcLXpFr62Y5eBV/0aGcgsHjEk8ibiBLnl5ktQEUyKPracejeH9 z7/ZIpezyc8uR8RS6rOntNd/52szWuK+3uA== X-ME-Sender: <xms:rY46ZYNPzt3POoBIlkCwfMSrx9d1Ye4ujqVV4kM8za8oDuMJy0pmYA> <xme:rY46Ze-oAfsksYiZIe1OdT7upUD0EyWY3c1iz5d6D6EZJrRECWVyfn602TLR_TLmg rTZe68Khk2sQcLT5g> X-ME-Received: <xmr:rY46ZfT_L1N76E0QRY9sgyx1f4ofZf7QSEak9cnRITLjpf3sF-rJOLU_CZW_viI6jDo9Ud6eMnGTW96L58bLq-IinQ> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrledvgdeljecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkfgjfhgggfgtsehtqhertddttddunecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepfefhjeeluedvvedtuddtuedtvefhieejtefhffeujefhteduudev tdektdeikeffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: <xmx:rY46ZQufOF3MB-YfsdLHbXGzDMmlJ9Gl1n4aGXIkkVpvtYYdkktYCw> <xmx:rY46ZQdeZ_RyX21q5HTF8GxV9QKCP-LOZ0_44R2eEh73ecPXzvUXfQ> <xmx:rY46ZU3v1y-EIwjdMEHYdZkp48o5weXFZ2XX2FMO2IThQnvywHZl3w> <xmx:ro46ZY4RQJfz4M51ZOoJHcFBn3pOUA4d2PmMujNEf-y0ue4jEH6moQ> Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 26 Oct 2023 12:07:08 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: Morten =?ISO-8859-1?Q?Br=F8rup?= <mb@smartsharesystems.com>, Bruce Richardson <bruce.richardson@intel.com> Cc: David Marchand <david.marchand@redhat.com>, dev@dpdk.org, stephen@networkplumber.org Subject: Re: [PATCH v3 0/2] allow creating thread with real-time priority Date: Thu, 26 Oct 2023 18:07:06 +0200 Message-ID: <2651241.BddDVKsqQX@thomas> In-Reply-To: <ZTqLnPs8o1ybKpFZ@bricha3-MOBL.ger.corp.intel.com> References: <20231024125416.798897-1-thomas@monjalon.net> <98CBD80474FA8B44BF855DF32C47DC35E9EF9D@smartserver.smartshare.dk> <ZTqLnPs8o1ybKpFZ@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org 26/10/2023 17:54, Bruce Richardson: > On Thu, Oct 26, 2023 at 04:59:51PM +0200, Morten Br=F8rup wrote: > > > From: Morten Br=F8rup [mailto:mb@smartsharesystems.com] > > > Sent: Thursday, 26 October 2023 16.50 > > >=20 > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > Sent: Thursday, 26 October 2023 16.31 > > > > > > > > 26/10/2023 16:08, Morten Br=F8rup: > > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > > > Sent: Thursday, 26 October 2023 16.05 > > > > > > > > > > > > 26/10/2023 15:57, Morten Br=F8rup: > > > > > > > > From: Morten Br=F8rup [mailto:mb@smartsharesystems.com] > > > > > > > > Sent: Thursday, 26 October 2023 15.45 > > > > > > > > > > > > > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > > > > > > Sent: Thursday, 26 October 2023 15.37 > > > > > > > > > > > > > > > > > > 25/10/2023 18:31, Thomas Monjalon: > > > > > > > > > > Real-time thread priority was been forbidden on Unix > > > > > > > > > > because of problems they can cause. > > > > > > > > > > Warnings and helpers are added to avoid deadlocks, > > > > > > > > > > so real-time can be allowed on all systems. > > > > > > > > > > > > > > > > > > Unit test is failing: > > > > > > > > > DPDK:fast-tests / threads_autotest TIMEOUT 600.01 s > > > > > > > > > > > > > > > > > > It is seen in only 1 target (maybe the failure occurence = is > > > random): > > > > > > > > > Debian 11 (Buster) (ARM) | PASS > > > > > > > > > Fedora 37 (ARM) | PASS > > > > > > > > > CentOS Stream 9 (ARM) | FAIL > > > > > > > > > Fedora 38 (ARM) | PASS > > > > > > > > > Fedora 38 (ARM Clang) | PASS > > > > > > > > > Ubuntu 20.04 (ARM) | PASS > > > > > > > > > > > > > > > > > > I need to send a v4 with new implementation and better co= mments. > > > > > > > > > The Unix sleep will be upgraded from 1 ns to 1 us in case= it makes > > > a > > > > > > > > > difference. > > > > > > > > > > > > > > > > It will not make a difference. The kernel will go through t= he > > > sleeping > > > > > > steps, > > > > > > > > then wake up again and see the real-time thread is ready to= run, and > > > > then > > > > > > > > immediately schedule it. > > > > > > > > > > > > > > > > For testing purposes, consider sleeping 10 milliseconds or = something > > > > > > > > significant like that. > > > > > > > > > > > > > > A bit more details... > > > > > > > > > > > > > > In our recent tests, nanosleep() itself took around 50 us. So= you need > > > > to > > > > > > sleep longer than that for your thread not to be runnable when = the > > > > nanosleep() > > > > > > wakes up again, because 50 us has already passed in "nanosleep > > > overhead". > > > > > > > 10 milliseconds provides plenty of margin, and corresponds to= 10 > > > jiffies > > > > on > > > > > > a 1000 Hz kernel. (I don't know if it makes any difference for = the > > > kernel > > > > > > scheduler if the timer crosses a jiffy border or not.) > > > > > > > > > > > > 10 ms looks like an eternity. > > > > > > > > > > Agree. It is only for functional testing, not for production! > > > > > > > > Realtime thread won't make any sense if we have to insert a long sl= eep. > > >=20 > > > It seems David came to our rescue here! > > >=20 > > > I have just tried running our test again with prctl(PR_SET_TIMERSLACK= ) of 1 > > > ns, and the nanosleep(1 ns) delay dropped from ca. 50 us to ca. 2.5 u= s. > > >=20 > > > The timeout parameter to epoll_wait() is in milliseconds, which is us= eless for > > > low-latency. > > > Perhaps real-time threads can be used with epoll() combined with time= rfd for > > > nanosecond resolution timeout. > >=20 > > Or epoll_pwait2(), which has nanosecond resolution timeout. > >=20 > > Unfortunately, rte_epoll_wait() is not an experimental API anymore, so = we cannot change its timeout parameter from milliseconds to micro- or nanos= econds. We would have to introduce a new API for this. > >=20 >=20 > Just an idea - can we change the timeout parameter to float rather than i= nt, > and then use function versioning for backward compatibility for any > binaries passing int? > That way the actual meaning of the parameter doesn't change, but it still > allows sub-millisecond values (all-be-it with some loss of accuracy due to > float). Sorry I'm not following why you want to use rte_epoll_wait()? If the realtime thread has some blocking system calls, no sleep is needed I think. =46or average realtime thread, I suggest the API rte_thread_yield_realtime() which could wait for 1 ms or less by default. =46or smaller sleep, we can use PR_SET_TIMERSLACK and rte_delay_us_sleep(). If we provide an API for PR_SET_TIMERSLACK, we could adapt the duration of rte_thread_yield_realtime() dynamically after calling prctl().