From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D9BB41BBE; Fri, 3 Feb 2023 16:16:20 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4A91442D2F; Fri, 3 Feb 2023 16:16:20 +0100 (CET) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by mails.dpdk.org (Postfix) with ESMTP id 3E4E24021E; Fri, 3 Feb 2023 16:16:18 +0100 (CET) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id E7D365C00E0; Fri, 3 Feb 2023 10:16:17 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 03 Feb 2023 10:16:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1675437377; x= 1675523777; bh=5YEqF3rptnhTrzEeMvSIPoxh9QMc3hMAS/J7Xu8IKeI=; b=o aHrjntMNfWjNyaI+Mb/lQ4YYTyfl0GZLbfIPNmd/di3DZO9XMR3/pvqD3v/OrK7k HabD7zs/fIU9Z5/jw6xHImb41cn+YSszYNYkBm14q4/UmBYi0as3hXHplq1WdxA3 RI3CrUbcFzb8K796EYKbJLreaajIbmrCXqeqHuB6CoKHyks+XzZfh8GtJSSoCQ5n 9ODmEbbo/5vR3K1Ce/CmSb5tfNWpjI0wdA2V1SBx7CVPXVGga3ezpWh4K8Xxrfg1 BLoQFOLRDGeCvoYlRmN9P6JiKbgPNb+lEh2hMRFmr7JA3Z+Ac7BMLeTQX1fdpYf1 OW25MCoAEYlCXeopwv+Fg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1675437377; x= 1675523777; bh=5YEqF3rptnhTrzEeMvSIPoxh9QMc3hMAS/J7Xu8IKeI=; b=n ry/Zr2Rk2QE4B9RMSV+Jch+qLcFShB5LKJiQK023NRuXsfBFoZUMKfqyeXnWbX2z IDKDmpJ+zv0aJVmIIpTvgg+PERbCF+ZljjdJxN0v8Lc9s3J+F1Pg8gc9rmD8mGN+ OzX4Z5A9K9pmMeSttp4EbXKe5Q7TgdNrK8ZJ5M2iY4nuBqZJpEEnO7w66AA61ZYY 8A4BIe85K3YF1iL2yYBfKq1HurIx5x9BEB6ldUy84rNN58Qx5jVwStizln4kw2RP owEoknTMvOtO0pGxMkE7Tj83zz7+KL6eRUi2LOqECGoiw76OYGmtlDF+mxObzc56 86Z+Y1hmskGT6InHTjhaw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudegtddgjedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpedtjeeiieefhedtfffgvdelteeufeefheeujefgueetfedttdei kefgkeduhedtgfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 3 Feb 2023 10:16:16 -0500 (EST) From: Thomas Monjalon To: David Marchand , "Van Haaren, Harry" Cc: "dev@dpdk.org" , "dpdklab@iol.unh.edu" , "ci@dpdk.org" , "Honnappa.Nagarahalli@arm.com" , "mattias. ronnblom" , Morten =?ISO-8859-1?Q?Br=F8rup?= , Tyler Retzlaff , Aaron Conole Subject: Re: [PATCH v3] test/service: fix spurious failures by extending timeout Date: Fri, 03 Feb 2023 16:16:15 +0100 Message-ID: <21760850.EfDdHjke4D@thomas> In-Reply-To: References: <20221006081729.578475-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 03/02/2023 16:03, Van Haaren, Harry: > From: Van Haaren, Harry > > > The timeout approach just does not have its place in a functional test. > > > Either this test is rewritten, or it must go to the performance tests > > > list so that we stop getting false positives. > > > Can you work on this? > > > > I'll investigate various approaches on Thursday and reply here with suggested > > next steps. > > I've identified 3 checks that fail in CI (from the above log outputs), all 3 cases > Have different dlays: 100 ms delay, 200 ms delay and 1000ms. > In the CI, the service-core just hasn't been scheduled (yet) and causes the "failure". > > Option 1) > One option is to while(1) loop, waiting for the service-thread to be scheduled. This can be > seen as "increasing the timeout", however in this case the test-case would be errored > not in the test-code, but in the meson-test runner as a timeout (with a 10sec default?) > The benefit here is that massively increasing (~1sec or less to 10 sec) will cover all/many > of the CI timeouts. > > Option 2) > Move to perf-tests, and not run these in a noisy-CI environment where the results are not > consistent enough to have value. This would mean that the tests are not run in CI for the > 3 checks in question are below, they all *require* the service core to be scheduled: > service_attr_get() -> requires service core to run for service stats to increment > service_lcore_attr_get() -> requires service core to run for lcore stats to increment > service_lcore_start_stop() -> requires service to run to to ensure service-func itself executes. > > I don't see how we can "improve" option 2 to not require the service-thread to be scheduled by the OS.. > And the only way to make the OS schedule it in the CI more consistently is to give it more time? We are talking about seconds. There are setups where scheduling a thread is taking seconds? > Thoughts and input welcomed, I'm happy to make the code changes themselves, its small effort > For both option 1 & 2. For time-sensitive tests, yes they should be in perf tests category. As David said earlier, no timeout approach in functional tests.