From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A630441D63 for ; Mon, 27 Feb 2023 17:08:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F6C4410FB; Mon, 27 Feb 2023 17:08:53 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6A156410DC for ; Mon, 27 Feb 2023 17:08:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677514130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=; b=UX2e1B8F13+mroSjq2mu+ctqxKMMHEi/6meWo2iGxNB7ZrGn1odXP17utcvQ+pDI5LO95e MTyNrCBMYjx8G/T0DUS2uk9+5xrTo0TkG2Gt/MFTme2zDy0/WaAvQpXYsNgpaxDs7euHUa Rpkfc0pPDPv14ZJZjtqhvJuzZK7bsqE= Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-213-R5US7lc8Pr61hwI1IvNOUQ-1; Mon, 27 Feb 2023 11:08:47 -0500 X-MC-Unique: R5US7lc8Pr61hwI1IvNOUQ-1 Received: by mail-pl1-f199.google.com with SMTP id ki15-20020a170903068f00b0019ce282dc68so3794298plb.6 for ; Mon, 27 Feb 2023 08:08:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=; b=OnK3cdTtbGzPSTI6Uy3ubE0MWwvhq6DLjsqajFsxyj6Nr/qzvqp8bWg7emagWTxMQS /QsyIzrk64CSwMZK0wSFqR556DMsPP0zR7R9kGqz75Gny90UL2ItWobjo5kp2JCm2e3D Am7F7hcXodWxW4TqxfQTWj63fGTk2zrKz8sHak0GuwPCTAGVL12k8W9EjZ7JRbc4HVWK OsHZMKr9msDqgMyKtxG1VklXjFi1Jj60XOV7QzMc6VHo2MqqY3dUoarclG2Bufl45Yq5 u25NfPuDso3p7xwai9GTB0JjawNGT+ocGZKCS7Ddz4HXsyI4TQSJQIZ0YQ/h5Vv8ldC0 BrKA== X-Gm-Message-State: AO0yUKUi9UAEonEWPZ751o5Ps8/s/UwcQt0jtltln8PX7tDde7G+lzdx Fu04oLmltXH+m3nNRgpdn7CAajNuXO7T6TUI9RGpzpUjwVd684NZMs8mHiy9e3YroJ6ALstRjpB yI9EvioSxONLO9NIp2w== X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id j12-20020a17090a3e0c00b002308d09962bmr5037627pjc.7.1677514126526; Mon, 27 Feb 2023 08:08:46 -0800 (PST) X-Google-Smtp-Source: AK7set+vS8XR8ywxbiriQ1UkVRvsQRn3oh8pSeG1fzrbYm+dWWOMYOPCYB2KgpyOaiyFyFvK27X1vCpDtwYhhY+cvok= X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id j12-20020a17090a3e0c00b002308d09962bmr5037617pjc.7.1677514126204; Mon, 27 Feb 2023 08:08:46 -0800 (PST) MIME-Version: 1.0 References: <20230224173637.243266-1-harry.van.haaren@intel.com> In-Reply-To: <20230224173637.243266-1-harry.van.haaren@intel.com> From: David Marchand Date: Mon, 27 Feb 2023 17:08:35 +0100 Message-ID: Subject: Re: [PATCH] service: split tests to perf and autotest to avoid spurious CI failures To: Harry van Haaren Cc: dev@dpdk.org, dpdklab@iol.unh.edu, ci@dpdk.org, Honnappa.Nagarahalli@arm.com, mattias.ronnblom@ericsson.com, mb@smartsharesystems.com, roretzla@linux.microsoft.com, aconole@redhat.com, bruce.richardson@intel.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Hello, On Fri, Feb 24, 2023 at 7:04 PM Harry van Haaren wrote: > > On some CI runs, some service-cores tests spuriously fail as the > service lcore thread is not actually scheduled by the OS in the > given amount of time. > > Increasing timeouts has not resolved the issue in the CI, so the > solution in this patch is to move them to a separate perf test > suite. > > Signed-off-by: Harry van Haaren > > --- > > See DPDK ML discussion in this thread: > http://mails.dpdk.org/archives/dev/2023-February/263523.html > --- > app/test/meson.build | 1 + > app/test/test_service_cores.c | 32 +++++++++++++++++++++++++++----- > 2 files changed, 28 insertions(+), 5 deletions(-) > > diff --git a/app/test/meson.build b/app/test/meson.build > index f34d19e3c3..2db5ccf4ff 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -287,6 +287,7 @@ perf_test_names = [ > 'pie_perf', > 'distributor_perf_autotest', > 'pmd_perf_autotest', > + 'service_perf_autotest', > 'stack_perf_autotest', > 'stack_lf_perf_autotest', > 'rand_perf_autotest', > diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c > index 637fcd7cf9..06653dfdef 100644 > --- a/app/test/test_service_cores.c > +++ b/app/test/test_service_cores.c > @@ -1022,17 +1022,12 @@ static struct unit_test_suite service_tests = { > TEST_CASE_ST(dummy_register, NULL, service_name), > TEST_CASE_ST(dummy_register, NULL, service_get_by_name), > TEST_CASE_ST(dummy_register, NULL, service_dump), > - TEST_CASE_ST(dummy_register, NULL, service_attr_get), > - TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get), > TEST_CASE_ST(dummy_register, NULL, service_probe_capability), > TEST_CASE_ST(dummy_register, NULL, service_start_stop), > TEST_CASE_ST(dummy_register, NULL, service_lcore_add_del), > - TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop), > TEST_CASE_ST(dummy_register, NULL, service_lcore_en_dis_able), > TEST_CASE_ST(dummy_register, NULL, service_mt_unsafe_poll), > TEST_CASE_ST(dummy_register, NULL, service_mt_safe_poll), > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe), > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe), > TEST_CASE_ST(dummy_register, NULL, service_may_be_active), > TEST_CASE_ST(dummy_register, NULL, service_active_two_cores), > TEST_CASES_END() /**< NULL terminate unit test array */ > @@ -1046,3 +1041,30 @@ test_service_common(void) > } > > REGISTER_TEST_COMMAND(service_autotest, test_service_common); > + > + > +/* The tests below have been split from the auto-test suite, as the What is the auto-test suite? Plus "as the when" reads strange. In the end, I don't think it helps much to have this comment in the code. The commitlog is supposed to tell the story, so I would simply remove this comment. > + * when they are run in a cloud CI environment they can give false-positive > + * errors, due to the service-cores not being scheduled by the OS. > + */ > +static struct unit_test_suite service_perf_tests = { > + .suite_name = "service core test suite", Maybe add "performance" in the name, so we have a uniquely named testsuite object. > + .setup = testsuite_setup, > + .teardown = testsuite_teardown, > + .unit_test_cases = { > + TEST_CASE_ST(dummy_register, NULL, service_attr_get), > + TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get), > + TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop), Looking at service_lcore_running_check(), don't you think service_may_be_active() and service_active_two_cores() are also subject to race? > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe), > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe), > + TEST_CASES_END() /**< NULL terminate unit test array */ > + } > +}; > + > +static int > +test_service_perf(void) > +{ > + return unit_test_suite_runner(&service_perf_tests); > +} > + > +REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf); > -- > 2.34.1 > -- David Marchand