From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B006241D63; Mon, 27 Feb 2023 17:08:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9177440A84; Mon, 27 Feb 2023 17:08:50 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 4DBD240A7D for ; Mon, 27 Feb 2023 17:08:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677514128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=; b=ViTabvo5Vs7lP/h57slmTmZr6YPbn700EK+dGGBP86C23F72uARs6rjgAryQA9lEsYqxwH 1yzxq7EjYzl7t8/k4zt88wCeUwuXK5UXric7F/3qMZB3xmFLdE8uZ3kdW/E+t19BeWHk92 ijc/STut1nWHPy5LoyeytOvsh0Yn4v8= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-590-yJ3Br27yOTqpK2caHURdpw-1; Mon, 27 Feb 2023 11:08:47 -0500 X-MC-Unique: yJ3Br27yOTqpK2caHURdpw-1 Received: by mail-pj1-f72.google.com with SMTP id gm13-20020a17090b100d00b0023704a72ca5so1804195pjb.4 for ; Mon, 27 Feb 2023 08:08:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=; b=HzA1QQInGmKs/QOFkZbEbsfxRym/8Eai8pNNzR9lS8bbbLNzNFSM+HkMXIBaOJ4wh0 YxmolcTKluINS6yYtDB9fUBdjqJXEzOoCSGdNjBlpsp6+/7+nlpe1N+BdsbHbdNsjO/2 E3LVm+KdiqMXXcn+1fhHnKTbdWn9RI/9fBxDis4o8r1oZZ4trmr2qxkMG2f/MbmZILRA PNQlE6qFI510HDAAfTNI1R/zWoHwWLiTauwj416OUGwTffsHBIpj08uNo3uf2wNiaznY HU1AC2DR6zr8sTOMC/F5/uitD9gfwhliFICMEk9Hy1Lx4bPOrpVj2GnvtDRrsScxq1JJ 0bDQ== X-Gm-Message-State: AO0yUKU3JU00I61RG+3UlSPqaiHxL2VQXGPKU8Bf2POz98n8iz2MZ44s vsuHbr8R55IPWs+VkmiohV6nU2A1UTCkjKtNhAbAiJZUN16yz/sAa3qPNu1B/kMCzgJuEjc+fNr MCmDwCDPtf7ZW711vdCc= X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id j12-20020a17090a3e0c00b002308d09962bmr5037629pjc.7.1677514126527; Mon, 27 Feb 2023 08:08:46 -0800 (PST) X-Google-Smtp-Source: AK7set+vS8XR8ywxbiriQ1UkVRvsQRn3oh8pSeG1fzrbYm+dWWOMYOPCYB2KgpyOaiyFyFvK27X1vCpDtwYhhY+cvok= X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id j12-20020a17090a3e0c00b002308d09962bmr5037617pjc.7.1677514126204; Mon, 27 Feb 2023 08:08:46 -0800 (PST) MIME-Version: 1.0 References: <20230224173637.243266-1-harry.van.haaren@intel.com> In-Reply-To: <20230224173637.243266-1-harry.van.haaren@intel.com> From: David Marchand Date: Mon, 27 Feb 2023 17:08:35 +0100 Message-ID: Subject: Re: [PATCH] service: split tests to perf and autotest to avoid spurious CI failures To: Harry van Haaren Cc: dev@dpdk.org, dpdklab@iol.unh.edu, ci@dpdk.org, Honnappa.Nagarahalli@arm.com, mattias.ronnblom@ericsson.com, mb@smartsharesystems.com, roretzla@linux.microsoft.com, aconole@redhat.com, bruce.richardson@intel.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hello, On Fri, Feb 24, 2023 at 7:04 PM Harry van Haaren wrote: > > On some CI runs, some service-cores tests spuriously fail as the > service lcore thread is not actually scheduled by the OS in the > given amount of time. > > Increasing timeouts has not resolved the issue in the CI, so the > solution in this patch is to move them to a separate perf test > suite. > > Signed-off-by: Harry van Haaren > > --- > > See DPDK ML discussion in this thread: > http://mails.dpdk.org/archives/dev/2023-February/263523.html > --- > app/test/meson.build | 1 + > app/test/test_service_cores.c | 32 +++++++++++++++++++++++++++----- > 2 files changed, 28 insertions(+), 5 deletions(-) > > diff --git a/app/test/meson.build b/app/test/meson.build > index f34d19e3c3..2db5ccf4ff 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -287,6 +287,7 @@ perf_test_names = [ > 'pie_perf', > 'distributor_perf_autotest', > 'pmd_perf_autotest', > + 'service_perf_autotest', > 'stack_perf_autotest', > 'stack_lf_perf_autotest', > 'rand_perf_autotest', > diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c > index 637fcd7cf9..06653dfdef 100644 > --- a/app/test/test_service_cores.c > +++ b/app/test/test_service_cores.c > @@ -1022,17 +1022,12 @@ static struct unit_test_suite service_tests = { > TEST_CASE_ST(dummy_register, NULL, service_name), > TEST_CASE_ST(dummy_register, NULL, service_get_by_name), > TEST_CASE_ST(dummy_register, NULL, service_dump), > - TEST_CASE_ST(dummy_register, NULL, service_attr_get), > - TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get), > TEST_CASE_ST(dummy_register, NULL, service_probe_capability), > TEST_CASE_ST(dummy_register, NULL, service_start_stop), > TEST_CASE_ST(dummy_register, NULL, service_lcore_add_del), > - TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop), > TEST_CASE_ST(dummy_register, NULL, service_lcore_en_dis_able), > TEST_CASE_ST(dummy_register, NULL, service_mt_unsafe_poll), > TEST_CASE_ST(dummy_register, NULL, service_mt_safe_poll), > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe), > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe), > TEST_CASE_ST(dummy_register, NULL, service_may_be_active), > TEST_CASE_ST(dummy_register, NULL, service_active_two_cores), > TEST_CASES_END() /**< NULL terminate unit test array */ > @@ -1046,3 +1041,30 @@ test_service_common(void) > } > > REGISTER_TEST_COMMAND(service_autotest, test_service_common); > + > + > +/* The tests below have been split from the auto-test suite, as the What is the auto-test suite? Plus "as the when" reads strange. In the end, I don't think it helps much to have this comment in the code. The commitlog is supposed to tell the story, so I would simply remove this comment. > + * when they are run in a cloud CI environment they can give false-positive > + * errors, due to the service-cores not being scheduled by the OS. > + */ > +static struct unit_test_suite service_perf_tests = { > + .suite_name = "service core test suite", Maybe add "performance" in the name, so we have a uniquely named testsuite object. > + .setup = testsuite_setup, > + .teardown = testsuite_teardown, > + .unit_test_cases = { > + TEST_CASE_ST(dummy_register, NULL, service_attr_get), > + TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get), > + TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop), Looking at service_lcore_running_check(), don't you think service_may_be_active() and service_active_two_cores() are also subject to race? > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe), > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe), > + TEST_CASES_END() /**< NULL terminate unit test array */ > + } > +}; > + > +static int > +test_service_perf(void) > +{ > + return unit_test_suite_runner(&service_perf_tests); > +} > + > +REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf); > -- > 2.34.1 > -- David Marchand