From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62FC041DC3 for ; Fri, 3 Mar 2023 09:37:49 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3E52C40687; Fri, 3 Mar 2023 09:37:49 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 21EB8400D6 for ; Fri, 3 Mar 2023 09:37:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677832667; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XxqBXGYLDpv13KucGxmwmMM5XgnO+0koMeQbn7JdIFk=; b=CEAasZ3r3HjHnnAmh9bjbO3iQANPI9/FNyD8C83Je4/9PJKpUWEVWNLGpELJwdW1PX9CO9 8EoE0NHLTLkXM6lPRrhiw8f883CP/y40sVx9VSk4tDXBuOUZsN61BeEO9F4Dzjq5bl7zhJ 237hAQ0GazxPtfk2J3lgSOiVXqCsp54= Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-245-9A51fCqDNuiP0J0RuDdQ8g-1; Fri, 03 Mar 2023 03:37:44 -0500 X-MC-Unique: 9A51fCqDNuiP0J0RuDdQ8g-1 Received: by mail-pl1-f199.google.com with SMTP id s8-20020a170902b18800b0019c92f56a8aso1067883plr.22 for ; Fri, 03 Mar 2023 00:37:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677832663; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XxqBXGYLDpv13KucGxmwmMM5XgnO+0koMeQbn7JdIFk=; b=vrt9+nxflvDoKPKSK54sY3diLJVknWXwQbXxG8gYr7sIWRt3aL+oit82e/9rLrlWrQ yYgbWH8zcnzVyzU87N8Js0VHXMtGFvesfG6lwto3XbIZwyPhXgZLdpwIPfbxv76bdp8o vkJKtk6vf+pTP4m4J5/lRNIWg1iSCcB0zwLEryTpxpcsVUTvMWDNktqM9MR8bcOTh93x gcn33A9tymUXa/oRuXfwmrIzTp5BvX6LNBEcvvzqV2Yo9nveE7sK0P3wxjDfk+P37+E+ YnK15QS86KvTW4OZ+uId4p37ebtXrutuq71GmfwjNJC9sHhQOVwYiyVHQfAe9mMCW6u5 JK+w== X-Gm-Message-State: AO0yUKW3C4F0R8sTH40q9YQPXD1lf5GHy1bTPSl2jHeXfQKphq2AhEQ9 yFwauIPJga76SfPhR8Ws2pd0KMbiOZeRlcL8QjX2OrmLpiFFsdtIEZLobMyzb+4nQ1xGxn/o3qR EeUGondHWJ21YkMPcmw== X-Received: by 2002:a17:903:2581:b0:19a:8bc7:d814 with SMTP id jb1-20020a170903258100b0019a8bc7d814mr430941plb.13.1677832663226; Fri, 03 Mar 2023 00:37:43 -0800 (PST) X-Google-Smtp-Source: AK7set+WkhX8BsjhyLGahNGvEO4p2Xd/3nfhVqBGTP3c3ZUY1Fbw3NDSein0XkLFynj54J6CjxL+Pt1sSbaKyk6GJNw= X-Received: by 2002:a17:903:2581:b0:19a:8bc7:d814 with SMTP id jb1-20020a170903258100b0019a8bc7d814mr430931plb.13.1677832662903; Fri, 03 Mar 2023 00:37:42 -0800 (PST) MIME-Version: 1.0 References: <20230224173637.243266-1-harry.van.haaren@intel.com> In-Reply-To: From: David Marchand Date: Fri, 3 Mar 2023 09:37:31 +0100 Message-ID: Subject: Re: [PATCH] service: split tests to perf and autotest to avoid spurious CI failures To: Harry van Haaren Cc: dev@dpdk.org, dpdklab@iol.unh.edu, ci@dpdk.org, Honnappa.Nagarahalli@arm.com, mattias.ronnblom@ericsson.com, mb@smartsharesystems.com, roretzla@linux.microsoft.com, aconole@redhat.com, bruce.richardson@intel.com, "Mcnamara, John" X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On Mon, Feb 27, 2023 at 5:08=E2=80=AFPM David Marchand wrote: > > Hello, > > On Fri, Feb 24, 2023 at 7:04 PM Harry van Haaren > wrote: > > > > On some CI runs, some service-cores tests spuriously fail as the > > service lcore thread is not actually scheduled by the OS in the > > given amount of time. > > > > Increasing timeouts has not resolved the issue in the CI, so the > > solution in this patch is to move them to a separate perf test > > suite. > > > > Signed-off-by: Harry van Haaren > > > > --- > > > > See DPDK ML discussion in this thread: > > http://mails.dpdk.org/archives/dev/2023-February/263523.html > > --- > > app/test/meson.build | 1 + > > app/test/test_service_cores.c | 32 +++++++++++++++++++++++++++----- > > 2 files changed, 28 insertions(+), 5 deletions(-) > > > > diff --git a/app/test/meson.build b/app/test/meson.build > > index f34d19e3c3..2db5ccf4ff 100644 > > --- a/app/test/meson.build > > +++ b/app/test/meson.build > > @@ -287,6 +287,7 @@ perf_test_names =3D [ > > 'pie_perf', > > 'distributor_perf_autotest', > > 'pmd_perf_autotest', > > + 'service_perf_autotest', > > 'stack_perf_autotest', > > 'stack_lf_perf_autotest', > > 'rand_perf_autotest', > > diff --git a/app/test/test_service_cores.c b/app/test/test_service_core= s.c > > index 637fcd7cf9..06653dfdef 100644 > > --- a/app/test/test_service_cores.c > > +++ b/app/test/test_service_cores.c > > @@ -1022,17 +1022,12 @@ static struct unit_test_suite service_tests = =3D { > > TEST_CASE_ST(dummy_register, NULL, service_name), > > TEST_CASE_ST(dummy_register, NULL, service_get_by_name)= , > > TEST_CASE_ST(dummy_register, NULL, service_dump), > > - TEST_CASE_ST(dummy_register, NULL, service_attr_get), > > - TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_g= et), > > TEST_CASE_ST(dummy_register, NULL, service_probe_capabi= lity), > > TEST_CASE_ST(dummy_register, NULL, service_start_stop), > > TEST_CASE_ST(dummy_register, NULL, service_lcore_add_de= l), > > - TEST_CASE_ST(dummy_register, NULL, service_lcore_start_= stop), > > TEST_CASE_ST(dummy_register, NULL, service_lcore_en_dis= _able), > > TEST_CASE_ST(dummy_register, NULL, service_mt_unsafe_po= ll), > > TEST_CASE_ST(dummy_register, NULL, service_mt_safe_poll= ), > > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt= _safe), > > - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt= _unsafe), > > TEST_CASE_ST(dummy_register, NULL, service_may_be_activ= e), > > TEST_CASE_ST(dummy_register, NULL, service_active_two_c= ores), > > TEST_CASES_END() /**< NULL terminate unit test array */ > > @@ -1046,3 +1041,30 @@ test_service_common(void) > > } > > > > REGISTER_TEST_COMMAND(service_autotest, test_service_common); > > + > > + > > +/* The tests below have been split from the auto-test suite, as the > > What is the auto-test suite? > Plus "as the when" reads strange. > > In the end, I don't think it helps much to have this comment in the code. > The commitlog is supposed to tell the story, so I would simply remove > this comment. > > > > + * when they are run in a cloud CI environment they can give false-pos= itive > > + * errors, due to the service-cores not being scheduled by the OS. > > + */ > > +static struct unit_test_suite service_perf_tests =3D { > > + .suite_name =3D "service core test suite", > > Maybe add "performance" in the name, so we have a uniquely named > testsuite object. > > > > + .setup =3D testsuite_setup, > > + .teardown =3D testsuite_teardown, > > + .unit_test_cases =3D { > > + TEST_CASE_ST(dummy_register, NULL, service_attr_get), > > + TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_g= et), > > + TEST_CASE_ST(dummy_register, NULL, service_lcore_start_= stop), > > Looking at service_lcore_running_check(), don't you think > service_may_be_active() and service_active_two_cores() are also > subject to race? > > > > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt= _safe), > > + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt= _unsafe), > > + TEST_CASES_END() /**< NULL terminate unit test array */ > > + } > > +}; > > + > > +static int > > +test_service_perf(void) > > +{ > > + return unit_test_suite_runner(&service_perf_tests); > > +} > > + > > +REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf); > > -- > > 2.34.1 > > ping. --=20 David Marchand