From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id B006241D63;
Mon, 27 Feb 2023 17:08:50 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 9177440A84;
Mon, 27 Feb 2023 17:08:50 +0100 (CET)
Received: from us-smtp-delivery-124.mimecast.com
(us-smtp-delivery-124.mimecast.com [170.10.133.124])
by mails.dpdk.org (Postfix) with ESMTP id 4DBD240A7D
for <dev@dpdk.org>; Mon, 27 Feb 2023 17:08:49 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
s=mimecast20190719; t=1677514128;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
in-reply-to:in-reply-to:references:references;
bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=;
b=ViTabvo5Vs7lP/h57slmTmZr6YPbn700EK+dGGBP86C23F72uARs6rjgAryQA9lEsYqxwH
1yzxq7EjYzl7t8/k4zt88wCeUwuXK5UXric7F/3qMZB3xmFLdE8uZ3kdW/E+t19BeWHk92
ijc/STut1nWHPy5LoyeytOvsh0Yn4v8=
Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com
[209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS
(version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id
us-mta-590-yJ3Br27yOTqpK2caHURdpw-1; Mon, 27 Feb 2023 11:08:47 -0500
X-MC-Unique: yJ3Br27yOTqpK2caHURdpw-1
Received: by mail-pj1-f72.google.com with SMTP id
gm13-20020a17090b100d00b0023704a72ca5so1804195pjb.4
for <dev@dpdk.org>; Mon, 27 Feb 2023 08:08:47 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=cc:to:subject:message-id:date:from:in-reply-to:references
:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id
:reply-to;
bh=pL2vDsQlGTfWvUN+eCnLC085dHmfZ/vdiZLG6lqhwPQ=;
b=HzA1QQInGmKs/QOFkZbEbsfxRym/8Eai8pNNzR9lS8bbbLNzNFSM+HkMXIBaOJ4wh0
YxmolcTKluINS6yYtDB9fUBdjqJXEzOoCSGdNjBlpsp6+/7+nlpe1N+BdsbHbdNsjO/2
E3LVm+KdiqMXXcn+1fhHnKTbdWn9RI/9fBxDis4o8r1oZZ4trmr2qxkMG2f/MbmZILRA
PNQlE6qFI510HDAAfTNI1R/zWoHwWLiTauwj416OUGwTffsHBIpj08uNo3uf2wNiaznY
HU1AC2DR6zr8sTOMC/F5/uitD9gfwhliFICMEk9Hy1Lx4bPOrpVj2GnvtDRrsScxq1JJ
0bDQ==
X-Gm-Message-State: AO0yUKU3JU00I61RG+3UlSPqaiHxL2VQXGPKU8Bf2POz98n8iz2MZ44s
vsuHbr8R55IPWs+VkmiohV6nU2A1UTCkjKtNhAbAiJZUN16yz/sAa3qPNu1B/kMCzgJuEjc+fNr
MCmDwCDPtf7ZW711vdCc=
X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id
j12-20020a17090a3e0c00b002308d09962bmr5037629pjc.7.1677514126527;
Mon, 27 Feb 2023 08:08:46 -0800 (PST)
X-Google-Smtp-Source: AK7set+vS8XR8ywxbiriQ1UkVRvsQRn3oh8pSeG1fzrbYm+dWWOMYOPCYB2KgpyOaiyFyFvK27X1vCpDtwYhhY+cvok=
X-Received: by 2002:a17:90a:3e0c:b0:230:8d09:962b with SMTP id
j12-20020a17090a3e0c00b002308d09962bmr5037617pjc.7.1677514126204; Mon, 27 Feb
2023 08:08:46 -0800 (PST)
MIME-Version: 1.0
References: <20230224173637.243266-1-harry.van.haaren@intel.com>
In-Reply-To: <20230224173637.243266-1-harry.van.haaren@intel.com>
From: David Marchand <david.marchand@redhat.com>
Date: Mon, 27 Feb 2023 17:08:35 +0100
Message-ID: <CAJFAV8yayLKk-w8sAAfm54dVJPFZvdmuX4OFQuhN7gmAKGPdhg@mail.gmail.com>
Subject: Re: [PATCH] service: split tests to perf and autotest to avoid
spurious CI failures
To: Harry van Haaren <harry.van.haaren@intel.com>
Cc: dev@dpdk.org, dpdklab@iol.unh.edu, ci@dpdk.org,
Honnappa.Nagarahalli@arm.com, mattias.ronnblom@ericsson.com,
mb@smartsharesystems.com, roretzla@linux.microsoft.com, aconole@redhat.com,
bruce.richardson@intel.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset="UTF-8"
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Hello,
On Fri, Feb 24, 2023 at 7:04 PM Harry van Haaren
<harry.van.haaren@intel.com> wrote:
>
> On some CI runs, some service-cores tests spuriously fail as the
> service lcore thread is not actually scheduled by the OS in the
> given amount of time.
>
> Increasing timeouts has not resolved the issue in the CI, so the
> solution in this patch is to move them to a separate perf test
> suite.
>
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
>
> ---
>
> See DPDK ML discussion in this thread:
> http://mails.dpdk.org/archives/dev/2023-February/263523.html
> ---
> app/test/meson.build | 1 +
> app/test/test_service_cores.c | 32 +++++++++++++++++++++++++++-----
> 2 files changed, 28 insertions(+), 5 deletions(-)
>
> diff --git a/app/test/meson.build b/app/test/meson.build
> index f34d19e3c3..2db5ccf4ff 100644
> --- a/app/test/meson.build
> +++ b/app/test/meson.build
> @@ -287,6 +287,7 @@ perf_test_names = [
> 'pie_perf',
> 'distributor_perf_autotest',
> 'pmd_perf_autotest',
> + 'service_perf_autotest',
> 'stack_perf_autotest',
> 'stack_lf_perf_autotest',
> 'rand_perf_autotest',
> diff --git a/app/test/test_service_cores.c b/app/test/test_service_cores.c
> index 637fcd7cf9..06653dfdef 100644
> --- a/app/test/test_service_cores.c
> +++ b/app/test/test_service_cores.c
> @@ -1022,17 +1022,12 @@ static struct unit_test_suite service_tests = {
> TEST_CASE_ST(dummy_register, NULL, service_name),
> TEST_CASE_ST(dummy_register, NULL, service_get_by_name),
> TEST_CASE_ST(dummy_register, NULL, service_dump),
> - TEST_CASE_ST(dummy_register, NULL, service_attr_get),
> - TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get),
> TEST_CASE_ST(dummy_register, NULL, service_probe_capability),
> TEST_CASE_ST(dummy_register, NULL, service_start_stop),
> TEST_CASE_ST(dummy_register, NULL, service_lcore_add_del),
> - TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop),
> TEST_CASE_ST(dummy_register, NULL, service_lcore_en_dis_able),
> TEST_CASE_ST(dummy_register, NULL, service_mt_unsafe_poll),
> TEST_CASE_ST(dummy_register, NULL, service_mt_safe_poll),
> - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe),
> - TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe),
> TEST_CASE_ST(dummy_register, NULL, service_may_be_active),
> TEST_CASE_ST(dummy_register, NULL, service_active_two_cores),
> TEST_CASES_END() /**< NULL terminate unit test array */
> @@ -1046,3 +1041,30 @@ test_service_common(void)
> }
>
> REGISTER_TEST_COMMAND(service_autotest, test_service_common);
> +
> +
> +/* The tests below have been split from the auto-test suite, as the
What is the auto-test suite?
Plus "as the when" reads strange.
In the end, I don't think it helps much to have this comment in the code.
The commitlog is supposed to tell the story, so I would simply remove
this comment.
> + * when they are run in a cloud CI environment they can give false-positive
> + * errors, due to the service-cores not being scheduled by the OS.
> + */
> +static struct unit_test_suite service_perf_tests = {
> + .suite_name = "service core test suite",
Maybe add "performance" in the name, so we have a uniquely named
testsuite object.
> + .setup = testsuite_setup,
> + .teardown = testsuite_teardown,
> + .unit_test_cases = {
> + TEST_CASE_ST(dummy_register, NULL, service_attr_get),
> + TEST_CASE_ST(dummy_register, NULL, service_lcore_attr_get),
> + TEST_CASE_ST(dummy_register, NULL, service_lcore_start_stop),
Looking at service_lcore_running_check(), don't you think
service_may_be_active() and service_active_two_cores() are also
subject to race?
> + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_safe),
> + TEST_CASE_ST(dummy_register, NULL, service_app_lcore_mt_unsafe),
> + TEST_CASES_END() /**< NULL terminate unit test array */
> + }
> +};
> +
> +static int
> +test_service_perf(void)
> +{
> + return unit_test_suite_runner(&service_perf_tests);
> +}
> +
> +REGISTER_TEST_COMMAND(service_perf_autotest, test_service_perf);
> --
> 2.34.1
>
--
David Marchand