From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ED6BEA034F; Fri, 12 Nov 2021 15:11:11 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5CF3D4113E; Fri, 12 Nov 2021 15:11:11 +0100 (CET) Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by mails.dpdk.org (Postfix) with ESMTP id 0EDCC410E3 for ; Fri, 12 Nov 2021 15:11:10 +0100 (CET) Received: by mail-ed1-f44.google.com with SMTP id w1so38321927edd.10 for ; Fri, 12 Nov 2021 06:11:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tdeyFZbw+xrsYXBFPtSQAjnhhx3Dab5lGeDVBFQhsjo=; b=DY7AUigcv1Xh4CLRTvlK4GdkFvYwRsq15LfYDkn9rru1aqzHUX3GDlkDQY6drxb9QI FIOvU7BmcoFOxpyc1XpVE4drS1s6uHVqaQcKRhXz6bIcFKlJEpDUjPrMl3/zY1rWoCRK Xp9kwcEoggglLDnmrq16W/vduNSt4i6AcF1lI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tdeyFZbw+xrsYXBFPtSQAjnhhx3Dab5lGeDVBFQhsjo=; b=j2lrG4Yr8CLSGc8WnNZEerXZdzv5vn2k4KvdnqjmRidiZj8+nO/+y1nI9PjIxjQzYa b20HCSCjudOOSwsxFbQfmvT88RS0TNNVs8i5qLOJ/DB9zIA34kjzwGeJ9xD2dTyB1Ctj eqCGMvyVlXKtU6rHVGoRUWWwuJKCPI4qg4Cd8BaPj/1749QE0j3NjtDRzO8+Avup416m mL/6oqYKBaQRiTfBpyw2h8b3XfWPOqy2K3GeJr/d5YCFAXuQnBRsCktppqA0knPxteda AnrMbQb4K1vQf3RLQ0Y6i5mlMfTohFvs235YSLDD7opTzbBOy3e9pWsQ6PiPTZQNJ60N QsXg== X-Gm-Message-State: AOAM532DP52dBrB0nhQOUW4eS0U+7iM/AozmUIYoyjAxPV6pWa1yNMDv pejF80MIWEQnkf+15UYZErUPbXR33m8kLuZY1fb2tA== X-Google-Smtp-Source: ABdhPJxBNNSBbPJSyCSSgnDS0uFlAF/r+E7n/hDZCz+RueNzRhWcSWzRbyLWv1N1SN/OXP5XSFgQCpdXv6Lw8alA5m0= X-Received: by 2002:a05:6402:331:: with SMTP id q17mr21917006edw.58.1636726269629; Fri, 12 Nov 2021 06:11:09 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Lincoln Lavoie Date: Fri, 12 Nov 2021 09:10:56 -0500 Message-ID: Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures To: David Marchand Cc: Cristian Dumitrescu , dev , Aaron Conole , Thomas Monjalon , "Yigit, Ferruh" , ci@dpdk.org Content-Type: multipart/alternative; boundary="000000000000e573bd05d0980784" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000e573bd05d0980784 Content-Type: text/plain; charset="UTF-8" On Fri, Nov 12, 2021 at 8:52 AM David Marchand wrote: > On Fri, Oct 8, 2021 at 9:24 AM wrote: > > > > https://bugs.dpdk.org/show_bug.cgi?id=826 > > > > Bug ID: 826 > > Summary: red_autotest random failures > > Product: DPDK > > Version: unspecified > > Hardware: All > > OS: All > > Status: UNCONFIRMED > > Severity: normal > > Priority: Normal > > Component: other > > Assignee: cristian.dumitrescu@intel.com > > Reporter: david.marchand@redhat.com > > CC: dev@dpdk.org, jasvinder.singh@intel.com > > Target Milestone: --- > > > > A recent failure can be found at: > > https://lab.dpdk.org/results/dashboard/patchsets/19223/ > > > > 50/94 DPDK:fast-tests / red_autotest FAIL > 0.86s > > exit status 1 > > > functional test 6 : use several queues (each with its own run-time data), > use several RED configurations (such that each > configuration is sharte_red by multiple queues), > increase average queue size to target level, > dequeue all packets until queue is empty, > confirm that average queue size is computed correctly > while queue is empty > (this is a larger scale version of functional test 3) > > queue config q avg before q avg after expected > difference % tolerance % result > 0 0 1022.0000 1022.0000 1016.0627 > 0.5843 5.0000 pass > 1 0 1022.0000 1022.0000 1016.0627 > 0.5843 5.0000 pass > 2 1 1022.0000 1022.0000 1010.1483 > 1.1733 5.0000 pass > 3 1 1022.0000 937.1660 1010.1483 > 7.2249 5.0000 fail > > -------------------------------------------------------------------------- > > > > This failure keeps on popping in the CI. > The bug report is one month old, with no reply. > > > I sent a proposal of removing red_autotest from the list executed by the > CI. > > https://patchwork.dpdk.org/project/dpdk/patch/20211027140458.2502-2-david.marchand@redhat.com/ > > It might be the best solution waiting for an analysis. > > > -- > David Marchand > > Hi David, My understanding is, removing the test would require removing it from the DPDK unit tests, we are just running the fast-tests suite for the unit tests. DPDK's unit test structure / framework does not allow removing or customizing the suite of tests beyond the suites. In the lab, Brandon has been looking into and trying different configurations for running the tests within the containers along the lines of the CPU pinning requirements that might be assumed by the unit tests. So far, everything he has tried has still had the similar failures / issues. We are still looking into it, so the bug is not sitting without action, just no final resolution. Cheers, Lincoln -- *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m) --000000000000e573bd05d0980784 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, Nov 12, 2021 at 8:52 AM David Marchand <= david.marchand@redhat.com&= gt; wrote:
On Fr= i, Oct 8, 2021 at 9:24 AM <bugzilla@dpdk.org> wrote:
>
> https://bugs.dpdk.org/show_bug.cgi?id=3D826
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Bug ID: 826
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Summary: red_autotest random = failures
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Product: DPDK
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Version: unspecified
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Hardware: All
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0OS: All >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Status: UNCONFIRMED
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Severity: normal
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Priority: Normal
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Component: other
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Assignee: cristian.dumitrescu@intel.co= m
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Reporter: david.marchand@redhat.com >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0CC: dev@dpdk.org, jasvinder.singh@intel.co= m
>=C2=A0 =C2=A0Target Milestone: ---
>
> A recent failure can be found at:
> https://lab.dpdk.org/results/dashboard/p= atchsets/19223/
>
> 50/94 DPDK:fast-tests / red_autotest=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0FAIL=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 0.86s
>=C2=A0 exit status 1


functional test 6 : use several queues (each with its own run-time data), =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 use several RED configurations (s= uch that each
configuration is sharte_red by multiple queues),
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 increase average queue size to ta= rget level,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dequeue all packets until queue i= s empty,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 confirm that average queue size i= s computed correctly
while queue is empty
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (this is a larger scale version o= f functional test 3)

queue=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 config=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0q avg before=C2=A0 =C2=A0q avg after=C2=A0 =C2=A0 expected
=C2=A0 =C2=A0 difference %=C2=A0 =C2=A0tolerance %=C2=A0 =C2=A0 result
0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2= =A0 =C2=A0 1016.0627
=C2=A0 =C2=A0 0.5843=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05.0000=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0pass
1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2= =A0 =C2=A0 1016.0627
=C2=A0 =C2=A0 0.5843=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05.0000=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0pass
2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2= =A0 =C2=A0 1010.1483
=C2=A0 =C2=A0 1.1733=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05.0000=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0pass
3=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 1022.0000=C2=A0 =C2=A0 =C2=A0 937.1660=C2=A0 =C2= =A0 =C2=A0 =C2=A01010.1483
=C2=A0 =C2=A0 7.2249=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A05.0000=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0fail
-------------------------------------<fail>--------------------------= -----------



This failure keeps on popping in the CI.
The bug report is one month old, with no reply.


I sent a proposal of removing red_autotest from the list executed by the CI= .
https:= //patchwork.dpdk.org/project/dpdk/patch/20211027140458.2502-2-david.marchan= d@redhat.com/

It might be the best solution waiting for an analysis.


--
David Marchand


Hi David,

My understanding is, removing the test would require removing it from= the DPDK unit tests, we are just running the fast-tests suite for the unit= tests.=C2=A0 DPDK's unit test structure / framework does not allow rem= oving or customizing the suite of tests beyond the suites.=C2=A0

In the lab, Brandon has been lookin= g into and trying different configurations for running the tests within the= containers along the lines of the CPU pinning requirements that might be a= ssumed by the unit tests. So far, everything he has tried has still had the= similar failures / issues.=C2=A0 We are still looking into it, so the bug = is not sitting without action, just no final resolution.

Cheers,
Lincoln--
Lincoln Lavoi= e
Principal Engineer, Broadband Technologies
21= Madbury Rd., Ste. 100, Durham, NH 03824
+1-603-674-2755 (m)

--000000000000e573bd05d0980784--