From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 88C1841DC8 for ; Fri, 3 Mar 2023 15:33:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 69C3440EE3; Fri, 3 Mar 2023 15:33:45 +0100 (CET) Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by mails.dpdk.org (Postfix) with ESMTP id 2FF2040ED9 for ; Fri, 3 Mar 2023 15:33:44 +0100 (CET) Received: by mail-ot1-f45.google.com with SMTP id l16-20020a9d4c10000000b006944b17058cso1316502otf.2 for ; Fri, 03 Mar 2023 06:33:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1677854023; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=OBAGC01axCnCKzMaMXNysyOxxIRodtzosJpPks/7zt4=; b=JxJbg9lUYzV+X6PcG5Pa+ypKhFgYc6S9xFJf1XHtTdHjWhVCgUsBwnFYSVoE/P+g8y ZBG1CglUv9Av480nFIfdJu1QMAEq4trj6n22IPNWnjO8JQx9dcAtimBLEvG/BFm1/vCN yPpLidjc0HMl6LN5AKknP2EQGso9ciqFSgxUc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677854023; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OBAGC01axCnCKzMaMXNysyOxxIRodtzosJpPks/7zt4=; b=KlBO2vgZUEClJZg/ea3PIjDQsr7nLzhURDP/fFv2dJgmJfreGAKyzUWef4TxKTvVY/ kgy2HiDfzk4h5BVts0rKoMV+GWtlyvwOp6wAnisPgjX9tyg8TkwmvyWrOO4ufEGKQk9G x8uuWDTkacHqfuQlJlwCeji6KVS7c9vQOJRmX5EqDPN5MW57itJbvuqndV7pvsXQym6k doMVv6z2/lY/oR3Dh7tYVHNz8tXXVheqJO60mypEIrg296ZuNLtHAD12Kw/7BU4HmZeL sztKyn30XzqOUUC77PZe5OOfznqG5yz+LukiNq0wHohBCOJSD3FOOgQL0XMNyE+U0SB6 3F8w== X-Gm-Message-State: AO0yUKXWSnx+q/vjCuseQtRrNfGi+uY01zVcjPTWIWpeiFRMm2t33RWO 8YEQhDuBV9qTsm5pC1k8YSQ0xchird3bkSmPPFy9yQ== X-Google-Smtp-Source: AK7set9LdS0FoVyayN2OZ4OedqxtnS9CW5EdazrWjGU90L1PhDnzJ4vLR6uCgoyShIJmUt/KCBnpG0GW1rOZvwrIkoY= X-Received: by 2002:a9d:5f87:0:b0:68d:7557:f74c with SMTP id g7-20020a9d5f87000000b0068d7557f74cmr486102oti.7.1677854023374; Fri, 03 Mar 2023 06:33:43 -0800 (PST) MIME-Version: 1.0 References: <08944dac-937f-5433-6ce5-fb6fbb2536ed@redhat.com> <7c0fae6e-638a-a806-0029-b228c4956870@redhat.com> In-Reply-To: <7c0fae6e-638a-a806-0029-b228c4956870@redhat.com> From: Patrick Robb Date: Fri, 3 Mar 2023 09:33:32 -0500 Message-ID: Subject: Re: UNH CI skipped tests To: Kevin Traynor Cc: Lincoln Lavoie , ci@dpdk.org, Aaron Conole , David Marchand , Jeremy Spewock Content-Type: multipart/alternative; boundary="0000000000000c739d05f5ffd547" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --0000000000000c739d05f5ffd547 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Kevin, My thinking was that allowing new testruns simply due to a jenkinsfile change might produce testrun results which yielded no value for you or others. But, it sounds like you do not feel this way, and I agree having a testrun kick off due to Jenkinsfile changes would provide some debugging information to us UNH folks. So, since it sounds like it's alright with you, I'm going to go ahead and undo my previous changes on this front. And yes I'm sure all of those new tests popping up all of a sudden was unexpected... Some of those were tests run on a server which was temporarily down (ARM-Gigabyte), so once that went back online we were able to run additional tests. And then you know the others which were failing were never intended to run on 21.11. Thanks, Patrick On Fri, Mar 3, 2023 at 4:59=E2=80=AFAM Kevin Traynor = wrote: > On 02/03/2023 18:49, Patrick Robb wrote: > > Hi Kevin, > > > > Hi Patrick, > > > The FIPS test has the same issue as the cryptodev tests - it cannot run > on > > 21.11 due to a lacking dependency. We disabled the testing on the norma= l > > 21.11 branch but I missed 21.11-staging. Sorry about the oversight. All= 3 > > tests are now disabled on 21.11-staging. > > > > Cool, thanks. > > > The reason the two runs happened is that we were inadvertently polling > > DPDK-Stable AND our jenkinsfile repo for commits, and triggering new > builds > > for commits to either repo. I have disabled polling to our Jenkinsfile > > repo, which should limit new builds to just commits to DPDK-Stable. > > > > I'm not sure that was the reason for the second test run. I did that > deliberately... > > - I ran the initial run on the original commit (pass) > - I added a new dependency/build impacting commit and saw new tests and > failures > - I wanted to understand if the new tests and failures were related to > the single commit allowing additional tests to run, so... > - I force pushed back to the original commit for a second run on the > original commit. There I also saw the new tests so was able to confirm > it was unrelated to the new commit. > > Just a thought wrt Jenkins repo, I'm not sure how often it gets updated > and modifies tests. If a Jenkins repo update triggers a run, then it > will be the only delta from the previous build, so won't that make it > easier to debug if there is some issue? > > Otherwise dpdk-stable and Jenkins updates will be first tested together, > which may make it harder to debug failures. > > I will leave it to your more better judgement the best approach wrt > triggering builds on jenkins updates. Either is fine with me. > > thanks, > Kevin. > > > > Best, > > Patrick > > > > On Thu, Mar 2, 2023 at 9:04=E2=80=AFAM Kevin Traynor > wrote: > > > >> On 02/03/2023 13:22, Lincoln Lavoie wrote: > >>> Hi Kevin, > >>> > >>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be running on the > >> older > >>> LTS branches, because they don't include the required patches that we= re > >>> released as part of 22.11. So, you can ignore those failures. We'll > make > >>> sure those tests are excluded from future runs on the older > >>> staging branches. > >> > >> ok, cool, thanks. > >> > >>> > >>> In terms of the two runs, I'm not sure of the cause and we'll have to > >> look > >>> into that. > >>> > >> > >> No problem, it's not urgent or blocking. I will keep a closer eye on t= he > >> tests ran in future and just force a re-run if necessary. > >> > >> thanks, > >> Kevin. > >> > >>> Cheers, > >>> Lincoln > >>> > >>> On Thu, Mar 2, 2023 at 5:04=E2=80=AFAM Kevin Traynor > >> wrote: > >>> > >>>> Hi, > >>>> > >>>> I have a question about UNH CI periodic runs. I had 2x runs of CI on > >>>> 21.11-staging on the same commit, a few days apart. > >>>> > >>>> The issue I see is that the first test run came back all green, so I > >>>> assume good and I can push to 21.11 branch. However, the second run > >>>> comes back with additional tests that showed failures. > >>>> > >>>> So I'm wondering why there are additional tests in the second run? a= nd > >>>> if/how skipped tests are being reported? > >>>> > >>>> At least with the fips tests I have seen previously so I don't think > >>>> they are all newly enabled tests in the days in-between. > >>>> > >>>> Details below. > >>>> > >>>> thanks, > >>>> Kevin. > >>>> > >>>> Initial test run: > >>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23476/ > >>>> > >>>> Second test run: > >>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23560/ > >>>> > >>>> Additional tests in the second run: > >>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not reported in > >>>> dashboard?) > >>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc_autotest > >> (fail) > >>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_snow3g_autote= st > >>>> (fail) > >>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_mbuf > >>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_tests > >>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile > >>>> Ubuntu 20.04 ARM SVE - lpm_autotest > >>>> > >>>> > >>> > >> > >> > > > > --=20 Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu --0000000000000c739d05f5ffd547 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Kevin,

My thinking was that allowing= new testruns simply due to a jenkinsfile change might produce testrun resu= lts which yielded no value for you or others. But, it sounds like you do no= t feel this way, and I agree having a testrun kick off due to Jenkinsfile c= hanges would provide some debugging information to us UNH folks. So, since = it sounds like it's alright with you, I'm going to go ahead and und= o my previous changes on this front.=C2=A0

And yes= I'm sure all of those new tests popping up all of a sudden was unexpec= ted... Some of those were tests run on a server which was temporarily down = (ARM-Gigabyte), so once that went back online we were able to run additiona= l tests. And then you know the others which were failing were never intende= d to run on 21.11.=C2=A0

Thanks,
Patrick



On Fri, Mar 3, 2023 at 4:59=E2=80=AFAM Kevin Traynor <= ktraynor@redhat.com> wrote:
On 02/03/2023 18:= 49, Patrick Robb wrote:
> Hi Kevin,
>

Hi Patrick,

> The FIPS test has the same issue as the cryptodev tests - it cannot ru= n on
> 21.11 due to a lacking dependency. We disabled the testing on the norm= al
> 21.11 branch but I missed 21.11-staging. Sorry about the oversight. Al= l 3
> tests are now disabled on 21.11-staging.
>

Cool, thanks.

> The reason the two runs happened is that we were inadvertently polling=
> DPDK-Stable AND our jenkinsfile repo for commits, and triggering new b= uilds
> for commits to either repo. I have disabled polling to our Jenkinsfile=
> repo, which should limit new builds to just commits to DPDK-Stable. >

I'm not sure that was the reason for the second test run. I did that deliberately...

- I ran the initial run on the original commit (pass)
- I added a new dependency/build impacting commit and saw new tests and failures
- I wanted to understand if the new tests and failures were related to
the single commit allowing additional tests to run, so...
- I force pushed back to the original commit for a second run on the
original commit. There I also saw the new tests so was able to confirm
it was unrelated to the new commit.

Just a thought wrt Jenkins repo, I'm not sure how often it gets updated=
and modifies tests. If a Jenkins repo update triggers a run, then it
will be the only delta from the previous build, so won't that make it <= br> easier to debug if there is some issue?

Otherwise dpdk-stable and Jenkins updates will be first tested together, which may make it harder to debug failures.

I will leave it to your more better judgement the best approach wrt
triggering builds on jenkins updates. Either is fine with me.

thanks,
Kevin.


> Best,
> Patrick
>
> On Thu, Mar 2, 2023 at 9:04=E2=80=AFAM Kevin Traynor <ktraynor@redhat.com> wro= te:
>
>> On 02/03/2023 13:22, Lincoln Lavoie wrote:
>>> Hi Kevin,
>>>
>>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be runn= ing on the
>> older
>>> LTS branches, because they don't include the required patc= hes that were
>>> released as part of 22.11. So, you can ignore those failures.= =C2=A0 We'll make
>>> sure those tests are excluded from future runs on the older >>> staging branches.
>>
>> ok, cool, thanks.
>>
>>>
>>> In terms of the two runs, I'm not sure of the cause and we= 'll have to
>> look
>>> into that.
>>>
>>
>> No problem, it's not urgent or blocking. I will keep a closer = eye on the
>> tests ran in future and just force a re-run if necessary.
>>
>> thanks,
>> Kevin.
>>
>>> Cheers,
>>> Lincoln
>>>
>>> On Thu, Mar 2, 2023 at 5:04=E2=80=AFAM Kevin Traynor <ktraynor@redhat.com= >
>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have a question about UNH CI periodic runs. I had 2x run= s of CI on
>>>> 21.11-staging on the same commit, a few days apart.
>>>>
>>>> The issue I see is that the first test run came back all g= reen, so I
>>>> assume good and I can push to 21.11 branch. However, the s= econd run
>>>> comes back with additional tests that showed failures.
>>>>
>>>> So I'm wondering why there are additional tests in the= second run? and
>>>> if/how skipped tests are being reported?
>>>>
>>>> At least with the fips tests I have seen previously so I d= on't think
>>>> they are all newly enabled tests in the days in-between. >>>>
>>>> Details below.
>>>>
>>>> thanks,
>>>> Kevin.
>>>>
>>>> Initial test run:
>>>> https://dpdkdash= board.iol.unh.edu/results/dashboard/tarballs/23476/
>>>>
>>>> Second test run:
>>>> https://dpdkdash= board.iol.unh.edu/results/dashboard/tarballs/23560/
>>>>
>>>> Additional tests in the second run:
>>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not repor= ted in
>>>> dashboard?)
>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc= _autotest
>> (fail)
>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_sno= w3g_autotest
>>>> (fail)
>>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_= mbuf
>>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_= tests
>>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile >>>> Ubuntu 20.04 ARM SVE - lpm_autotest
>>>>
>>>>
>>>
>>
>>
>



--

Patrick Rob= b

Technical Service Manager

UNH InterOpera= bility Laboratory

21 Madbury Rd, Suite 100, Durham, NH 0= 3824

www.iol.unh.edu


--0000000000000c739d05f5ffd547--