Re: UNH CI skipped tests

[Patrick Robb <probb@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 5465 bytes --]

Hi Kevin,

My thinking was that allowing new testruns simply due to a jenkinsfile
change might produce testrun results which yielded no value for you or
others. But, it sounds like you do not feel this way, and I agree having a
testrun kick off due to Jenkinsfile changes would provide some debugging
information to us UNH folks. So, since it sounds like it's alright with
you, I'm going to go ahead and undo my previous changes on this front.

And yes I'm sure all of those new tests popping up all of a sudden was
unexpected... Some of those were tests run on a server which was
temporarily down (ARM-Gigabyte), so once that went back online we were able
to run additional tests. And then you know the others which were failing
were never intended to run on 21.11.

Thanks,
Patrick



On Fri, Mar 3, 2023 at 4:59 AM Kevin Traynor <ktraynor@redhat.com> wrote:

> On 02/03/2023 18:49, Patrick Robb wrote:
> > Hi Kevin,
> >
>
> Hi Patrick,
>
> > The FIPS test has the same issue as the cryptodev tests - it cannot run
> on
> > 21.11 due to a lacking dependency. We disabled the testing on the normal
> > 21.11 branch but I missed 21.11-staging. Sorry about the oversight. All 3
> > tests are now disabled on 21.11-staging.
> >
>
> Cool, thanks.
>
> > The reason the two runs happened is that we were inadvertently polling
> > DPDK-Stable AND our jenkinsfile repo for commits, and triggering new
> builds
> > for commits to either repo. I have disabled polling to our Jenkinsfile
> > repo, which should limit new builds to just commits to DPDK-Stable.
> >
>
> I'm not sure that was the reason for the second test run. I did that
> deliberately...
>
> - I ran the initial run on the original commit (pass)
> - I added a new dependency/build impacting commit and saw new tests and
> failures
> - I wanted to understand if the new tests and failures were related to
> the single commit allowing additional tests to run, so...
> - I force pushed back to the original commit for a second run on the
> original commit. There I also saw the new tests so was able to confirm
> it was unrelated to the new commit.
>
> Just a thought wrt Jenkins repo, I'm not sure how often it gets updated
> and modifies tests. If a Jenkins repo update triggers a run, then it
> will be the only delta from the previous build, so won't that make it
> easier to debug if there is some issue?
>
> Otherwise dpdk-stable and Jenkins updates will be first tested together,
> which may make it harder to debug failures.
>
> I will leave it to your more better judgement the best approach wrt
> triggering builds on jenkins updates. Either is fine with me.
>
> thanks,
> Kevin.
>
>
> > Best,
> > Patrick
> >
> > On Thu, Mar 2, 2023 at 9:04 AM Kevin Traynor <ktraynor@redhat.com>
> wrote:
> >
> >> On 02/03/2023 13:22, Lincoln Lavoie wrote:
> >>> Hi Kevin,
> >>>
> >>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be running on the
> >> older
> >>> LTS branches, because they don't include the required patches that were
> >>> released as part of 22.11. So, you can ignore those failures.  We'll
> make
> >>> sure those tests are excluded from future runs on the older
> >>> staging branches.
> >>
> >> ok, cool, thanks.
> >>
> >>>
> >>> In terms of the two runs, I'm not sure of the cause and we'll have to
> >> look
> >>> into that.
> >>>
> >>
> >> No problem, it's not urgent or blocking. I will keep a closer eye on the
> >> tests ran in future and just force a re-run if necessary.
> >>
> >> thanks,
> >> Kevin.
> >>
> >>> Cheers,
> >>> Lincoln
> >>>
> >>> On Thu, Mar 2, 2023 at 5:04 AM Kevin Traynor <ktraynor@redhat.com>
> >> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I have a question about UNH CI periodic runs. I had 2x runs of CI on
> >>>> 21.11-staging on the same commit, a few days apart.
> >>>>
> >>>> The issue I see is that the first test run came back all green, so I
> >>>> assume good and I can push to 21.11 branch. However, the second run
> >>>> comes back with additional tests that showed failures.
> >>>>
> >>>> So I'm wondering why there are additional tests in the second run? and
> >>>> if/how skipped tests are being reported?
> >>>>
> >>>> At least with the fips tests I have seen previously so I don't think
> >>>> they are all newly enabled tests in the days in-between.
> >>>>
> >>>> Details below.
> >>>>
> >>>> thanks,
> >>>> Kevin.
> >>>>
> >>>> Initial test run:
> >>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23476/
> >>>>
> >>>> Second test run:
> >>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23560/
> >>>>
> >>>> Additional tests in the second run:
> >>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not reported in
> >>>> dashboard?)
> >>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc_autotest
> >> (fail)
> >>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_snow3g_autotest
> >>>> (fail)
> >>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_mbuf
> >>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_tests
> >>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile
> >>>> Ubuntu 20.04 ARM SVE - lpm_autotest
> >>>>
> >>>>
> >>>
> >>
> >>
> >
>
>

-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu

[-- Attachment #2: Type: text/html, Size: 9118 bytes --]