DPDK CI discussions
 help / color / mirror / Atom feed
From: Kevin Traynor <ktraynor@redhat.com>
To: Patrick Robb <probb@iol.unh.edu>
Cc: Lincoln Lavoie <lylavoie@iol.unh.edu>,
	ci@dpdk.org, Aaron Conole <aconole@redhat.com>,
	David Marchand <david.marchand@redhat.com>,
	Jeremy Spewock <jspewock@iol.unh.edu>
Subject: Re: UNH CI skipped tests
Date: Fri, 3 Mar 2023 15:15:30 +0000	[thread overview]
Message-ID: <44234778-6720-76f4-8546-297994419643@redhat.com> (raw)
In-Reply-To: <CAJvnSUCq_KDav7BB_yw7jqadBFhNNC+26iavrCRx8VXa5PpN7Q@mail.gmail.com>

On 03/03/2023 14:33, Patrick Robb wrote:
> Hi Kevin,
> 
> My thinking was that allowing new testruns simply due to a jenkinsfile
> change might produce testrun results which yielded no value for you or
> others. But, it sounds like you do not feel this way, and I agree having a
> testrun kick off due to Jenkinsfile changes would provide some debugging
> information to us UNH folks. So, since it sounds like it's alright with
> you, I'm going to go ahead and undo my previous changes on this front.
> 

Fine by me.

> And yes I'm sure all of those new tests popping up all of a sudden was
> unexpected... Some of those were tests run on a server which was
> temporarily down (ARM-Gigabyte), so once that went back online we were able
> to run additional tests. And then you know the others which were failing
> were never intended to run on 21.11.
> 

ok, I think that's all the differences explained.

It might be worth flagging in the dashboard if some valid tests were 
skipped for a case like this. I've no idea if that's easy to do, so not 
sure if it's worth the effort.

Thanks Lincoln and Patrick for the quick response and explanations,

Kevin.

> Thanks,
> Patrick
> 
> 
> 
> On Fri, Mar 3, 2023 at 4:59 AM Kevin Traynor <ktraynor@redhat.com> wrote:
> 
>> On 02/03/2023 18:49, Patrick Robb wrote:
>>> Hi Kevin,
>>>
>>
>> Hi Patrick,
>>
>>> The FIPS test has the same issue as the cryptodev tests - it cannot run
>> on
>>> 21.11 due to a lacking dependency. We disabled the testing on the normal
>>> 21.11 branch but I missed 21.11-staging. Sorry about the oversight. All 3
>>> tests are now disabled on 21.11-staging.
>>>
>>
>> Cool, thanks.
>>
>>> The reason the two runs happened is that we were inadvertently polling
>>> DPDK-Stable AND our jenkinsfile repo for commits, and triggering new
>> builds
>>> for commits to either repo. I have disabled polling to our Jenkinsfile
>>> repo, which should limit new builds to just commits to DPDK-Stable.
>>>
>>
>> I'm not sure that was the reason for the second test run. I did that
>> deliberately...
>>
>> - I ran the initial run on the original commit (pass)
>> - I added a new dependency/build impacting commit and saw new tests and
>> failures
>> - I wanted to understand if the new tests and failures were related to
>> the single commit allowing additional tests to run, so...
>> - I force pushed back to the original commit for a second run on the
>> original commit. There I also saw the new tests so was able to confirm
>> it was unrelated to the new commit.
>>
>> Just a thought wrt Jenkins repo, I'm not sure how often it gets updated
>> and modifies tests. If a Jenkins repo update triggers a run, then it
>> will be the only delta from the previous build, so won't that make it
>> easier to debug if there is some issue?
>>
>> Otherwise dpdk-stable and Jenkins updates will be first tested together,
>> which may make it harder to debug failures.
>>
>> I will leave it to your more better judgement the best approach wrt
>> triggering builds on jenkins updates. Either is fine with me.
>>
>> thanks,
>> Kevin.
>>
>>
>>> Best,
>>> Patrick
>>>
>>> On Thu, Mar 2, 2023 at 9:04 AM Kevin Traynor <ktraynor@redhat.com>
>> wrote:
>>>
>>>> On 02/03/2023 13:22, Lincoln Lavoie wrote:
>>>>> Hi Kevin,
>>>>>
>>>>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be running on the
>>>> older
>>>>> LTS branches, because they don't include the required patches that were
>>>>> released as part of 22.11. So, you can ignore those failures.  We'll
>> make
>>>>> sure those tests are excluded from future runs on the older
>>>>> staging branches.
>>>>
>>>> ok, cool, thanks.
>>>>
>>>>>
>>>>> In terms of the two runs, I'm not sure of the cause and we'll have to
>>>> look
>>>>> into that.
>>>>>
>>>>
>>>> No problem, it's not urgent or blocking. I will keep a closer eye on the
>>>> tests ran in future and just force a re-run if necessary.
>>>>
>>>> thanks,
>>>> Kevin.
>>>>
>>>>> Cheers,
>>>>> Lincoln
>>>>>
>>>>> On Thu, Mar 2, 2023 at 5:04 AM Kevin Traynor <ktraynor@redhat.com>
>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have a question about UNH CI periodic runs. I had 2x runs of CI on
>>>>>> 21.11-staging on the same commit, a few days apart.
>>>>>>
>>>>>> The issue I see is that the first test run came back all green, so I
>>>>>> assume good and I can push to 21.11 branch. However, the second run
>>>>>> comes back with additional tests that showed failures.
>>>>>>
>>>>>> So I'm wondering why there are additional tests in the second run? and
>>>>>> if/how skipped tests are being reported?
>>>>>>
>>>>>> At least with the fips tests I have seen previously so I don't think
>>>>>> they are all newly enabled tests in the days in-between.
>>>>>>
>>>>>> Details below.
>>>>>>
>>>>>> thanks,
>>>>>> Kevin.
>>>>>>
>>>>>> Initial test run:
>>>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23476/
>>>>>>
>>>>>> Second test run:
>>>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23560/
>>>>>>
>>>>>> Additional tests in the second run:
>>>>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not reported in
>>>>>> dashboard?)
>>>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc_autotest
>>>> (fail)
>>>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_snow3g_autotest
>>>>>> (fail)
>>>>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_mbuf
>>>>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_tests
>>>>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile
>>>>>> Ubuntu 20.04 ARM SVE - lpm_autotest
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
> 


      reply	other threads:[~2023-03-03 15:15 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-02 10:04 Kevin Traynor
2023-03-02 13:22 ` Lincoln Lavoie
2023-03-02 14:04   ` Kevin Traynor
2023-03-02 18:49     ` Patrick Robb
2023-03-03  9:59       ` Kevin Traynor
2023-03-03 14:33         ` Patrick Robb
2023-03-03 15:15           ` Kevin Traynor [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=44234778-6720-76f4-8546-297994419643@redhat.com \
    --to=ktraynor@redhat.com \
    --cc=aconole@redhat.com \
    --cc=ci@dpdk.org \
    --cc=david.marchand@redhat.com \
    --cc=jspewock@iol.unh.edu \
    --cc=lylavoie@iol.unh.edu \
    --cc=probb@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).