Re: UNH CI skipped tests

[Kevin Traynor <ktraynor@redhat.com>]

On 02/03/2023 18:49, Patrick Robb wrote:
> Hi Kevin,
> 

Hi Patrick,

> The FIPS test has the same issue as the cryptodev tests - it cannot run on
> 21.11 due to a lacking dependency. We disabled the testing on the normal
> 21.11 branch but I missed 21.11-staging. Sorry about the oversight. All 3
> tests are now disabled on 21.11-staging.
> 

Cool, thanks.

> The reason the two runs happened is that we were inadvertently polling
> DPDK-Stable AND our jenkinsfile repo for commits, and triggering new builds
> for commits to either repo. I have disabled polling to our Jenkinsfile
> repo, which should limit new builds to just commits to DPDK-Stable.
> 

I'm not sure that was the reason for the second test run. I did that 
deliberately...

- I ran the initial run on the original commit (pass)
- I added a new dependency/build impacting commit and saw new tests and 
failures
- I wanted to understand if the new tests and failures were related to 
the single commit allowing additional tests to run, so...
- I force pushed back to the original commit for a second run on the 
original commit. There I also saw the new tests so was able to confirm 
it was unrelated to the new commit.

Just a thought wrt Jenkins repo, I'm not sure how often it gets updated 
and modifies tests. If a Jenkins repo update triggers a run, then it 
will be the only delta from the previous build, so won't that make it 
easier to debug if there is some issue?

Otherwise dpdk-stable and Jenkins updates will be first tested together, 
which may make it harder to debug failures.

I will leave it to your more better judgement the best approach wrt 
triggering builds on jenkins updates. Either is fine with me.

thanks,
Kevin.


> Best,
> Patrick
> 
> On Thu, Mar 2, 2023 at 9:04 AM Kevin Traynor <ktraynor@redhat.com> wrote:
> 
>> On 02/03/2023 13:22, Lincoln Lavoie wrote:
>>> Hi Kevin,
>>>
>>> The FIPS and crypto (ZUC / SNOW) testing shouldn't be running on the
>> older
>>> LTS branches, because they don't include the required patches that were
>>> released as part of 22.11. So, you can ignore those failures.  We'll make
>>> sure those tests are excluded from future runs on the older
>>> staging branches.
>>
>> ok, cool, thanks.
>>
>>>
>>> In terms of the two runs, I'm not sure of the cause and we'll have to
>> look
>>> into that.
>>>
>>
>> No problem, it's not urgent or blocking. I will keep a closer eye on the
>> tests ran in future and just force a re-run if necessary.
>>
>> thanks,
>> Kevin.
>>
>>> Cheers,
>>> Lincoln
>>>
>>> On Thu, Mar 2, 2023 at 5:04 AM Kevin Traynor <ktraynor@redhat.com>
>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have a question about UNH CI periodic runs. I had 2x runs of CI on
>>>> 21.11-staging on the same commit, a few days apart.
>>>>
>>>> The issue I see is that the first test run came back all green, so I
>>>> assume good and I can push to 21.11 branch. However, the second run
>>>> comes back with additional tests that showed failures.
>>>>
>>>> So I'm wondering why there are additional tests in the second run? and
>>>> if/how skipped tests are being reported?
>>>>
>>>> At least with the fips tests I have seen previously so I don't think
>>>> they are all newly enabled tests in the days in-between.
>>>>
>>>> Details below.
>>>>
>>>> thanks,
>>>> Kevin.
>>>>
>>>> Initial test run:
>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23476/
>>>>
>>>> Second test run:
>>>> https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/23560/
>>>>
>>>> Additional tests in the second run:
>>>> Ubuntu 20.04 VM - dpdk_fips_validation (warning, not reported in
>>>> dashboard?)
>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_zuc_autotest
>> (fail)
>>>> NA NA (Linux container host) 10000 Mbps - cryptodev_sw_snow3g_autotest
>>>> (fail)
>>>> Arm Intel XL710-QDA2 4000 Mbps - lpm_autotest, unit_tests_mbuf
>>>> Arm Broadcom 25000 Mbps - unit_tests_mbuf,nic_single_core_tests
>>>> Ubuntu 20.04 ARM GCC Cross compile - dpdk_meson_compile
>>>> Ubuntu 20.04 ARM SVE - lpm_autotest
>>>>
>>>>
>>>
>>
>>
>