Re: [dpdk-ci] [dpdk-dev] [PATCH 00/14] Unit tests fixes for CI

[Aaron Conole <aconole@redhat.com>]

Thomas Monjalon <thomas@monjalon.net> writes:

> 04/06/2019 17:49, Michael Santana Francisco:
>> On 6/4/19 4:59 AM, David Marchand wrote:
>> > - the "perf" tests are taking way too long for my taste,

+1 here.

>> 
>> We should still fix them. However I don't know if we should be running 
>> the perf test for every job and every patch on travis. It takes too 
>> long. The travis queue will be delayed too far behind for it to be of 
>> any use.
>> 
>> OTOH we could have one job as part of the travis build dedicated to 
>> running tests (or just perf test). It's still time consuming but better 
>> than running the test on every travis job. For this to work we would 
>> need to decreased the timeout for the perf tests as the timeout for it 
>> and the travis are both 10 minutes
>
> +Cc ci@dpdk.org
>
> I don't think we should run the perf tests in basic CI like Travis.
> We can run perf tests if the purpose is to compare the performance
> with previous releases, as some other tests in the community lab.

+1 - some of the perf tests aren't going to complete in any sort of
reasonable time.  While we could claim it's a separate problem, we
should also not enable something that will make the travis runs so much
longer.

I do like the idea of running tests in the travis build, and I think it
would make sense to have just a single job for it (or maybe one for
clang and one for gcc?  maybe even that is overkill).

I would rather not do performance tests during the travis run, though.
It doesn't really make sense.  Travis isn't any kind of an 'optimized'
environment, so I don't know what 'performance' should mean.