[dpdk-ci] Compiling dpdk in non compilation tests

[dpdk-ci] Compiling dpdk in non compilation tests

[David Marchand]

Looking at CI reports this morning, I noticed this on the series "ice:
update base code".

https://lab.dpdk.org/results/dashboard/patchsets/13992/

- Compilation is failing on OpenSuse, but I think this is because its
toolchain does not have AVX512 support.
This seems to be a regression introduced by the series, indeed, cool.


- On Ubuntu 18.04, build is fine, but unit tests are ko.
Looking into the unit tests zipped logs, I can see that:
* the unit tests job builds its own binaries,
* the unit tests environment is not the same as the build test
environment (only copying significant differences below):

Build:
The Meson build system
Version: 0.55.3
...
C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the host machine: cc ld.bfd 2.30
...
Run-time dependency libpcap found: YES 1.8.1
Has header "pcap.h" with dependency libpcap: YES
...
Fetching value of define "__AVX2__" : 1
Fetching value of define "__AVX512BW__" : 1
Fetching value of define "__AVX512CD__" : 1
Fetching value of define "__AVX512DQ__" : 1
Fetching value of define "__AVX512F__" : 1
Fetching value of define "__AVX512VL__" : 1
...
Run-time dependency jansson found: YES 2.11
...

vs

Unit tests:
The Meson build system
Version: 0.53.1
...
C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the host machine: cc GNU ld.bfd 2.30
...
Run-time dependency libpcap found: NO (tried pkgconfig)
Library pcap found: NO
...
Fetching value of define "__AVX2__" :
Fetching value of define "__AVX512BW__" :
Fetching value of define "__AVX512CD__" :
Fetching value of define "__AVX512DQ__" :
Fetching value of define "__AVX512F__" :
Fetching value of define "__AVX512VL__" :
...
Run-time dependency jansson found: NO (tried pkgconfig and cmake)
...


The Unit tests env does not have a AVX512 enabled toolchain, or
something in meson disables it, hence the compilation fails and the
unit tests are marked as failed while it has nothing to do with
functional tests.


Please rework the CI pipeline so that the Unit tests job only does
functional tests, while binaries are built by the Build test job.


Thanks.

-- 
David Marchand

Re: [dpdk-ci] Compiling dpdk in non compilation tests

[Richardson, Bruce]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Sunday, October 25, 2020 8:36 AM
> To: dpdklab@iol.unh.edu; Lincoln Lavoie <lylavoie@iol.unh.edu>
> Cc: ci@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Richardson, Bruce
> <bruce.richardson@intel.com>
> Subject: Compiling dpdk in non compilation tests
> 
> Looking at CI reports this morning, I noticed this on the series "ice:
> update base code".
> 
> https://lab.dpdk.org/results/dashboard/patchsets/13992/
> 
> - Compilation is failing on OpenSuse, but I think this is because its
> toolchain does not have AVX512 support.
> This seems to be a regression introduced by the series, indeed, cool.
> 
> 
> - On Ubuntu 18.04, build is fine, but unit tests are ko.
> Looking into the unit tests zipped logs, I can see that:
> * the unit tests job builds its own binaries,
> * the unit tests environment is not the same as the build test
> environment (only copying significant differences below):
> 
> Build:
> The Meson build system
> Version: 0.55.3
> ...
> C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
> 7.5.0-3ubuntu1~18.04) 7.5.0")
> C linker for the host machine: cc ld.bfd 2.30
> ...
> Run-time dependency libpcap found: YES 1.8.1
> Has header "pcap.h" with dependency libpcap: YES
> ...
> Fetching value of define "__AVX2__" : 1
> Fetching value of define "__AVX512BW__" : 1
> Fetching value of define "__AVX512CD__" : 1
> Fetching value of define "__AVX512DQ__" : 1
> Fetching value of define "__AVX512F__" : 1
> Fetching value of define "__AVX512VL__" : 1
> ...
> Run-time dependency jansson found: YES 2.11
> ...
> 
> vs
> 
> Unit tests:
> The Meson build system
> Version: 0.53.1
> ...
> C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
> 7.5.0-3ubuntu1~18.04) 7.5.0")
> C linker for the host machine: cc GNU ld.bfd 2.30
> ...
> Run-time dependency libpcap found: NO (tried pkgconfig)
> Library pcap found: NO
> ...
> Fetching value of define "__AVX2__" :
> Fetching value of define "__AVX512BW__" :
> Fetching value of define "__AVX512CD__" :
> Fetching value of define "__AVX512DQ__" :
> Fetching value of define "__AVX512F__" :
> Fetching value of define "__AVX512VL__" :
> ...
> Run-time dependency jansson found: NO (tried pkgconfig and cmake)
> ...
> 
> 
> The Unit tests env does not have a AVX512 enabled toolchain, or
> something in meson disables it, hence the compilation fails and the
> unit tests are marked as failed while it has nothing to do with
> functional tests.
> 

FYI: The original release of Ubuntu 18.04 had broken AVX-512 support
in binutils. However, the fix for that issue has recently been 
backported to 18.04, so running "apt-get update && apt-get upgrade"
should fix that support.

/Bruce

PS: Yes, I'm old-skool and use apt-get rather than just "apt". I
don't like how plain apt does fancy things with the terminal for
the progress bar! 😊

Re: [dpdk-ci] [dpdklab] RE: Compiling dpdk in non compilation tests

[Brandon Lo]

[-- Attachment #1: Type: text/plain, Size: 3553 bytes --]

Hi everyone,

Thank you for submitting.
I have started a bug ticket for this so we can track it:
https://bugs.dpdk.org/show_bug.cgi?id=566

Thanks,
Brandon

On Tue, Oct 27, 2020 at 5:17 AM Richardson, Bruce <
bruce.richardson@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Sunday, October 25, 2020 8:36 AM
> > To: dpdklab@iol.unh.edu; Lincoln Lavoie <lylavoie@iol.unh.edu>
> > Cc: ci@dpdk.org; Thomas Monjalon <thomas@monjalon.net>; Richardson,
> Bruce
> > <bruce.richardson@intel.com>
> > Subject: Compiling dpdk in non compilation tests
> >
> > Looking at CI reports this morning, I noticed this on the series "ice:
> > update base code".
> >
> > https://lab.dpdk.org/results/dashboard/patchsets/13992/
> >
> > - Compilation is failing on OpenSuse, but I think this is because its
> > toolchain does not have AVX512 support.
> > This seems to be a regression introduced by the series, indeed, cool.
> >
> >
> > - On Ubuntu 18.04, build is fine, but unit tests are ko.
> > Looking into the unit tests zipped logs, I can see that:
> > * the unit tests job builds its own binaries,
> > * the unit tests environment is not the same as the build test
> > environment (only copying significant differences below):
> >
> > Build:
> > The Meson build system
> > Version: 0.55.3
> > ...
> > C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
> > 7.5.0-3ubuntu1~18.04) 7.5.0")
> > C linker for the host machine: cc ld.bfd 2.30
> > ...
> > Run-time dependency libpcap found: YES 1.8.1
> > Has header "pcap.h" with dependency libpcap: YES
> > ...
> > Fetching value of define "__AVX2__" : 1
> > Fetching value of define "__AVX512BW__" : 1
> > Fetching value of define "__AVX512CD__" : 1
> > Fetching value of define "__AVX512DQ__" : 1
> > Fetching value of define "__AVX512F__" : 1
> > Fetching value of define "__AVX512VL__" : 1
> > ...
> > Run-time dependency jansson found: YES 2.11
> > ...
> >
> > vs
> >
> > Unit tests:
> > The Meson build system
> > Version: 0.53.1
> > ...
> > C compiler for the host machine: cc (gcc 7.5.0 "cc (Ubuntu
> > 7.5.0-3ubuntu1~18.04) 7.5.0")
> > C linker for the host machine: cc GNU ld.bfd 2.30
> > ...
> > Run-time dependency libpcap found: NO (tried pkgconfig)
> > Library pcap found: NO
> > ...
> > Fetching value of define "__AVX2__" :
> > Fetching value of define "__AVX512BW__" :
> > Fetching value of define "__AVX512CD__" :
> > Fetching value of define "__AVX512DQ__" :
> > Fetching value of define "__AVX512F__" :
> > Fetching value of define "__AVX512VL__" :
> > ...
> > Run-time dependency jansson found: NO (tried pkgconfig and cmake)
> > ...
> >
> >
> > The Unit tests env does not have a AVX512 enabled toolchain, or
> > something in meson disables it, hence the compilation fails and the
> > unit tests are marked as failed while it has nothing to do with
> > functional tests.
> >
>
> FYI: The original release of Ubuntu 18.04 had broken AVX-512 support
> in binutils. However, the fix for that issue has recently been
> backported to 18.04, so running "apt-get update && apt-get upgrade"
> should fix that support.
>
> /Bruce
>
> PS: Yes, I'm old-skool and use apt-get rather than just "apt". I
> don't like how plain apt does fancy things with the terminal for
> the progress bar! 😊
>


-- 

Brandon Lo

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

blo@iol.unh.edu

www.iol.unh.edu

[-- Attachment #2: Type: text/html, Size: 6359 bytes --]