[dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

[dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

[Juraj Linkeš]

Hi DPDK devs,

I'm hitting a build failure with dpdk 20.02.1 with rte_ethdev.h that's happening seemingly at random:
make install T=arm64-armv8a-linuxapp-gcc -j
In file included from /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_class_eth.c:13:0:
/tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10: fatal error: rte_ethdev_core.h: No such file or directory
#include <rte_ethdev_core.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
make[5]: *** [rte_class_eth.o] Error 1
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [librte_ethdev] Error 2
make[3]: *** [lib] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2

Full log here: https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-dpdk-perf-verify-master-3n-tsh/11/archives/log.html.gz

This doesn't happen every time. It took me five builds on two machines (for a total of 10 builds) to hit this issue, but it did eventually happen. I tried this with 20.02 at first and it seemed to be happening more frequently, but I don't have enough data to really tell if there's an actual difference in frequency.

We're not doing anything special, just downloading and extracting the archive, then setting CONFIG_RTE_LIBRTE_MLX5_PMD and CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC to y in config/common_base and then running make install T=arm64-armv8a-linuxapp-gcc -j. As mentioned in the subject, the build server is a Taishan ARM server.

Any ideas on how to fix (or workaround) this? Is there a chance meson build would fix this? If so, how do we make sure we build testpmd with mxl5 support and how do we set CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC with meson build?

Thanks,
Juraj

Re: [dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

[Jerin Jacob]

On Mon, Jun 1, 2020 at 6:15 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>
> Hi DPDK devs,
>
> I'm hitting a build failure with dpdk 20.02.1 with rte_ethdev.h that's happening seemingly at random:
> make install T=arm64-armv8a-linuxapp-gcc -j
> In file included from /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_class_eth.c:13:0:
> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10: fatal error: rte_ethdev_core.h: No such file or directory
> #include <rte_ethdev_core.h>
>           ^~~~~~~~~~~~~~~~~~~


Try as  #include "rte_ethdev_core.h" instead of #include <rte_ethdev_core.h>

> compilation terminated.
> make[5]: *** [rte_class_eth.o] Error 1
> make[5]: *** Waiting for unfinished jobs....
> make[4]: *** [librte_ethdev] Error 2
> make[3]: *** [lib] Error 2
> make[2]: *** [all] Error 2
> make[1]: *** [pre_install] Error 2
> make: *** [install] Error 2
>
> Full log here: https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-dpdk-perf-verify-master-3n-tsh/11/archives/log.html.gz
>

Re: [dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

[Juraj Linkeš]

Hi Jerin,

I forgot to mention that I'm using GCC 8 and this fixed the issue. A bit of googling illuminated why this could be so, according to https://gcc.gnu.org/onlinedocs/cpp/Include-Syntax.html, the angle brackets directive looks for files in a standard list of system directories while the quotes directive looks in the directory containing the current file first. What it doesn't really explain is the random failues. Any idea why it sometimes built fine and sometimes it errored? Some race condition perhaps?

Thanks,
Juraj

-----Original Message-----
From: Jerin Jacob <jerinjacobk@gmail.com> 
Sent: Monday, June 1, 2020 4:37 PM
To: Juraj Linkeš <juraj.linkes@pantheon.tech>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

On Mon, Jun 1, 2020 at 6:15 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>
> Hi DPDK devs,
>
> I'm hitting a build failure with dpdk 20.02.1 with rte_ethdev.h that's happening seemingly at random:
> make install T=arm64-armv8a-linuxapp-gcc -j In file included from 
> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_class_eth.c:13:0:
> /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10: 
> fatal error: rte_ethdev_core.h: No such file or directory #include <rte_ethdev_core.h>
>           ^~~~~~~~~~~~~~~~~~~


Try as  #include "rte_ethdev_core.h" instead of #include <rte_ethdev_core.h>

> compilation terminated.
> make[5]: *** [rte_class_eth.o] Error 1
> make[5]: *** Waiting for unfinished jobs....
> make[4]: *** [librte_ethdev] Error 2
> make[3]: *** [lib] Error 2
> make[2]: *** [all] Error 2
> make[1]: *** [pre_install] Error 2
> make: *** [install] Error 2
>
> Full log here: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-dpdk-perf-verify-master-3n-tsh/11/archives/log.html.gz
>

Re: [dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server

[Jerin Jacob]

On Tue, Jun 2, 2020 at 12:43 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
>
> Hi Jerin,
>
> I forgot to mention that I'm using GCC 8 and this fixed the issue. A bit of googling illuminated why this could be so, according to https://gcc.gnu.org/onlinedocs/cpp/Include-Syntax.html, the angle brackets directive looks for files in a standard list of system directories while the quotes directive looks in the directory containing the current file first. What it doesn't really explain is the random failues. Any idea why it sometimes built fine and sometimes it errored? Some race condition perhaps?

it is racy due to the fact that rte_ethdev_core.h need to show up in
build/include/ before .c file compiles in order to make #include <>
works. In case of #include "", that dependency is not there.
Makefile file infrastructure is not defining the dependency anywhere.


>
> Thanks,
> Juraj
>
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Monday, June 1, 2020 4:37 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Non-deterministic build failure in rte_ethdev.h:4287:10 on Taishan ARM server
>
> On Mon, Jun 1, 2020 at 6:15 PM Juraj Linkeš <juraj.linkes@pantheon.tech> wrote:
> >
> > Hi DPDK devs,
> >
> > I'm hitting a build failure with dpdk 20.02.1 with rte_ethdev.h that's happening seemingly at random:
> > make install T=arm64-armv8a-linuxapp-gcc -j In file included from
> > /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_class_eth.c:13:0:
> > /tmp/openvpp-testing/dpdk/lib/librte_ethdev/rte_ethdev.h:4287:10:
> > fatal error: rte_ethdev_core.h: No such file or directory #include <rte_ethdev_core.h>
> >           ^~~~~~~~~~~~~~~~~~~
>
>
> Try as  #include "rte_ethdev_core.h" instead of #include <rte_ethdev_core.h>
>
> > compilation terminated.
> > make[5]: *** [rte_class_eth.o] Error 1
> > make[5]: *** Waiting for unfinished jobs....
> > make[4]: *** [librte_ethdev] Error 2
> > make[3]: *** [lib] Error 2
> > make[2]: *** [all] Error 2
> > make[1]: *** [pre_install] Error 2
> > make: *** [install] Error 2
> >
> > Full log here:
> > https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-dpdk-perf-verify-master-3n-tsh/11/archives/log.html.gz
> >