[dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Bruce Richardson]

Static builds can take a lot of space, so reduce the number of examples
built when doing those static builds.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 .ci/linux-build.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index d500c4c00..2e0971dee 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -42,9 +42,15 @@ if [ "$BUILD_32BIT" = "1" ]; then
     export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig"
 fi
 
+if [ "$DEF_LIB" = "static" ] ; then
+    OPTS="$OPTS -Dexamples=l2fwd,l3fwd"
+else
+    OPTS="$OPTS -Dexamples=all"
+fi
+
 OPTS="$OPTS --default-library=$DEF_LIB"
 OPTS="$OPTS --buildtype=debugoptimized"
-meson build --werror -Dexamples=all $OPTS
+meson build --werror $OPTS
 ninja -C build
 
 if [ "$AARCH64" != "1" ]; then
-- 
2.24.1

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Aaron Conole]

Bruce Richardson <bruce.richardson@intel.com> writes:

> Static builds can take a lot of space, so reduce the number of examples
> built when doing those static builds.
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---

It looks good to me.  I'll try a run with Ruifeng's no-huge series and
see if I can trigger errors on my side (and then see what happens with
the cache sizes, too).  Might take some time as I adjust some things
here.

>  .ci/linux-build.sh | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d500c4c00..2e0971dee 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -42,9 +42,15 @@ if [ "$BUILD_32BIT" = "1" ]; then
>      export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig"
>  fi
>  
> +if [ "$DEF_LIB" = "static" ] ; then
> +    OPTS="$OPTS -Dexamples=l2fwd,l3fwd"
> +else
> +    OPTS="$OPTS -Dexamples=all"
> +fi
> +
>  OPTS="$OPTS --default-library=$DEF_LIB"
>  OPTS="$OPTS --buildtype=debugoptimized"
> -meson build --werror -Dexamples=all $OPTS
> +meson build --werror $OPTS
>  ninja -C build
>  
>  if [ "$AARCH64" != "1" ]; then

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Thomas Monjalon]

16/03/2020 18:09, Bruce Richardson:
> Static builds can take a lot of space, so reduce the number of examples
> built when doing those static builds.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> +if [ "$DEF_LIB" = "static" ] ; then
> +    OPTS="$OPTS -Dexamples=l2fwd,l3fwd"
> +else
> +    OPTS="$OPTS -Dexamples=all"
> +fi

I was thinking about doing this kind of change in devtools/test-meson-builds.sh
What do you think?

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[David Marchand]

On Mon, Mar 16, 2020 at 6:53 PM Aaron Conole <aconole@redhat.com> wrote:
>
> Bruce Richardson <bruce.richardson@intel.com> writes:
>
> > Static builds can take a lot of space, so reduce the number of examples
> > built when doing those static builds.
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
>
> It looks good to me.  I'll try a run with Ruifeng's no-huge series and
> see if I can trigger errors on my side (and then see what happens with
> the cache sizes, too).  Might take some time as I adjust some things
> here.

- The disk quota issue was reproduced on master without Ruifeng patches:
https://travis-ci.com/github/DPDK/dpdk/jobs/298584616
https://travis-ci.com/github/DPDK/dpdk/jobs/298858638


And for this, the cache is not really big.
cache-arm64-linux-bionic-a550182015679db90d1e0a2460464d106712e1c2c29f3efc7db1dc1b6fd86ebb--compiler-gcc.tgz
   last modified: 2020-03-16 11:28:59  size: 160.84 MiB
I remember bigger caches in the past.

I deleted this job cache and restarted the first job, and it passed.
Restarted again, and then it failed o_O.


- So I went and restarted a couple of times both of those jobs
(leaving the cache as is, which now contains 60MB of data).
I get random failures.
It does not seem to depend on the system: it failed once on
f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com, then
worked later.

hostname: f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com: KO
hostname: edb47637-c344-4a03-8538-711046469ad4@14975.lxd-arm64-01-com: ok
hostname: 386fc0aa-a84a-4a77-b2bd-e1e754bfe51d@14975.lxd-arm64-01-com: ok
hostname: 43c67d46-c94c-498a-816f-2eddce2a58a6@14975.lxd-arm64-01-com: ok
hostname: f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com: ok

hostname: a59a18bb-6f7f-4c0f-bb98-a8075cb61063@6138.lxd-arm64-02-com: KO
hostname: 743ab211-a1ba-480e-be85-2b1c01ce8b86@14975.lxd-arm64-01-com: ok
hostname: 8ac56afe-df63-464a-8c85-9b2e8b141b94@25560.lxd-arm64-03-com: ok
hostname: 306788d5-7614-4e9d-856a-1043cb4aeaf4@6138.lxd-arm64-02-com: KO
hostname: b6c5307c-7e65-4feb-a46a-7e2bc9a17fb8@6138.lxd-arm64-02-com: ok


-- 
David Marchand

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Mon, Mar 16, 2020 at 6:53 PM Aaron Conole <aconole@redhat.com> wrote:
>>
>> Bruce Richardson <bruce.richardson@intel.com> writes:
>>
>> > Static builds can take a lot of space, so reduce the number of examples
>> > built when doing those static builds.
>> >
>> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
>> > ---
>>
>> It looks good to me.  I'll try a run with Ruifeng's no-huge series and
>> see if I can trigger errors on my side (and then see what happens with
>> the cache sizes, too).  Might take some time as I adjust some things
>> here.
>
> - The disk quota issue was reproduced on master without Ruifeng patches:
> https://travis-ci.com/github/DPDK/dpdk/jobs/298584616
> https://travis-ci.com/github/DPDK/dpdk/jobs/298858638

Okay - guess I should just ack those patches, then.  They look good otherwise.

>
> And for this, the cache is not really big.
> cache-arm64-linux-bionic-a550182015679db90d1e0a2460464d106712e1c2c29f3efc7db1dc1b6fd86ebb--compiler-gcc.tgz
>    last modified: 2020-03-16 11:28:59  size: 160.84 MiB
> I remember bigger caches in the past.
>
> I deleted this job cache and restarted the first job, and it passed.
> Restarted again, and then it failed o_O.
>
>
> - So I went and restarted a couple of times both of those jobs
> (leaving the cache as is, which now contains 60MB of data).
> I get random failures.
> It does not seem to depend on the system: it failed once on
> f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com, then
> worked later.

Seems it's all arm64 builds that are failing.  I went to file an issue,
but looks like you beat me to it :)

https://travis-ci.community/t/disk-quota-exceeded-on-arm64/7619

I'll help promote the issue however I can

> hostname: f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com: KO
> hostname: edb47637-c344-4a03-8538-711046469ad4@14975.lxd-arm64-01-com: ok
> hostname: 386fc0aa-a84a-4a77-b2bd-e1e754bfe51d@14975.lxd-arm64-01-com: ok
> hostname: 43c67d46-c94c-498a-816f-2eddce2a58a6@14975.lxd-arm64-01-com: ok
> hostname: f432ae31-ccac-4caa-92ca-8156fabad86f@6138.lxd-arm64-02-com: ok
>
> hostname: a59a18bb-6f7f-4c0f-bb98-a8075cb61063@6138.lxd-arm64-02-com: KO
> hostname: 743ab211-a1ba-480e-be85-2b1c01ce8b86@14975.lxd-arm64-01-com: ok
> hostname: 8ac56afe-df63-464a-8c85-9b2e8b141b94@25560.lxd-arm64-03-com: ok
> hostname: 306788d5-7614-4e9d-856a-1043cb4aeaf4@6138.lxd-arm64-02-com: KO
> hostname: b6c5307c-7e65-4feb-a46a-7e2bc9a17fb8@6138.lxd-arm64-02-com: ok

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Bruce Richardson]

On Mon, Mar 16, 2020 at 07:37:01PM +0100, Thomas Monjalon wrote:
> 16/03/2020 18:09, Bruce Richardson:
> > Static builds can take a lot of space, so reduce the number of examples
> > built when doing those static builds.
> > 
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> > ---
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> > +if [ "$DEF_LIB" = "static" ] ; then
> > +    OPTS="$OPTS -Dexamples=l2fwd,l3fwd"
> > +else
> > +    OPTS="$OPTS -Dexamples=all"
> > +fi
> 
> I was thinking about doing this kind of change in devtools/test-meson-builds.sh
> What do you think?
> 

No major objection. It's probably less likely to be an issue for people
running their own test-builds, though if it saves time and space and
doesn't much affect the number of issues we are going to hit/miss, I don't
see why not do so.

Re: [dpdk-dev] [RFC PATCH] ci: reduce examples built for static builds

[Thomas Monjalon]

16/03/2020 18:09, Bruce Richardson:
> Static builds can take a lot of space, so reduce the number of examples
> built when doing those static builds.
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>

No objection after one month, it is a RFC but I see nothing to change.

Applied, thanks