DPDK CI discussions
 help / color / Atom feed
* [dpdk-ci] Meson failing on master
@ 2019-12-04 11:08 David Marchand
  2019-12-04 15:01 ` Jeremy Plsek
  0 siblings, 1 reply; 3+ messages in thread
From: David Marchand @ 2019-12-04 11:08 UTC (permalink / raw)
  To: Jeremy Plsek; +Cc: ci, Bruce Richardson, Aaron Conole

Hello Jeremy,

I noticed that meson is failing for a patch on master.
Example: https://lab.dpdk.org/results/dashboard/patchsets/8804/

The artifact gives the non-obvious (to me) error log:

Build targets in project: 810
Option default_library is: shared [default: static]
Found ninja-1.8.2 at /usr/bin/ninja
ninja: error: manifest 'build.ninja' still dirty after 100 tries

[1] Error running command.

Talked to Bruce, and he suspects something on the filesystem level.
Is this issue happening on a system with the overlayfs stuff you mentioned?


--
David Marchand


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-ci] Meson failing on master
  2019-12-04 11:08 [dpdk-ci] Meson failing on master David Marchand
@ 2019-12-04 15:01 ` Jeremy Plsek
  2019-12-04 15:09   ` Richardson, Bruce
  0 siblings, 1 reply; 3+ messages in thread
From: Jeremy Plsek @ 2019-12-04 15:01 UTC (permalink / raw)
  To: David Marchand; +Cc: ci, Bruce Richardson, Aaron Conole

On Wed, Dec 4, 2019 at 6:09 AM David Marchand <david.marchand@redhat.com> wrote:
>
> Hello Jeremy,
>
> I noticed that meson is failing for a patch on master.
> Example: https://lab.dpdk.org/results/dashboard/patchsets/8804/
>
> The artifact gives the non-obvious (to me) error log:
>
> Build targets in project: 810
> Option default_library is: shared [default: static]
> Found ninja-1.8.2 at /usr/bin/ninja
> ninja: error: manifest 'build.ninja' still dirty after 100 tries
>
> [1] Error running command.
>
> Talked to Bruce, and he suspects something on the filesystem level.
> Is this issue happening on a system with the overlayfs stuff you mentioned?
>

Hi David,

This started happening last week as well
(https://lab.dpdk.org/results/dashboard/patchsets/8711/), so I
disabled it temporarily.
I enabled it yesterday, and it was fine for a few patch sets
(https://lab.dpdk.org/results/dashboard/patchsets/8803/), but, as you
mentioned, it broke again. I've disabled it again for future tests
until it can be resolved.

This is not related to overlayfs work I've been working on (the
overlayfs work was for the bare metal machines). These tests run in a
container, using podman, with a mount to the jenkins workspace. The
containers run on a no-so-special Ubuntu 18.04 VM.

I've noticed similar issues for other CI
(https://github.com/ninja-build/ninja/issues/1599,
https://github.com/neovim/neovim/issues/10213), which, for the most
part, wasn't really resolved or gave a solution, and may not be
related to our issue.

Hmmm... I was able to reproduce it with the tarball that "failed" most
recently (ps 8804), but it passed with the older "failed" tarball (ps
8711). The difference between them that I noticed, was that when
untarring, it (ps 8804) would complain that the date was in the
future.

$ date
Wed Dec  4 14:28:18 UTC 2019
$ ls -l --full-time
...
-rw-rw-r--  1 jenkins-local jenkins-local   12 2019-12-04
15:03:12.000000000 +0000 VERSION

I tried to just update the files to the current time...

$ find . -exec touch {} +

And now it compiles... Hmmm... Yes, it seems that causes it? Updating
the date fixed it? Interesting bug.

So there are a few issues to look into:
1. Why the dates in the tarball are wrong in the first place (my
issue. or just add a dirty fix to the compile script to just update
the date after extracting)
2. Why the date of the file matters so much in meson/ninja (I
understand that build systems typically use the date to check whether
to recompile a file, but it should be more graceful about handling
this type of in-the-future problem)

I'll see if I can make a simple example and maybe make a bug report.
(At the minimum meson/ninja should fail gracefully rather than retry
100 times.)

>
> --
> David Marchand
>


-- 
Jeremy Plsek
UNH InterOperability Laboratory

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-ci] Meson failing on master
  2019-12-04 15:01 ` Jeremy Plsek
@ 2019-12-04 15:09   ` Richardson, Bruce
  0 siblings, 0 replies; 3+ messages in thread
From: Richardson, Bruce @ 2019-12-04 15:09 UTC (permalink / raw)
  To: Jeremy Plsek, David Marchand; +Cc: ci, Aaron Conole



> -----Original Message-----
> From: Jeremy Plsek <jplsek@iol.unh.edu>
> Sent: Wednesday, December 4, 2019 3:01 PM
> To: David Marchand <david.marchand@redhat.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Aaron
> Conole <aconole@redhat.com>
> Subject: Re: Meson failing on master
> 
> On Wed, Dec 4, 2019 at 6:09 AM David Marchand <david.marchand@redhat.com>
> wrote:
> >
> > Hello Jeremy,
> >
> > I noticed that meson is failing for a patch on master.
> > Example: https://lab.dpdk.org/results/dashboard/patchsets/8804/
> >
> > The artifact gives the non-obvious (to me) error log:
> >
> > Build targets in project: 810
> > Option default_library is: shared [default: static] Found ninja-1.8.2
> > at /usr/bin/ninja
> > ninja: error: manifest 'build.ninja' still dirty after 100 tries
> >
> > [1] Error running command.
> >
> > Talked to Bruce, and he suspects something on the filesystem level.
> > Is this issue happening on a system with the overlayfs stuff you
> mentioned?
> >
> 
> Hi David,
> 
> This started happening last week as well
> (https://lab.dpdk.org/results/dashboard/patchsets/8711/), so I disabled it
> temporarily.
> I enabled it yesterday, and it was fine for a few patch sets
> (https://lab.dpdk.org/results/dashboard/patchsets/8803/), but, as you
> mentioned, it broke again. I've disabled it again for future tests until
> it can be resolved.
> 
> This is not related to overlayfs work I've been working on (the overlayfs
> work was for the bare metal machines). These tests run in a container,
> using podman, with a mount to the jenkins workspace. The containers run on
> a no-so-special Ubuntu 18.04 VM.
> 
> I've noticed similar issues for other CI (https://github.com/ninja-
> build/ninja/issues/1599,
> https://github.com/neovim/neovim/issues/10213), which, for the most part,
> wasn't really resolved or gave a solution, and may not be related to our
> issue.
> 
> Hmmm... I was able to reproduce it with the tarball that "failed" most
> recently (ps 8804), but it passed with the older "failed" tarball (ps
> 8711). The difference between them that I noticed, was that when
> untarring, it (ps 8804) would complain that the date was in the future.
> 
> $ date
> Wed Dec  4 14:28:18 UTC 2019
> $ ls -l --full-time
> ...
> -rw-rw-r--  1 jenkins-local jenkins-local   12 2019-12-04
> 15:03:12.000000000 +0000 VERSION
> 
> I tried to just update the files to the current time...
> 
> $ find . -exec touch {} +
> 
> And now it compiles... Hmmm... Yes, it seems that causes it? Updating the
> date fixed it? Interesting bug.
> 
> So there are a few issues to look into:
> 1. Why the dates in the tarball are wrong in the first place (my issue. or
> just add a dirty fix to the compile script to just update the date after
> extracting) 2. Why the date of the file matters so much in meson/ninja (I
> understand that build systems typically use the date to check whether to
> recompile a file, but it should be more graceful about handling this type
> of in-the-future problem)
> 
> I'll see if I can make a simple example and maybe make a bug report.
> (At the minimum meson/ninja should fail gracefully rather than retry
> 100 times.)
> 
Interesting. 
Could this be due to people sending patches from machines with incorrect date/time settings on them? The headers for that particular patch are below:

Return-Path: <dev-bounces@dpdk.org>
X-Original-To: patchwork@inbox.dpdk.org
Delivered-To: patchwork@inbox.dpdk.org
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id E3389A04B5;
	Wed,  4 Dec 2019 08:26:33 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 18C421BE82;
	Wed,  4 Dec 2019 08:26:33 +0100 (CET)
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
 by dpdk.org (Postfix) with ESMTP id 470015B3C
 for <dev@dpdk.org>; Wed,  4 Dec 2019 08:26:31 +0100 (CET)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
 03 Dec 2019 23:26:28 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.69,276,1571727600"; d="scan'208";a="236182863"
Received: from dpdk-jiangcheng.sh.intel.com ([10.67.119.194])
 by fmsmga004.fm.intel.com with ESMTP; 03 Dec 2019 23:26:27 -0800
From: Cheng Jiang <Cheng1.jiang@intel.com>
To: dev@dpdk.org
Cc: maxime.coquelin@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com,
 Cheng Jiang <Cheng1.jiang@intel.com>
Date: Wed,  4 Dec 2019 15:03:12 +0000
Message-Id: <20191204150312.32697-1-Cheng1.jiang@intel.com>
X-Mailer: git-send-email 2.17.1
Subject: [dpdk-dev] [DPDK] net/virtio: packed ring notification data feature
	support
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

I notice, for example that the patch date is "4 Dec 2019 15:03:12 +0000", while it was received on "4 Dec 2019 08:26:33 +0100". Therefore, this is a patch from the future. This would also explain why you get passed now with the older failing tarball as it's no longer from the future, meaning any files it changes will be older than the build.ninja file generated by meson.

Regards,
/Bruce

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-04 11:08 [dpdk-ci] Meson failing on master David Marchand
2019-12-04 15:01 ` Jeremy Plsek
2019-12-04 15:09   ` Richardson, Bruce

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox