DPDK patches and discussions
 help / color / mirror / Atom feed
From: Jeremy Plsek <jplsek@iol.unh.edu>
To: David Marchand <david.marchand@redhat.com>
Cc: ci@dpdk.org, Thomas Monjalon <thomas@monjalon.net>,
	dev <dev@dpdk.org>,  Michael Santana <maicolgabriel@hotmail.com>,
	Aaron Conole <aconole@redhat.com>,
	Bruce Richardson <bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [dpdk-ci] [PATCH 4/4] ci: reorganise Travis jobs
Date: Thu, 20 Feb 2020 14:38:43 -0500	[thread overview]
Message-ID: <CA+xUZB6FW80se6VMN22ydtNOHwD_nMW1H=oOQ_01Na=VbhnfOw@mail.gmail.com> (raw)
In-Reply-To: <CAJFAV8w5gxiB7z5wAXS+Hxzqkk-TSPVDJedKB32DRqouQCxm7w@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 3853 bytes --]

On Thu, Feb 20, 2020 at 11:02 AM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >>
> > >> 19/02/2020 22:39, Aaron Conole:
> > >> > David Marchand <david.marchand@redhat.com> writes:
> > >> >
> > >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> > >> > > in Travis.
> > >> > >
> > >> > > Since meson enables automatically the relevant components, there is not
> > >> > > much gain in testing with extra_packages vs required_packages only.
> > >> > >
> > >> > > For a given arch/compiler/env combination, compilation is first tested
> > >> > > in all jobs that run tests or build the docs or run the ABI checks.
> > >> > > In the same context, for jobs that accumulates running tests, building
> > >> > > the docs etc..., those steps are independent and can be split to save
> > >> > > some cpu on Travis.
> > >> > >
> > >> > > With this, we go down from 21 to 15 jobs.
> > >> > >
> > >> > > Note: this patch requires a flush of the existing caches in Travis.
> > >> > >
> > >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > >> > > ---
> > >> >
> > >> > In general, I think the idea with required vs. extra was to have a build
> > >> > that did the minimum required, and one that did all the packages (to
> > >> > allow a minimum vs. full DPDK).
> > >> >
> > >> > At least, that's from
> > >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> > >>
> > >> I think the benefit of a minimum build is to have a quick report,
> > >> and easy to setup.
> > >
> > > Yes, Travis serves as a first gate when submitting patches.
> > > But since Travis is best effort/free, we can't have a full coverage.
> > >
> > >
> > >> > Not sure if that's still something anyone cares about.
> > >>
> > >> Given that Travis knows how to satisfy the dependencies,
> > >> and that we must wait for all jobs to finish,
> > >> I don't see any benefit of a minimal setup.
> > >
> > > This minimal setup also tests that dpdk dependencies are correct.
> > > If a change makes something rely on libX and libX is in the packages
> > > always installed in Travis, the missing dependency would not get
> > > caught.
> > >
> > > But here, this adds too many jobs.
> > >
> > > UNH, Intel and other CIs should step in and fill this kind of gap.
> >
> > Okay, makes sense to me.  Are one of these CI providers offering to
> > cover this?
>
> Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.
>
> For the CI guys, which packages are installed on the systems/vms that
> do compilation tests?

For our compile tests, we compile with pretty much all dpdk
dependencies that can easily be installed from the distro's repository
(the dependencies we installed are based on the meson output). The
thought process was to test compiling as much of dpdk as possible
(with constraints mentioned before).

Here are the dependencies we install on some of the setups:

Ubuntu 18.04: make gcc ninja-build pkg-config libnuma-dev
librdmacm-dev librdmacm1 rdma-core libelf-dev libmnl-dev libpcap-dev
libcrypto++-dev libjansson-dev
(meson installed with pip)

CentOS 8 and Fedora 31: diffutils gcc make ninja-build meson
pkg-config numactl-devel librdmacm rdma-core-devel
elfutils-libelf-devel libmnl-devel libpcap-devel jansson-devel

openSuse Leap 15: gcc make ninja pkg-config libnuma-devel librdmacm1
rdma-core-devel libelf-devel libmnl-devel libpcap-devel
libjansson-devel
(meson installed with pip)

> Is it possible to have a summary of the different setups?
>
I've attached an overview/summary of what we do at the UNH-IOL DPDK
Community Lab.
>
> Thanks.

>
> --
> David Marchand
>

[-- Attachment #2: DPDK Lab Summary February 2020.pdf --]
[-- Type: application/pdf, Size: 846111 bytes --]

  reply	other threads:[~2020-02-21 16:49 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-19 19:41 [dpdk-dev] [PATCH 0/4] Reorganise " David Marchand
2020-02-19 19:41 ` [dpdk-dev] [PATCH 1/4] ci: remove unnecessary dependency on Linux headers David Marchand
2020-02-20 10:44   ` Thomas Monjalon
2020-02-20 14:37   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 2/4] ci: fix Travis config warnings David Marchand
2020-02-20 11:03   ` Thomas Monjalon
2020-02-20 12:09     ` David Marchand
2020-02-20 16:46     ` David Marchand
2020-02-20 21:01       ` Aaron Conole
2020-02-21 10:17         ` Thomas Monjalon
2020-02-20 14:36   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 3/4] ci: use an explicit list of Travis jobs David Marchand
2020-02-20 11:05   ` Thomas Monjalon
2020-02-20 14:35   ` Aaron Conole
2020-02-19 19:41 ` [dpdk-dev] [PATCH 4/4] ci: reorganise " David Marchand
2020-02-19 21:39   ` Aaron Conole
2020-02-20 10:42     ` Thomas Monjalon
2020-02-20 12:22       ` David Marchand
2020-02-20 14:35         ` Aaron Conole
2020-02-20 16:01           ` David Marchand
2020-02-20 19:38             ` Jeremy Plsek [this message]
2020-02-20  0:18   ` dwilder
2020-02-20 11:07   ` Thomas Monjalon
2020-02-20 21:05 ` [dpdk-dev] [PATCH 0/4] Reorganise " David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CA+xUZB6FW80se6VMN22ydtNOHwD_nMW1H=oOQ_01Na=VbhnfOw@mail.gmail.com' \
    --to=jplsek@iol.unh.edu \
    --cc=aconole@redhat.com \
    --cc=bruce.richardson@intel.com \
    --cc=ci@dpdk.org \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=maicolgabriel@hotmail.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).