From: Marco Varlese <mvarlese@suse.de>
To: Luca Boccassi <bluca@debian.org>, dev@dpdk.org
Cc: stable@dpdk.org, thomas@monjalon.net
Subject: Re: [dpdk-dev] Regression tests for stable releases from companies involved in DPDK
Date: Fri, 01 Jun 2018 13:04:14 +0200 [thread overview]
Message-ID: <ecff038d3ae204f42225e0702de2b1c687520c15.camel@suse.de> (raw)
In-Reply-To: <1527847000.6997.66.camel@debian.org>
On Fri, 2018-06-01 at 10:56 +0100, Luca Boccassi wrote:
> On Fri, 2018-06-01 at 10:17 +0200, Marco Varlese wrote:
> > Hi Luca,
> >
> > On Thu, 2018-05-31 at 11:26 +0100, Luca Boccassi wrote:
> > > Hello all,
> > >
> > > At this morning's release meeting (minutes coming soon from John),
> > > we
> > > briefly discussed the state of the regression testing for stable
> > > releases and agreed we need to formalise the process.
> > >
> > > At the moment we have a firm commitment from Intel and Mellanox to
> > > test
> > > all stable branches (and if I heard correctly from NXP as well?
> > > Please
> > > confirm!). AT&T committed to run regressions on the 16.11 branch.
> > >
> > > Here's what we need in order to improve the quality of the stable
> > > releases process:
> > >
> > > 1) More commitments to help from other companies involved in the
> > > DPDK
> > > community. At the cost of re-stating the obvious, improving the
> > > quality
> > > of stable releases is for everyone's benefit, as a lot of customers
> > > and
> > > projects rely on the stable or LTS releases for their production
> > > environments.
> >
> > Do you have a list of steps (test-cases?) which are carried out for
> > stable
> > regression? I think it is necessary in order to understand the effort
> > involved
> > before committing to it.
>
> This is left intentionally vague - apart from the unit tests, we don't
> have a public formal "production" test suite, so each company has its
> own, that in general is tailored to test their own product.
>
> For example, AT&T builds a virtual router, and the regression tests
> cover the functionality of that product that are implemented via DPDK.
>
> I imagine companies developing NICs and PMDs will have regression tests
> that exercise those network cards in production. Likewise an enterprise
> distribution like Ubuntu can test their build of OVS, for example.
>
> So what we are asking is, for those companies/groups that use DPDK, if
> they can help us test that the stable candidate releases are working
> correctly with their products/hardware/projects. The hope is that among
> everybody there is enough coverage of various functionalities and PMDs.
> I'd love to have a formalised, public, all-encompassing test suite
> (something that was discussed in Dublin the year before last I believe)
> but we have to work with what we've got.
>
> Does this make sense?
Thanks for taking the time to explain it. Now, it is much clearer in my mind.
>
> > >
> > > 2) A formalised deadline - the current proposal is 10 days from the
> > > "xx.yy patches review and test" email, which was just sent for
> > > 16.11.
> > > For the involved companies, please let us know if 10 days is
> > > enough. In
> > > terms of scheduling, this period will always start within a week
> > > from
> > > the mainline final release. Again, the signal is the "xx.yy patches
> > > review and test" appearing in the inbox, which will detail the
> > > deadline.
> >
> > Again, I believe it depends on what needs to be tested (and how) in
> > order to
> > comment on "how much time is required".
>
> See above - it will vary from stakeholder to stakeholder, and that's
> why I asked if that timeframe is workable.
Undestood.
>
Cheers,
--
Marco V
SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg
next prev parent reply other threads:[~2018-06-01 11:04 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-31 10:26 Luca Boccassi
2018-06-01 4:38 ` [dpdk-dev] [dpdk-stable] " Christian Ehrhardt
2018-06-01 9:57 ` Luca Boccassi
2018-06-01 8:17 ` [dpdk-dev] " Marco Varlese
2018-06-01 9:56 ` Luca Boccassi
2018-06-01 11:04 ` Marco Varlese [this message]
2018-06-04 5:24 ` Shreyansh Jain
2018-06-04 8:38 ` [dpdk-dev] [dpdk-stable] " Luca Boccassi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ecff038d3ae204f42225e0702de2b1c687520c15.camel@suse.de \
--to=mvarlese@suse.de \
--cc=bluca@debian.org \
--cc=dev@dpdk.org \
--cc=stable@dpdk.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).