From: Bruce Richardson <bruce.richardson@intel.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
dev@dpdk.org, john.mcnamara@intel.com, reshma.pattan@intel.com,
Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
Subject: Re: [dpdk-dev] [PATCH 0/7] Make unit tests great again
Date: Wed, 13 Jun 2018 11:29:05 +0100 [thread overview]
Message-ID: <20180613102905.GB9784@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <0191e8b6-1823-7004-2829-d8973e33a1ae@intel.com>
On Wed, Jun 13, 2018 at 09:38:32AM +0100, Burakov, Anatoly wrote:
> On 12-Jun-18 2:07 PM, Thomas Monjalon wrote:
> > +Cc Jananee
> >
> > 07/06/2018 23:01, Anatoly Burakov:
> > > Previously, unit tests were running in groups. There were
> > > technical reasons why that was the case (mostly having to do
> > > with limiting memory), but it was hard to maintain and update
> > > the autotest script.
> > >
> > > In 18.05, limiting of memory at DPDK startup was no longer
> > > necessary, as DPDK allocates memory at runtime as needed. This
> > > has the implication that the old test grouping can now be
> > > retired and replaced with a more sensible way of running unit
> > > tests (using multiprocessing pool of workers and a queue of
> > > tests). This patchset accomplishes exactly that.
> > >
> > > This patchset conflicts with some of the earlier work on
> > > autotests [1] [2] [3], but i think it presents a cleaner
> > > solution for some of the problems highlighted by those patch
> > > series. I can integrate those patches into this series if
> > > need be.
> > >
> > > [1] http://dpdk.org/dev/patchwork/patch/40370/
> > > [2] http://dpdk.org/dev/patchwork/patch/40371/
> > > [3] http://dpdk.org/dev/patchwork/patch/40372/
> >
> > It may be interesting to work on lists of tests as done
> > in the following patch:
> > http://dpdk.org/dev/patchwork/patch/40373/
> >
> > The idea is to split tests in several categories:
> > - basic and short test
> > - longer and lower priority
> > - performance test
> > As a long term solution, we can think about making category an attribute
> > inside the test itself?
> >
>
> These test categories do not conflict with my patchset as they ultimately
> rely on white/blacklisting, which will continue to work as before.
>
> In my view, it really boils down to two things - either tests can be run in
> parallel with others (i.e. their result won't be affected by another
> independent DPDK test app instance), or not. On top of that, we can use
> blacklisting or whitelisting to define which tests will actually be run
> (i.e. define any "categories" we want), but their (non-)parallelism must
> always be respected to get good test results.
>
Have you looked at: http://mesonbuild.com/Unit-tests.html, at it would be
good to transition away from our own custom script infrastructure for the
tests.
There is already some support for running the unit tests using "meson
test", but it could do with some more cleanup e.g. to move the tests into
suitable suites (corresponding to the categories Thomas has suggested). We
could also do with specifying properly what tests are parallel-safe and
what aren't.
next prev parent reply other threads:[~2018-06-13 10:29 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-07 21:01 Anatoly Burakov
2018-06-07 21:01 ` [dpdk-dev] [PATCH 1/7] autotest: fix printing Anatoly Burakov
2018-06-07 21:01 ` [dpdk-dev] [PATCH 2/7] autotest: fix invalid code on reports Anatoly Burakov
2018-06-07 21:01 ` [dpdk-dev] [PATCH 3/7] autotest: make autotest runner python 2/3 compliant Anatoly Burakov
2018-06-07 21:01 ` [dpdk-dev] [PATCH 4/7] autotest: visually separate different test categories Anatoly Burakov
2018-06-07 21:01 ` [dpdk-dev] [PATCH 5/7] autotest: improve filtering Anatoly Burakov
2018-06-07 21:02 ` [dpdk-dev] [PATCH 6/7] autotest: remove autotest grouping Anatoly Burakov
2018-06-07 21:02 ` [dpdk-dev] [PATCH 7/7] autotest: properly parallelize unit tests Anatoly Burakov
2018-06-12 13:07 ` [dpdk-dev] [PATCH 0/7] Make unit tests great again Thomas Monjalon
2018-06-13 8:38 ` Burakov, Anatoly
2018-06-13 10:29 ` Bruce Richardson [this message]
2018-07-13 16:19 ` [dpdk-dev] [PATCH v2 00/10] " Reshma Pattan
2018-07-13 16:19 ` [dpdk-dev] [PATCH v2 01/10] autotest: fix printing Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 02/10] autotest: fix invalid code on reports Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 03/10] autotest: make autotest runner python 2/3 compliant Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 04/10] autotest: visually separate different test categories Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 05/10] autotest: improve filtering Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 06/10] autotest: remove autotest grouping Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 07/10] autotest: properly parallelize unit tests Reshma Pattan
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 08/10] autotest: add new test cases to autotest Reshma Pattan
2018-07-13 16:41 ` Burakov, Anatoly
2018-07-16 13:29 ` Pattan, Reshma
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 09/10] autotest: update result for skipped test cases Reshma Pattan
2018-07-13 16:42 ` Burakov, Anatoly
2018-07-13 16:20 ` [dpdk-dev] [PATCH v2 10/10] mk: update make targets for classified testcases Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 0/9] Make unit tests great again Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 1/9] autotest: fix printing Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 2/9] autotest: fix invalid code on reports Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 3/9] autotest: make autotest runner python 2/3 compliant Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 4/9] autotest: visually separate different test categories Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 5/9] autotest: improve filtering Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 6/9] autotest: remove autotest grouping Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 7/9] autotest: properly parallelize unit tests Reshma Pattan
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 8/9] autotest: update autotest test case list Reshma Pattan
2018-07-16 15:16 ` Burakov, Anatoly
2018-07-17 9:18 ` Pattan, Reshma
2018-07-17 9:23 ` Burakov, Anatoly
2018-07-17 9:45 ` Pattan, Reshma
2018-07-17 10:10 ` Burakov, Anatoly
2018-07-16 14:12 ` [dpdk-dev] [PATCH v3 9/9] mk: update make targets for classified testcases Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 0/9] Make unit tests great again Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 1/9] autotest: fix printing Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 2/9] autotest: fix invalid code on reports Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 3/9] autotest: make autotest runner python 2/3 compliant Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 4/9] autotest: visually separate different test categories Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 5/9] autotest: improve filtering Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 6/9] autotest: remove autotest grouping Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 7/9] autotest: properly parallelize unit tests Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 8/9] autotest: update autotest test case list Reshma Pattan
2018-07-17 10:39 ` [dpdk-dev] [PATCH v4 9/9] mk: update make targets for classified testcases Reshma Pattan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180613102905.GB9784@bricha3-MOBL.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=jananeex.m.parthasarathy@intel.com \
--cc=john.mcnamara@intel.com \
--cc=reshma.pattan@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).