From: "Parthasarathy, JananeeX M" <jananeex.m.parthasarathy@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"Burakov, Anatoly" <anatoly.burakov@intel.com>,
"Pattan, Reshma" <reshma.pattan@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2] app/test: enhance sanity script
Date: Sun, 20 May 2018 17:41:50 +0000 [thread overview]
Message-ID: <7AE31235A30B41498D1C31348DC858BD5B3D64D6@IRSMSX103.ger.corp.intel.com> (raw)
In-Reply-To: <12834353.uf7ZmsVvok@xps>
Hello Thomas,
-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net]
Sent: Friday, May 18, 2018 7:18 PM
To: Parthasarathy, JananeeX M <jananeex.m.parthasarathy@intel.com>
Cc: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>; Pattan, Reshma <reshma.pattan@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2] app/test: enhance sanity script
18/05/2018 15:32, Parthasarathy, JananeeX M:
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > 15/05/2018 18:14, Parthasarathy, JananeeX M:
> > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> >
> > I think we need to keep a fast subset of tests.
> >
> > Please check in mk/rte.sdktest.mk:
> > PERFLIST=ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf
> > test-fast: BLACKLIST=-$(PERFLIST)
>
> We have added the below test cases to blacklist in mk/rte.sdktest.mk
> reciprocal_division,reciprocal_division_perf,lpm_perf,red_all
>
> Now time taken is around 4mins.
> Please let us know is there any minimum time limit for test execution.
> Is this time duration fine?
4 minutes, it's a bit long for developer oriented tests.
We need to distinguish which tests are enough for quick tests, and which ones can be in another category for full validation purpose.
Please can you make a summary of which tests you are adding, and propose a priority classification.
Thanks
Please find below the test cases newly added to sanity script (autotest_data.py) with priority classification.
P1 - Main test cases
P2 - Cryptodev test cases which requires dependent libraries (Can be added if the environment is set accordingly)
P3 - Perf test cases and others which takes more time (> 10s) (these will be in the blacklist test cases)
P4 - Logging/Dump test cases (to be checked whether these are really necessary)
The blacklist is updated in makefile "rte.sdktest.mk" with below list of test cases to reduce the total test execution time.
Blacklist test cases are ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf,reciprocal_division,reciprocal_division_perf,lpm_perf,
red_all,barrier,hash_multiwriter,timer_racecond,efd,hash_functions,eventdev_selftest_sw,member_perf,efd_perf,lpm6_perf,red_perf,distributor_perf
Currently after the above blacklist update, the total execution time of "make test-fast" is 54s.
Eventdev selftest octeontx
P1
Event ring autotest
P1
Table autotest
P1
Flow classify autotest
P1
Dump struct sizes
P4
Dump mempool
P4
Dump malloc stats
P4
Dump devargs
P4
Dump log types
P4
Dump_ring
P4
Quit
P4
Set rxtx anchor
P4
Set rxtx sc
P4
Event eth rx adapter autotest
P1
Rawdev autotest
P1
Kvargs autotest
P1
Dump physmem
P1
Dump memzone
P4
Devargs autotest
P4
Link bonding autotest
P1
Link bonding mode4 autotest
P1
Link bonding rssconf autotest
P1
Crc autotest
P1
Distributor autotest
P1
Reorder autotest
P1
Barrier autotest
P3
User delay
P1
Bitmap test
P1
Hash scaling autotest
P1
Hash multiwriter autotest
P3
Service autotest
P1
Timer racecond autotest
P3
Member autotest
P3
Efd_autotest
P3
Thash autotest
P1
Hash function autotest
P3
Cryptodev sw mrvl autotest
P2
Cryptodev dpaa2 sec autotest
P2
Cryptodev dpaa sec autotest
P2
Cryptodev qat autotest
P2
Cryptodev aesni mb autotest
P2
Cryptodev openssl autotest
P2
Cryptodev scheduler autotest
P2
Cryptodev aesni gcm autotest
P2
Cryptodev null autotest
P2
Cryptodev sw snow3g autotest
P2
Cryptodev sw kasumi autotest
P2
Cryptodev_sw_zuc_autotest
P2
Reciprocal division
P3
Red all
P3
Mempool performance autotest
P3
Memcpy performance autotest
P3
Hash performance autotest
P3
Timer performance autotest
P3
Reciprocal division perf
P3
Member perf autotest
P3
Efd perf autotest
P3
Lpm perf autotest
P3
Lpm6 perf autotest
P3
Red_perf
P3
Distributor perf autotest
P3
Ring pmd perf autotest
P1
Pmd perf autotest
P1
Ring performance autotest
P1
Thanks
M.P.Jananee
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
next prev parent reply other threads:[~2018-05-20 17:41 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-27 13:20 [dpdk-dev] [PATCH] " Reshma Pattan
2018-04-27 13:41 ` Burakov, Anatoly
2018-04-27 13:56 ` Burakov, Anatoly
2018-05-03 10:03 ` Parthasarathy, JananeeX M
2018-05-03 10:54 ` Burakov, Anatoly
2018-05-10 10:36 ` [dpdk-dev] [PATCH v2] " Jananee Parthasarathy
2018-05-13 22:12 ` Thomas Monjalon
2018-05-14 15:26 ` Pattan, Reshma
2018-05-14 15:29 ` Thomas Monjalon
2018-05-15 16:14 ` Parthasarathy, JananeeX M
2018-05-15 16:17 ` Thomas Monjalon
2018-05-18 13:32 ` Parthasarathy, JananeeX M
2018-05-18 13:48 ` Thomas Monjalon
2018-05-20 17:41 ` Parthasarathy, JananeeX M [this message]
2018-05-20 19:21 ` Thomas Monjalon
2018-05-15 15:58 ` [dpdk-dev] [PATCH v3 0/3] enhance sanity scripts Jananee Parthasarathy
2018-05-15 15:58 ` [dpdk-dev] [PATCH v3 1/3] app/test: add new test cases to sanity script Jananee Parthasarathy
2018-05-15 16:12 ` Burakov, Anatoly
2018-05-15 15:58 ` [dpdk-dev] [PATCH v3 2/3] app/test: update result for skipped test cases Jananee Parthasarathy
2018-05-15 15:58 ` [dpdk-dev] [PATCH v3 3/3] app/test: enhance freebsd support in sanity script Jananee Parthasarathy
2018-05-23 11:41 ` [dpdk-dev] [PATCH v4 0/4] enhance autotest config and test targets Jananee Parthasarathy
2018-05-23 11:41 ` [dpdk-dev] [PATCH v4 1/4] app/test: enhance autotest config Jananee Parthasarathy
2018-05-23 11:41 ` [dpdk-dev] [PATCH v4 2/4] app/test: update result for skipped test cases Jananee Parthasarathy
2018-05-23 11:41 ` [dpdk-dev] [PATCH v4 3/4] app/test: enhance freebsd support in autotest config Jananee Parthasarathy
2018-05-23 11:41 ` [dpdk-dev] [PATCH v4 4/4] mk: update make targets for classified testcases Jananee Parthasarathy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7AE31235A30B41498D1C31348DC858BD5B3D64D6@IRSMSX103.ger.corp.intel.com \
--to=jananeex.m.parthasarathy@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=reshma.pattan@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).