From: Ruifeng Wang <Ruifeng.Wang@arm.com>
To: Ciara Power <ciara.power@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "thomas@monjalon.net" <thomas@monjalon.net>,
"declan.doherty@intel.com" <declan.doherty@intel.com>,
"gakhil@marvell.com" <gakhil@marvell.com>,
"aconole@redhat.com" <aconole@redhat.com>,
"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
"anoobj@marvell.com" <anoobj@marvell.com>,
"asomalap@amd.com" <asomalap@amd.com>,
"Ajit Khaparde (ajit.khaparde@broadcom.com)"
<ajit.khaparde@broadcom.com>, "g.singh@nxp.com" <g.singh@nxp.com>,
nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
Date: Sun, 25 Apr 2021 09:24:47 +0000 [thread overview]
Message-ID: <AM5PR0802MB24657058D3CBC88823D18DB99E439@AM5PR0802MB2465.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <20210423161820.2135053-1-ciara.power@intel.com>
> -----Original Message-----
> From: Ciara Power <ciara.power@intel.com>
> Sent: Saturday, April 24, 2021 12:18 AM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; declan.doherty@intel.com; gakhil@marvell.com;
> aconole@redhat.com; hemant.agrawal@nxp.com; anoobj@marvell.com;
> Ruifeng Wang <Ruifeng.Wang@arm.com>; asomalap@amd.com; Ajit
> Khaparde (ajit.khaparde@broadcom.com) <ajit.khaparde@broadcom.com>;
> g.singh@nxp.com; Ciara Power <ciara.power@intel.com>
> Subject: [PATCH v3 0/7] test: refactor crypto unit test framework
>
> The current crypto unit test framework is not granular enough to accurately
> track unit test results. This is caused by one testcase in a suite actually
> running multiple testcases, but only returning one result.
>
> The approach taken in this patchset allows a test suite have a list of sub-
> testsuites, and/or a list of testcases as previously used.
> The unit test suite runner can then recursively iterate and run the sub-
> testsuites, until it reaches a suite with testcases, and it then runs each
> testcase as it had done previously.
> In the case of a testsuite with both testcases and sub-testsuites, the
> testcases are executed first before iterating through the sub-testsuites.
>
> By allowing this further breakdown into sub-testsuites, a refactor of the
> crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites
> can be used in place of the testcases that had multiple testcases hidden on a
> sub level.
> The blockcipher tests previously had these hidden testcases, but are now
> sub-testsuites that are dynamically created and added to a parent test suite,
> allowing for each testcase status to be reported directly to the runner.
> The cryptodev test suite is broken down into smaller suites that are used as
> sub-testsuites, which allows for more flexibility choosing which sub-
> testsuites should run for the current device.
> The introduction of sub-testsuites also allows for more precise
> setup/teardown functions, that can check the capabilities required to run its
> testcases.
>
> For example, when running the cryptodev_aesni_mb_autotest, the parent
> Cryptodev Test Suite is executed.
> Various sub-testsuites are added to the parent test suite, such as the static
> suites of testcases that were once in the cryptodev_testsuite, and
> blockcipher suites.
> The unit test runner can then run the Cryptodev parent test suite, which in
> turn will run the sub-testsuites.
>
> The user is now required to create vdevs via EAL commandline args, this is no
> longer done within the test app for crypto autotests.
>
> Documentation will need to be added at a later stage, adding to the test
> document that isn't yet merged. [1]
>
> [1]
> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-
> aconole@redhat.com/
>
>
> v3:
> - Added support for a testsuite having both a list of testcases,
> and a list of sub-testsuites.
> - Replaced PMD based parent testsuites with a cryptodev testsuite
> used by all autotests, with the exception of scheduler autotest.
> - Setup functions were added for all sub-testsuites, within which
> required capability support is checked.
> - The setup functions no longer create vdevs if needed,
> this must be done by the user when running the test.
> - Patch added to standardise return values for skipped testcases.
> v2:
> - Added macro in place of testcase/testsuite loops.
> - Added more detail in the summary output.
> - Moved testcase counts to the testsuite structure.
> - Flattened testsuite structure to remove union.
> - Added patch for fix of blockcipher test return value.
> - Squashed release note into last patch.
>
>
> Ciara Power (7):
> app/test: refactor of unit test suite runner
> test: introduce parent testsuite format
> test/crypto: refactor to use sub-testsuites
> test/crypto: replace unsupported with skipped
> test/crypto: move testsuite params to header file
> test/crypto: fix return value on test skipped
> test/crypto: dynamically build blockcipher suite
>
> app/test/test.c | 176 +-
> app/test/test.h | 23 +-
> app/test/test_cryptodev.c | 2324 ++++++++++++++----------
> app/test/test_cryptodev.h | 32 +
> app/test/test_cryptodev_asym.c | 111 +-
> app/test/test_cryptodev_blockcipher.c | 423 ++++-
> app/test/test_cryptodev_blockcipher.h | 12 +-
> app/test/test_ipsec.c | 32 +-
> doc/guides/rel_notes/release_21_05.rst | 5 +
> 9 files changed, 2000 insertions(+), 1138 deletions(-)
>
> --
> 2.25.1
Tested armv8 cryptodev, and result looks good.
Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>
next prev parent reply other threads:[~2021-04-25 9:25 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-23 16:18 Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format Ciara Power
2021-04-23 16:32 ` Aaron Conole
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites Ciara Power
2021-04-30 20:59 ` Doherty, Declan
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 4/7] test/crypto: replace unsupported with skipped Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 5/7] test/crypto: move testsuite params to header file Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 6/7] test/crypto: fix return value on test skipped Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 7/7] test/crypto: dynamically build blockcipher suite Ciara Power
2021-04-25 9:24 ` Ruifeng Wang [this message]
2021-04-30 21:00 ` [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Doherty, Declan
2021-05-05 10:41 ` Hemant Agrawal
2021-05-05 12:20 ` Hemant Agrawal
2021-05-08 15:38 ` [dpdk-dev] [EXT] " Akhil Goyal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM5PR0802MB24657058D3CBC88823D18DB99E439@AM5PR0802MB2465.eurprd08.prod.outlook.com \
--to=ruifeng.wang@arm.com \
--cc=aconole@redhat.com \
--cc=ajit.khaparde@broadcom.com \
--cc=anoobj@marvell.com \
--cc=asomalap@amd.com \
--cc=ciara.power@intel.com \
--cc=declan.doherty@intel.com \
--cc=dev@dpdk.org \
--cc=g.singh@nxp.com \
--cc=gakhil@marvell.com \
--cc=hemant.agrawal@nxp.com \
--cc=nd@arm.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).