DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ciara Power <ciara.power@intel.com>
To: dev@dpdk.org
Cc: declan.doherty@intel.com, gakhil@marvell.com, aconole@redhat.com,
	hemant.agrawal@nxp.com, anoobj@marvell.com, ruifeng.wang@arm.com,
	asomalap@amd.com, ajit.khaparde@broadcom.com, g.singh@nxp.com,
	Ciara Power <ciara.power@intel.com>
Subject: [dpdk-dev] [PATCH v2 0/6] test: refactor crypto unit test framework
Date: Fri,  2 Apr 2021 14:24:18 +0000
Message-ID: <20210402142424.1353789-1-ciara.power@intel.com> (raw)

v2:
  - Added macro in place of testcase/testsuite loops.
  - Added more detail in the summary output.
  - Moved testcase counts to the testsuite structure.
  - Flattened testsuite structure to remove union.
  - Added patch for fix of blockcipher test return value.
  - Squashed release note into last patch.

The current crypto unit test framework is not granular enough to
accurately track unit test results. This is caused by one testcase
in a suite actually running multiple testcases, but only returning
one result.
 
The approach taken in this patchset allows a test suite have either a
list of sub-testsuites, or a list of testcases as previously used.
The unit test suite runner can then recursively iterate and run the
sub-testsuites, until it reaches a suite with testcases,
and it then runs each testcase as it had done previously.
 
By allowing this further breakdown into sub-testsuites,
a refactor of the crypto unit tests solves the issue of inaccurate
reporting, as sub-testsuites can be used in place of the testcases
that had multiple testcases hidden on a sub level.
The blockcipher tests previously had these hidden testcases,
but are now sub-testsuites that are dynamically created and added to a
parent test suite, allowing for each testcase status to be reported
directly to the runner.
The cryptodev test suite is broken down into smaller suites that are
used as sub-testsuites, which allows for more flexibility choosing which
sub-testsuites should run for the current device autotest.
The introduction of sub-testsuites also allows for more precise
setup/teardown functions, rather than general ones loaded with
conditions as was seen with the initial setup function used for all
crypto testsuites.
 
For example, when running the cryptodev_aesni_mb_autotest, 
the AESNI MB parent test suite has its own setup function to initialise
the AESNI MB device.
Various sub-testsuites are added to the parent test suite, such as some
of the static suites that were once in the cryptodev_testsuite,
and blockcipher suites.
The unit test runner can then run the AESNI MB parent test suite,
which in turn will run the sub-testsuites.
 
Documentation will be added in a later version of the patchset,
adding to the test document that isn't yet merged. [1]

---
[1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/

Ciara Power (6):
  app/test: refactor of unit test suite runner
  test: introduce parent testsuite format
  test/crypto: refactor to use sub-testsuites
  test/crypto: move testsuite params to header file
  test/crypto: fix return value on test skipped
  test/crypto: dynamically build blockcipher suite

 app/test/test.c                        |  226 ++-
 app/test/test.h                        |   23 +-
 app/test/test_cryptodev.c              | 1964 ++++++++++++++++++------
 app/test/test_cryptodev.h              |   20 +
 app/test/test_cryptodev_asym.c         |   93 +-
 app/test/test_cryptodev_blockcipher.c  |  127 +-
 app/test/test_cryptodev_blockcipher.h  |   12 +-
 app/test/test_ipsec.c                  |   32 +-
 doc/guides/rel_notes/release_21_05.rst |    5 +
 9 files changed, 1831 insertions(+), 671 deletions(-)

-- 
2.25.1


             reply	other threads:[~2021-04-02 14:24 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-02 14:24 Ciara Power [this message]
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 1/6] app/test: refactor of unit test suite runner Ciara Power
2021-04-05 12:10   ` Aaron Conole
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 2/6] test: introduce parent testsuite format Ciara Power
2021-04-05 12:30   ` Aaron Conole
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 3/6] test/crypto: refactor to use sub-testsuites Ciara Power
2021-04-13 17:51   ` [dpdk-dev] [EXT] " Akhil Goyal
2021-04-13 18:17     ` Thomas Monjalon
2021-04-14 11:12     ` Doherty, Declan
2021-04-14 11:18       ` Akhil Goyal
2021-04-14 11:20         ` Thomas Monjalon
2021-04-14 11:22           ` Akhil Goyal
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 4/6] test/crypto: move testsuite params to header file Ciara Power
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 5/6] test/crypto: fix return value on test skipped Ciara Power
2021-04-13 17:07   ` [dpdk-dev] [EXT] " Akhil Goyal
2021-04-02 14:24 ` [dpdk-dev] [PATCH v2 6/6] test/crypto: dynamically build blockcipher suite Ciara Power
2021-04-06  1:34 ` [dpdk-dev] [PATCH v2 0/6] test: refactor crypto unit test framework Ruifeng Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210402142424.1353789-1-ciara.power@intel.com \
    --to=ciara.power@intel.com \
    --cc=aconole@redhat.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=anoobj@marvell.com \
    --cc=asomalap@amd.com \
    --cc=declan.doherty@intel.com \
    --cc=dev@dpdk.org \
    --cc=g.singh@nxp.com \
    --cc=gakhil@marvell.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=ruifeng.wang@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git