DPDK patches and discussions
 help / color / mirror / Atom feed
* Re: [dpdk-dev] [PATCH 5/6] test/crypto: dynamically build blockcipher suite
@ 2021-03-31 14:46 Akhil Goyal
  2021-04-02 14:34 ` Power, Ciara
  0 siblings, 1 reply; 3+ messages in thread
From: Akhil Goyal @ 2021-03-31 14:46 UTC (permalink / raw)
  To: Ciara Power, dev
  Cc: declan.doherty, hemant.agrawal, Anoob Joseph, ruifeng.wang,
	asomalap, ajit.khaparde, g.singh, aconole

Hi Ciara,

Nice work. I have below comment on the output of the results.

++ more crypto PMD maintainers.

> In the existing implementation, the blockcipher test cases are being run
> and reported as one test case per type, even though multiple test cases
> are hidden in each. For example, "test_AES_chain_all" runs 46 test cases.
> Each blockcipher type should have a testsuite instead.
> 
> The blockcipher testsuite is dynamically built, depending on the
> blockcipher type chosen. The testcase struct is modified to allow
> running a testcase with data, which is used for data required when
> running each blockcipher testcase.
> 
> The blockcipher testsuites are added dynamically to parent testsuites
> as sub-testsuites where needed.
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>
> ---
When I run the patchset, The output I got is 
+ Sub Testsuites Total :       24 
 + Sub Testsuites Skipped :      2
 + Sub Testsuites Passed :      22
 + Sub Testsuites Failed :       0      

Is it possible for you to give more details in the end like 
Number of tests passed/failed/skipped/unsupported as well for each of the suite.

The reason is that, people normally check the report in the end.
For example: 
Crypto General(A/X passed, B/X skipped, C/X failed)
//one line for each of the suite.


Regards,
Akhil

^ permalink raw reply	[flat|nested] 3+ messages in thread
* [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework
@ 2021-03-16 14:32 Ciara Power
  2021-03-16 14:32 ` [dpdk-dev] [PATCH 5/6] test/crypto: dynamically build blockcipher suite Ciara Power
  0 siblings, 1 reply; 3+ messages in thread
From: Ciara Power @ 2021-03-16 14:32 UTC (permalink / raw)
  To: dev; +Cc: declan.doherty, Ciara Power

The current crypto unit test framework is not granular enough to
accurately track unit test results. This is caused by one testcase
in a suite actually running multiple testcases, but only returning
one result.
 
The approach taken in this patchset allows a test suite have either a
list of sub-testsuites, or a list of testcases as previously used.
The unit test suite runner can then recursively iterate and run the
sub-testsuites, until it reaches a suite with testcases,
and it then runs each testcase as it had done previously.
 
By allowing this further breakdown into sub-testsuites,
a refactor of the crypto unit tests solves the issue of inaccurate
reporting, as sub-testsuites can be used in place of the testcases
that had multiple testcases hidden on a sub level.
The blockcipher tests previously had these hidden testcases,
but are now sub-testsuites that are dynamically created and added to a
parent test suite, allowing for each testcase status to be reported
directly to the runner.
The cryptodev test suite is broken down into smaller suites that are
used as sub-testsuites, which allows for more flexibility choosing which
sub-testsuites should run for the current device autotest.
The introduction of sub-testsuites also allows for more precise
setup/teardown functions, rather than general ones loaded with
conditions as was seen with the initial setup function used for all
crypto testsuites.
 
For example, when running the cryptodev_aesni_mb_autotest, 
the AESNI MB parent test suite has its own setup function to initialise
the AESNI MB device.
Various sub-testsuites are added to the parent test suite, such as some
of the static suites that were once in the cryptodev_testsuite,
and blockcipher suites.
The unit test runner can then run the AESNI MB parent test suite,
which in turn will run the sub-testsuites.
 
Documentation will be added in a later version of the patchset,
adding to the test document that isn't yet merged. [1]

---
[1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/

Ciara Power (6):
  app/test: refactor of unit test suite runner
  test: introduce parent testsuite format
  test/crypto: refactor to use sub-testsuites
  test/crypto: move testsuite params to header file
  test/crypto: dynamically build blockcipher suite
  doc: add unit test suite change to release notes

 app/test/test.c                        |  168 +-
 app/test/test.h                        |   22 +-
 app/test/test_bitratestats.c           |    4 +-
 app/test/test_compressdev.c            |    4 +-
 app/test/test_cryptodev.c              | 2020 ++++++++++++++++++------
 app/test/test_cryptodev.h              |   20 +
 app/test/test_cryptodev_asym.c         |  105 +-
 app/test/test_cryptodev_blockcipher.c  |  121 +-
 app/test/test_cryptodev_blockcipher.h  |   12 +-
 app/test/test_ethdev_link.c            |    4 +-
 app/test/test_event_crypto_adapter.c   |    4 +-
 app/test/test_event_eth_rx_adapter.c   |    8 +-
 app/test/test_event_eth_tx_adapter.c   |    4 +-
 app/test/test_event_timer_adapter.c    |    4 +-
 app/test/test_eventdev.c               |    4 +-
 app/test/test_fbarray.c                |    4 +-
 app/test/test_fib.c                    |    8 +-
 app/test/test_fib6.c                   |    8 +-
 app/test/test_graph.c                  |    4 +-
 app/test/test_graph_perf.c             |    4 +-
 app/test/test_ipfrag.c                 |    4 +-
 app/test/test_ipsec.c                  |   36 +-
 app/test/test_ipsec_sad.c              |    4 +-
 app/test/test_latencystats.c           |    4 +-
 app/test/test_link_bonding.c           |    4 +-
 app/test/test_link_bonding_mode4.c     |    4 +-
 app/test/test_link_bonding_rssconf.c   |    4 +-
 app/test/test_metrics.c                |    4 +-
 app/test/test_pmd_ring.c               |    4 +-
 app/test/test_reorder.c                |    4 +-
 app/test/test_rib.c                    |    8 +-
 app/test/test_rib6.c                   |    8 +-
 app/test/test_security.c               |    4 +-
 app/test/test_service_cores.c          |    4 +-
 app/test/test_trace.c                  |    4 +-
 doc/guides/rel_notes/release_21_05.rst |    5 +
 36 files changed, 1898 insertions(+), 739 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-04-02 14:34 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-31 14:46 [dpdk-dev] [PATCH 5/6] test/crypto: dynamically build blockcipher suite Akhil Goyal
2021-04-02 14:34 ` Power, Ciara
  -- strict thread matches above, loose matches on Subject: below --
2021-03-16 14:32 [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework Ciara Power
2021-03-16 14:32 ` [dpdk-dev] [PATCH 5/6] test/crypto: dynamically build blockcipher suite Ciara Power

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).