DPDK patches and discussions
 help / color / mirror / Atom feed
From: Aaron Conole <aconole@redhat.com>
To: "Doherty\, Declan" <declan.doherty@intel.com>
Cc: Ciara Power <ciara.power@intel.com>,  dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework
Date: Wed, 31 Mar 2021 10:43:17 -0400	[thread overview]
Message-ID: <f7tft0bv0xm.fsf@dhcp-25.97.bos.redhat.com> (raw)
In-Reply-To: <e889dba0-9514-f906-be5b-24842b780d18@intel.com> (Declan Doherty's message of "Tue, 30 Mar 2021 17:15:17 +0100")

"Doherty, Declan" <declan.doherty@intel.com> writes:

> Hey Aaron,
>
> based on the work you've been doing on the unit test documentation we
> would appreciate if you could take a look over this patchset and get
> your thoughts. The primary drive here is to make it easier to get a
> clear picture of what is being executed in the cryptodev testsuite, as
> at the moment there are test suites masquerading as unit tests and the
> output doesn't reflect the actual number of unit tests being executed.

I sent some comments - thanks :)

> Thanks
> Declan
>
> On 16/03/2021 2:32 PM, Ciara Power wrote:
>> The current crypto unit test framework is not granular enough to
>> accurately track unit test results. This is caused by one testcase
>> in a suite actually running multiple testcases, but only returning
>> one result.
>>   The approach taken in this patchset allows a test suite have
>> either a
>> list of sub-testsuites, or a list of testcases as previously used.
>> The unit test suite runner can then recursively iterate and run the
>> sub-testsuites, until it reaches a suite with testcases,
>> and it then runs each testcase as it had done previously.
>>   By allowing this further breakdown into sub-testsuites,
>> a refactor of the crypto unit tests solves the issue of inaccurate
>> reporting, as sub-testsuites can be used in place of the testcases
>> that had multiple testcases hidden on a sub level.
>> The blockcipher tests previously had these hidden testcases,
>> but are now sub-testsuites that are dynamically created and added to a
>> parent test suite, allowing for each testcase status to be reported
>> directly to the runner.
>> The cryptodev test suite is broken down into smaller suites that are
>> used as sub-testsuites, which allows for more flexibility choosing which
>> sub-testsuites should run for the current device autotest.
>> The introduction of sub-testsuites also allows for more precise
>> setup/teardown functions, rather than general ones loaded with
>> conditions as was seen with the initial setup function used for all
>> crypto testsuites.
>>   For example, when running the cryptodev_aesni_mb_autotest,
>> the AESNI MB parent test suite has its own setup function to initialise
>> the AESNI MB device.
>> Various sub-testsuites are added to the parent test suite, such as some
>> of the static suites that were once in the cryptodev_testsuite,
>> and blockcipher suites.
>> The unit test runner can then run the AESNI MB parent test suite,
>> which in turn will run the sub-testsuites.
>>   Documentation will be added in a later version of the patchset,
>> adding to the test document that isn't yet merged. [1]
>>
>> ---
>> [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/
>>
>> Ciara Power (6):
>>    app/test: refactor of unit test suite runner
>>    test: introduce parent testsuite format
>>    test/crypto: refactor to use sub-testsuites
>>    test/crypto: move testsuite params to header file
>>    test/crypto: dynamically build blockcipher suite
>>    doc: add unit test suite change to release notes
>>
>>   app/test/test.c                        |  168 +-
>>   app/test/test.h                        |   22 +-
>>   app/test/test_bitratestats.c           |    4 +-
>>   app/test/test_compressdev.c            |    4 +-
>>   app/test/test_cryptodev.c              | 2020 ++++++++++++++++++------
>>   app/test/test_cryptodev.h              |   20 +
>>   app/test/test_cryptodev_asym.c         |  105 +-
>>   app/test/test_cryptodev_blockcipher.c  |  121 +-
>>   app/test/test_cryptodev_blockcipher.h  |   12 +-
>>   app/test/test_ethdev_link.c            |    4 +-
>>   app/test/test_event_crypto_adapter.c   |    4 +-
>>   app/test/test_event_eth_rx_adapter.c   |    8 +-
>>   app/test/test_event_eth_tx_adapter.c   |    4 +-
>>   app/test/test_event_timer_adapter.c    |    4 +-
>>   app/test/test_eventdev.c               |    4 +-
>>   app/test/test_fbarray.c                |    4 +-
>>   app/test/test_fib.c                    |    8 +-
>>   app/test/test_fib6.c                   |    8 +-
>>   app/test/test_graph.c                  |    4 +-
>>   app/test/test_graph_perf.c             |    4 +-
>>   app/test/test_ipfrag.c                 |    4 +-
>>   app/test/test_ipsec.c                  |   36 +-
>>   app/test/test_ipsec_sad.c              |    4 +-
>>   app/test/test_latencystats.c           |    4 +-
>>   app/test/test_link_bonding.c           |    4 +-
>>   app/test/test_link_bonding_mode4.c     |    4 +-
>>   app/test/test_link_bonding_rssconf.c   |    4 +-
>>   app/test/test_metrics.c                |    4 +-
>>   app/test/test_pmd_ring.c               |    4 +-
>>   app/test/test_reorder.c                |    4 +-
>>   app/test/test_rib.c                    |    8 +-
>>   app/test/test_rib6.c                   |    8 +-
>>   app/test/test_security.c               |    4 +-
>>   app/test/test_service_cores.c          |    4 +-
>>   app/test/test_trace.c                  |    4 +-
>>   doc/guides/rel_notes/release_21_05.rst |    5 +
>>   36 files changed, 1898 insertions(+), 739 deletions(-)
>>


  reply	other threads:[~2021-03-31 14:43 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-16 14:32 Ciara Power
2021-03-16 14:32 ` [dpdk-dev] [PATCH 1/6] app/test: refactor of unit test suite runner Ciara Power
2021-03-31 14:42   ` Aaron Conole
2021-03-16 14:32 ` [dpdk-dev] [PATCH 2/6] test: introduce parent testsuite format Ciara Power
2021-03-31 14:42   ` Aaron Conole
2021-03-16 14:32 ` [dpdk-dev] [PATCH 3/6] test/crypto: refactor to use sub-testsuites Ciara Power
2021-03-16 14:32 ` [dpdk-dev] [PATCH 4/6] test/crypto: move testsuite params to header file Ciara Power
2021-03-16 14:32 ` [dpdk-dev] [PATCH 5/6] test/crypto: dynamically build blockcipher suite Ciara Power
2021-03-16 14:32 ` [dpdk-dev] [PATCH 6/6] doc: add unit test suite change to release notes Ciara Power
2021-03-30 16:15 ` [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework Doherty, Declan
2021-03-31 14:43   ` Aaron Conole [this message]
2021-04-02 14:32     ` Power, Ciara
2021-04-01  3:13 ` Ruifeng Wang
2021-04-02 14:29   ` Power, Ciara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f7tft0bv0xm.fsf@dhcp-25.97.bos.redhat.com \
    --to=aconole@redhat.com \
    --cc=ciara.power@intel.com \
    --cc=declan.doherty@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).