From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CB355A054F; Tue, 16 Mar 2021 15:33:02 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 87E0D26005B; Tue, 16 Mar 2021 15:33:02 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id D89824069F for ; Tue, 16 Mar 2021 15:33:00 +0100 (CET) IronPort-SDR: lYhqcopjIXg8owFW9v/KPGWbpB2qM8tC7PNgegT4tuPdHenm/4dQQvbgwKUnsdaY8wfB5a0PeN 0ruvVZeuqnIg== X-IronPort-AV: E=McAfee;i="6000,8403,9924"; a="185906428" X-IronPort-AV: E=Sophos;i="5.81,251,1610438400"; d="scan'208";a="185906428" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Mar 2021 07:32:59 -0700 IronPort-SDR: 1aWNfoKumoSWwEB5FlxOARzT1EGe4INMNxa/6coSGyJMcXyjhaZ0g+2omswoKrtL8Eo5TN0Cyz jKKBmyyb5rjw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,251,1610438400"; d="scan'208";a="373805401" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.223.148]) by orsmga006.jf.intel.com with ESMTP; 16 Mar 2021 07:32:58 -0700 From: Ciara Power To: dev@dpdk.org Cc: declan.doherty@intel.com, Ciara Power Date: Tue, 16 Mar 2021 14:32:47 +0000 Message-Id: <20210316143253.3849182-1-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The current crypto unit test framework is not granular enough to accurately track unit test results. This is caused by one testcase in a suite actually running multiple testcases, but only returning one result. The approach taken in this patchset allows a test suite have either a list of sub-testsuites, or a list of testcases as previously used. The unit test suite runner can then recursively iterate and run the sub-testsuites, until it reaches a suite with testcases, and it then runs each testcase as it had done previously. By allowing this further breakdown into sub-testsuites, a refactor of the crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites can be used in place of the testcases that had multiple testcases hidden on a sub level. The blockcipher tests previously had these hidden testcases, but are now sub-testsuites that are dynamically created and added to a parent test suite, allowing for each testcase status to be reported directly to the runner. The cryptodev test suite is broken down into smaller suites that are used as sub-testsuites, which allows for more flexibility choosing which sub-testsuites should run for the current device autotest. The introduction of sub-testsuites also allows for more precise setup/teardown functions, rather than general ones loaded with conditions as was seen with the initial setup function used for all crypto testsuites. For example, when running the cryptodev_aesni_mb_autotest, the AESNI MB parent test suite has its own setup function to initialise the AESNI MB device. Various sub-testsuites are added to the parent test suite, such as some of the static suites that were once in the cryptodev_testsuite, and blockcipher suites. The unit test runner can then run the AESNI MB parent test suite, which in turn will run the sub-testsuites. Documentation will be added in a later version of the patchset, adding to the test document that isn't yet merged. [1] --- [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ Ciara Power (6): app/test: refactor of unit test suite runner test: introduce parent testsuite format test/crypto: refactor to use sub-testsuites test/crypto: move testsuite params to header file test/crypto: dynamically build blockcipher suite doc: add unit test suite change to release notes app/test/test.c | 168 +- app/test/test.h | 22 +- app/test/test_bitratestats.c | 4 +- app/test/test_compressdev.c | 4 +- app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ app/test/test_cryptodev.h | 20 + app/test/test_cryptodev_asym.c | 105 +- app/test/test_cryptodev_blockcipher.c | 121 +- app/test/test_cryptodev_blockcipher.h | 12 +- app/test/test_ethdev_link.c | 4 +- app/test/test_event_crypto_adapter.c | 4 +- app/test/test_event_eth_rx_adapter.c | 8 +- app/test/test_event_eth_tx_adapter.c | 4 +- app/test/test_event_timer_adapter.c | 4 +- app/test/test_eventdev.c | 4 +- app/test/test_fbarray.c | 4 +- app/test/test_fib.c | 8 +- app/test/test_fib6.c | 8 +- app/test/test_graph.c | 4 +- app/test/test_graph_perf.c | 4 +- app/test/test_ipfrag.c | 4 +- app/test/test_ipsec.c | 36 +- app/test/test_ipsec_sad.c | 4 +- app/test/test_latencystats.c | 4 +- app/test/test_link_bonding.c | 4 +- app/test/test_link_bonding_mode4.c | 4 +- app/test/test_link_bonding_rssconf.c | 4 +- app/test/test_metrics.c | 4 +- app/test/test_pmd_ring.c | 4 +- app/test/test_reorder.c | 4 +- app/test/test_rib.c | 8 +- app/test/test_rib6.c | 8 +- app/test/test_security.c | 4 +- app/test/test_service_cores.c | 4 +- app/test/test_trace.c | 4 +- doc/guides/rel_notes/release_21_05.rst | 5 + 36 files changed, 1898 insertions(+), 739 deletions(-) -- 2.25.1