From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3E261A0A02; Mon, 17 May 2021 17:55:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0766C4014E; Mon, 17 May 2021 17:55:47 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 85C7940041 for ; Mon, 17 May 2021 17:55:45 +0200 (CEST) IronPort-SDR: LbKcmYKDExG4iIn7wGLYV2mOfdelvtbo+iHRzyhFZ/SHPYI8kFCU/cGB0OG4yfMw6IB1bERStx alkZXYJwnEcw== X-IronPort-AV: E=McAfee;i="6200,9189,9987"; a="200187136" X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="200187136" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2021 08:55:44 -0700 IronPort-SDR: aOlkemwpJXft5YEDwZs2otNg0s28EbTUU+G/6TfQmv1o6LdZqO8Y8FSv7P8ko40R9tNvrHoBnE uv7BA5F6/tjg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="394504699" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.223.148]) by orsmga006.jf.intel.com with ESMTP; 17 May 2021 08:55:42 -0700 From: Ciara Power To: dev@dpdk.org Cc: roy.fan.zhang@intel.com, declan.doherty@intel.com, aconole@redhat.com, Ciara Power Date: Mon, 17 May 2021 15:55:17 +0000 Message-Id: <20210517155517.806841-1-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH] doc/guides: add details for new test structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The testing guide is now updated to include details about using sub-testsuites. Some example code is given to demonstrate how they can be used. A note is also added to highlight the need for using vdev EAL args when running cryptodev tests. Depends-on: patch-88751 ("guides: add a guide for developing unit tests") Signed-off-by: Ciara Power --- doc/guides/contributing/testing.rst | 89 ++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/doc/guides/contributing/testing.rst b/doc/guides/contributing/testing.rst index 0757d71ad0..1188d63a97 100644 --- a/doc/guides/contributing/testing.rst +++ b/doc/guides/contributing/testing.rst @@ -114,13 +114,21 @@ for interacting with the test harness: 2. unit_test_suite_runner(struct unit_test_suite \*) Returns a runner for a full test suite object, which contains - a test suite name, setup, tear down, and vector of unit test - cases. + a test suite name, setup, tear down, a pointer to a list of + sub-testsuites, and vector of unit test cases. Each test suite has a setup and tear down function that runs at the beginning and end of the test suite execution. Each unit test has a similar function for test case setup and tear down. +Each testsuite may use a nested list of sub-testsuites, +which are iterated by the unit_test_suite_runner. +This support allows for better granularity when designing test suites. +The sub-testsuites list can also be used in parallel with the vector +of testcases, in this case the testcases will be run, +and then each sub-testsuite is executed. To see an example of a +testsuite using sub-testsuites, see *app/test/test_cryptodev.c*. + Test cases are added to the `.unit_test_cases` element of the unit test suite structure. Ex: @@ -165,6 +173,70 @@ test suite structure. Ex: The above code block is a small example that can be used to create a complete test suite with test case. +Sub-testsuites can be added to the `.unit_test_suites` element of +the unit test suite structure. Ex: + +.. code-block:: c + :linenos: + + static int testsuite_setup(void) { return TEST_SUCCESS; } + static void testsuite_teardown(void) { } + + static int ut_setup(void) { return TEST_SUCCESS; } + static void ut_teardown(void) { } + + static int test_case_first(void) { return TEST_SUCCESS; } + + static struct unit_test_suite example_parent_testsuite = { + .suite_name = "EXAMPLE PARENT TEST SUITE", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = {TEST_CASES_END()} + }; + + static int sub_testsuite_setup(void) { return TEST_SUCCESS; } + static void sub_testsuite_teardown(void) { } + + static struct unit_test_suite example_sub_testsuite = { + .suite_name = "EXAMPLE SUB TEST SUITE", + .setup = sub_testsuite_setup, + .teardown = sub_testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(ut_setup, ut_teardown, test_case_first), + + TEST_CASES_END(), /**< NULL terminate unit test array */ + }, + }; + + static struct unit_test_suite end_testsuite = { + .suite_name = NULL, + .setup = NULL, + .teardown = NULL, + .unit_test_suites = NULL + }; + + static int example_tests() + { + uint8_t ret, i = 0; + struct unit_test_suite *sub_suites[] = { + &example_sub_testsuite, + &end_testsuite /**< NULL test suite to indicate end of list */ + }; + + example_parent_testsuite.unit_test_suites = + malloc(sizeof(struct unit_test_suite *) * RTE_DIM(sub_suites)); + + for (i = 0; i < RTE_DIM(sub_suites); i++) + example_parent_testsuite.unit_test_suites[i] = sub_suites[i]; + + ret = unit_test_suite_runner(&example_parent_testsuite); + free(example_parent_testsuite.unit_test_suites); + + return ret; + } + + REGISTER_TEST_COMMAND(example_autotest, example_tests); + Designing a test ---------------- @@ -241,3 +313,16 @@ be run as part of the appropriate class (fast, perf, driver, etc.). Some of these default test suites are run during continuous integration tests, making regression checking automatic for new patches submitted to the project. + + +Running Cryptodev Tests +----------------------- + +When running cryptodev tests, the user must create any required virtual +device via EAL args, as this is not automatically done by the test. + +.. note:: + + The cryptodev_scheduler_autotest is the only exception to this. + This vdev will be created automatically by the test app, + as it requires a more complex setup than other vdevs. -- 2.25.1