From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 150A1A0C50; Fri, 16 Jul 2021 15:40:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 855C14067B; Fri, 16 Jul 2021 15:40:30 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id 9E97040151 for ; Fri, 16 Jul 2021 15:40:28 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10046"; a="208914915" X-IronPort-AV: E=Sophos;i="5.84,245,1620716400"; d="scan'208";a="208914915" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2021 06:40:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,245,1620716400"; d="scan'208";a="496773505" Received: from silpixa00400355.ir.intel.com (HELO silpixa00400355.ger.corp.intel.com) ([10.237.223.24]) by FMSMGA003.fm.intel.com with ESMTP; 16 Jul 2021 06:40:25 -0700 From: Ciara Power To: dev@dpdk.org Cc: roy.fan.zhang@intel.com, declan.doherty@intel.com, aconole@redhat.com, Ciara Power Date: Fri, 16 Jul 2021 13:40:22 +0000 Message-Id: <20210716134022.61315-1-ciara.power@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210517155517.806841-1-ciara.power@intel.com> References: <20210517155517.806841-1-ciara.power@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2] doc/guides: add details for new test structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The testing guide is now updated to include details about using sub-testsuites. Some example code is given to demonstrate how they can be used. A note is also added to highlight the need for using vdev EAL args when running cryptodev tests. Depends-on: patch-95866 ("guides: add a guide for developing unit tests") Signed-off-by: Ciara Power --- v2: - Updated to depend on v4 of testing doc patch. - Added examples for running with cryptodev vdev args. - Small formatting fixes. --- doc/guides/contributing/unit_test.rst | 93 ++++++++++++++++++++++++++- 1 file changed, 91 insertions(+), 2 deletions(-) diff --git a/doc/guides/contributing/unit_test.rst b/doc/guides/contributing/unit_test.rst index b274bd5f93..265d58d9ea 100644 --- a/doc/guides/contributing/unit_test.rst +++ b/doc/guides/contributing/unit_test.rst @@ -185,13 +185,21 @@ for interacting with the test harness: 2. unit_test_suite_runner(struct unit_test_suite \*) Returns a runner for a full test suite object, which contains - a test suite name, setup, tear down, and vector of unit test - cases. + a test suite name, setup, tear down, a pointer to a list of + sub-testsuites, and vector of unit test cases. Each test suite has a setup and tear down function that runs at the beginning and end of the test suite execution. Each unit test has a similar function for test case setup and tear down. +Each testsuite may use a nested list of sub-testsuites, +which are iterated by the unit_test_suite_runner. +This support allows for better granularity when designing test suites. +The sub-testsuites list can also be used in parallel with the vector +of testcases, in this case the testcases will be run, +and then each sub-testsuite is executed. To see an example of a +testsuite using sub-testsuites, see *app/test/test_cryptodev.c*. + Test cases are added to the `.unit_test_cases` element of the appropriate unit test suite structure. An example of both a test suite and a case: @@ -236,6 +244,70 @@ unit test suite structure. An example of both a test suite and a case: The above code block is a small example that can be used to create a complete test suite with test case. +Sub-testsuites can be added to the `.unit_test_suites` element of +the unit test suite structure, for example: + +.. code-block:: c + :linenos: + + static int testsuite_setup(void) { return TEST_SUCCESS; } + static void testsuite_teardown(void) { } + + static int ut_setup(void) { return TEST_SUCCESS; } + static void ut_teardown(void) { } + + static int test_case_first(void) { return TEST_SUCCESS; } + + static struct unit_test_suite example_parent_testsuite = { + .suite_name = "EXAMPLE PARENT TEST SUITE", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = {TEST_CASES_END()} + }; + + static int sub_testsuite_setup(void) { return TEST_SUCCESS; } + static void sub_testsuite_teardown(void) { } + + static struct unit_test_suite example_sub_testsuite = { + .suite_name = "EXAMPLE SUB TEST SUITE", + .setup = sub_testsuite_setup, + .teardown = sub_testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(ut_setup, ut_teardown, test_case_first), + + TEST_CASES_END(), /**< NULL terminate unit test array */ + }, + }; + + static struct unit_test_suite end_testsuite = { + .suite_name = NULL, + .setup = NULL, + .teardown = NULL, + .unit_test_suites = NULL + }; + + static int example_tests() + { + uint8_t ret, i = 0; + struct unit_test_suite *sub_suites[] = { + &example_sub_testsuite, + &end_testsuite /**< NULL test suite to indicate end of list */ + }; + + example_parent_testsuite.unit_test_suites = + malloc(sizeof(struct unit_test_suite *) * RTE_DIM(sub_suites)); + + for (i = 0; i < RTE_DIM(sub_suites); i++) + example_parent_testsuite.unit_test_suites[i] = sub_suites[i]; + + ret = unit_test_suite_runner(&example_parent_testsuite); + free(example_parent_testsuite.unit_test_suites); + + return ret; + } + + REGISTER_TEST_COMMAND(example_autotest, example_tests); + Designing a test ---------------- @@ -325,3 +397,20 @@ In general, when a test is added to the `dpdk-test` application, it probably should be added to a meson test suite, but the choice is left to maintainers and individual developers. Preference is to add tests to the meson test suites. + + +Running Cryptodev Tests +----------------------- + +When running cryptodev tests, the user must create any required virtual +device via EAL args, as this is not automatically done by the test:: + + $ ./build/app/test/dpdk-test --vdev crypto_aesni_mb + $ meson test -C build --suite driver-tests \ + --test-args="--vdev crypto_aesni_mb" + +.. note:: + + The cryptodev_scheduler_autotest is the only exception to this. + This vdev will be created automatically by the test app, + as it requires a more complex setup than other vdevs. -- 2.25.1