From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E001A034F; Wed, 31 Mar 2021 16:43:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4E440140F0D; Wed, 31 Mar 2021 16:43:28 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mails.dpdk.org (Postfix) with ESMTP id 27813140EF2 for ; Wed, 31 Mar 2021 16:43:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1617201805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OJIPl+6yIPK8jR7dV2F5tyGZ649gzE69M9LZfv4b4EI=; b=Rf0PV+3WwcqxIsmk2d+rVCCDGaE5GQq6y3k732m3guZaj/9I6Ac/PwvJm1BE8y3Z3XbS6b c3GaibCG24eteqfn5nsakxkX+AQrRFIGiQQJcBmTkxlegcr++RWN/sxz8DlMTVq6aaqqvL bkA2wbRw2UGXfhAL37UVxuqTNjJxitc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-458-w9vfk3MqO--dGMJpuN7JPA-1; Wed, 31 Mar 2021 10:43:21 -0400 X-MC-Unique: w9vfk3MqO--dGMJpuN7JPA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 89F1310866A3; Wed, 31 Mar 2021 14:43:20 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (ovpn-113-108.rdu2.redhat.com [10.10.113.108]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E7C4817A6A; Wed, 31 Mar 2021 14:43:19 +0000 (UTC) From: Aaron Conole To: "Doherty\, Declan" Cc: Ciara Power , dev@dpdk.org References: <20210316143253.3849182-1-ciara.power@intel.com> Date: Wed, 31 Mar 2021 10:43:17 -0400 In-Reply-To: (Declan Doherty's message of "Tue, 30 Mar 2021 17:15:17 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=aconole@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" "Doherty, Declan" writes: > Hey Aaron, > > based on the work you've been doing on the unit test documentation we > would appreciate if you could take a look over this patchset and get > your thoughts. The primary drive here is to make it easier to get a > clear picture of what is being executed in the cryptodev testsuite, as > at the moment there are test suites masquerading as unit tests and the > output doesn't reflect the actual number of unit tests being executed. I sent some comments - thanks :) > Thanks > Declan > > On 16/03/2021 2:32 PM, Ciara Power wrote: >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase >> in a suite actually running multiple testcases, but only returning >> one result. >> The approach taken in this patchset allows a test suite have >> either a >> list of sub-testsuites, or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub-testsuites, until it reaches a suite with testcases, >> and it then runs each testcase as it had done previously. >> By allowing this further breakdown into sub-testsuites, >> a refactor of the crypto unit tests solves the issue of inaccurate >> reporting, as sub-testsuites can be used in place of the testcases >> that had multiple testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, >> but are now sub-testsuites that are dynamically created and added to a >> parent test suite, allowing for each testcase status to be reported >> directly to the runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing which >> sub-testsuites should run for the current device autotest. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, rather than general ones loaded with >> conditions as was seen with the initial setup function used for all >> crypto testsuites. >> For example, when running the cryptodev_aesni_mb_autotest, >> the AESNI MB parent test suite has its own setup function to initialise >> the AESNI MB device. >> Various sub-testsuites are added to the parent test suite, such as some >> of the static suites that were once in the cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the AESNI MB parent test suite, >> which in turn will run the sub-testsuites. >> Documentation will be added in a later version of the patchset, >> adding to the test document that isn't yet merged. [1] >> >> --- >> [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ >> >> Ciara Power (6): >> app/test: refactor of unit test suite runner >> test: introduce parent testsuite format >> test/crypto: refactor to use sub-testsuites >> test/crypto: move testsuite params to header file >> test/crypto: dynamically build blockcipher suite >> doc: add unit test suite change to release notes >> >> app/test/test.c | 168 +- >> app/test/test.h | 22 +- >> app/test/test_bitratestats.c | 4 +- >> app/test/test_compressdev.c | 4 +- >> app/test/test_cryptodev.c | 2020 ++++++++++++++++++------ >> app/test/test_cryptodev.h | 20 + >> app/test/test_cryptodev_asym.c | 105 +- >> app/test/test_cryptodev_blockcipher.c | 121 +- >> app/test/test_cryptodev_blockcipher.h | 12 +- >> app/test/test_ethdev_link.c | 4 +- >> app/test/test_event_crypto_adapter.c | 4 +- >> app/test/test_event_eth_rx_adapter.c | 8 +- >> app/test/test_event_eth_tx_adapter.c | 4 +- >> app/test/test_event_timer_adapter.c | 4 +- >> app/test/test_eventdev.c | 4 +- >> app/test/test_fbarray.c | 4 +- >> app/test/test_fib.c | 8 +- >> app/test/test_fib6.c | 8 +- >> app/test/test_graph.c | 4 +- >> app/test/test_graph_perf.c | 4 +- >> app/test/test_ipfrag.c | 4 +- >> app/test/test_ipsec.c | 36 +- >> app/test/test_ipsec_sad.c | 4 +- >> app/test/test_latencystats.c | 4 +- >> app/test/test_link_bonding.c | 4 +- >> app/test/test_link_bonding_mode4.c | 4 +- >> app/test/test_link_bonding_rssconf.c | 4 +- >> app/test/test_metrics.c | 4 +- >> app/test/test_pmd_ring.c | 4 +- >> app/test/test_reorder.c | 4 +- >> app/test/test_rib.c | 8 +- >> app/test/test_rib6.c | 8 +- >> app/test/test_security.c | 4 +- >> app/test/test_service_cores.c | 4 +- >> app/test/test_trace.c | 4 +- >> doc/guides/rel_notes/release_21_05.rst | 5 + >> 36 files changed, 1898 insertions(+), 739 deletions(-) >>