From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B7DA6A0A02; Thu, 13 May 2021 15:28:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53ADC4067E; Thu, 13 May 2021 15:28:27 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id D602F4003F for <dev@dpdk.org>; Thu, 13 May 2021 15:28:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620912504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=oavbrCw/vmuQh860sBBoZFAp5C44QYh52sFnKNIn3GQ=; b=W4GwPbPbDvW2z1sIrXEyKofFbd2cHZw2RBytJUfqrsq3t0kJYrxJWiaXhV+j9Mnrb66+1n YR3llWQFx6kpeHc2Jw6KKMAehvjqchvjl9w3GNZOUrabUActfGnkmtctq6SZE2zOYQMRGC qzdsCX+3vbIIMgMw8RZMJTczo2APPfQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-155-U3OloNpDPCOp7WsMxFbQAA-1; Thu, 13 May 2021 09:28:22 -0400 X-MC-Unique: U3OloNpDPCOp7WsMxFbQAA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AC553107ACE3; Thu, 13 May 2021 13:28:20 +0000 (UTC) Received: from RHTPC1VM0NT (ovpn-114-213.rdu2.redhat.com [10.10.114.213]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C8B6760CC6; Thu, 13 May 2021 13:28:18 +0000 (UTC) From: Aaron Conole <aconole@redhat.com> To: Akhil Goyal <gakhil@marvell.com> Cc: Ciara Power <ciara.power@intel.com>, "dev@dpdk.org" <dev@dpdk.org>, "declan.doherty@intel.com" <declan.doherty@intel.com>, "hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>, Anoob Joseph <anoobj@marvell.com>, "ruifeng.wang@arm.com" <ruifeng.wang@arm.com>, "asomalap@amd.com" <asomalap@amd.com>, "ajit.khaparde@broadcom.com" <ajit.khaparde@broadcom.com>, "g.singh@nxp.com" <g.singh@nxp.com>, "roy.fan.zhang@intel.com" <roy.fan.zhang@intel.com>, Ilya Maximets <imaximet@redhat.com> References: <20210512113655.568814-1-ciara.power@intel.com> <MW2PR18MB2284B2DCD4FB8BEFEA8AE9B1D8529@MW2PR18MB2284.namprd18.prod.outlook.com> Date: Thu, 13 May 2021 09:28:17 -0400 In-Reply-To: <MW2PR18MB2284B2DCD4FB8BEFEA8AE9B1D8529@MW2PR18MB2284.namprd18.prod.outlook.com> (Akhil Goyal's message of "Wed, 12 May 2021 13:15:46 +0000") Message-ID: <f7tr1iasqz2.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=aconole@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [dpdk-dev] [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> Akhil Goyal <gakhil@marvell.com> writes: > Hi Ciara, > > I see that CI is reporting one error in autotest. Can you please check > if it is because of your patchset? > > 9/96 DPDK:fast-tests / cycles_autotest FAIL 1.42s (exit status 255 or > signal 127 SIGinvalid) Just to note, I'm looking into that test case a bit more. It would be nice if the failures were asserts that listed the values. I also wonder if it's possible that we hit some kind of delay on wake up (especially on busy systems like CI systems) and should have a flag to disable this. I can see how it's possible for this test case to give a false positive in some system configurations but not sure how best to remedy that situation. > 12:33:15 DPDK_TEST=cycles_autotest MALLOC_PERTURB_=44 > /home/jenkins-local/jenkins-agent/workspace/Ubuntu18.04-Unit-Test-DPDK/dpdk/build/app/test/dpdk-test > '-l 0-3' --file-prefix=cycles_autotest > ----------------------------------- output ----------------------------------- > stdout: > RTE>>cycles_autotest > > delay_us is not accurate: too long > Test Failed > > > Regards, > Akhil > >> -----Original Message----- >> From: Ciara Power <ciara.power@intel.com> >> Sent: Wednesday, May 12, 2021 5:07 PM >> To: dev@dpdk.org >> Cc: declan.doherty@intel.com; Akhil Goyal <gakhil@marvell.com>; >> aconole@redhat.com; hemant.agrawal@nxp.com; Anoob Joseph >> <anoobj@marvell.com>; ruifeng.wang@arm.com; asomalap@amd.com; >> ajit.khaparde@broadcom.com; g.singh@nxp.com; roy.fan.zhang@intel.com; >> Ciara Power <ciara.power@intel.com> >> Subject: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework >> >> External Email >> >> ---------------------------------------------------------------------- >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase >> in a suite actually running multiple testcases, but only returning >> one result. >> >> The approach taken in this patchset allows a test suite have a >> list of sub-testsuites, and/or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub-testsuites, until it reaches a suite with testcases, >> and it then runs each testcase as it had done previously. >> In the case of a testsuite with both testcases and sub-testsuites, >> the testcases are executed first before iterating through the >> sub-testsuites. >> >> By allowing this further breakdown into sub-testsuites, >> a refactor of the crypto unit tests solves the issue of inaccurate >> reporting, as sub-testsuites can be used in place of the testcases >> that had multiple testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, >> but are now sub-testsuites that are dynamically created and added to a >> parent test suite, allowing for each testcase status to be reported >> directly to the runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing which >> sub-testsuites should run for the current device. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, that can check the capabilities required to >> run its testcases. >> >> For example, when running the cryptodev_aesni_mb_autotest, >> the parent Cryptodev Test Suite is executed. >> Various sub-testsuites are added to the parent test suite, such as >> the static suites of testcases that were once in the cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the Cryptodev parent test suite, >> which in turn will run the sub-testsuites. >> >> The user is now required to create vdevs via EAL commandline args, >> this is no longer done within the test app for crypto autotests. >> >> Documentation will need to be added at a later stage, >> adding to the test document that isn't yet merged. [1] >> >> [1] https://urldefense.proofpoint.com/v2/url?u=https- >> 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536- >> 2D1-2Daconole- >> 40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PR >> wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=XItKx7UnOATOiVC3kRLlV3Woh >> rLeZbmlqShu_py452U&s=BpOMigsNnvWMV739- >> fMxEvVERJieNSnfnnmZdiMLMJg&e= >> >> >> v4: >> - Fixed some return value bugs related to patch 4. >> - Rebased onto main. >> v3: >> - Added support for a testsuite having both a list of testcases, >> and a list of sub-testsuites. >> - Replaced PMD based parent testsuites with a cryptodev testsuite >> used by all autotests, with the exception of scheduler autotest. >> - Setup functions were added for all sub-testsuites, within which >> required capability support is checked. >> - The setup functions no longer create vdevs if needed, >> this must be done by the user when running the test. >> - Patch added to standardise return values for skipped testcases. >> v2: >> - Added macro in place of testcase/testsuite loops. >> - Added more detail in the summary output. >> - Moved testcase counts to the testsuite structure. >> - Flattened testsuite structure to remove union. >> - Added patch for fix of blockcipher test return value. >> - Squashed release note into last patch. >> >> Ciara Power (7): >> app/test: refactor of unit test suite runner >> test: introduce parent testsuite format >> test/crypto: refactor to use sub-testsuites >> test/crypto: replace unsupported with skipped >> test/crypto: move testsuite params to header file >> test/crypto: fix return value on test skipped >> test/crypto: dynamically build blockcipher suite >> >> app/test/test.c | 176 +- >> app/test/test.h | 23 +- >> app/test/test_cryptodev.c | 2333 ++++++++++++++---------- >> app/test/test_cryptodev.h | 32 + >> app/test/test_cryptodev_asym.c | 111 +- >> app/test/test_cryptodev_blockcipher.c | 423 ++++- >> app/test/test_cryptodev_blockcipher.h | 12 +- >> app/test/test_ipsec.c | 32 +- >> doc/guides/rel_notes/release_21_05.rst | 5 + >> 9 files changed, 2004 insertions(+), 1143 deletions(-) >> >> -- >> 2.25.1