From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A7B82A0C41; Wed, 12 May 2021 17:42:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32AED4003F; Wed, 12 May 2021 17:42:21 +0200 (CEST) Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com [66.111.4.224]) by mails.dpdk.org (Postfix) with ESMTP id BE2904003E for ; Wed, 12 May 2021 17:42:19 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id 20D0E5805EB; Wed, 12 May 2021 11:42:19 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 12 May 2021 11:42:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm1; bh= BwmLYUF6tUdKjkj2mm3KO7jbjtbBGvuJ316VQQNIJtE=; b=c2JUOol9Nb1EzDiW aZbXXdwBPFoCSBe82zda4w+9d8Oiorqv+iHVYXaNOac2uCO4jbfCJHnX+sycTaFJ cudKAyV8prrT9v2mFBpYDa3ND6NM03992//0g5Gpssgn0IMjowPCIZSf72dwgNp8 zmzj1QC63698SjQgEkhHMLia0Q2Bph+a0vqQAS+4CgUwdThNX91VpVu6Vg/IdQy1 vciFEYYpmdgFH57rdPdcGorBwt+x1A54HAIfj7xOEtieZhhAiadZMXJ9IUke+uEM WRRvehc2X+pnOIUFmPq+GvLqY1ZSaglGN3XHZ8bYskh18qM27rrqJj0IDBFclnjP h3pVpA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=BwmLYUF6tUdKjkj2mm3KO7jbjtbBGvuJ316VQQNIJ tE=; b=sKd9C8yp4qNH6ImXeMiXANGVtBuiD5gA6s1CjaYJ7tSp32k9THbVCrdWu lkj3IDM7RQp6wxEdXkh+o0SfuwuFndClBq3pYFxqiKMpFR8BlE7C6wcipXORCYOb kFuTT/6K/PCix2xTVrwnYCxoPTub3qO/29UcKuMOd+iaywDWftYLjMNae+d6Tma5 VCv+OPeaa+U6zXHqXnpb22qd2N1m93tcJTuaBqbwTTxtlzf+/R4aMp2/KyLy+NDL l0LBUILdgCLn2b4vjxaWEQssfWqaJsii5YmeeLJqG0A3BgppqJVzJSoVdrVqLQxE JHyRbES5RUN63NZQGoq6pPIRqQIiQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdehvddgkeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtufertddttddvnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepjeegfeeffeeuveejhfdvhfekveelfeejhfdvueetgeeutddvjeet teekudevhedunecuffhomhgrihhnpehprhhoohhfphhoihhnthdrtghomhdpughpughkrd horhhgnecukfhppeejjedrudefgedrvddtfedrudekgeenucevlhhushhtvghrufhiiigv pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghlohhnrd hnvght X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 12 May 2021 11:42:15 -0400 (EDT) From: Thomas Monjalon To: Ciara Power , Akhil Goyal Cc: "dev@dpdk.org" , "declan.doherty@intel.com" , "aconole@redhat.com" , "hemant.agrawal@nxp.com" , Anoob Joseph , "ruifeng.wang@arm.com" , "asomalap@amd.com" , "ajit.khaparde@broadcom.com" , "g.singh@nxp.com" , "roy.fan.zhang@intel.com" Date: Wed, 12 May 2021 17:42:13 +0200 Message-ID: <17245202.EenqpPWYet@thomas> In-Reply-To: References: <20210512113655.568814-1-ciara.power@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" I see this patchset is merged in the crypto tree but I guess it is not definitive. Am I right it is too late to do such refactoring after -rc2? What is the intent? 12/05/2021 15:15, Akhil Goyal: > Hi Ciara, > > I see that CI is reporting one error in autotest. Can you please check if it is because of your patchset? > > 9/96 DPDK:fast-tests / cycles_autotest FAIL 1.42s (exit status 255 or signal 127 SIGinvalid) > 12:33:15 DPDK_TEST=cycles_autotest MALLOC_PERTURB_=44 /home/jenkins-local/jenkins-agent/workspace/Ubuntu18.04-Unit-Test-DPDK/dpdk/build/app/test/dpdk-test '-l 0-3' --file-prefix=cycles_autotest > ----------------------------------- output ----------------------------------- > stdout: > RTE>>cycles_autotest > > delay_us is not accurate: too long > Test Failed > > > Regards, > Akhil > > > -----Original Message----- > > From: Ciara Power > > Sent: Wednesday, May 12, 2021 5:07 PM > > To: dev@dpdk.org > > Cc: declan.doherty@intel.com; Akhil Goyal ; > > aconole@redhat.com; hemant.agrawal@nxp.com; Anoob Joseph > > ; ruifeng.wang@arm.com; asomalap@amd.com; > > ajit.khaparde@broadcom.com; g.singh@nxp.com; roy.fan.zhang@intel.com; > > Ciara Power > > Subject: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework > > > > External Email > > > > ---------------------------------------------------------------------- > > The current crypto unit test framework is not granular enough to > > accurately track unit test results. This is caused by one testcase > > in a suite actually running multiple testcases, but only returning > > one result. > > > > The approach taken in this patchset allows a test suite have a > > list of sub-testsuites, and/or a list of testcases as previously used. > > The unit test suite runner can then recursively iterate and run the > > sub-testsuites, until it reaches a suite with testcases, > > and it then runs each testcase as it had done previously. > > In the case of a testsuite with both testcases and sub-testsuites, > > the testcases are executed first before iterating through the > > sub-testsuites. > > > > By allowing this further breakdown into sub-testsuites, > > a refactor of the crypto unit tests solves the issue of inaccurate > > reporting, as sub-testsuites can be used in place of the testcases > > that had multiple testcases hidden on a sub level. > > The blockcipher tests previously had these hidden testcases, > > but are now sub-testsuites that are dynamically created and added to a > > parent test suite, allowing for each testcase status to be reported > > directly to the runner. > > The cryptodev test suite is broken down into smaller suites that are > > used as sub-testsuites, which allows for more flexibility choosing which > > sub-testsuites should run for the current device. > > The introduction of sub-testsuites also allows for more precise > > setup/teardown functions, that can check the capabilities required to > > run its testcases. > > > > For example, when running the cryptodev_aesni_mb_autotest, > > the parent Cryptodev Test Suite is executed. > > Various sub-testsuites are added to the parent test suite, such as > > the static suites of testcases that were once in the cryptodev_testsuite, > > and blockcipher suites. > > The unit test runner can then run the Cryptodev parent test suite, > > which in turn will run the sub-testsuites. > > > > The user is now required to create vdevs via EAL commandline args, > > this is no longer done within the test app for crypto autotests. > > > > Documentation will need to be added at a later stage, > > adding to the test document that isn't yet merged. [1] > > > > [1] https://urldefense.proofpoint.com/v2/url?u=https- > > 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536- > > 2D1-2Daconole- > > 40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PR > > wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=XItKx7UnOATOiVC3kRLlV3Woh > > rLeZbmlqShu_py452U&s=BpOMigsNnvWMV739- > > fMxEvVERJieNSnfnnmZdiMLMJg&e= > > > > > > v4: > > - Fixed some return value bugs related to patch 4. > > - Rebased onto main. > > v3: > > - Added support for a testsuite having both a list of testcases, > > and a list of sub-testsuites. > > - Replaced PMD based parent testsuites with a cryptodev testsuite > > used by all autotests, with the exception of scheduler autotest. > > - Setup functions were added for all sub-testsuites, within which > > required capability support is checked. > > - The setup functions no longer create vdevs if needed, > > this must be done by the user when running the test. > > - Patch added to standardise return values for skipped testcases. > > v2: > > - Added macro in place of testcase/testsuite loops. > > - Added more detail in the summary output. > > - Moved testcase counts to the testsuite structure. > > - Flattened testsuite structure to remove union. > > - Added patch for fix of blockcipher test return value. > > - Squashed release note into last patch. > > > > Ciara Power (7): > > app/test: refactor of unit test suite runner > > test: introduce parent testsuite format > > test/crypto: refactor to use sub-testsuites > > test/crypto: replace unsupported with skipped > > test/crypto: move testsuite params to header file > > test/crypto: fix return value on test skipped > > test/crypto: dynamically build blockcipher suite > > > > app/test/test.c | 176 +- > > app/test/test.h | 23 +- > > app/test/test_cryptodev.c | 2333 ++++++++++++++---------- > > app/test/test_cryptodev.h | 32 + > > app/test/test_cryptodev_asym.c | 111 +- > > app/test/test_cryptodev_blockcipher.c | 423 ++++- > > app/test/test_cryptodev_blockcipher.h | 12 +- > > app/test/test_ipsec.c | 32 +- > > doc/guides/rel_notes/release_21_05.rst | 5 + > > 9 files changed, 2004 insertions(+), 1143 deletions(-) > > > > -- > > 2.25.1 > >