From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A83E1A0C43; Wed, 12 May 2021 15:15:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2FC1E4003F; Wed, 12 May 2021 15:15:59 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id B97604003E for ; Wed, 12 May 2021 15:15:57 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 14CDEY1O017420; Wed, 12 May 2021 06:15:49 -0700 Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2109.outbound.protection.outlook.com [104.47.58.109]) by mx0a-0016f401.pphosted.com with ESMTP id 38fw8ybvsp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 May 2021 06:15:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CvyYU5reCWBVfOBV0QOfUsWetH3nPik7V/qiDNAoHMEImWdCKqc4EOU2UtIvBeqF0PBh9oFXb75N3Q/dMINA1sCCsqgTQmXsgqIiYMsXrvGWPx/HYitX2b6GtpklwBPQWfuXXAvOZ7Mj1gHJ+DwEEq3guFNbVBCGohHjknXmHmvsQCqWDUrwT8d90jY0uuaIRenynCRPkGxnIzTEcAAnc7f9V8C7YwdpIeQSV0BNWtKDmEEcgV3mNlccMN0JC5mjz7aaOvJp0guMImfxCT07ZUIHyCpbTB/WymHf8et1J5wAFJwS7kx3qIQV8HMkHk/sjOhXqYapBKzZNkrEPcjysQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lyIxtrZ7OtnSKDryjR9OkK4U7mBFm6jXQPKlUWFkaQM=; b=YyMMIvHb2dnYMapxyytjyidRXFyk+oCI+dw/Bl56bmJMcUGI2nyg6WxVU2fiH1frXAsZlj0UaOFSMvMipeDKDrNMOFKSCEmuRvfOsiWraOevWlMaHMYnlKrrxT5/+CdaU+7ZlaDP2Hyrq7zgGLjWS697EbUOa+3NgNArznB3J34poLn+k1xAalnD2jURatOyGZjegUdiaYH1vuqslYxs8492Q55JqlkKZLF9nDEzz92+7d8ssxdJHQRoNuCsQkldT+8o1wmGBbYQtLmqB06hx+VJ9VGIYAkqk+qM/i0rOt+sBdWT7I5D0bghofW/aCJiZv1qZuXT4tSN0DeiaG2s3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lyIxtrZ7OtnSKDryjR9OkK4U7mBFm6jXQPKlUWFkaQM=; b=j4XiWJihRIBH6PXnqu84/ilzY/C5ZbVoIwsWbEk0tloNY9NvhodFC5bCvGiVui2ow/ukSeDte4Y2t7XVPBPFr2bxMCZ7Oxhsju0dPKBnCInbXy/3xrVPkYgzB8uztZaOOiGlYDrP9wSCKM7tzaj9yD5B3eex40KGLpDIWFx1Lp8= Received: from MW2PR18MB2284.namprd18.prod.outlook.com (2603:10b6:907:10::16) by CO6PR18MB4019.namprd18.prod.outlook.com (2603:10b6:5:34c::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 13:15:47 +0000 Received: from MW2PR18MB2284.namprd18.prod.outlook.com ([fe80::3168:cb00:6607:743f]) by MW2PR18MB2284.namprd18.prod.outlook.com ([fe80::3168:cb00:6607:743f%7]) with mapi id 15.20.4129.025; Wed, 12 May 2021 13:15:47 +0000 From: Akhil Goyal To: Ciara Power , "dev@dpdk.org" CC: "declan.doherty@intel.com" , "aconole@redhat.com" , "hemant.agrawal@nxp.com" , Anoob Joseph , "ruifeng.wang@arm.com" , "asomalap@amd.com" , "ajit.khaparde@broadcom.com" , "g.singh@nxp.com" , "roy.fan.zhang@intel.com" Thread-Topic: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework Thread-Index: AQHXRyMpBHYJ3lYgOEWFM9JYGyDjnarf004g Date: Wed, 12 May 2021 13:15:46 +0000 Message-ID: References: <20210512113655.568814-1-ciara.power@intel.com> In-Reply-To: <20210512113655.568814-1-ciara.power@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [171.48.51.1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5e162e77-1a37-4e73-4452-08d915480e5e x-ms-traffictypediagnostic: CO6PR18MB4019: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7jsIS4AWtYjxf4eYPUZ1ngn1/b5SkOxFHSI02yx716dnxdEPAdBoBJpxubo7F6k4vT5ekbWjSHwymOxC3+HnO4LJavmXIResB5jgSWxOiQHuKnTBzl+PD89Enrxj2sRHXViAbNt5lJ41lrJVvq+QxMFbFs8qWNEtnEEOcfTgZSEpw3LVPxTlkEfhCgJ1uDr1P63CJrWsRlLIpmJL5naQytDBy6Jq7fj/abEn5wbO5zKNiNBCO4TKORTMcEzPqGvVHEcU7+77E4YSMpgFog77iEbKpWPqTys1UnFiEB02hV48lpi+2jEWenW8zo633B9DHImlRgF8Z6ZXT0Fnq2QpLZo5G1HgLNyY95rMCD6ShzyJLlbhb0mfSDFG7JSCt/5KBiI7xHec1oJu1PPAGykpZtCMl+gt3b74ENVEQpiWKCvsQVs+scZ4TLxoW+R7NBf9wSStQs5/6/ChNd6Bx+M3wCxOIPzW8FGk9eBtgt3/EbP+QnHJyRncr4jB7s94e9kI3fMMF4I0s0wL6jku3LRxxAd2XTkH52qHGbYKMoygSIenuqjBbud+sS8SE6zhM31cO1qAnScRvviqPl8cKBY8XsM2ZOn7hI11psjtgpIX3RWHiQ+ZtJ6+bQ56VVhTliT/dr4frWlsDwCJo9DFmyBBXguldmmDVgMqouqshe80NlUkB3+9RAP5e1k9Yd6EpNwR x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR18MB2284.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(346002)(376002)(396003)(366004)(478600001)(66476007)(8676002)(6506007)(19627235002)(55016002)(83380400001)(66446008)(9686003)(71200400001)(86362001)(54906003)(52536014)(316002)(66556008)(64756008)(4326008)(38100700002)(2906002)(7416002)(5660300002)(66946007)(53546011)(33656002)(76116006)(966005)(26005)(186003)(7696005)(110136005)(122000001)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?3NJXnaB3ouxQ6XcwSoufPUkyFXoekmxxyiA/4uqohRT2XgGXsr8fEFOlfWVZ?= =?us-ascii?Q?6y+PctxP66ueECTP7vN5o1zr1WnKU+X0UT6huxz7+mGVhqRkOQ+pd0nhoGqE?= =?us-ascii?Q?ukSeoSFO9J25js5hZFjZNbMEJQqfs14fcx7VO0oJq30V4fHr7WqFw+nOt6hy?= =?us-ascii?Q?eyQOnl20CW2PdXLoDDYXuZK8+6sIRB81h9xyZE+nCysJmyE+9WRQ0TTkhch9?= =?us-ascii?Q?irpqIjViDpLbJ2nDaAqXhyCqeEOB6z+KuPUe2IQbFa0PlyluML6FlV9RZTlA?= =?us-ascii?Q?gNYfN0Ql7eEveD0E9hgkg/ZDRwsKbuvb+EwfeuTmIlMez7wSnuc4sK6N8kmf?= =?us-ascii?Q?c+6l38qTocbJJgAlh3RjvQkwTqCigQqPLyVsRg49gVa9xTfGxY9EQnCuEBTm?= =?us-ascii?Q?AHzr4sFgfWmdcbGjqnhwTkynLUQl7JFYp5VgydHtWcfBq1mQTjG8/HQZ2orU?= =?us-ascii?Q?WPt3PpiOF5rKlS+S7yqF3aC2yXJ0UdUJr7lZAGZknQtGrGatCsE/JIuj3xoy?= =?us-ascii?Q?gNinIi4zT6XYBf6NNlKxKkv/MNcsinZtBlt1cbMFeZUntxTDEW7WGaSOFPEE?= =?us-ascii?Q?8lZZv41dSRSeGVtENQiKqTrv1zViQZNIiYjT07ag+IWTUFleURfCp9JF/joL?= =?us-ascii?Q?9u6MjYuNYGuQDgrBFyULMQA6BMe7xCoKg43kh1GJskrI4/kQwn9lfY+JaXRV?= =?us-ascii?Q?tDyYAZNrIbmV1EJOEknFZaYS8/CSdVhyuQqcVrM25XBqD5sajQqBuOrY0MEK?= =?us-ascii?Q?1D4kFUtvbg9gao2fqrn02j+ZzzkOO6OzK7R9gQVY0hnVk/OBWm/8GnAPGt9X?= =?us-ascii?Q?bnHON+wh+3leNyO0/msN4eCIa2EHyLwJchmF8YYm0e1pX45NJYiaJzhecoCC?= =?us-ascii?Q?5pCdydvu370QkeqYIY3AcqkesCNnwOw3TpzumWtcLpABrf5H+Ohx46P4WkXa?= =?us-ascii?Q?xUQR0+jnGhVKZ51fVpHbAFSznYwWOzPvRfjSayl0xwoPpv1wS3k+9hZYkh8u?= =?us-ascii?Q?z3ja5OM9efVRuc8aFiGyhQzW4L3XDOog3VeCWYCsctjT31/PEzd92FNwvBiq?= =?us-ascii?Q?4wzQxI8Qodo5FR7MawlgEtr5U0xRK1Ii8YJjby3PEyJZHVNK7sU7Vgeq00gx?= =?us-ascii?Q?Ix5bfxlbfttxcEm8Ej748tq+rMr93IZvtb4yU4rmfbb7pj7A7+xAWCjRgvci?= =?us-ascii?Q?9S5S/ce20c1FqjlegEMFpJZ4Ie/uSNB0xE9nVLsf5zUrwawhceIcMCgUmJ4K?= =?us-ascii?Q?l/g9hpSbh6bLPoiERp1yCOMOygREBxV6w/nPuVW+4uNhO28ZCiXbUKnNJ8Wg?= =?us-ascii?Q?WrE=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2284.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5e162e77-1a37-4e73-4452-08d915480e5e X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2021 13:15:46.8955 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tDIfJjvhHvn/pHBSsoYsZqcRw3d7c67HZuQ01TqB0nK9ionn4dV7HtOLIi904BjwKyEI5MU+vyuO6YHFNQTewA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR18MB4019 X-Proofpoint-GUID: S8uk-Tivi03impg9PyWsQHDRvhZ4LNmm X-Proofpoint-ORIG-GUID: S8uk-Tivi03impg9PyWsQHDRvhZ4LNmm X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-05-12_06:2021-05-12, 2021-05-12 signatures=0 Subject: Re: [dpdk-dev] [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ciara, I see that CI is reporting one error in autotest. Can you please check if i= t is because of your patchset? 9/96 DPDK:fast-tests / cycles_autotest FAIL 1.42s= (exit status 255 or signal 127 SIGinvalid) 12:33:15 DPDK_TEST=3Dcycles_autotest MALLOC_PERTURB_=3D44 /home/jenkins-loc= al/jenkins-agent/workspace/Ubuntu18.04-Unit-Test-DPDK/dpdk/build/app/test/d= pdk-test '-l 0-3' --file-prefix=3Dcycles_autotest ----------------------------------- output --------------------------------= --- stdout: RTE>>cycles_autotest delay_us is not accurate: too long Test Failed Regards, Akhil > -----Original Message----- > From: Ciara Power > Sent: Wednesday, May 12, 2021 5:07 PM > To: dev@dpdk.org > Cc: declan.doherty@intel.com; Akhil Goyal ; > aconole@redhat.com; hemant.agrawal@nxp.com; Anoob Joseph > ; ruifeng.wang@arm.com; asomalap@amd.com; > ajit.khaparde@broadcom.com; g.singh@nxp.com; roy.fan.zhang@intel.com; > Ciara Power > Subject: [EXT] [PATCH v4 0/7] test: refactor crypto unit test framework >=20 > External Email >=20 > ---------------------------------------------------------------------- > The current crypto unit test framework is not granular enough to > accurately track unit test results. This is caused by one testcase > in a suite actually running multiple testcases, but only returning > one result. >=20 > The approach taken in this patchset allows a test suite have a > list of sub-testsuites, and/or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the > sub-testsuites, until it reaches a suite with testcases, > and it then runs each testcase as it had done previously. > In the case of a testsuite with both testcases and sub-testsuites, > the testcases are executed first before iterating through the > sub-testsuites. >=20 > By allowing this further breakdown into sub-testsuites, > a refactor of the crypto unit tests solves the issue of inaccurate > reporting, as sub-testsuites can be used in place of the testcases > that had multiple testcases hidden on a sub level. > The blockcipher tests previously had these hidden testcases, > but are now sub-testsuites that are dynamically created and added to a > parent test suite, allowing for each testcase status to be reported > directly to the runner. > The cryptodev test suite is broken down into smaller suites that are > used as sub-testsuites, which allows for more flexibility choosing which > sub-testsuites should run for the current device. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, that can check the capabilities required to > run its testcases. >=20 > For example, when running the cryptodev_aesni_mb_autotest, > the parent Cryptodev Test Suite is executed. > Various sub-testsuites are added to the parent test suite, such as > the static suites of testcases that were once in the cryptodev_testsuite, > and blockcipher suites. > The unit test runner can then run the Cryptodev parent test suite, > which in turn will run the sub-testsuites. >=20 > The user is now required to create vdevs via EAL commandline args, > this is no longer done within the test app for crypto autotests. >=20 > Documentation will need to be added at a later stage, > adding to the test document that isn't yet merged. [1] >=20 > [1] https://urldefense.proofpoint.com/v2/url?u=3Dhttps- > 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536- > 2D1-2Daconole- > 40redhat.com_&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3DDnL7Si2wl_PR > wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=3DXItKx7UnOATOiVC3kRLlV3Woh > rLeZbmlqShu_py452U&s=3DBpOMigsNnvWMV739- > fMxEvVERJieNSnfnnmZdiMLMJg&e=3D >=20 >=20 > v4: > - Fixed some return value bugs related to patch 4. > - Rebased onto main. > v3: > - Added support for a testsuite having both a list of testcases, > and a list of sub-testsuites. > - Replaced PMD based parent testsuites with a cryptodev testsuite > used by all autotests, with the exception of scheduler autotest. > - Setup functions were added for all sub-testsuites, within which > required capability support is checked. > - The setup functions no longer create vdevs if needed, > this must be done by the user when running the test. > - Patch added to standardise return values for skipped testcases. > v2: > - Added macro in place of testcase/testsuite loops. > - Added more detail in the summary output. > - Moved testcase counts to the testsuite structure. > - Flattened testsuite structure to remove union. > - Added patch for fix of blockcipher test return value. > - Squashed release note into last patch. >=20 > Ciara Power (7): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: replace unsupported with skipped > test/crypto: move testsuite params to header file > test/crypto: fix return value on test skipped > test/crypto: dynamically build blockcipher suite >=20 > app/test/test.c | 176 +- > app/test/test.h | 23 +- > app/test/test_cryptodev.c | 2333 ++++++++++++++---------- > app/test/test_cryptodev.h | 32 + > app/test/test_cryptodev_asym.c | 111 +- > app/test/test_cryptodev_blockcipher.c | 423 ++++- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ipsec.c | 32 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 9 files changed, 2004 insertions(+), 1143 deletions(-) >=20 > -- > 2.25.1