From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32AFBA0C42; Sat, 8 May 2021 17:38:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A2F7B4013F; Sat, 8 May 2021 17:38:12 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id E963640041 for ; Sat, 8 May 2021 17:38:10 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 148Fc5kE031862; Sat, 8 May 2021 08:38:05 -0700 Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2109.outbound.protection.outlook.com [104.47.70.109]) by mx0b-0016f401.pphosted.com with ESMTP id 38dt5rgb5t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 08 May 2021 08:38:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AQ8aNxI2AHW8zUXT/iqgyi0qpdCdpcgNfYJZodc1dLNR3GbnsjNgFKvuztvAgY+hxK4Ulb9wLpW8RLneMtbRPvyLZzCX60A6qO1nKLd9bFF5iaKkzLyBUwRmrd4bwD9nXJwq4aKjcAW5ZAJebDytlMZgjZw8tcAUDLGyFwyhUFOw8LTZiHrUUY25cZBg4A7RgdlJzXWN+a1k4yusgLBnN34bm02f96KOBnbT2VKMel7K9VQzFEg/at+F2qaL4saMzRmvZUJqD/xYVKo+aviFl2Kuf4PMM0IxFfGEgkEN1OH1mhkay8iThUHXKtFjm2GtjCk1BH7qHWM/70TVpfaTAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lrTldxoxeJG6Llnnc3bIZqEjfGZ4Nxm53eF2GMJQAog=; b=DEY7UHzxt+doQjO1+FlPqOFg/fkt6GRer/J9TVrCLQ50VWJELNtfX1/Ykh50/mjCq60GDKlsRn6F+aL1ILZ8GeIvaLcUrr8txHNi85NowZDo3q0U/8yG0xrPCNrK8S2NTmsSKgb6dxQMs2oUjL/ef13wQWE6GKwS67UK7yHPcUx7CROroZ334j/trBBWdqxG135VmulvLcghM+wiTIkwaELmPQ/ym3A8rZyMfFS81tO5WYR6Lku2PyF6gO1ds99Zmo68vlikUF8wdIhcNmz9xWA2oBsBickgx1IS+9fQ31vyJ7NHEMrf4h2sUnqf3dAkl1phc385TufaaZyFViGrdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lrTldxoxeJG6Llnnc3bIZqEjfGZ4Nxm53eF2GMJQAog=; b=GJgoa3pCL7T/ueW8NpK4TQQoMdVuykesY1cXqr9kXIadCrpnmPIQlK+iy+1Uh9xlJC57ZGSJd+JdVmZDasjmKdeS23TN4mbhivlYr5yREzFT76Bp/zSPQ0lGgQs99txgyiue4qQHeHwucroRECsHUv7QG00MdQSn6dz9Wzk88PE= Received: from MW2PR18MB2284.namprd18.prod.outlook.com (2603:10b6:907:10::16) by CO6PR18MB3971.namprd18.prod.outlook.com (2603:10b6:5:34a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.26; Sat, 8 May 2021 15:38:03 +0000 Received: from MW2PR18MB2284.namprd18.prod.outlook.com ([fe80::3168:cb00:6607:743f]) by MW2PR18MB2284.namprd18.prod.outlook.com ([fe80::3168:cb00:6607:743f%7]) with mapi id 15.20.4108.030; Sat, 8 May 2021 15:38:03 +0000 From: Akhil Goyal To: Ciara Power , "dev@dpdk.org" CC: "thomas@monjalon.net" , "declan.doherty@intel.com" , "aconole@redhat.com" , "hemant.agrawal@nxp.com" , Anoob Joseph , "ruifeng.wang@arm.com" , "asomalap@amd.com" , "ajit.khaparde@broadcom.com" , "g.singh@nxp.com" Thread-Topic: [EXT] [PATCH v3 0/7] test: refactor crypto unit test framework Thread-Index: AQHXOFxZDzKoEwYRYU+dZplYwV0rJarZzthg Date: Sat, 8 May 2021 15:38:03 +0000 Message-ID: References: <20210423161820.2135053-1-ciara.power@intel.com> In-Reply-To: <20210423161820.2135053-1-ciara.power@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [171.48.51.1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5b845d53-15b5-47ce-ebfb-08d9123744d9 x-ms-traffictypediagnostic: CO6PR18MB3971: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WqjnL/VXWaZHoDrs/aeJLZoCVz//Luo6O+xbxK4anwQ38LKPd4ap1y8fr8THAmrrTitIAESUXyIXPIiVSJm5a3HdDDmNhcfURL/3LlnS9OANuHaiIwF56dMceD3V2OAl3tlWmkQJdLyzh/8IezspU7Sg6y7pJ9N+bIcRqSfwl4m0JRDeWQWv+EkCUJggadAskymD2+pQwfHXWuo1x4izAgwQWAXxDMUfU7pdTyfkvoU3j1fuE/w2KgeyDv8cJdWia9d5ARLlUF1fFj29hDm3ds5XPiVjaOkt224aH/0jVPasgDNzBgXRVNTB26OOYTfrrf6BiX4k3MiOfus+r5O944B6sux72ySv0676nH0WbQGy1UhidbCEzbklFBsQn5cEZVn+8KVr5dV5yy5xsCDuLsO9fw42Y/dyQ+3NjBYa291MDVoeEPKXs60bVedk1kT662u2fLce0Rm/zIfI0o1uBhs0NqvaUkBHET1ttLo1N6Df/GAoUzgiD3wpAqiLrK3lYZnrm7Dx0UE/SJY3UWgedJlKzNRRLXHaHjeBSyUJGtqDBRGAxnifVgM3wKBhnXJQEudYsy5sc5SGUEkZdZqsY6ckjURBVSwiUXde36Qhzlc1FX8U5TbH5w8N0GvszqCpp2LpM2y3i77VsFTyvganbj9WCHyT7jkWsP6txpYjjFMWX329+jUjB6ksaLTYAe4W x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR18MB2284.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(366004)(346002)(66946007)(86362001)(76116006)(8676002)(38100700002)(122000001)(186003)(52536014)(83380400001)(5660300002)(64756008)(66446008)(66556008)(66476007)(54906003)(71200400001)(7696005)(6506007)(110136005)(4326008)(26005)(7416002)(2906002)(55016002)(9686003)(33656002)(508600001)(966005)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?pVP2awPnIiyBzp84o42G52JslbFY3u9r2EBSvA5ejl8ymWBD5IZTXR0Zmj1X?= =?us-ascii?Q?CpGkCueq1mWt58WeacMJglfMQKe7SSLMh7LOv6U0mGhRARYFiOx5QtUYIlzT?= =?us-ascii?Q?tavyhqBIk8t/Zsg/7okEc9fRsU8QYRmuuX6RGMcySxQbzyrW841LpqyL2N+1?= =?us-ascii?Q?U4bgukIA/IajoNOGtOSnSoeAJ1mj8s3Z9xTykI7IIxS5Zh3z32abewiUlxM4?= =?us-ascii?Q?DnVP8f5RloNsSP/GOUNZ688QvQXFOd8yBV0Gn0QuUEn8+rvzun4y7A+gJay7?= =?us-ascii?Q?4uJcmMfvbcwXMC7jyRh7pnBZJbArQxJSxtNxUObn1PwTrTd4LSD14augzq1s?= =?us-ascii?Q?828+60cIQpltaTDIVJIaN6ptdCWwn5X5rutHqTxV1cOtjahTUB37zkoVXayI?= =?us-ascii?Q?d7PEq6+NmQOoG8S6joXiuYvgDobxT4Tr7DMdrFay89Je2jAXjikMuYoOR7fc?= =?us-ascii?Q?A8K5CbdZ/1zww4a6rsKRckFXOHKaw8RKUUnrNlVry+KxL5OO4poMd86WW1hc?= =?us-ascii?Q?LeW1dg7B2tg+aZNVnoRLBbU174jXeXWj2o+j3SidpnT0mRm3E/UVYDlECOXl?= =?us-ascii?Q?UjqmJl6xnuWGQxMDmHOlgHdBWVuN5oQEbKTJlVnJSD5gQCPDAYL5A1kV/OW8?= =?us-ascii?Q?REdF0G64kdBOaDsb8/WNWKeICpE4IjMvx4g4wbtQD/Uittq0J8esQD0ZmSU3?= =?us-ascii?Q?kk6/Qln1BBSnMRncEHUalRmDN2re9gZJ+9bjoD92HAVwouRTr+T4euApfrIn?= =?us-ascii?Q?QbGrem/b1zAXFs3UaAtFoqQNvZ3UdafgWn/MLvRiy9lTHmoAseXy+/DVN4OM?= =?us-ascii?Q?PM6Y38lNPxoBDGCKEoQD4JKfqa5RbL6u6CWv1StNAdmMKWenVRhIp39Bwr5X?= =?us-ascii?Q?3mInzcpqaabiLrMmA4NWh4IR3UczrxTduBSihsYSXCjl/LL0ac23EZc8gxjG?= =?us-ascii?Q?etEk/0XsyyIvf/xzv3+uyynXi90pAk5xkWd8Z5tUWxVGXLrJp3k7KIeOeIvk?= =?us-ascii?Q?cbPFJQztuvaqmswpxJstQjyWdE9DlNS77e+A/Iom3AdNze9cTfkLed6MyVn0?= =?us-ascii?Q?SilViYfi6+rP3vqdpkZQI1WzZw4SWcuAZzL+sHzDUpWgiDXO1BiQrLdGV82k?= =?us-ascii?Q?T8+RaaYjumtaQ2Z441MsSMoGrVGiaOCKFsbpaumNSAVTwSyHJdcV5lZ90mck?= =?us-ascii?Q?DrbCMcQG0pyH6VWWXwGNRaozjiRFYyFBvqXD/HaGTn5jgsf8TooteiaFqB4W?= =?us-ascii?Q?nFe0mIBKOiW3NGJIS8pS2fIdnmHHNdIMG1FO/i1WQCPObLTVC5K4kHam6TQS?= =?us-ascii?Q?Z5M=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2284.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b845d53-15b5-47ce-ebfb-08d9123744d9 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 May 2021 15:38:03.5900 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: PzpVHYsYCEPTNPMPuU6q9Orhq5xCJsSdJtdT4nsdDAXjIsgYInGXbz4QY+FXQv9Zr+ZBuh2Z03ZqZrGdrDn/Gw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR18MB3971 X-Proofpoint-ORIG-GUID: 77_-7P80qxt82IhKcQr4daSOkC4bGiMh X-Proofpoint-GUID: 77_-7P80qxt82IhKcQr4daSOkC4bGiMh X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-05-08_07:2021-05-06, 2021-05-08 signatures=0 Subject: Re: [dpdk-dev] [EXT] [PATCH v3 0/7] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ciara, I see that case " test_snow3g_auth_cipher_test_case_1 " was unsupported on marvell PMDs and was getting skipped, but now after your series it is causing segmentation fault. I guess there is some check missed for digest encrypted case. Can you check? > The current crypto unit test framework is not granular enough to > accurately track unit test results. This is caused by one testcase > in a suite actually running multiple testcases, but only returning > one result. >=20 > The approach taken in this patchset allows a test suite have a > list of sub-testsuites, and/or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the > sub-testsuites, until it reaches a suite with testcases, > and it then runs each testcase as it had done previously. > In the case of a testsuite with both testcases and sub-testsuites, > the testcases are executed first before iterating through the > sub-testsuites. >=20 > By allowing this further breakdown into sub-testsuites, > a refactor of the crypto unit tests solves the issue of inaccurate > reporting, as sub-testsuites can be used in place of the testcases > that had multiple testcases hidden on a sub level. > The blockcipher tests previously had these hidden testcases, > but are now sub-testsuites that are dynamically created and added to a > parent test suite, allowing for each testcase status to be reported > directly to the runner. > The cryptodev test suite is broken down into smaller suites that are > used as sub-testsuites, which allows for more flexibility choosing which > sub-testsuites should run for the current device. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, that can check the capabilities required to > run its testcases. >=20 > For example, when running the cryptodev_aesni_mb_autotest, > the parent Cryptodev Test Suite is executed. > Various sub-testsuites are added to the parent test suite, such as > the static suites of testcases that were once in the cryptodev_testsuite, > and blockcipher suites. > The unit test runner can then run the Cryptodev parent test suite, > which in turn will run the sub-testsuites. >=20 > The user is now required to create vdevs via EAL commandline args, > this is no longer done within the test app for crypto autotests. >=20 > Documentation will need to be added at a later stage, > adding to the test document that isn't yet merged. [1] >=20 > [1] https://urldefense.proofpoint.com/v2/url?u=3Dhttps- > 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536- > 2D1-2Daconole- > 40redhat.com_&d=3DDwIDAg&c=3DnKjWec2b6R0mOyPaz7xtfQ&r=3DDnL7Si2wl_PR > wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=3DKnksoVYw74RhGrlmzJAVAS_O > uxTVx-7GPFYdyP-qn14&s=3DuvfBln2-zGHif5yvsK0_QuZhdo4e- > 3MpVXIevmc5OBE&e=3D >=20 >=20 > v3: > - Added support for a testsuite having both a list of testcases, > and a list of sub-testsuites. > - Replaced PMD based parent testsuites with a cryptodev testsuite > used by all autotests, with the exception of scheduler autotest. > - Setup functions were added for all sub-testsuites, within which > required capability support is checked. > - The setup functions no longer create vdevs if needed, > this must be done by the user when running the test. > - Patch added to standardise return values for skipped testcases. > v2: > - Added macro in place of testcase/testsuite loops. > - Added more detail in the summary output. > - Moved testcase counts to the testsuite structure. > - Flattened testsuite structure to remove union. > - Added patch for fix of blockcipher test return value. > - Squashed release note into last patch. >=20 >=20 > Ciara Power (7): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: replace unsupported with skipped > test/crypto: move testsuite params to header file > test/crypto: fix return value on test skipped > test/crypto: dynamically build blockcipher suite >=20 > app/test/test.c | 176 +- > app/test/test.h | 23 +- > app/test/test_cryptodev.c | 2324 ++++++++++++++---------- > app/test/test_cryptodev.h | 32 + > app/test/test_cryptodev_asym.c | 111 +- > app/test/test_cryptodev_blockcipher.c | 423 ++++- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ipsec.c | 32 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 9 files changed, 2000 insertions(+), 1138 deletions(-) >=20 > -- > 2.25.1