From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 36565A0A02; Tue, 6 Apr 2021 03:34:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B37E6406A2; Tue, 6 Apr 2021 03:34:20 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130052.outbound.protection.outlook.com [40.107.13.52]) by mails.dpdk.org (Postfix) with ESMTP id EB1D14068B for ; Tue, 6 Apr 2021 03:34:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=34kMM2PazOhAtU3C/lM5DQn75FLBiJOBBfC7/YOrZYo=; b=ty95PpsMX2hjnxBv4jKVWWJcOGn5/rj7+6RDehzwFPddZY25q+SqhumNQxIJAmL6OlZ6X0nCCaTU+9018ZvpNA7+dXV7ts5i1AOwSwEXiZyQeLOTYmcrmeqcxdvw1ZzzM0/jZxuAW+/UQk8Z2TGbwNDUZQ8Ra+lAqnxrfG2D3F0= Received: from DB9PR02CA0029.eurprd02.prod.outlook.com (2603:10a6:10:1d9::34) by AM9PR08MB6210.eurprd08.prod.outlook.com (2603:10a6:20b:282::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.27; Tue, 6 Apr 2021 01:34:18 +0000 Received: from DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:1d9:cafe::b6) by DB9PR02CA0029.outlook.office365.com (2603:10a6:10:1d9::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.27 via Frontend Transport; Tue, 6 Apr 2021 01:34:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT011.mail.protection.outlook.com (10.152.20.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.29 via Frontend Transport; Tue, 6 Apr 2021 01:34:17 +0000 Received: ("Tessian outbound 82c2d58b350b:v90"); Tue, 06 Apr 2021 01:34:17 +0000 X-CR-MTA-TID: 64aa7808 Received: from cc927ff409c6.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0C71728A-4350-4D31-BD28-C0E647F9E67D.1; Tue, 06 Apr 2021 01:34:12 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id cc927ff409c6.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 06 Apr 2021 01:34:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=alVSkwYPqsM7M/2LhdNIY6A7GXVZtNHj5gc0+NeXls15K919I/6r+kDAnsrkq9CLNXYSYkLe9tclUcs8VxW2jbRakjdztsQkwE/9GO6C/NBm4wj3tP2YWpAd7XdgcmXUf3YjlmeNR1OwmpSDjhzMtixGrkTNeVu9dkAui/Pa7fmEIp32kxe3NgM+HjhLlpwGY3tRfJ786h1QKCf2B8jOvHkt07ZTlI5ojPM3n62rVJfVbdFqurg8yN0EdnumZPD8m2UiPz9xAeQWfM9mKT2B/gnyX8SakfbTIYoYU3F2qc1bMcMiJSz1YuEOVv4mVJ2xLqZrthkOXnUlsoLr1L9j+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=34kMM2PazOhAtU3C/lM5DQn75FLBiJOBBfC7/YOrZYo=; b=BfLfjy0Q6V2UlmbxUvHlkDsW3fhCfNKJX2T329nkC8hYjQwS96yPjoDB0jIIiEgd2BGqkg+h+4RLsoV6hmYSc09J6+FNFHw2hcR8w10DntRYdOxCD38LzJgVnaJvJSGNjrvu2wrtcO544IqUcSNia2a1O15R/+Tphk/YvCTqTxlte9JSQJhZg62USQBGXDK+sas2/AwclkdOIlVVOxrYmcaNtbPbjX1Tlm1vZXscv+JdTCmiXq8FhFYzvmsXiRmI3Pd1eBDzaPFGbaleCOWLVBQzsks9iYMH6GSh7Ald8JrnX+9qHJw4rqcfG/G7nGeDICjTHLfTO/gBI2xqq55G1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=34kMM2PazOhAtU3C/lM5DQn75FLBiJOBBfC7/YOrZYo=; b=ty95PpsMX2hjnxBv4jKVWWJcOGn5/rj7+6RDehzwFPddZY25q+SqhumNQxIJAmL6OlZ6X0nCCaTU+9018ZvpNA7+dXV7ts5i1AOwSwEXiZyQeLOTYmcrmeqcxdvw1ZzzM0/jZxuAW+/UQk8Z2TGbwNDUZQ8Ra+lAqnxrfG2D3F0= Received: from AM5PR0802MB2465.eurprd08.prod.outlook.com (2603:10a6:203:9f::22) by AM6PR08MB5286.eurprd08.prod.outlook.com (2603:10a6:20b:aa::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.28; Tue, 6 Apr 2021 01:34:08 +0000 Received: from AM5PR0802MB2465.eurprd08.prod.outlook.com ([fe80::5da6:1eb2:8524:17e0]) by AM5PR0802MB2465.eurprd08.prod.outlook.com ([fe80::5da6:1eb2:8524:17e0%11]) with mapi id 15.20.3999.032; Tue, 6 Apr 2021 01:34:08 +0000 From: Ruifeng Wang To: Ciara Power , "dev@dpdk.org" CC: "declan.doherty@intel.com" , "gakhil@marvell.com" , "aconole@redhat.com" , "hemant.agrawal@nxp.com" , "anoobj@marvell.com" , "asomalap@amd.com" , "Ajit Khaparde (ajit.khaparde@broadcom.com)" , "g.singh@nxp.com" , nd Thread-Topic: [PATCH v2 0/6] test: refactor crypto unit test framework Thread-Index: AQHXJ8wBcF64b8MVEEW/F6BihhWgVKqmugsw Date: Tue, 6 Apr 2021 01:34:08 +0000 Message-ID: References: <20210402142424.1353789-1-ciara.power@intel.com> In-Reply-To: <20210402142424.1353789-1-ciara.power@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: C7E0377A9F0A4441806A61FB5F1A2185.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [203.126.0.112] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 51066d1d-d9fa-42e6-185d-08d8f89c183f x-ms-traffictypediagnostic: AM6PR08MB5286:|AM9PR08MB6210: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: uNK9md5ff0wX9qkKsQa98V2cbh0cEXUS/2z2VY+22qg/Tk3zXG9RywPnGBiRh5fDjxOFHNTB+nXwmzZLgHMW4zP8F8IPLHhEuSVDzrTK8U80P6IaCG/o1YXxc47cudGo3l4dBV3/40PbQyFHYYYdeMuX1i5df7phgXKSo5fqOWQEFVf5Sl8OwePITf9QaVLTX09pXqbsscv2qanNeTBO9We31sh+2bXghmrqc2DdFFif/gE1JKeZG3Y6594cJ8yTSpBiD8cGKwAX7d46gCqrq8fKhihNlGCjKCsNDZQ9JZjOxu1t1SoTAVo7f3+kBNJKoMtWehn2ZqH5oplBXBBNoR2HB/Z5Qt6ubHOgG3d4pgGLtOEqTnKmVsnfILYGghPPodL5rXULZaOD/nQZ4fqafONMr2FUonVx5zd4K40M50MpuNYu4/5yDl5aJbCmEGmwmi+QGqsAPxIxsW82IEHVQr8W7YNPOF2dm3cVwUqhxh7n4ZXWGVIs5wGfWmGWHeqENxwl8ocfsfM2hDS7ofiT5mBNkdn42/za1n3rOwJ6dbRAjfnt5QBEG62vGolVBNmbPfjvPMR0jiZpvdBwBgjhUwR6LwdVHSQCtsIdetCGEcXz4p8OReGPf+sXH4uf2Q97+Fn3D2DMhW7Vu/1i4R/5ms2mdekrcIlGO+bS00zJoWagnfDZGVnBC4trFjBjBW/mTzweyPv8wfC2aggxvk84ahvfn6ARTTw96VzLyFERfLq1Ws1Ryp0EOTb4GolSsQqA X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM5PR0802MB2465.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(396003)(136003)(366004)(39850400004)(83380400001)(33656002)(5660300002)(2906002)(52536014)(76116006)(54906003)(71200400001)(66556008)(66446008)(66476007)(316002)(64756008)(7696005)(66946007)(7416002)(53546011)(8936002)(9686003)(38100700001)(6506007)(8676002)(4326008)(186003)(110136005)(86362001)(55016002)(966005)(26005)(478600001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?MEpRJHLZRhtcTF+DpQAUc74Oe5R21BS272D88cvEGqhZi+xk7TMyBxB7nr87?= =?us-ascii?Q?TrcasXQWcW3fyAFvDvZ+vSj4PLTb7XAjaZv+dqGP362gKS3lBoSZ7LfNvqQa?= =?us-ascii?Q?rYWGohYmWr4qqP3kr9E+OvfFNd7I0PdqOmS9GwoMPIVJ8d8xZw0eK3TqhzQ8?= =?us-ascii?Q?RVkbrSrMjJRhtsV6AhJNX9oSs2lE25OVm4OX2z3OxtQdcbdBRwgP4Cv5y5Uy?= =?us-ascii?Q?ZJH4bq7tK7CK3FFvrX/sNlsZfi5JYFEsDy9gUeCz6sZjE0QmiJyIDRkQCP3W?= =?us-ascii?Q?umIzY3hRNitHSsaYWSJIBg6zGXBZE/Pi4+C0KclKcW35gCJK1BZcYJv5D1U/?= =?us-ascii?Q?ZUnGR8WYu/zUSclfP7aof0JfVfEEMOdtMc5ZWWmwMw/SdUUyl61KmhF8jQt+?= =?us-ascii?Q?Av4lCLFetpmsdDPksSm2858gavzVwKF8GkzSH6IQH+g96IYTzro8m4T8h0MB?= =?us-ascii?Q?Wp7ediSD/LOoj5aNgUg/rkyy1dRTLTzHdEgX1JQmO+2g5k6JDjWgn5T7/s/1?= =?us-ascii?Q?UZFcPKtPMxIF/7jnPaYdtO4sDqj+Foda7f3VVslhLVI7jDvuk2vzjMe7JMD9?= =?us-ascii?Q?Or1yGuCMiwLcC0kjyWGK6z3UbjiSzaKLPPlQ+BRK4t0NxwAWSiJ+yj4XLUkv?= =?us-ascii?Q?FAJf5zP0a/QFDmWflWhox8UzG5ECov5iawWOe21/YMp72gzRm4iL4Im853Ah?= =?us-ascii?Q?RTF7zRg/17jHwLXTBJOIi8tM+BtPK1eIlpbHw4B1EvH8zlaDPE8s92CIGGvr?= =?us-ascii?Q?VUNuObb+cuzMoLXGhn6T22ovnk7HURZSF2iIEFuhyYKnk/aC7rJmiwFKGH5Y?= =?us-ascii?Q?hcgqqDaAzlwWXRHxOu/1q1k1215mTwRcFOYfpQZIfAFLNgA2ZwMrBGRzKz1e?= =?us-ascii?Q?jVF/5CyBXYB4mO1jz9n1hM6hPn+1FTKLX4RUjviN95mLvatR/k6Z5m7du2MQ?= =?us-ascii?Q?JN/7u4UG0Jd93JcQh5A1O0auOG82CXawhtxiX1hPwlGK8iSveP3UefGnf/09?= =?us-ascii?Q?CrU4Ethq90HbA58OcHMpARBFDJZUd0Do76XG2oHoiWzYpWJG9v+gexK/VSTB?= =?us-ascii?Q?U3gQYYj4VS5TMwgNlp5rzEuCuC6g0jL2tvp0UK92ivN4DEkuiiwc4Z/RZJDB?= =?us-ascii?Q?zIPTMcqB+fbx4uVsKiMZcjnuegYeeyjBUhyiTiUq0RSi/EsVBah1zb8Mz/r3?= =?us-ascii?Q?3zcKQaSwEin0VSPtu32CazqXOFPDI7d0CLCI7QbneP1M+qzG7whyvnFdYkjr?= =?us-ascii?Q?gu1bbajteXbKPkFRkZXs9q0YEBORNPsXTKEmpLpgTPX0Tq5VLoONUAxn19pM?= =?us-ascii?Q?1QqIzsBVpoi9EQxcsQDkLCd6?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5286 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 4244ab11-1e2a-4a23-5a8f-08d8f89c12f1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NkHe3HRjmOW0z/K6HPo/BTa91nSvaWB0wcTNv7waNRcdm/sBzWxe4syaPHNcPiEGSLJI7+Ve2i95WNtjMW48g2yAkPKjL3g+q/mjTFOkZlhysZPhdjSJazNXJFTaZtla+s9hpn6qfbL5y+4+fgDRgEM7HSLy65Rom2JaMcY1PXimb5+EngPx5UJvf2ZNJhMMrQDJXltNvWwIn8ZCrKo1CaMXgxGgaUC5WBltAhqTIxPjfM2LS1Z+dpdwOzV++VdLGJ+YgOWps/wHBxvoCu19tZ8uh/ZTX7H7RRfAsX8l2R1H/QFgHiiZZBPSaBgUAAEKEAx9sNSwRjb+qp9Sp5bYmRgUyvRAqnZSsUK4o6MyPkm4haspFwRPsV83oandiEkDNH5DvVftBml15ugqxXsOqgd/hkjEOBPcPZq7bhTEBZVMerVuySVNvOgUigw4YqXIta77RxuSJMJI7o3CQ/ppCXIgqtEHAWoA+WY1L+RNB0kris5rMqrRvK6Ox5meI7WtjX/iKmO4feMUkqht0kCh+UFU7SYj9ZxgprOj5bGRlrRCRj5I0qCAcr3guZwCwrr7hkGVQ+8RL8B0caIQW0FLHagv939lyUT7GeJ3QoDMXIEQ/1RZ0uvjUIUHAGs3WgriGnrfzXniOmV3pBi44Sj2kIp4btYKlJy9aNRrpPNSxhxFrYHiy0ETDyDkCbcqFAiGQZF1pOZUUoFMbSw4o5z9ApwF4/g0dOIn5o4dQGwK96Q= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39850400004)(136003)(36840700001)(46966006)(36860700001)(9686003)(70206006)(110136005)(86362001)(478600001)(316002)(966005)(82310400003)(83380400001)(54906003)(6506007)(26005)(4326008)(8676002)(53546011)(336012)(186003)(356005)(52536014)(2906002)(47076005)(8936002)(33656002)(82740400003)(5660300002)(55016002)(70586007)(81166007)(7696005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2021 01:34:17.8180 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51066d1d-d9fa-42e6-185d-08d8f89c183f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6210 Subject: Re: [dpdk-dev] [PATCH v2 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Ciara Power > Sent: Friday, April 2, 2021 10:24 PM > To: dev@dpdk.org > Cc: declan.doherty@intel.com; gakhil@marvell.com; aconole@redhat.com; > hemant.agrawal@nxp.com; anoobj@marvell.com; Ruifeng Wang > ; asomalap@amd.com; Ajit Khaparde > (ajit.khaparde@broadcom.com) ; > g.singh@nxp.com; Ciara Power > Subject: [PATCH v2 0/6] test: refactor crypto unit test framework >=20 > v2: > - Added macro in place of testcase/testsuite loops. > - Added more detail in the summary output. > - Moved testcase counts to the testsuite structure. > - Flattened testsuite structure to remove union. > - Added patch for fix of blockcipher test return value. > - Squashed release note into last patch. >=20 > The current crypto unit test framework is not granular enough to accurate= ly > track unit test results. This is caused by one testcase in a suite actual= ly > running multiple testcases, but only returning one result. >=20 > The approach taken in this patchset allows a test suite have either a lis= t of > sub-testsuites, or a list of testcases as previously used. > The unit test suite runner can then recursively iterate and run the sub- > testsuites, until it reaches a suite with testcases, and it then runs eac= h > testcase as it had done previously. >=20 > By allowing this further breakdown into sub-testsuites, a refactor of the > crypto unit tests solves the issue of inaccurate reporting, as sub-testsu= ites > can be used in place of the testcases that had multiple testcases hidden = on a > sub level. > The blockcipher tests previously had these hidden testcases, but are now > sub-testsuites that are dynamically created and added to a parent test su= ite, > allowing for each testcase status to be reported directly to the runner. > The cryptodev test suite is broken down into smaller suites that are used= as > sub-testsuites, which allows for more flexibility choosing which sub- > testsuites should run for the current device autotest. > The introduction of sub-testsuites also allows for more precise > setup/teardown functions, rather than general ones loaded with conditions > as was seen with the initial setup function used for all crypto testsuite= s. >=20 > For example, when running the cryptodev_aesni_mb_autotest, the AESNI > MB parent test suite has its own setup function to initialise the AESNI M= B > device. > Various sub-testsuites are added to the parent test suite, such as some o= f > the static suites that were once in the cryptodev_testsuite, and blockcip= her > suites. > The unit test runner can then run the AESNI MB parent test suite, which i= n > turn will run the sub-testsuites. >=20 > Documentation will be added in a later version of the patchset, adding to= the > test document that isn't yet merged. [1] >=20 > --- > [1] > https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1- > aconole@redhat.com/ >=20 > Ciara Power (6): > app/test: refactor of unit test suite runner > test: introduce parent testsuite format > test/crypto: refactor to use sub-testsuites > test/crypto: move testsuite params to header file > test/crypto: fix return value on test skipped > test/crypto: dynamically build blockcipher suite >=20 > app/test/test.c | 226 ++- > app/test/test.h | 23 +- > app/test/test_cryptodev.c | 1964 ++++++++++++++++++------ > app/test/test_cryptodev.h | 20 + > app/test/test_cryptodev_asym.c | 93 +- > app/test/test_cryptodev_blockcipher.c | 127 +- > app/test/test_cryptodev_blockcipher.h | 12 +- > app/test/test_ipsec.c | 32 +- > doc/guides/rel_notes/release_21_05.rst | 5 + > 9 files changed, 1831 insertions(+), 671 deletions(-) >=20 > -- > 2.25.1 Tested against armv8crypto PMD and the result looks good. Tested-by: Ruifeng Wang