From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DD35CA0524; Wed, 5 May 2021 12:41:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 68FA940040; Wed, 5 May 2021 12:41:28 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2052.outbound.protection.outlook.com [40.107.21.52]) by mails.dpdk.org (Postfix) with ESMTP id 79CBB4003C for ; Wed, 5 May 2021 12:41:26 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YwhXg6P1wEL+E/G/MFQOlBi8MfQbhUheEoZn7Aq+sW7JyINoc7k8LbHb/9zSSWGS1pJ9l96M3X+SBnYolYQFTlyLwBWsuCkItXISn6rpHXk/3KRkF1KUkwga1EDG2BgEktxz9y+cSPVTeEYhKMmeiw6EkXDDeq+UmFAka0ZEZSdGDT5fPEDVhv2Fa5lcniCL1Xth2859zEKVS3lWqkBevP1KURcxi7R7izytu1Uj/r4Fm0Cm6PgVzXtBkxzKo59AAH8oNXu6nVI3+2F45nuWH5ttwoifGU1fj/BDQk+h0QP0syiKT9XWhsTLo1EvssCtmtF/E+bad9BM1zISpbrR6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6qD+YI2M8DRqq/BMGeONeseS2xTevIwyqdMv5DyU7Xc=; b=Sbb+gHm1bAt8YNZSaDzb+jsjvQR9s5I6ftUhhEDhzk16yffMfqVu605lLjfv/cY3UswZbdsmf55j2DytlsXosfMYTeK12k/LDfgNOF+z6+n0yetRiQQnccQzLhMNQJwyhDGliLYHrwYpW2syF/l0DLXCa/BzoxZT6jzMhOtjwmTkObSRy8ci5M2T90tM/zHw4uM0Dsw4jPIEaKiqI0UKMN75saOePOGt+Dat5uQMG2Yw9117+xy1Tx8qulLDO4ZsgcfsR1RI03kvZYggj1nKKVMeatsCjJQ/Jo+4RTWLbe256l/YG8RkvDu4oIZDR70vPzcBspGBkQtF103aq2QtNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oss.nxp.com; dmarc=pass action=none header.from=oss.nxp.com; dkim=pass header.d=oss.nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NXP1.onmicrosoft.com; s=selector2-NXP1-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6qD+YI2M8DRqq/BMGeONeseS2xTevIwyqdMv5DyU7Xc=; b=iXmUGDrTJBmF4dBYOP94pMkfSG6odtC0HPCXRjPhUNv2fYUojG0rKQY9hAsF8vKZArXYypJXmE8A9N46tR83smGTKeEW7aw2nT+Re+cXOAQE+h0gsyepfRv7zgUmZ0O5SQ5gPH2L1OZS3JkL7nHi0U6+ZRwlLO9pQpx92ckPISU= Authentication-Results: nxp.com; dkim=none (message not signed) header.d=none;nxp.com; dmarc=none action=none header.from=oss.nxp.com; Received: from AM6PR04MB4456.eurprd04.prod.outlook.com (2603:10a6:20b:22::25) by AM6PR04MB6645.eurprd04.prod.outlook.com (2603:10a6:20b:fe::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Wed, 5 May 2021 10:41:24 +0000 Received: from AM6PR04MB4456.eurprd04.prod.outlook.com ([fe80::ad9e:a38e:e84e:bf55]) by AM6PR04MB4456.eurprd04.prod.outlook.com ([fe80::ad9e:a38e:e84e:bf55%6]) with mapi id 15.20.4065.039; Wed, 5 May 2021 10:41:24 +0000 To: "Doherty, Declan" , Ciara Power , dev@dpdk.org Cc: thomas@monjalon.net, gakhil@marvell.com, aconole@redhat.com, hemant.agrawal@nxp.com, anoobj@marvell.com, ruifeng.wang@arm.com, asomalap@amd.com, ajit.khaparde@broadcom.com, g.singh@nxp.com References: <20210423161820.2135053-1-ciara.power@intel.com> <365ff71b-92fd-dd9d-9bea-0b579629c810@intel.com> From: Hemant Agrawal Message-ID: <14441025-f9e1-a057-35a2-7866e95a0ee6@oss.nxp.com> Date: Wed, 5 May 2021 16:11:14 +0530 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 In-Reply-To: <365ff71b-92fd-dd9d-9bea-0b579629c810@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [223.178.219.73] X-ClientProxiedBy: MA1PR01CA0166.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:71::36) To AM6PR04MB4456.eurprd04.prod.outlook.com (2603:10a6:20b:22::25) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.1.2] (223.178.219.73) by MA1PR01CA0166.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:71::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25 via Frontend Transport; Wed, 5 May 2021 10:41:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 84c91ce1-d434-4923-03e0-08d90fb25434 X-MS-TrafficTypeDiagnostic: AM6PR04MB6645: X-MS-Exchange-SharedMailbox-RoutingAgent-Processed: True X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PhkdMLadQ5ORtDz6bGNbWKKwY3Tag3GbfiLUUppE6uV9RTYDfucqrw1TGnOLlAri10Tmr7a+wlGtp71e3IMbPIGbq3T054jNW5mPnnXAxLxTaOrEdhpXf2YQB33qPkkrJvtE11a79U/3qwDu543Mez8SgX60TwsQleol66WCDC/WSvfGea4w3odoB6/RBSYkYMMKiUcOVUg5n2OYE6h70KKg4IrlhbVDKPxxW9AUWs8Ys3pZe02kcOmrVrOle6NCoLTZtBU+d4ziVixo3x6ckNRRiA8psprl1Ef39F44L4TPFSlQTPJTrHC2hUBNZ8mzqN773vIAKuMz3U2dv4+klR2xWOSEwcCtvUmGxXcuV/U0mGo37+1CZ23Y5YERHHxiNj72l5d3q/I6+Br7shlrhUzsmsnbPnRrhmZycQ7zT2f7ySTzfrsSRvKLjoPcv6bsWvafUh9IXVGY4/fwOm62uJziKaaITtTxOId8fIzwUVRdRuJ4SDYHTQRomcBhq6ONFCKx4HfdobRh8ZWI/j1WjXkIGRTZ8/Y1W9MbhppHFPskD5CvhyC9u6YsNfMUmMtYPsZCFoOjdEuF2/1WwndAAe8FyoniyHrFnxX56wUN2od+Pxqp8pyo7uq89WeZBjegfADE5lC/2YjMZU+NHua8QKXvHt7tFsRndjkfhOZT12fD8ErOkpjCD7Y8JbjsVQIejHRd4i9FAjLd5luJzDe4kmxEbn2xfcrALg1W7PVSL/WVMJvlNvl0oDhtZ5wUrDzk/cndZw6vOMRPoRiYsZxmz+iMDvREzz4Jb/xo0OKLG+Q= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB4456.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(396003)(376002)(366004)(136003)(346002)(6486002)(7416002)(83380400001)(26005)(8676002)(110136005)(86362001)(6666004)(44832011)(2616005)(956004)(16526019)(5660300002)(186003)(66946007)(66556008)(66476007)(2906002)(8936002)(31696002)(4326008)(38350700002)(52116002)(316002)(31686004)(16576012)(38100700002)(966005)(478600001)(53546011)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?ZnprQmVub2NJTkFPVDhjcGQyNWV5ZERYMnZYRDI5d3NIdGc4RkpTOGdyMG9k?= =?utf-8?B?S0xqYnNvdW16dk0rdlZtVkhKU2FWOUQyYVJWSHo0UjZweEpYU05kR0pvOTlw?= =?utf-8?B?UkxZQ2c3ZzFGSlBReU03T01lQW9MYmxRclBXMTMrNlZzcDZwa2FsM1F1NGtI?= =?utf-8?B?K2RrSzZQbCt4MzJuc2toQ25yTzljZ09SVFNmVU9WSTNmRG9QNXpnb3lHZzht?= =?utf-8?B?dm00cDNmTm5oem1LRTNPVCtrTGRuaGJMeWc3Rmh3R2d5ZFBUeUtibDFLSkxV?= =?utf-8?B?YVRQT01vdmpFa0ZVV2twSUJrMlI1cTZlTnpYWFZ4dnd5MDBTOVRVY21HOUpU?= =?utf-8?B?SlExTzQya2hYUXI5YnZLNUxYb1NTNDA5bzYrTG1JN3p3UGpxSjcvb1VneXVV?= =?utf-8?B?bzRDUWR2MkJ6RDZkVFcxbitMUG1sY2Q4YzdEL0lPOGI5N2hvTTllU25ldjNz?= =?utf-8?B?cFc0NkxGT3B5ZzBORlhDbFlRZGVRQ3FmTjlaWnV0SXowaVp0YmZXcjdLT1VS?= =?utf-8?B?cFlMRVREamxDWFRCVnU5QkJwd2dhSk1ETWhrcHpkeldFQ0szNVpoUGpIdVhC?= =?utf-8?B?dmJlYWo2N1BGTldQU3JySElmRUtIK2xtUnJNbjZQZ2d0VFo2cmpOa3lFZ2cy?= =?utf-8?B?Rm5uVk9lejVMbWF0QzlPNmxuRHIrNGRsL0VYdWVFZWdXQ1JsMGhndllZRTNx?= =?utf-8?B?UmxNeDV6eHpXVG8rMTVrUXlnc0pkOUlqc1hWNTJLK1BoMVVtU3pUYXl6WUp4?= =?utf-8?B?eDcwOGZodXVTRDdIWXU5OE5xUVJmQlhNc0hBYmwvdHZkMG9LaFFDa1gvc1Bn?= =?utf-8?B?eVF5RlVjZ0xhdG1iVkVBd2NpT1FETUs0MmIwQXUrb0N2UFpiZWNOY0FvRWRS?= =?utf-8?B?SkE0NWlhTXAwTExJc0ZoMXQ4M1o4ZENvMThodGlFb3dhMExUa2pIMSswbUNp?= =?utf-8?B?MGRNT2VoL2xJQlYwa2ZVWDFYQk5FODlLTThyT0xpTEVXRWRzT21OdkxtN0NP?= =?utf-8?B?dHgxNncvSFFZdndKNVBMRlMyUE1qaFpWY3dNYjAvYnNDTnZDQitONnpPbW8x?= =?utf-8?B?ejVYTTVNaUN2V1k0bFYwcTJqWFlHc3lTc0FIamQrbHhUKzVoWHVYa1JFelQr?= =?utf-8?B?VEJwY2V2QU10WnIzUGMrK2pFNEhWNXNRMk10Z2hoRzY1TUVYcjVveEtkbHh6?= =?utf-8?B?a0dpRFhlTEhBRnlsTDljYVRJdEU4M1NCZ2FDbzI0VGd6ZlVDUlc4cWdnaUpH?= =?utf-8?B?N0FRMEorNmQ2UkcxUGxxZEU1ait3b2dsSTlhS3YwWDE5WUNrb1h3Rnp2amNX?= =?utf-8?B?WnBsN0E2RXB5SUZrbStJVVNoL1UzNWZpZTg1OXJJdzh4RndjNzBtU0Q0aTFC?= =?utf-8?B?TWc2OXN6bWZNV3hIY0ZNckZvTDFZRlErL3BTUWlYQVZhZ2NMZlRiaUtoa2NJ?= =?utf-8?B?R29EOFRNbW1ZNEg4eWxxMVlSS3JnU1FIVmZBZjRBUFYzcnZYbWFPMHRya3Nj?= =?utf-8?B?UGZMNENGc2RrU1FPME9ESUREVUlkaWN0WFp1Q3Vla2dtekpHK21TZlJSZDgw?= =?utf-8?B?cVd4S0FVUmVlb3RaU0pUVFpQbGtlckNNdlAxN1JRaUYrN0R3ZG4vb21VWVE1?= =?utf-8?B?cU9LTHRmQjVXbXRzekRQcy9pdTNaYlEyQUQ3S2FxNXFXbGhsUHJSM2piWmVC?= =?utf-8?B?Z1VVd1hUZUNPbkt5WEg3N0w4THVuZVJQT0tuOGM4SHlYQU5JYW5CQktjbVFw?= =?utf-8?Q?suZ0bRd8OIyiLMSlQUKKHPusDq+BsiOAp8c4G8w?= X-OriginatorOrg: oss.nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84c91ce1-d434-4923-03e0-08d90fb25434 X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB4456.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2021 10:41:24.5053 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yxiFj8LN6i9P1K6Ca/ppFEc/arDixHZnnvGwZXwBtqnSeFYJ8t9tos90G/IgPuHHWPLy3/H76KWJI2yKWRxIcg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB6645 Subject: Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: hemant.agrawal@nxp.com Errors-To: dev-bounces@dpdk.org Sender: "dev" Sorry for top posting. Akhil,     Will you please hold for a day on this patch? we are seeing few failures on dpaa2 with this patch, currently debugging them. Regards, Hemant On 5/1/2021 2:30 AM, Doherty, Declan wrote: > > > On 23/04/2021 5:18 PM, Ciara Power wrote: >> The current crypto unit test framework is not granular enough to >> accurately track unit test results. This is caused by one testcase >> in a suite actually running multiple testcases, but only returning >> one result. >>   The approach taken in this patchset allows a test suite have a >> list of sub-testsuites, and/or a list of testcases as previously used. >> The unit test suite runner can then recursively iterate and run the >> sub-testsuites, until it reaches a suite with testcases, >> and it then runs each testcase as it had done previously. >> In the case of a testsuite with both testcases and sub-testsuites, >> the testcases are executed first before iterating through the >> sub-testsuites. >>   By allowing this further breakdown into sub-testsuites, >> a refactor of the crypto unit tests solves the issue of inaccurate >> reporting, as sub-testsuites can be used in place of the testcases >> that had multiple testcases hidden on a sub level. >> The blockcipher tests previously had these hidden testcases, >> but are now sub-testsuites that are dynamically created and added to a >> parent test suite, allowing for each testcase status to be reported >> directly to the runner. >> The cryptodev test suite is broken down into smaller suites that are >> used as sub-testsuites, which allows for more flexibility choosing which >> sub-testsuites should run for the current device. >> The introduction of sub-testsuites also allows for more precise >> setup/teardown functions, that can check the capabilities required to >> run its testcases. >>   For example, when running the cryptodev_aesni_mb_autotest, >> the parent Cryptodev Test Suite is executed. >> Various sub-testsuites are added to the parent test suite, such as >> the static suites of testcases that were once in the >> cryptodev_testsuite, >> and blockcipher suites. >> The unit test runner can then run the Cryptodev parent test suite, >> which in turn will run the sub-testsuites. >> >> The user is now required to create vdevs via EAL commandline args, >> this is no longer done within the test app for crypto autotests. >> >> Documentation will need to be added at a later stage, >> adding to the test document that isn't yet merged. [1] >> >> [1] >> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/ >> >> >> v3: >>    - Added support for a testsuite having both a list of testcases, >>      and a list of sub-testsuites. >>    - Replaced PMD based parent testsuites with a cryptodev testsuite >>      used by all autotests, with the exception of scheduler autotest. >>    - Setup functions were added for all sub-testsuites, within which >>      required capability support is checked. >>    - The setup functions no longer create vdevs if needed, >>      this must be done by the user when running the test. >>    - Patch added to standardise return values for skipped testcases. >> v2: >>    - Added macro in place of testcase/testsuite loops. >>    - Added more detail in the summary output. >>    - Moved testcase counts to the testsuite structure. >>    - Flattened testsuite structure to remove union. >>    - Added patch for fix of blockcipher test return value. >>    - Squashed release note into last patch. >> >> >> Ciara Power (7): >>    app/test: refactor of unit test suite runner >>    test: introduce parent testsuite format >>    test/crypto: refactor to use sub-testsuites >>    test/crypto: replace unsupported with skipped >>    test/crypto: move testsuite params to header file >>    test/crypto: fix return value on test skipped >>    test/crypto: dynamically build blockcipher suite >> >>   app/test/test.c                        |  176 +- >>   app/test/test.h                        |   23 +- >>   app/test/test_cryptodev.c              | 2324 ++++++++++++++---------- >>   app/test/test_cryptodev.h              |   32 + >>   app/test/test_cryptodev_asym.c         |  111 +- >>   app/test/test_cryptodev_blockcipher.c  |  423 ++++- >>   app/test/test_cryptodev_blockcipher.h  |   12 +- >>   app/test/test_ipsec.c                  |   32 +- >>   doc/guides/rel_notes/release_21_05.rst |    5 + >>   9 files changed, 2000 insertions(+), 1138 deletions(-) >> > > Series Acked-by: Declan Doherty