From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7DDD9A0548; Fri, 2 Apr 2021 16:32:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 00ACA4069E; Fri, 2 Apr 2021 16:32:20 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 7A53840142 for ; Fri, 2 Apr 2021 16:32:18 +0200 (CEST) IronPort-SDR: H37luKPni+4l4baMpGuVkVeI2tNz6awtafkGnPqy/p4qRpeEH93rYsSX8yVthjWNTaMsRNHuil mXOCYpVqWaAA== X-IronPort-AV: E=McAfee;i="6000,8403,9942"; a="172510991" X-IronPort-AV: E=Sophos;i="5.81,300,1610438400"; d="scan'208";a="172510991" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2021 07:32:17 -0700 IronPort-SDR: f6Uh5przWJLOote2Pa1VPsFuHyt2pwEhB5mOj302agMdIQK8VhuiJSoWzJfUnltNvukzoreURj 7ZHZF8ovqx0g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,300,1610438400"; d="scan'208";a="413202312" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by fmsmga008.fm.intel.com with ESMTP; 02 Apr 2021 07:32:17 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Fri, 2 Apr 2021 07:32:16 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2 via Frontend Transport; Fri, 2 Apr 2021 07:32:16 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.48) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2106.2; Fri, 2 Apr 2021 07:32:16 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jsxI4D/Jw5mhnXXxRk0If/8fQLNsz1xCJ5t9Ev/WZRy55/CUhfjP1HaGPVd14Azvw7UeFT1ozQ2eULvHDnBHeNZ77Hyd3DRG5nXSBckd2qdFMp8eD8Dul7Atz9f/eN51Y8LNSz+YgKdukWkF1LUo+n0uW08UtHQ8nbrRtBM5p0uFDCtMGwY3MdtY+DkHUw/zS1s8YrElTYy3RAr8m95lBPtQW0qDZMmFlTLZE1Jr34Be5DFxvqFEexQoaEnqo+ZJY7eK1HegycbWfiPH63FI/slkEMSlH+gkcfMwJwg2gfz4LkABBaG3PRuN8WdxrXGlZndxBVWqyDkJ3i0QoFyv+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ltLuV3trV76JAc7O9yitDQa/COqBA6gLcABd2LuwVdY=; b=NK5AGuHz9HUlL+x1+XWfwqi1cG04cUwGmO+d9N36gMoAuB1bZCSyrZbHqCi1hgR88GqzNUnJrxgK1jKUPkXcjXHvADyj1EaZG6eoaTr1UpV2M1y1sE8OjZ8O6dB0m0SPEXlcHmMW1jVPZwZQAgsu6PxFjJtmAeWIVjQT9kAZJjXTqpHaVkDjF1AA7Xj+06lfAI4FDf/M4tzilhSlCLkOFjW5PGidhmeMj+csjPbE1/k47HtVkhA1iJX+Ux3umePVLGTHaV2crZJQu8CGEneNuoYMPKStoSE/WCQZHTbKn8BFSoUmlK60Jo5sp4SvRjDJiTrm+/ySzxqTCRwtxXiP2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ltLuV3trV76JAc7O9yitDQa/COqBA6gLcABd2LuwVdY=; b=ipu06t4MZsc+m3rHlLndnb2/gB8hit+2i9eZPxrtTOiiwT2LVEbSW6QbPL/WpsuYaX2o7k0BqFU4XcjPJLNCbaSWpk11jMyA8qrt/GuR8mXOLk22LkC6mZg0UOAR76UbO4EzsEOJIs+aS+dXZGe2oWikJplkQgKe63EZbc7Q7Nc= Received: from MN2PR11MB3821.namprd11.prod.outlook.com (2603:10b6:208:f7::24) by MN2PR11MB4726.namprd11.prod.outlook.com (2603:10b6:208:269::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3999.28; Fri, 2 Apr 2021 14:32:14 +0000 Received: from MN2PR11MB3821.namprd11.prod.outlook.com ([fe80::546c:75a5:fe9d:de32]) by MN2PR11MB3821.namprd11.prod.outlook.com ([fe80::546c:75a5:fe9d:de32%5]) with mapi id 15.20.3999.028; Fri, 2 Apr 2021 14:32:14 +0000 From: "Power, Ciara" To: Aaron Conole , "Doherty, Declan" CC: "dev@dpdk.org" Thread-Topic: [PATCH 0/6] test: refactor crypto unit test framework Thread-Index: AQHXGnFLRYIOl55I7k+g4UEd1VfPDqqcytyAgAF4u++AAyEGQA== Date: Fri, 2 Apr 2021 14:32:14 +0000 Message-ID: References: <20210316143253.3849182-1-ciara.power@intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [37.228.239.233] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e083339c-0cbd-4a9a-537e-08d8f5e41c01 x-ms-traffictypediagnostic: MN2PR11MB4726: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: d7+c6YC9wU9mFqBa4L/J6UuplgyElf4CGfdmgfA2SH+T8iAGGSPhE2Op06dZ2jl7zKu+3QpkQFcfhZLIhxbIwW6uSagC5kPea+et3pVnpxFJK+c828dcJVA2ZkROB59ozlg9tBWaNieUyoeDqdVxzDgE79kwcMOpwzINNXwE1GbgvbKbNu/Xqr/k/M1OZK218zkflVqn+bnkKeMw7qaEjd7gg1s9LATwDdUJc9kwMrPuVNu4wJ5vdDlZixAMxQ4+HKTU3C25zl9EfY5VA2FXpFQS76GWWLeOrxzcOLQfnbai8znQecfQj2D0u4OR28UpQd/Z0Tbox8mO0LQojbL/gNcfr5uxNeT70tcBUl8rVKCEghLcAU55shFqp+1A1Q6hrYAy6DNN4PXkZbJuuzxdmfd3poYSSsSazDxtrCtJVudjbHk1Z0pYyQigZ6YdZFDugWEEdoLcWpOiK8kxFt3aW7N2mE1nwkTDchPLhBPMKd4ae8Wwx1pewYwb3L7OKGSdzbw+uF28Oh7jkNsIwUhW34Za4xheoW+1gJKE0cTdjvJIe70uapyqsF29Uub13QawvRy3bbD1osmpl2CXTrLa6K0ov3b0wUTlpc3+L1A9Uq0OhEXMRZUmeXSLCMJsULc3yPlQSgTbiS+Ls0rm9O1xQ17wcXsAGNaRK2xy1wPVKv9SjxCfjd+upFhR8r2Zf/RDf0bJDVbHaKENcr5kMjV9YfqGVuuQ1uWfbVGOApSNyT6ocrdYrLfUjKENVcT/post x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR11MB3821.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(396003)(346002)(366004)(136003)(376002)(186003)(4326008)(8676002)(7696005)(83380400001)(6506007)(53546011)(38100700001)(71200400001)(55016002)(26005)(9686003)(8936002)(86362001)(6636002)(76116006)(64756008)(5660300002)(66446008)(66556008)(33656002)(2906002)(110136005)(316002)(966005)(478600001)(52536014)(66476007)(66946007); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?8Vvmv9+S6Jaav/tLsO1evfrASRJUYlYni0ukbTFyaPGwCfbrckMxUmggjztk?= =?us-ascii?Q?alESE2yG+ZqP4UpdGnL3iaM0gIdsgayxRD/z16rd/ESXcdX1wKladbTMgGCc?= =?us-ascii?Q?GUXHjAGfIg9iFu54yif2wccSSWhM0eGYhF8Be4iD2nGIe2m+RMkq6SRHvMAp?= =?us-ascii?Q?Psp+0btPUDetr1WltzCPp7oMhk4/+wMBln8gpjdBE2+XCbGtVgcPvRqzBFPW?= =?us-ascii?Q?OVTxn/av3Kh5STSfK2gH718P222vz4ZDkx4t1YkLt+g4fOO98MxwaYaS9nYJ?= =?us-ascii?Q?tTNkrBKqAkHnIxMW9NGEhn2zgz71/OXfK6YsoP1JZ2S7qaVmlmcyeVPnCMSN?= =?us-ascii?Q?vup94VsxJXsXUQm3uTCpMFSvWOFrpoTMoPsyVQa6abZDCwDYhe25OLqpe9r6?= =?us-ascii?Q?DRmC3i0V2y90SKO6afdKjCmpSe6iYJfWdZ4GuSrwGTbVKfbGlBARDnRQ2Pv0?= =?us-ascii?Q?+WkdnI8LgCuk2kzL0kKOMzen/Z8WHM8iIFoZICLC4k4L01swncyhzuFR7wlJ?= =?us-ascii?Q?Ztrakopk8D3E4x8Hw/Ay+ejfvacnA6jeeBQXw9CNWC3XRWJo6LTIlvKWsz8V?= =?us-ascii?Q?+m9cKgCCWmywYmLl2KgxaMe1g3rTlpHRdYAYykeoZblQtdaTvT9uaD5Wv2q4?= =?us-ascii?Q?O6FdHQ4xNZZ2aQADh09/PyrNOiyF1O4uBUmhrKEPASmzbh2rnsqF/IK3Aged?= =?us-ascii?Q?Qwb05AerKzBUKVxg1im4/Tk4mGrpV5yqcwJf5qeSHQ4ZO3chdR/80rv5C8QN?= =?us-ascii?Q?af1cjKAJ0oRbOP88iYHk8A6xyDEoq9dnLZ9jBR3fWGNiCJvrzIz4LhXCFr/m?= =?us-ascii?Q?U09g4OlmqoOrEQO1hbu/GqssBhL3CFud4izcFLREyrztjGQZHOEXS/3R9JG3?= =?us-ascii?Q?QFH+YoWDq35DTBKIIyXtBToD6tL5O62A9eZgRCdLbnCIE4vYwLFWxjegB9Rw?= =?us-ascii?Q?PmGu3DKJe8qaZUtV5WGFXviwGmoj6xjz3oTvpjz3eQdXDpRqoGciQQsZGroD?= =?us-ascii?Q?pfdAhC+q6EVuyrwoF9PWe1aAsOIbvjYs0N4ttJM6MYtUvoRjt/C28Quj2wgU?= =?us-ascii?Q?dPb6mr7Enxs8np0aUBBkScpdngkX4ggRsoiqRBjc7HpTqAxORVl4qA48YKr9?= =?us-ascii?Q?QJUF8/DvVDtgs55LsuBhKwdlWRH9G9D83cG4jQsLAcX22hTo6hntbd4Zp7T8?= =?us-ascii?Q?neDIflJNFSoKDEMinWE69veYwjdcaGGsmfsUxiMLHu9IAfEhjWXFJrA0NDt1?= =?us-ascii?Q?i6xlAR6+jv0uKP/25PIpu+u4AFrAtajqJ5TJwI1XE9gxX1Hk/4i0hIpNCRoZ?= =?us-ascii?Q?Hyw4+xK5I/Oz6HNhG3usGM1j?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB3821.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e083339c-0cbd-4a9a-537e-08d8f5e41c01 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2021 14:32:14.3223 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6h1Bb4uaxUKDhqo2wXx94HSZopyVAdyNw7WfWe+YLVFKRyGPWTMmlAZe1eRNzTox/N5p41pD7XOV2za1oo8ZCw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4726 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH 0/6] test: refactor crypto unit test framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Aaron, >-----Original Message----- >From: Aaron Conole >Sent: Wednesday 31 March 2021 15:43 >To: Doherty, Declan >Cc: Power, Ciara ; dev@dpdk.org >Subject: Re: [PATCH 0/6] test: refactor crypto unit test framework > >"Doherty, Declan" writes: > >> Hey Aaron, >> >> based on the work you've been doing on the unit test documentation we >> would appreciate if you could take a look over this patchset and get >> your thoughts. The primary drive here is to make it easier to get a >> clear picture of what is being executed in the cryptodev testsuite, as >> at the moment there are test suites masquerading as unit tests and the >> output doesn't reflect the actual number of unit tests being executed. > >I sent some comments - thanks :) > Thanks for the review! I have sent a v2 including your suggested changes incorporated. https://patchwork.dpdk.org/project/dpdk/list/?series=3D16085 >> Thanks >> Declan >> >> On 16/03/2021 2:32 PM, Ciara Power wrote: >>> The current crypto unit test framework is not granular enough to >>> accurately track unit test results. This is caused by one testcase in >>> a suite actually running multiple testcases, but only returning one >>> result. >>> The approach taken in this patchset allows a test suite have either >>> a list of sub-testsuites, or a list of testcases as previously used. >>> The unit test suite runner can then recursively iterate and run the >>> sub-testsuites, until it reaches a suite with testcases, and it then >>> runs each testcase as it had done previously. >>> By allowing this further breakdown into sub-testsuites, a refactor >>> of the crypto unit tests solves the issue of inaccurate reporting, as >>> sub-testsuites can be used in place of the testcases that had >>> multiple testcases hidden on a sub level. >>> The blockcipher tests previously had these hidden testcases, but are >>> now sub-testsuites that are dynamically created and added to a parent >>> test suite, allowing for each testcase status to be reported directly >>> to the runner. >>> The cryptodev test suite is broken down into smaller suites that are >>> used as sub-testsuites, which allows for more flexibility choosing >>> which sub-testsuites should run for the current device autotest. >>> The introduction of sub-testsuites also allows for more precise >>> setup/teardown functions, rather than general ones loaded with >>> conditions as was seen with the initial setup function used for all >>> crypto testsuites. >>> For example, when running the cryptodev_aesni_mb_autotest, the >>> AESNI MB parent test suite has its own setup function to initialise >>> the AESNI MB device. >>> Various sub-testsuites are added to the parent test suite, such as >>> some of the static suites that were once in the cryptodev_testsuite, >>> and blockcipher suites. >>> The unit test runner can then run the AESNI MB parent test suite, >>> which in turn will run the sub-testsuites. >>> Documentation will be added in a later version of the patchset, >>> adding to the test document that isn't yet merged. [1] >>> >>> --- >>> [1] >>> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536- >1 >>> -aconole@redhat.com/ >>> >>> Ciara Power (6): >>> app/test: refactor of unit test suite runner >>> test: introduce parent testsuite format >>> test/crypto: refactor to use sub-testsuites >>> test/crypto: move testsuite params to header file >>> test/crypto: dynamically build blockcipher suite >>> doc: add unit test suite change to release notes >>> >>> app/test/test.c | 168 +- >>> app/test/test.h | 22 +- >>> app/test/test_bitratestats.c | 4 +- >>> app/test/test_compressdev.c | 4 +- >>> app/test/test_cryptodev.c | 2020 ++++++++++++++++++-----= - >>> app/test/test_cryptodev.h | 20 + >>> app/test/test_cryptodev_asym.c | 105 +- >>> app/test/test_cryptodev_blockcipher.c | 121 +- >>> app/test/test_cryptodev_blockcipher.h | 12 +- >>> app/test/test_ethdev_link.c | 4 +- >>> app/test/test_event_crypto_adapter.c | 4 +- >>> app/test/test_event_eth_rx_adapter.c | 8 +- >>> app/test/test_event_eth_tx_adapter.c | 4 +- >>> app/test/test_event_timer_adapter.c | 4 +- >>> app/test/test_eventdev.c | 4 +- >>> app/test/test_fbarray.c | 4 +- >>> app/test/test_fib.c | 8 +- >>> app/test/test_fib6.c | 8 +- >>> app/test/test_graph.c | 4 +- >>> app/test/test_graph_perf.c | 4 +- >>> app/test/test_ipfrag.c | 4 +- >>> app/test/test_ipsec.c | 36 +- >>> app/test/test_ipsec_sad.c | 4 +- >>> app/test/test_latencystats.c | 4 +- >>> app/test/test_link_bonding.c | 4 +- >>> app/test/test_link_bonding_mode4.c | 4 +- >>> app/test/test_link_bonding_rssconf.c | 4 +- >>> app/test/test_metrics.c | 4 +- >>> app/test/test_pmd_ring.c | 4 +- >>> app/test/test_reorder.c | 4 +- >>> app/test/test_rib.c | 8 +- >>> app/test/test_rib6.c | 8 +- >>> app/test/test_security.c | 4 +- >>> app/test/test_service_cores.c | 4 +- >>> app/test/test_trace.c | 4 +- >>> doc/guides/rel_notes/release_21_05.rst | 5 + >>> 36 files changed, 1898 insertions(+), 739 deletions(-) >>>