From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9200A425FC; Tue, 19 Sep 2023 10:36:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3499E402DA; Tue, 19 Sep 2023 10:36:47 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id AB84C4026E for ; Tue, 19 Sep 2023 10:36:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695112605; x=1726648605; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=dHqujv1sgdUxpLEsr3/+O5C+VOeRNeDmi5AzUSw8yA0=; b=i8NaN35o63TOCCFY8ye1SY1jG7NiA/33iABAMt5Q6hHtL8JwFwsGsUMc 2XyWuq0r0SZ6m7gedka3mdJWL/mS5v+RdbdL+gDRyVr6GU6OiKbVaIxtK nyTheZvoZ7nKg1ZHfbasYlH45TxvPq1BOOs04NybSSGt8S0Dcwf54Hid6 eKqvJU5Zyl7XPxqUvOqYE7PnhAUTECT4SfC0JiSKVmGRbLYxAIDwRYF2t rd3W23UsxyZVgiCKixZ1rpuyTD9HqwEzZAqhRxhtDgIFYWaAI4dLYYOTa 8I2xoakWKvjZ3inYDlpUTvdNNqT2GU4C68CFJWN5f9EJm+HJGnhR5B7YC A==; X-IronPort-AV: E=McAfee;i="6600,9927,10837"; a="446360679" X-IronPort-AV: E=Sophos;i="6.02,159,1688454000"; d="scan'208";a="446360679" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Sep 2023 01:36:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10837"; a="889410686" X-IronPort-AV: E=Sophos;i="6.02,159,1688454000"; d="scan'208";a="889410686" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmsmga001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 19 Sep 2023 01:35:58 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Tue, 19 Sep 2023 01:36:43 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Tue, 19 Sep 2023 01:36:43 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32 via Frontend Transport; Tue, 19 Sep 2023 01:36:43 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.176) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.32; Tue, 19 Sep 2023 01:36:42 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WQKiZi2XazYdC/qXSUYgbGul7mFmbFApBVfAnwFw0pSSLy5TVnpbRMbs0CK1r2KX5lOdewb5eYAxq0ySjxBpMS7vj67oJxX5GVFKqr4tqBnwHZdqhXYhIZRREjrjJlkvJ/onA0Ga8l7CMCtUWuGjRqMrUWQAWjWaw10TIENQw2NfICuFmsTOAX1h5idEveGOBlCxzTO3au4jGlpPLPaBv1ZRpB3gP8QhNur5R0zacnVZlMOVoLYwwUFQYU0vF+49pH2j1aSGR4+1+cO/l1f8nPelPJM6yayvJeNNxawHD8ZkCxee2WbmEBCeqndRrmYT5Ej3p4bv7xTOovnxZ/UL+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qZD6lCV7t36+U0rtwYVPqeQbEx6XQe1MvVwn0gxCjxA=; b=Zlhjou0d8Zq490RSuirVmEI9NiszUtiHogKQoDR77RIO5TAVOhbGBx/ja75RooxP4XwqthI1uaMj/Czkej6fzKKucTKjBcQxZE48UbrhtgACvVpqaHc45t/XsQQjuww/uJLEewC4BcPrxOA/nq6H5YutqTP+h0m0BOMxOpVJnZPP3LFaMonweK02+dnSvb5R1MNPp84tDzpiI8B9hB0oKioEdMsPNeSRxV7/sAEqjncUGUuyuqJb1PLF4pvu3I+sKPv7mu71MzbY8sZnXnbiFDm0+E7XUFG/eoYNr+E4fWcwv3oSvPJLPgBDpHYG5aPN2BbCa9ci8ZF5kQyokW5TYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by PH0PR11MB5878.namprd11.prod.outlook.com (2603:10b6:510:14c::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.26; Tue, 19 Sep 2023 08:36:40 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::43d1:af60:464:347%5]) with mapi id 15.20.6768.029; Tue, 19 Sep 2023 08:36:40 +0000 Date: Tue, 19 Sep 2023 09:36:32 +0100 From: Bruce Richardson To: David Marchand CC: Subject: Re: [PATCH 0/2] add checks for tests not in a suite Message-ID: References: <20230915115206.132198-1-bruce.richardson@intel.com> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: DB7PR03CA0085.eurprd03.prod.outlook.com (2603:10a6:10:72::26) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|PH0PR11MB5878:EE_ X-MS-Office365-Filtering-Correlation-Id: bf705fe2-74bc-400f-0114-08dbb8eb8bbb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eHfKp9Lag/vwwoUNmTYeqga6Orz9W33PPG69BJ71ZbCZ4CNNi1GZjympmMWmIjsf2AOiEIyHqOWhO1Ok7HsKVr8iGi+MtTUzGduVb53pTy2heiHZS2ZaS5KDjKKmb2fGlIohV9/C92BuXgRGjD9bZznXkobJlTN8lDu3gKi+YBfp7gXIf33BM+obdffczlwZilI03Bd3m5apWb0m2LSpFrDMnCNvA4FIzMNiA4mhMp7Ci30ieZJWOPDFjWscthPpSb5ina0r3IHY7hXgf0g0rOWwx+fxWEIfjdXXm4ZyDjVL9UrDA9jitOhlrJpNpHgJa5cmIvgUvG4uOmmdFVTw4GU7kflfh6lEJs0fL7Sl0cR82o1Qj6fyS0YwxkHELldlivK/etbYrF+mjN381e4hvNaNP5XttSyKh/uaHjxSKbdCNnwba9yYa+ynkq2ihpRHDPZFqHhhORfF619bZpzycGPLc3PyhiOKYjanxNTgLBslTYenDR2SlZ94LYgWhmMkz3529zdz1D90v1DWSczIS+WsIbx7SOTDRoumybJZITiISPLu+muCW+cut1KRKzRh X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(346002)(366004)(136003)(376002)(39860400002)(396003)(1800799009)(186009)(451199024)(26005)(8936002)(82960400001)(8676002)(4326008)(83380400001)(2906002)(44832011)(5660300002)(86362001)(6506007)(53546011)(6486002)(478600001)(6666004)(6916009)(316002)(6512007)(66946007)(66476007)(66556008)(38100700002)(41300700001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eGR5MnhuSG5MeWdpejdtb2NPVGJBY0lseGVBZEt5Y3Z2b2tOalBaRzBrV3lq?= =?utf-8?B?RUJKTDFwSlg5ckZXVjBSTDE5UExJQVFGQXBWZzFodmlLTjdsMldXNS9IWlZj?= =?utf-8?B?UXVQNk5QVHBCclhiQkVHNEg2RnNpc3B3MXY5a2d0ZGJ4YW1tSUg0eDNxT3N4?= =?utf-8?B?OXBrdkJpdnFJNms1UmczY2hVUndSOGZsc3M4WndDUjEzOUwrcWtrSGEvazYy?= =?utf-8?B?cVNIVVo3UjdFZzh3eUN0b0FiZTFNaTdMTXNlUnJLNU52WFBUbmNoc0ErNkVw?= =?utf-8?B?L2hKdFZXS0t2enVESmNHSmZjN3FKdFB6dUtDVk1GNEVIN3E3NW50RDlWV1Y4?= =?utf-8?B?SWd1TTRzb3dJWnlIcUpFRGh4c2h5L1pNTXNOR1hNR3VTQTVFV2hFMklzb0hO?= =?utf-8?B?R2k0cTBTTFdwekFyQ241Y2NLamFSYjF0YVo5K1prTmQxWXpScmgwZ2FPcEpD?= =?utf-8?B?WkxTTExiM0JIbERGQ1lHelBMMHc3OW5rY2xkZkFUYnYzbFhkZGU3NDgrNy9x?= =?utf-8?B?aFozbWRqUytyY3g1U2dXTC9uOEFqNDdFWDI2VnZWOHdDYzdpNkNzdVhmRTlx?= =?utf-8?B?ei9GWU04YkdGMFI2ODBya3lZTTBKek82Q0VOdmJIRDlRTE9Jcll6VGZLNlp6?= =?utf-8?B?M29wclV2RStlV2ZxQWtrV2xMZWJuekUrd1lnWFErTkJ4c0loOE5FTXFDSVpP?= =?utf-8?B?aE9DVVFMZUUwTzBQU3R3VDJkNHlnbk1vbFlYaEZTZERJRnFxN3hZWEpQbHlO?= =?utf-8?B?R3MxeEcrNXVpdzhyNmF3aHBsYnJhUnRreDJoekVRQlcyQzRxbEdsVjFDWDU1?= =?utf-8?B?WmlvaVRyVFI4Ujd2d3FmM1l2TkxXM2F2dFM3NWl3RVpneDhudE5OeXRMTkc5?= =?utf-8?B?NEdqM1pzWForTDdJbDJPSnVESlFxM0YzV01oTXpzTmhKcTM4bTNBcEduR1FC?= =?utf-8?B?cllRMi9laG80Qm04NjNZUXdJSVVSUVduUUhoK2lGRjdZcUZWbXBZb0ZFOTVF?= =?utf-8?B?Wlc4dkNMMEhiNldjWmdxMllZb2xVa1BwVU9sa2VJYXBWRWxYNW1Ba0dmenUx?= =?utf-8?B?Y2dUbm83eW56T2RHVlJLcXQ2azhqalhiclV2RjFjYURLemRiOGhIMU9TOHo3?= =?utf-8?B?cGhTeW9sazFKaGZERlRJOVVTVzlkWEkyamNmK2YwNFgvcHEyZU1CTDBhN0ds?= =?utf-8?B?N0JvWE5ycU9SZ3R3ckJKMFdpbEpndjZ1OTFWenVyUnVuTHhyZnh1bWFmaThh?= =?utf-8?B?V095emZQZFBZMUFhMGNSYXAxQ1dLQzJ3OU8rK0VEZjBscTVxbm1BREFoVkMr?= =?utf-8?B?WXlhT0w2VTBXa3dZZXNHSnowZ0hUaHgwSnl1aUdRMXVwSzJSMlp2TnA1U3Uv?= =?utf-8?B?ZWlXcXBsZGhId0RnWTNaWTQ1bldqclBNZnl4OWUwMHNBNmtlSVE4TTI3Sk44?= =?utf-8?B?S25TU1lHWjVkeEJtVmJLZlpOZjN6eGpnSjdramNqTGdmWmh4cEhyMTJSU0NW?= =?utf-8?B?cU1pSFdNYTlVRERIMk5SUmtLOEZxUWpVbHBOY25NVGtxN0xSaU5xWHptbEVC?= =?utf-8?B?UXlla09HTk0zZUwzdHZocEp5QzVKdSt3ZFFaT1hBc3RaYnpXdzVVYmlyeUs5?= =?utf-8?B?Um0xWW4zOU9McFlJbkZXUlRoejVGaVhSRXFkVWtLZGpVQTBndHlTbG5aNUsv?= =?utf-8?B?ZjM4eG1tb1YxdWllZ3VDWDZtaXRSMWZ3aElKcy9GeERsQjlpc0tpR0NmSHk4?= =?utf-8?B?cjJaOTlBUnpkcmxaZy9hNjI0QWlKd3FEZGI5VEU5RjVEWUt3RXpwSTY2alVF?= =?utf-8?B?eXVZb3NIZTNHcEhlOEhEM0JRZC85c2JQSGZQOHNYNFhaaWoxZ2lBUFFkWHJN?= =?utf-8?B?cXhuT1RSdnRqc0ZNTitlZGtMSTQ3OUN0Y0RZczlDNGtPbU1oQjNPdmRuS0dZ?= =?utf-8?B?c1d0bHZVNE1PeTRPRStucCtHaGRqTVBXM2NBVE10UlFYQlpOMW0vMkdOL0s5?= =?utf-8?B?NHpac2N4T1VxVDU3NGVEaitoWGNSSWFSMGlYK3FSOVFhcExwVzZpZG4wS3B5?= =?utf-8?B?OGhmZDZkMmZaN0xhbFp3YVk0Vjdxd05JM0JCRW5kakNFUm5oR1VkUUFmdEw0?= =?utf-8?B?a3o1STJwK3lZZFdIeTBYMTFkeW85MjJCQnF1b2V3WlNnOW5NalFJZXFQMWNq?= =?utf-8?B?TkE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: bf705fe2-74bc-400f-0114-08dbb8eb8bbb X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2023 08:36:40.6049 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GsiicG/CxrS8nFaHMYWYcS0nhr+Zu+uWH0uJ6q2z1k8KroNGobtw8IQkjPfveshy06boBJiVkh0xUxMCMTaelPsVcfMhre5Oar6ABVgfICg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5878 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Sep 19, 2023 at 10:29:07AM +0200, David Marchand wrote: > On Fri, Sep 15, 2023 at 1:52 PM Bruce Richardson > wrote: > > > > To help ensure that we don't have "orphaned" tests not in any test > > suites we can add the following checks: > > > > * In developer-mode builds, emit a warning for each test defined using > > REGISTER_TEST_COMMAND > > * In checkpatches, add a check to prevent the addition of new tests > > using the REGISTER_TEST_COMMAND macro > > > > Bruce Richardson (2): > > app/test: emit warning for tests not in a test suite > > devtools: check for tests added without a test suite > > > > app/test/suites/meson.build | 13 ++++++++++++- > > buildtools/get-test-suites.py | 12 +++++++++--- > > devtools/checkpatches.sh | 8 ++++++++ > > 3 files changed, 29 insertions(+), 4 deletions(-) > > The "non_suite_tests" testsuite returned by > buildtools/get-test-suites.py is a bit strange, as it is not a > testsuite from meson pov. Yeah, it is a bit strange, and I'm open to new ideas on other solutions. I did it that way to avoid having yet another script to scan the files - I figured it was faster (in terms of runtime, not dev time) to do the scanning when the files are already being opened and processed by this one. Of course, if we can get the un-suitened [:-)] test cases down to zero, we can theoretically drop this check in future, and just use the checkpatch one. /Bruce