From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D48A43080; Wed, 16 Aug 2023 13:02:18 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 73A8D4325B; Wed, 16 Aug 2023 13:02:18 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id B975440EF0; Wed, 16 Aug 2023 13:02:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692183735; x=1723719735; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=bpZR4YpN+kC7BU648So/89erB4TIdW252adGS3TTcM0=; b=YpS/vP2OZ9+3L+/047WeNHaxs72kGX9Rch/RF1ZtMTodqcKQCL5YmZou 6EzUQJRpVQBZDXy81Mqh3bizqgyn+MccS7ZIeFZBzZgqe6tBNCbHmR6HM ny4c6LL/H0/eDH4Jqx96HIrHm1ne/PewN6dzD0y7Yeo0oRVGWOeUJB85C 2D9Y8DT9zhTgWa71OFm6hLIXqt/cVb7bvAR164nprAXFadwGD3AgR7Wa/ R5J69JgvJ9TmrLqxnqrLj2XtCxdP8lxXbuZ7jWhUZUJhlqvwSGctjHQjn 2AAQ35vNd1sRrPWuHBULC4MwttzLF415gaApcXNOUKrYvYY93D8Z4nI0A w==; X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="352089503" X-IronPort-AV: E=Sophos;i="6.01,176,1684825200"; d="scan'208";a="352089503" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Aug 2023 04:02:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="737266439" X-IronPort-AV: E=Sophos;i="6.01,176,1684825200"; d="scan'208";a="737266439" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by fmsmga007.fm.intel.com with ESMTP; 16 Aug 2023 04:02:12 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 16 Aug 2023 04:02:12 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Wed, 16 Aug 2023 04:02:12 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.109) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Wed, 16 Aug 2023 04:02:11 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aIgdP9Wa/QrNoQ/3lJV5UovbaSti6T4Ajj2xq+EDOhTJBOOccKx15dY5Nb+Q/Av85JleKtdB8lrye2aO0crP09RLhNqCqsekBTkUBl/VspBbvqCNcnY8KAyKplVRqrX+A0jqFw8bVQ5XxLMDRADTEaIvAtt4wj4dYyw1EO+WmyoCqem3zVS6FxndARCor5Ej5DhuIFKvdCD7i5ZK70r0Y3yUuWAzS83rufbKOvbVZGNrtYEKoiVKhCnjcF8P9DSeEmeGUIfu2o/xiOC71AltOuFw3lJvJ2phSMYAW17u91hkAGMkMK7dux33eYMwl99XsQ6BaIkJH3s8h35R2/dkmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EQgimdQoluXMEhPWWtyd8KJoKHcjXd00IMMnhEQbCSs=; b=VZD28O0O6zUkoJgJpJTHSrm52Wzyy5WPdo43dqWK90W+fo1QeApr2ck+OV3VvjudmLa7HGMeoT1me/iD1zFrM8sxX+jFUafkuv/7qFpcv4AIlk2SIFAmv2wxGCw1b2XaJT/KAB/uwlodl9+dNGFuvT5bgXVRceHpocmpb+ZuV3OiNWkvZul+UBA6+xx4VJBplNxpaGbO8ZCIecG7cudHTy9nE10GRv0xHDas2v/tAtn+sIpJlZ97TFWHRXSQu5Jsn6Kavc0gKThv7DjSyZBEK1J990sA7zrxdiEBg1AvnakQiemLLcZZR6p5ifxPDpKZ2LuVDR2iUg3YxY68Kr697w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA3PR11MB7583.namprd11.prod.outlook.com (2603:10b6:806:306::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Wed, 16 Aug 2023 11:02:10 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::9802:65f0:c441:780f]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::9802:65f0:c441:780f%7]) with mapi id 15.20.6678.025; Wed, 16 Aug 2023 11:02:10 +0000 Date: Wed, 16 Aug 2023 12:02:03 +0100 From: Bruce Richardson To: CC: , Morten =?iso-8859-1?Q?Br=F8rup?= Subject: Re: [PATCH v5 05/10] app/test: define unit tests suites based on test macros Message-ID: References: <20230721115125.55137-1-bruce.richardson@intel.com> <20230815151053.996469-1-bruce.richardson@intel.com> <20230815151053.996469-6-bruce.richardson@intel.com> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20230815151053.996469-6-bruce.richardson@intel.com> X-ClientProxiedBy: DUZPR01CA0155.eurprd01.prod.exchangelabs.com (2603:10a6:10:4bd::11) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA3PR11MB7583:EE_ X-MS-Office365-Filtering-Correlation-Id: b85c22f7-afd2-4336-ca35-08db9e483d0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JISe1iqqcHAIwRKk+QYne8Cd4sIt/tuCq4U7gY4s3/GsRwCQZIGujeV2DHh2AW78kbGwLmn8JhdSWbimwBnmaNbS4g2tVjGc4dT2NjiqnmvYmiJ6C4n0V8fmtmCrMvoMbBZEgHccYDemyEh0fvGiJ4ZLkPVgqE92s00qK/U+JyfToug651ji1si6LLbm5kQhJzGDci29rEKku0Vd8GllbHTDOEKEOltOsmDWYrsNobAdvCTAeYYvaVV2548i7Agsa66zIu9OOrPQU8iwG3+jOQEZccyRYIawIpv0lGEUgJMTsbdPVJKfdM/aEeirSdmzS30lodDYq8uxHllU1rEYDoQN1twpH3zwsDDe1CId3ALNAKxM2o2/NcYXLgw9H1vV8jV6n1rkKDjnbez5tX8WDDGiNa67F/a6sCFkrZ45uwtmL/bJ0EqDr2vUt16AfJ/NEDS+Ik5MfYd9RTRXgixfcnn0o9t1iVVq9hkt9Ho5A+VfTy0CLIe+MTExNK/VQ3mxAPfbHgfRPUISS5vVHF6EWjSOm/3MH0Mx+KZ+JX5nBGjBnvzqIkx9K0MqLEgcxeu+gQnaG9nj4uY+VHBw74V+Wl/GY1sk2jLv8umnBAQSHAA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(396003)(136003)(366004)(346002)(376002)(1800799009)(451199024)(186009)(316002)(6916009)(66946007)(66476007)(66556008)(41300700001)(5660300002)(44832011)(66574015)(38100700002)(8676002)(4326008)(8936002)(82960400001)(2906002)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(6666004)(6486002)(67856001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?FRXnxt9zNsXcwiMZMrtfxk4LKYqD3zeQvXqGAqCudpE2aTUPWQ8s0Iz1Es?= =?iso-8859-1?Q?gVAtkf6nxIKXBu30pV2t1Nc+515dEaRlcGfESzmO1DIVDekluay3Y5xnxO?= =?iso-8859-1?Q?MVaky+KZDo4fLwquR9pZZdZlPbW3q7cUpMJpsEuXLzVYzDqRz/kB9yJDCd?= =?iso-8859-1?Q?sH70Laxbkwjyz5v3dgHGj6MVCMwWA8F0ZbsfRt69jyZAUnN5/nFo/a09e7?= =?iso-8859-1?Q?jsJvW+KeUZYVTbEkAEYY1syxwXzeEXdU15yrbxfeRkd3Ccsb2/UsTjCFV5?= =?iso-8859-1?Q?2FCUy++6CnMz39sTUndI2U5gGi7Mag/Fx/Z8RmAS0gbHYFmQDMhurQzCTZ?= =?iso-8859-1?Q?VyT8+HO3/0X8L6I4kENnMzZC3oJjqdcpttCgG8Hff+ZD5/t4mJ8RjoPZGM?= =?iso-8859-1?Q?SpRuf0Iw3XZ8nDd5CX+7JHqO2IzqMt+96b0FVP2T3hWgWdLNVI5OE3y6Pr?= =?iso-8859-1?Q?RYWlKCBsMayKzzKHuJPwllHEaBsFyD+Tz3CBWhHS5ZN/hMtrHnm0XGLx05?= =?iso-8859-1?Q?Fv0/QvqlyM5BNnTFljEmpp+t09MePVOmVRpnpO84raopjL/vTbOWv//EMS?= =?iso-8859-1?Q?grATPP65CpsFJmJgL5r/ldHMwI2JKS8Vlim1ul3Vf75WsP/YxTJS5RoFqy?= =?iso-8859-1?Q?s6Y+gxNpM4O4btNpLhpSkq6lXk3yu2uUg65P3aHjuv/U73sjGVWrw34jt3?= =?iso-8859-1?Q?OupxbdZufLc7lkerFRC4e5eIjdmWnkQddEdwhFPBTk01HQRZ3a5UfMD9g+?= =?iso-8859-1?Q?p5h+IFTQW44QMT5Q92i+7aW9WHAnNfrPZ53QsTx82e8a437KdfQyHxBf+g?= =?iso-8859-1?Q?glJFx3ncug6LB61nkVvBI5yNEynLovBvVKcJSVp423XlVRcbMCQIdzxhfF?= =?iso-8859-1?Q?K5JnYXPpubHYDX6CMnA/v+04BN0utOLP1hTapD6HWve2ENDiF9hhDjtrkI?= =?iso-8859-1?Q?NHa/v2ihKBHKH+Mfb9FjE2GkkyuZG7abFtXPzjqmYKHSAP1sqklfloISbg?= =?iso-8859-1?Q?IKFQQJFji7KArQKyMLDIkQobjX1/WR33JEo38IcDqXp12rD5cnrvVhKlE3?= =?iso-8859-1?Q?LQVj+7cfazOueGxAW1Al/IefyC2+pqR0pdJm98jAkS81ke82sqO3DFXUZE?= =?iso-8859-1?Q?XGX64ppFmvR9B2VM2bIkxklWUtRP0qHZnVIM5BhFnew5M+BShCgckB3odR?= =?iso-8859-1?Q?4hGDAZUZjTlRwj2yP7ICnlTug/hHJUvlKnfiuXsHf1piLEdndJp5u6gbsK?= =?iso-8859-1?Q?1h52Rx+zGfdB6paPcWQ87iGi+rzp1TKYG+HHMfqjoG/SNQPg+RXxpgHGRg?= =?iso-8859-1?Q?dNHXuMBy34WesDoyN5HDcximqQvnAg2pZBJKDF/JMllnQTuCZL+N2XpP0X?= =?iso-8859-1?Q?F923MWLaTf/Jpp1VxWY8bea1GuGv9H2/Zifn74zBtC6BBzzeNXF0I1/Lf/?= =?iso-8859-1?Q?E48jBhV9KLG8Semo7HHcuKntDIpX0qTSP4yQFbmyLYKzEQZrDQLu767t4W?= =?iso-8859-1?Q?tL4BDRehikh1NfxSaVal5OKH2nVqY5+fQBMgxH6h/kAoKXOVsWEcmv1Ec4?= =?iso-8859-1?Q?DgXB9mc6KEAI2UO/DmMxOvI5vaYpSs40mPpmCD6Ysp7lR2dDMHgZRqj1Pc?= =?iso-8859-1?Q?DyJr9KHZXM3CQxYtHz0gwej0YCAw9vm2F1G8dAFxbhbDyAvl1/U18EEw?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: b85c22f7-afd2-4336-ca35-08db9e483d0c X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 11:02:10.3533 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2RrsFlCxAn69FD6A1Y7ma2qsHNx1919AWe0/xpEzgUavpJUujtg/9OneysViJKUR53Veap0zSvPgIlJOhqfJp1iZiITm/XHDqbm6YO+nls8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR11MB7583 X-OriginatorOrg: intel.com X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On Tue, Aug 15, 2023 at 04:10:49PM +0100, Bruce Richardson wrote: > Rather than having the test suites listed out in the meson.build files > and having to have them enabled/disabled selectively based on what libs > are being built, pull the tests to run from the source files which were > added to the build. > > Most test suites require no additional info other than the list of test > names in the suite. However the fast-test are special that they have > additional parameters associated with them. This requires some > additional work in the test extraction script and in the meson.build > file for processing the output. > > Signed-off-by: Bruce Richardson > Acked-by: Morten Brørup > --- > app/meson.build | 7 +- > app/test/suites/meson.build | 74 +++++++++++++++++++ > buildtools/get-test-suites.py | 33 +++++++++ > .../has-hugepages.py | 0 > buildtools/meson.build | 2 + > 5 files changed, 115 insertions(+), 1 deletion(-) > create mode 100644 app/test/suites/meson.build > create mode 100644 buildtools/get-test-suites.py > rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%) > > diff --git a/app/meson.build b/app/meson.build > index 0d8b618e7f..c14dc80892 100644 > --- a/app/meson.build > +++ b/app/meson.build > @@ -101,7 +101,7 @@ foreach app:apps > link_libs = dpdk_static_libraries + dpdk_drivers > endif > > - executable('dpdk-' + name, > + exec = executable('dpdk-' + name, > sources, > c_args: cflags, > link_args: ldflags, > @@ -110,4 +110,9 @@ foreach app:apps > include_directories: includes, > install_rpath: join_paths(get_option('prefix'), driver_install_path), > install: true) > + if name == 'test' > + dpdk_test = exec > + autotest_sources = sources > + subdir('test/suites') # define the pre-canned test suites > + endif > endforeach > diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build > new file mode 100644 > index 0000000000..ec74d8adf2 > --- /dev/null > +++ b/app/test/suites/meson.build > @@ -0,0 +1,74 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2023 Intel Corporation > + > +# some perf tests (eg: memcpy perf autotest)take very long > +# to complete, so timeout to 10 minutes > +timeout_seconds = 600 > +timeout_seconds_fast = 10 > + > +test_no_huge_args = ['--no-huge', '-m', '2048'] > +has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0' > +message('hugepage availability: @0@'.format(has_hugepage)) > + > +# process source files to determine the different unit test suites > +# - fast_tests > +# - perf_tests > +# - driver_tests > +test_suites = run_command(get_test_suites_cmd, autotest_sources, > + check: true).stdout().strip().split() > +foreach suite:test_suites > + # simple cases - tests without parameters or special handling > + suite = suite.split('=') > + suite_name = suite[0] > + suite_tests = suite[1].split(',') > + if suite_name != 'fast-tests' > + # simple cases - tests without parameters or special handling > + foreach t: suite_tests > + test(t, dpdk_test, > + env: ['DPDK_TEST=' + t], > + timeout: timeout_seconds, > + is_parallel: false, > + suite: suite_name) > + endforeach > + else > + # special fast-test handling here > + foreach t: suite_tests > + params = t.split(':') > + test_name = params[0] > + nohuge = params[1] == 'true' > + asan = params[2] == 'true' > + > + test_args = [] > + if nohuge > + test_args += test_no_huge_args > + elif not has_hugepage > + continue #skip this tests > + endif > + if not asan and (get_option('b_sanitize') == 'address' > + or get_option('b_sanitize') == 'address,undefined') > + continue # skip this test > + endif > + > + if get_option('default_library') == 'shared' > + test_args += ['-d', dpdk_drivers_build_dir] > + endif These lines here seem to be exposing a bug in the mempool unit tests for shared builds, which is why we have a CI failure. The mempool unit tests use the mempool "create_empty" API, and then call the populate APIs to finish setting up the mempool. However, the create_empty API does not explicitly configure a particular set of mempool ops for the new mempool, leaving the ops field at 0. This means that unless the app explicitly sets other ops, the mempool will use the ops from whatever driver registers itself first. This occurs even when the driver is unsuitable, e.g. on my Intel system, the dpaa2 is first on the list, leading to failures in setting up and using the mempool. In v6 of this set, I intend to fix this, by changing the create empty API to explicitly set the ring driver as default for new mempools. It's the most likely to work for common cases. /Bruce