From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 950A243007; Tue, 8 Aug 2023 10:46:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8E66443247; Tue, 8 Aug 2023 10:46:35 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 3589640A87 for ; Tue, 8 Aug 2023 10:46:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691484392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RBE1LPjXMQ398D7IGzJyvGqiyX3fuIhmqyfxJdJ2Jyc=; b=WmLHpNJDYB8E8ni1Kah9/SE5aigzs+LuqAXFVQwhE8X7uAHXzw9iJh6a1Rd/VBSv8GjZXl vllO493+k51yTo3C5Tx12jTyW5O4jAtKUxFav+2W1X2iEJ5yOsgdRrmPBq0W9S6M+NqHU4 P8wC8Gk37197yngOBj2f/GJAshLqkio= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-h-sZlm4oO7y7UzNu3_ZMgA-1; Tue, 08 Aug 2023 04:46:31 -0400 X-MC-Unique: h-sZlm4oO7y7UzNu3_ZMgA-1 Received: by mail-lj1-f200.google.com with SMTP id 38308e7fff4ca-2b9ab2dd649so36816331fa.0 for ; Tue, 08 Aug 2023 01:46:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691484390; x=1692089190; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RBE1LPjXMQ398D7IGzJyvGqiyX3fuIhmqyfxJdJ2Jyc=; b=HmnPbEGta9/vAYLPvVlm4e8SHf8i48TMW0Ckx4iS5HV4kurSiYe86xJ/tzBz+3wpl/ 9NSETaG/a26i19XbifQ+ijixIoMhXoLxbbLPDGRGoC2Iv0jZX/cWp/XrEztm4uNfwyrd dMj9Fz7lEF9ZMjfTihLuEjpLKfUTQI1jc3kbL/v7zEoWEd6XJ221DIPET9P3wL2fSNXh 2jlWG9nAc4AtbPQGOWpNy16uoTy0FByi5GxLdDANY7+ul5graZ87CKKXlWWgy2t7zO7M UdTOzizyduof0ehAkKRubn+5lH8+LXBTWCSRqk7GaZRI28uWZYzww0NC9lepgjN2gj7C o6RQ== X-Gm-Message-State: AOJu0YxcNX9TKLxHT5LKDybkobV1TyzBvAL4xt8sbCdRaaVBKg0xBnZv GpX0wymaCO/BKI9NM3/wIeX6D5mAce0YgqW4g1ZF4briN2Np41T5lN+/FHQD+82DpiCx4vD/Bgx OboxrJxRVz/ffn4H3mA== X-Received: by 2002:a2e:3c02:0:b0:2b9:601d:2c0 with SMTP id j2-20020a2e3c02000000b002b9601d02c0mr2629736lja.25.1691484389922; Tue, 08 Aug 2023 01:46:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFfPmsXtjiLyfoHnWqC6W1YVaNpsg8jfyAhFqCJtAO6FlCT3GSgyCFeDO4QEXn+16F2FFlvz8VcLo6bEjZ2tAA= X-Received: by 2002:a2e:3c02:0:b0:2b9:601d:2c0 with SMTP id j2-20020a2e3c02000000b002b9601d02c0mr2629721lja.25.1691484389505; Tue, 08 Aug 2023 01:46:29 -0700 (PDT) MIME-Version: 1.0 References: <20230721115125.55137-1-bruce.richardson@intel.com> In-Reply-To: <20230721115125.55137-1-bruce.richardson@intel.com> From: David Marchand Date: Tue, 8 Aug 2023 10:46:17 +0200 Message-ID: Subject: Re: [RFC PATCH 0/5] replace build code for unit tests To: Bruce Richardson Cc: dev@dpdk.org, Tyler Retzlaff , Thomas Monjalon , =?UTF-8?Q?Morten_Br=C3=B8rup?= , Aaron Conole , ci@dpdk.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Hello Bruce, On Fri, Jul 21, 2023 at 1:51=E2=80=AFPM Bruce Richardson wrote: > > While working on series such as [1] to make DPDK builds more > configurable, the majority of the complexity involved centered on the > unit tests, and ensuring that the individual files were added/removed > from the build as the components were enabled/disabled. > > This complexity was exacerbated by: > * C files having multiple dependencies, specifically, > * Some libraries having one set of independent tests, while other > library tests requiring additional components > * Having to maintain lists of tests in various suites which also needed > to be updated, depending on component availability. > * Everything having to be checked via if-else clauses, since this was > not data-driven > > This RFC attempts to address these issues by doing the following: > * Creating a per-file dependency list for the autotests, allowing easy > checks of when a file could be built > * Tagging the various fast-tests/perf-tests/drivers-tests suites via > macros in the C code. Then scan the C-files we are building to > construct the test suites based on what is available I did not look into the implementation details too much but I like the idea= . > > > This RFC is just an early prototype, and probably needs cleanup and > validation of the dependency lists. Windows support is probably also > broken, given the fact that our current build file maintains custom > lists there. However, I would hope it should not be too big a job to > enable the unit tests for the windows components that are already > available. (Tests for any components not being build on windows are > already removed via the dependency lists). > > Another (hopefully less-impactful) side-effect of this change is that > the "test" binary has moved in the build folder. It is now put > along-side the other binaries in "build/app" rather than being in the > subdirectory "build/app/test". Cc: ci people. It may break some CI scripts but it is hard to tell which one. As far as testsuites we run in GHA or UNH are concerned, it is done through meson test so we are good on this side. > > Future work could also be done to scan the C files that are not being > build for tests. Those tests could then be stubbed out in an > auto-genereated file to always return "SKIPPED", meaning that all test > commands are always valid, irrespective of what components are present > or not. > > [1] http://patches.dpdk.org/project/dpdk/list/?series=3D28628&state=3D10 > > Bruce Richardson (5): > app/test: add new macros for various test types > app/test: tag tests with the test type > app/test: build using per-file dependency matrix > app/test: define unit tests suites based on test macros > app/test: add fast test suite to new build infrastructure > > app/meson.build | 15 +- > app/test/meson.build | 796 +++++------------- > app/test/suites/meson.build | 75 ++ > app/test/test.h | 9 +- > app/test/test_acl.c | 2 +- > app/test/test_atomic.c | 2 +- > app/test/test_barrier.c | 2 +- > app/test/test_bitmap.c | 2 +- > app/test/test_bitops.c | 2 +- > app/test/test_bpf.c | 4 +- > app/test/test_byteorder.c | 2 +- > app/test/test_cksum.c | 2 +- > app/test/test_cmdline.c | 2 +- > app/test/test_common.c | 2 +- > app/test/test_cpuflags.c | 2 +- > app/test/test_crc.c | 2 +- > app/test/test_cryptodev.c | 38 +- > app/test/test_cryptodev_asym.c | 2 +- > app/test/test_cycles.c | 2 +- > app/test/test_debug.c | 2 +- > app/test/test_devargs.c | 2 +- > app/test/test_distributor.c | 2 +- > app/test/test_distributor_perf.c | 2 +- > app/test/test_dmadev.c | 2 +- > app/test/test_eal_flags.c | 24 +- > app/test/test_eal_fs.c | 2 +- > app/test/test_efd.c | 2 +- > app/test/test_efd_perf.c | 2 +- > app/test/test_errno.c | 2 +- > app/test/test_ethdev_link.c | 2 +- > app/test/test_event_ring.c | 2 +- > app/test/test_eventdev.c | 2 +- > app/test/test_fbarray.c | 2 +- > app/test/test_fib.c | 4 +- > app/test/test_fib6.c | 4 +- > app/test/test_fib6_perf.c | 2 +- > app/test/test_fib_perf.c | 2 +- > app/test/test_func_reentrancy.c | 2 +- > app/test/test_hash.c | 2 +- > app/test/test_hash_functions.c | 2 +- > app/test/test_hash_multiwriter.c | 2 +- > app/test/test_hash_perf.c | 2 +- > app/test/test_hash_readwrite.c | 4 +- > app/test/test_hash_readwrite_lf_perf.c | 2 +- > app/test/test_interrupts.c | 2 +- > app/test/test_ipfrag.c | 2 +- > app/test/test_ipsec.c | 2 +- > app/test/test_ipsec_perf.c | 2 +- > app/test/test_kni.c | 2 +- > app/test/test_kvargs.c | 2 +- > app/test/test_lcores.c | 2 +- > app/test/test_logs.c | 2 +- > app/test/test_lpm.c | 2 +- > app/test/test_lpm6.c | 2 +- > app/test/test_lpm6_perf.c | 2 +- > app/test/test_lpm_perf.c | 2 +- > app/test/test_malloc.c | 2 +- > app/test/test_malloc_perf.c | 2 +- > app/test/test_mbuf.c | 2 +- > app/test/test_mcslock.c | 2 +- > app/test/test_member.c | 2 +- > app/test/test_member_perf.c | 2 +- > app/test/test_memcpy.c | 2 +- > app/test/test_memcpy_perf.c | 2 +- > app/test/test_memory.c | 2 +- > app/test/test_mempool.c | 2 +- > app/test/test_mempool_perf.c | 2 +- > app/test/test_memzone.c | 2 +- > app/test/test_meter.c | 2 +- > app/test/test_mp_secondary.c | 2 +- > app/test/test_per_lcore.c | 2 +- > app/test/test_pflock.c | 2 +- > app/test/test_pie.c | 6 +- > app/test/test_pmd_perf.c | 2 +- > app/test/test_power.c | 2 +- > app/test/test_power_cpufreq.c | 2 +- > app/test/test_power_intel_uncore.c | 2 +- > app/test/test_power_kvm_vm.c | 2 +- > app/test/test_prefetch.c | 2 +- > app/test/test_rand_perf.c | 2 +- > app/test/test_rcu_qsbr.c | 2 +- > app/test/test_rcu_qsbr_perf.c | 2 +- > app/test/test_reassembly_perf.c | 2 +- > app/test/test_reciprocal_division.c | 2 +- > app/test/test_reciprocal_division_perf.c | 2 +- > app/test/test_red.c | 4 +- > app/test/test_reorder.c | 2 +- > app/test/test_rib.c | 4 +- > app/test/test_rib6.c | 4 +- > app/test/test_ring.c | 2 +- > app/test/test_ring_perf.c | 2 +- > app/test/test_rwlock.c | 8 +- > app/test/test_sched.c | 2 +- > app/test/test_security.c | 2 +- > app/test/test_seqlock.c | 2 +- > app/test/test_service_cores.c | 4 +- > app/test/test_spinlock.c | 2 +- > app/test/test_stack.c | 4 +- > app/test/test_stack_perf.c | 4 +- > app/test/test_string_fns.c | 2 +- > app/test/test_tailq.c | 2 +- > app/test/test_thash.c | 2 +- > app/test/test_thash_perf.c | 2 +- > app/test/test_threads.c | 2 +- > app/test/test_ticketlock.c | 2 +- > app/test/test_timer.c | 2 +- > app/test/test_timer_perf.c | 2 +- > app/test/test_timer_racecond.c | 2 +- > app/test/test_trace.c | 2 +- > app/test/test_trace_perf.c | 2 +- > app/test/test_version.c | 2 +- > buildtools/get-test-suites.py | 33 + > .../has-hugepages.py | 0 > buildtools/meson.build | 2 + > 114 files changed, 478 insertions(+), 754 deletions(-) > create mode 100644 app/test/suites/meson.build > create mode 100644 buildtools/get-test-suites.py > rename app/test/has_hugepage.py =3D> buildtools/has-hugepages.py (100%) > > -- > 2.39.2 > --=20 David Marchand