From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 010B843064; Mon, 14 Aug 2023 20:21:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F1CF04326E; Mon, 14 Aug 2023 20:21:15 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 5AE6740A8A; Mon, 14 Aug 2023 20:21:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692037274; x=1723573274; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x8IHM2aO9IFBjlbI6ifkr9n5MxeMJAupRyRtXqKetSg=; b=DqTttZVtUuwnqGJASAFoygytB04ebI0hcqmxwFdTFBWSuGyHz8EZeEsG aWOtiLU9W6iBH525kBZI3wFXPJ2ccDoPYgYh1L9dPm2Vgo0Vk3aYgYCxO ro5gxcId21cVQMSaWp6QgiMWekgrR1NLg0jOoIoTg2AUZBmwG+NbK+evK AXifNPIDKUSiblTzh5srOJCc7Ll70KIVUX0p+hIy63AA1g7NojvoHAZpo pJI1qCmhXRiNkywPY1R5ZrGOqVSRs/bAvbZf7hUAaPz1RBOScqgERAMNM AhUcWaAOwKz+IJQ84eCuYunf3p/RnKSQ4UwiWxM2mwj+Y7c/kzHdF6W2+ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10802"; a="375831095" X-IronPort-AV: E=Sophos;i="6.01,173,1684825200"; d="scan'208";a="375831095" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2023 11:21:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10802"; a="733545432" X-IronPort-AV: E=Sophos;i="6.01,173,1684825200"; d="scan'208";a="733545432" Received: from silpixa00401385.ir.intel.com ([10.237.214.14]) by orsmga002.jf.intel.com with ESMTP; 14 Aug 2023 11:21:12 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: ci@dpdk.org, Bruce Richardson Subject: [PATCH v3 0/8] expand list of optional libraries Date: Mon, 14 Aug 2023 19:20:56 +0100 Message-Id: <20230814182104.470270-1-bruce.richardson@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721115125.55137-1-bruce.richardson@intel.com> References: <20230721115125.55137-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org This patchset contains changes to DPDK to make the build more configurable overall. The changes can be grouped into two areas: * Firstly, there are changes to make the build of the unit tests more flexible and maintainable. These 5 patches switch the unit tests from having separate hard-coded lists of files and tests, with lots of conditional logic to add/remove items from the list, to instead being a single list of files and dependencies per-file. The actual lists of test-suites are built up dynamically from the list of files whose dependencies are met and therefore will be part of the build. This rework enables... * The second set of patches which expands the list of libraries which can be disabled at build-time, while still giving a successful build of DPDK. Overall, after this set is applied, the list of optional libraries in DPDK is approx doubled in size. [15 additions giving 31 optional libs] NOTES: * as stated in the RFC, the test binary moves in the build as a result of this set - from app/test/dpdk-test to app/dpdk-test. * running the suites of tests no longer works after patch 4 until patch 5 is applied. If this is an issue, the two patches can be merged on apply. However, the individual tests are still runnable through the test binary - it's only through "meson test" that there are issues, so I'm keeping the patches separate since each is relatively complex in its own right. * A number of files in the test directory were not being built, and seem to have never been built with meson. This requires separate work, outside the scope of the patchset to fix. * Similarly, some unit tests were never added to test suites. These can now be found by using grep to find the old REGISTER_TEST_COMMAND macro. Again, adding orphan tests to suites is outside the scope of this set. V3: - Create symlink to old path to test binary, to ensure all automated CI, and any other use of the test binary still works. V2: - Changes since RFC: - merged two previous sets - reworking test build, and disabling libs - into one, since they depend on each other - Reworked the list of dependencies for each file in the tests, so that files that depend on other files have those deps called out. - Added list of external dependencies to the test builds - For disabling libs, merged patches disabling individual libs together into one patch, since all unit test build changes were unnecessary. This made each patch a single-line addition to the optional list in lib/meson.build. Bruce Richardson (8): app/test: add new macros for various test types app/test: tag tests with the test type app/test: make telemetry data test buildable on windows app/test: build using per-file dependency matrix app/test: define unit tests suites based on test macros examples/l3fwd: make eventdev an optional dependency build: make most device classes optional build: expand list of optional libraries app/meson.build | 15 +- app/test/meson.build | 823 +++++------------- app/test/suites/meson.build | 76 ++ app/test/test.h | 9 +- app/test/test_acl.c | 2 +- app/test/test_atomic.c | 2 +- app/test/test_barrier.c | 2 +- app/test/test_bitmap.c | 2 +- app/test/test_bitops.c | 2 +- app/test/test_bpf.c | 4 +- app/test/test_byteorder.c | 2 +- app/test/test_cksum.c | 2 +- app/test/test_cmdline.c | 2 +- app/test/test_common.c | 2 +- app/test/test_cpuflags.c | 2 +- app/test/test_crc.c | 2 +- app/test/test_cryptodev.c | 38 +- app/test/test_cryptodev_asym.c | 2 +- app/test/test_cycles.c | 2 +- app/test/test_debug.c | 2 +- app/test/test_devargs.c | 2 +- app/test/test_distributor.c | 2 +- app/test/test_distributor_perf.c | 2 +- app/test/test_dmadev.c | 2 +- app/test/test_eal_flags.c | 24 +- app/test/test_eal_fs.c | 2 +- app/test/test_efd.c | 2 +- app/test/test_efd_perf.c | 2 +- app/test/test_errno.c | 2 +- app/test/test_ethdev_link.c | 2 +- app/test/test_event_ring.c | 2 +- app/test/test_eventdev.c | 2 +- app/test/test_fbarray.c | 2 +- app/test/test_fib.c | 4 +- app/test/test_fib6.c | 4 +- app/test/test_fib6_perf.c | 2 +- app/test/test_fib_perf.c | 2 +- app/test/test_func_reentrancy.c | 2 +- app/test/test_hash.c | 2 +- app/test/test_hash_functions.c | 2 +- app/test/test_hash_multiwriter.c | 2 +- app/test/test_hash_perf.c | 2 +- app/test/test_hash_readwrite.c | 4 +- app/test/test_hash_readwrite_lf_perf.c | 2 +- app/test/test_interrupts.c | 2 +- app/test/test_ipfrag.c | 2 +- app/test/test_ipsec.c | 2 +- app/test/test_ipsec_perf.c | 2 +- app/test/test_kvargs.c | 2 +- app/test/test_lcores.c | 2 +- app/test/test_logs.c | 2 +- app/test/test_lpm.c | 2 +- app/test/test_lpm6.c | 2 +- app/test/test_lpm6_perf.c | 2 +- app/test/test_lpm_perf.c | 2 +- app/test/test_malloc.c | 2 +- app/test/test_malloc_perf.c | 2 +- app/test/test_mbuf.c | 2 +- app/test/test_mcslock.c | 2 +- app/test/test_member.c | 2 +- app/test/test_member_perf.c | 2 +- app/test/test_memcpy.c | 2 +- app/test/test_memcpy_perf.c | 2 +- app/test/test_memory.c | 2 +- app/test/test_mempool.c | 2 +- app/test/test_mempool_perf.c | 2 +- app/test/test_memzone.c | 2 +- app/test/test_meter.c | 2 +- app/test/test_mp_secondary.c | 2 +- app/test/test_per_lcore.c | 2 +- app/test/test_pflock.c | 2 +- app/test/test_pie.c | 6 +- app/test/test_pmd_perf.c | 2 +- app/test/test_power.c | 2 +- app/test/test_power_cpufreq.c | 2 +- app/test/test_power_intel_uncore.c | 2 +- app/test/test_power_kvm_vm.c | 2 +- app/test/test_prefetch.c | 2 +- app/test/test_rand_perf.c | 2 +- app/test/test_rcu_qsbr.c | 2 +- app/test/test_rcu_qsbr_perf.c | 2 +- app/test/test_reassembly_perf.c | 2 +- app/test/test_reciprocal_division.c | 2 +- app/test/test_reciprocal_division_perf.c | 2 +- app/test/test_red.c | 4 +- app/test/test_reorder.c | 2 +- app/test/test_rib.c | 4 +- app/test/test_rib6.c | 4 +- app/test/test_ring.c | 2 +- app/test/test_ring_perf.c | 2 +- app/test/test_rwlock.c | 8 +- app/test/test_sched.c | 2 +- app/test/test_security.c | 2 +- app/test/test_seqlock.c | 2 +- app/test/test_service_cores.c | 4 +- app/test/test_spinlock.c | 2 +- app/test/test_stack.c | 4 +- app/test/test_stack_perf.c | 4 +- app/test/test_string_fns.c | 2 +- app/test/test_tailq.c | 2 +- app/test/test_telemetry_data.c | 14 +- app/test/test_thash.c | 2 +- app/test/test_thash_perf.c | 2 +- app/test/test_threads.c | 2 +- app/test/test_ticketlock.c | 2 +- app/test/test_timer.c | 2 +- app/test/test_timer_perf.c | 2 +- app/test/test_timer_racecond.c | 2 +- app/test/test_trace.c | 2 +- app/test/test_trace_perf.c | 2 +- app/test/test_version.c | 2 +- buildtools/get-test-suites.py | 33 + .../has-hugepages.py | 0 buildtools/meson.build | 2 + examples/l3fwd/l3fwd_em.c | 2 + examples/l3fwd/l3fwd_em_hlm.h | 2 + examples/l3fwd/l3fwd_event.c | 2 + examples/l3fwd/l3fwd_event.h | 7 +- examples/l3fwd/l3fwd_event_generic.c | 2 + examples/l3fwd/l3fwd_event_internal_port.c | 2 + examples/l3fwd/l3fwd_fib.c | 2 + examples/l3fwd/l3fwd_lpm.c | 2 + examples/l3fwd/main.c | 65 +- examples/l3fwd/meson.build | 5 +- lib/meson.build | 15 + 125 files changed, 614 insertions(+), 764 deletions(-) create mode 100644 app/test/suites/meson.build create mode 100644 buildtools/get-test-suites.py rename app/test/has_hugepage.py => buildtools/has-hugepages.py (100%) -- 2.39.2