From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6C1045DB6; Wed, 27 Nov 2024 12:27:37 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 030C54042C; Wed, 27 Nov 2024 12:27:35 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 9F4DC40666 for ; Wed, 27 Nov 2024 12:27:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732706853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1PL+plvyxXjW40hzPZLa7TAWnh/z0Aai4uJ5uZGcDus=; b=icEjUG7IRv0Kf3TTreMRZ/QjBp4oHIv4IOVag710hDU/8sJzMHta4O3cFnrQZMMyYgOJwG CxWyswgAsdIwympSTdiIcksz66dDgzEq8FHWqjvXsEpWKdxtvzHPW/KCAmaAHO6KVpLzQO 1n8WtDmlt31jjBcaEVLK+z0jAr4+V50= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-654-0aL5FMZcMwmM0GcHdWCwmg-1; Wed, 27 Nov 2024 06:27:29 -0500 X-MC-Unique: 0aL5FMZcMwmM0GcHdWCwmg-1 X-Mimecast-MFC-AGG-ID: 0aL5FMZcMwmM0GcHdWCwmg Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C41E71956077; Wed, 27 Nov 2024 11:27:28 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.129]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E0B2B1956054; Wed, 27 Nov 2024 11:27:26 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, Aaron Conole , Michael Santana Subject: [RFC 6/6] buildtools: externally check exported headers Date: Wed, 27 Nov 2024 12:26:16 +0100 Message-ID: <20241127112617.1331125-7-david.marchand@redhat.com> In-Reply-To: <20241127112617.1331125-1-david.marchand@redhat.com> References: <20241127112617.1331125-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: cBz5KEwE_pUj7p5D-f1hetc5w6Fs7240GQf6bZPnsu0_1732706848 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org At the moment, the headers check (triggered via the check_includes meson option) is run "internally", iow with compilation flags and include path coming from the meson components. One issue is that both internal and public headers are usually stored in a single directory in the DPDK components. If a lib/foo library exports a header rte_foo.h (iow rte_foo.h is part of the headers list in lib/foo/meson.build) and this rte_foo.h includes an internal header foo_internal.h, then the headers check won't detect such an issue as rte_foo.h is compiled with -Ilib/foo. Reimplement the headers check by inspecting (compiling) all DPDK headers installed in a staging directory. To identify the directory where DPDK headers are, this check relies on pkg-config and skips EAL generic/ and internal/ headers. The existing headers check (though less accurate) is kept as is, as it provides a first level of check and may have existing users. Signed-off-by: David Marchand --- .ci/linux-build.sh | 7 ++-- buildtools/chkincs/Makefile | 77 +++++++++++++++++++++++++++++++++++ devtools/test-meson-builds.sh | 6 ++- 3 files changed, 85 insertions(+), 5 deletions(-) create mode 100644 buildtools/chkincs/Makefile diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index fdb5787621..3146d9ccd8 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -95,8 +95,6 @@ OPTS="$OPTS -Ddefault_library=$DEF_LIB" OPTS="$OPTS -Dbuildtype=$buildtype" if [ "$STDATOMIC" = "true" ]; then OPTS="$OPTS -Denable_stdatomic=true" -else - OPTS="$OPTS -Dcheck_includes=true" fi if [ "$MINI" = "true" ]; then OPTS="$OPTS -Denable_drivers=net/null" @@ -176,13 +174,16 @@ if [ "$RUN_TESTS" = "true" ]; then [ "$failed" != "true" ] fi -# Test examples compilation with an installed dpdk +# Test headers and examples compilation with an installed dpdk if [ "$BUILD_EXAMPLES" = "true" ]; then [ -d install ] || DESTDIR=$(pwd)/install meson install -C build export LD_LIBRARY_PATH=$(dirname $(find $(pwd)/install -name librte_eal.so)):$LD_LIBRARY_PATH export PATH=$(dirname $(find $(pwd)/install -name dpdk-devbind.py)):$PATH export PKG_CONFIG_PATH=$(dirname $(find $(pwd)/install -name libdpdk.pc)):$PKG_CONFIG_PATH export PKGCONF="pkg-config --define-prefix" + + make -C buildtools/chkincs O=build/buildtools/chkincs + find build/examples -maxdepth 1 -type f -name "dpdk-*" | while read target; do target=${target%%:*} diff --git a/buildtools/chkincs/Makefile b/buildtools/chkincs/Makefile new file mode 100644 index 0000000000..838819c617 --- /dev/null +++ b/buildtools/chkincs/Makefile @@ -0,0 +1,77 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2024 Red Hat, Inc. + +ifeq ($(V),) +Q ?= @ +else +Q = +endif + +O ?= build + +PKGCONF ?= pkg-config + +ifneq ($(shell $(PKGCONF) --exists libdpdk && echo 0),0) +$(error "no installation of DPDK found") +endif + +ifeq ($(I),) + +.PHONY: headers_list +.ONESHELL: +headers_list: + $(Q)for dir in $$($(PKGCONF) --cflags-only-I libdpdk); do + dir=$${dir##-I} + [ -e $$dir/rte_build_config.h ] || continue + $(MAKE) I="$$dir" O="$(O)" + break + done +else + +HEADERS := $(shell find $(I) -name "*.h" | grep -vE $(I)'/(generic|internal)/') +SRCS := $(patsubst $(I)/%.h, $(O)/%.c, $(HEADERS)) +.PRECIOUS: $(SRCS) + +PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null) +CFLAGS = $(shell $(PKGCONF) --cflags libdpdk) -Wall -Wextra -Werror +LDFLAGS = $(shell $(PKGCONF) --libs libdpdk) + +OBJS := $(patsubst $(I)/%.h, $(O)/%.o, $(HEADERS)) +OBJS_EXP := $(patsubst $(I)/%.h, $(O)/%+exp.o, $(HEADERS)) +OBJS_ALL := $(patsubst $(I)/%.h, $(O)/%+all.o, $(HEADERS)) + +all: $(OBJS) $(OBJS_EXP) $(OBJS_ALL) + +$(O): + $(Q)mkdir -p $@ + +$(O)/%.c: $(I)/%.h $(O) gen_c_file_for_header.py Makefile + $(Q)python3 gen_c_file_for_header.py $< $@ + +$(O)/%.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CC) $(CFLAGS) -o $@ -c $< + +$(O)/%+exp.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CC) $(CFLAGS) -DALLOW_EXPERIMENTAL_API -o $@ -c $< + +$(O)/%+all.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CC) $(CFLAGS) -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -o $@ -c $< + +CXXFLAGS = $(shell $(PKGCONF) --cflags libdpdk) -Wall -Wextra -Werror + +OBJS_CPP := $(patsubst $(I)/%.h, $(O)/%.cpp.o, $(HEADERS)) +OBJS_CPP_EXP := $(patsubst $(I)/%.h, $(O)/%.cpp+exp.o, $(HEADERS)) +OBJS_CPP_ALL := $(patsubst $(I)/%.h, $(O)/%.cpp+all.o, $(HEADERS)) + +all: $(OBJS_CPP) $(OBJS_CPP_EXP) $(OBJS_CPP_ALL) + +$(O)/%.cpp.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CXX) $(CXXFLAGS) -o $@ -c $< + +$(O)/%.cpp+exp.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CXX) $(CXXFLAGS) -DALLOW_EXPERIMENTAL_API -o $@ -c $< + +$(O)/%.cpp+all.o: $(O)/%.c Makefile $(PC_FILE) + $(Q)$(CXX) $(CXXFLAGS) -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -o $@ -c $< + +endif diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 4fff1f7177..c3cebead70 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -248,7 +248,7 @@ generic_isa='nehalem' if ! check_cc_flags "-march=$generic_isa" ; then generic_isa='corei7' fi -build build-x86-generic cc skipABI -Dcheck_includes=true \ +build build-x86-generic cc skipABI \ -Dlibdir=lib -Dcpu_instruction_set=$generic_isa $use_shared # 32-bit with default compiler @@ -318,9 +318,11 @@ if [ "$examples" = 'all' ]; then done | sort -u | tr '\n' ' ') fi -# if pkg-config defines the necessary flags, test building some examples +# if pkg-config defines the necessary flags, check headers and test building some examples if pkg-config --define-prefix libdpdk >/dev/null 2>&1; then export PKGCONF="pkg-config --define-prefix" + echo "## Checking exported headers" + $MAKE -C buildtools/chkincs O=$build_path/buildtools/chkincs -j for example in $examples; do echo "## Building $example" [ $example = helloworld ] && static=static || static= # save disk space -- 2.47.0