From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A2BE9463BB; Tue, 11 Mar 2025 10:57:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 932C0402D7; Tue, 11 Mar 2025 10:57:30 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6D71140268 for ; Tue, 11 Mar 2025 10:57:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741687048; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iitE3/xvHctmMRYmapZoPzETw4tFOXHTdpIDUqyK73c=; b=R3vPkPbBuejoac+DKftcOqwkXxIo2CfcxJ9eLlILxRPYZkzhHeTL0xEPIMNF6zMfZWVQQB TuLelYun4juynpSSKuvFY7FaB7XZdusI+36799sJ0kLeuzotsKvXFw+ktEG7IMW+UOj++U VzsfCYYfiJxDvz9ylchQItVbawsdD/U= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-517-XzWU0RGDPV-xYnPbkLPCKQ-1; Tue, 11 Mar 2025 05:57:26 -0400 X-MC-Unique: XzWU0RGDPV-xYnPbkLPCKQ-1 X-Mimecast-MFC-AGG-ID: XzWU0RGDPV-xYnPbkLPCKQ_1741687045 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4722D18001F7; Tue, 11 Mar 2025 09:57:25 +0000 (UTC) Received: from dmarchan.lan (unknown [10.45.225.29]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7FF521955BCB; Tue, 11 Mar 2025 09:57:22 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, andremue@linux.microsoft.com, Aaron Conole , Michael Santana Subject: [RFC v3 7/8] build: use dynamically generated version maps Date: Tue, 11 Mar 2025 10:56:05 +0100 Message-ID: <20250311095609.194523-8-david.marchand@redhat.com> In-Reply-To: <20250311095609.194523-1-david.marchand@redhat.com> References: <20250305212349.2036410-1-david.marchand@redhat.com> <20250311095609.194523-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qupk8hgsS5sn-7T4hYnYZLJWeLa_4NEoCcnpCn_FQxM_1741687045 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Switch to always dynamically generate version maps. As the map files get generated, tooling around checking, converting, updating etc.. static version maps can be removed. Signed-off-by: David Marchand --- .github/workflows/build.yml | 1 - MAINTAINERS | 7 - buildtools/check-symbols.sh | 33 +- buildtools/map-list-symbol.sh | 7 +- buildtools/map_to_win.py | 41 --- buildtools/meson.build | 1 - devtools/check-symbol-change.sh | 186 ----------- devtools/check-symbol-maps.sh | 101 ------ devtools/checkpatches.sh | 2 +- devtools/update-abi.sh | 46 --- devtools/update_version_map_abi.py | 210 ------------ doc/guides/contributing/abi_policy.rst | 21 +- doc/guides/contributing/coding_style.rst | 7 - .../contributing/img/patch_cheatsheet.svg | 303 ++++++++---------- doc/guides/contributing/patches.rst | 6 +- drivers/meson.build | 74 ++--- lib/meson.build | 73 ++--- 17 files changed, 188 insertions(+), 931 deletions(-) delete mode 100644 buildtools/map_to_win.py delete mode 100755 devtools/check-symbol-change.sh delete mode 100755 devtools/check-symbol-maps.sh delete mode 100755 devtools/update-abi.sh delete mode 100755 devtools/update_version_map_abi.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index fba46b920f..e97b5cdb8b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -31,7 +31,6 @@ jobs: failed= devtools/check-doc-vs-code.sh upstream/${{ env.REF_GIT_BRANCH }} || failed=true devtools/check-meson.py || failed=true - devtools/check-symbol-maps.sh || failed=true [ -z "$failed" ] ubuntu-vm-builds: name: ${{ join(matrix.config.*, '-') }} diff --git a/MAINTAINERS b/MAINTAINERS index 04772951d3..9474189035 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -88,7 +88,6 @@ M: Thomas Monjalon F: MAINTAINERS F: devtools/build-dict.sh F: devtools/check-abi.sh -F: devtools/check-abi-version.sh F: devtools/check-doc-vs-code.sh F: devtools/check-dup-includes.sh F: devtools/check-maintainers.sh @@ -96,17 +95,13 @@ F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-spdx-tag.sh F: devtools/check-symbol-change.py -F: devtools/check-symbol-change.sh -F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.sh F: devtools/git-log-fixes.sh F: devtools/load-devel-config F: devtools/parse-flow-support.sh F: devtools/process-iwyu.py -F: devtools/update-abi.sh F: devtools/update-patches.py -F: devtools/update_version_map_abi.py F: devtools/libabigail.abignore F: devtools/words-case.txt F: license/ @@ -166,7 +161,6 @@ M: Tyler Retzlaff F: lib/eal/common/ F: lib/eal/unix/ F: lib/eal/include/ -F: lib/eal/version.map F: doc/guides/prog_guide/env_abstraction_layer.rst F: app/test/test_alarm.c F: app/test/test_atomic.c @@ -396,7 +390,6 @@ Windows support M: Dmitry Kozlyuk M: Tyler Retzlaff F: lib/eal/windows/ -F: buildtools/map_to_win.py F: doc/guides/windows_gsg/ Windows memory allocation diff --git a/buildtools/check-symbols.sh b/buildtools/check-symbols.sh index b8ac24391e..0d6745ec14 100755 --- a/buildtools/check-symbols.sh +++ b/buildtools/check-symbols.sh @@ -7,29 +7,12 @@ OBJFILE=$2 ROOTDIR=$(readlink -f $(dirname $(readlink -f $0))/..) LIST_SYMBOL=$ROOTDIR/buildtools/map-list-symbol.sh -CHECK_SYMBOL_MAPS=$ROOTDIR/devtools/check-symbol-maps.sh - -# added check for "make -C test/" usage -if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ] -then - exit 0 -fi - -if [ -d $MAPFILE ] -then - exit 0 -fi - DUMPFILE=$(mktemp -t dpdk.${0##*/}.objdump.XXXXXX) trap 'rm -f "$DUMPFILE"' EXIT objdump -t $OBJFILE >$DUMPFILE ret=0 -if ! $CHECK_SYMBOL_MAPS $MAPFILE; then - ret=1 -fi - for SYM in `$LIST_SYMBOL -S EXPERIMENTAL $MAPFILE |cut -d ' ' -f 3` do if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && @@ -37,8 +20,7 @@ do $LIST_SYMBOL -s $SYM $MAPFILE | grep -q EXPERIMENTAL then cat >&2 <<- END_OF_MESSAGE - $SYM is not flagged as experimental - but is listed in version map + $SYM is not flagged as experimental but is exported as an experimental symbol Please add __rte_experimental to the definition of $SYM END_OF_MESSAGE ret=1 @@ -53,9 +35,8 @@ for SYM in `awk '{ do $LIST_SYMBOL -S EXPERIMENTAL -s $SYM -q $MAPFILE || { cat >&2 <<- END_OF_MESSAGE - $SYM is flagged as experimental - but is not listed in version map - Please add $SYM to the version map + $SYM is flagged as experimental but is not exported as an experimental symbol + Please add RTE_EXPORT_EXPERIMENTAL_SYMBOL to the definition of $SYM END_OF_MESSAGE ret=1 } @@ -67,8 +48,7 @@ do ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE then cat >&2 <<- END_OF_MESSAGE - $SYM is not flagged as internal - but is listed in version map + $SYM is not flagged as internal but is exported as an internal symbol Please add __rte_internal to the definition of $SYM END_OF_MESSAGE ret=1 @@ -83,9 +63,8 @@ for SYM in `awk '{ do $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { cat >&2 <<- END_OF_MESSAGE - $SYM is flagged as internal - but is not listed in version map - Please add $SYM to the version map + $SYM is flagged as internal but is not exported as an internal symbol + Please add RTE_EXPORT_INTERNAL_SYMBOL to the definition of $SYM END_OF_MESSAGE ret=1 } diff --git a/buildtools/map-list-symbol.sh b/buildtools/map-list-symbol.sh index 0829df4be5..962d5f3271 100755 --- a/buildtools/map-list-symbol.sh +++ b/buildtools/map-list-symbol.sh @@ -42,7 +42,6 @@ for file in $@; do cat "$file" |awk ' BEGIN { current_section = ""; - current_version = ""; if ("'$section'" == "all" && "'$symbol'" == "all" && "'$version'" == "") { ret = 0; } else { @@ -54,15 +53,11 @@ for file in $@; do current_section = $1; } } - /.*}/ { current_section = ""; current_version = ""; } - /^\t# added in / { - current_version=$4; - } + /.*}/ { current_section = ""; } /^[^}].*[^:*];/ { if (current_section == "") { next; } - symbol_version = current_version if (/^[^}].*[^:*]; # added in /) { symbol_version = $5 } diff --git a/buildtools/map_to_win.py b/buildtools/map_to_win.py deleted file mode 100644 index aa1752cacd..0000000000 --- a/buildtools/map_to_win.py +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2019 Intel Corporation - -import sys - - -def is_function_line(ln): - return ln.startswith('\t') and ln.endswith(';\n') and ":" not in ln and "# WINDOWS_NO_EXPORT" not in ln - -# MinGW keeps the original .map file but replaces per_lcore* to __emutls_v.per_lcore* -def create_mingw_map_file(input_map, output_map): - with open(input_map) as f_in, open(output_map, 'w') as f_out: - f_out.writelines([lines.replace('per_lcore', '__emutls_v.per_lcore') for lines in f_in.readlines()]) - -def main(args): - if not args[1].endswith('version.map') or \ - not args[2].endswith('exports.def') and \ - not args[2].endswith('mingw.map'): - return 1 - - if args[2].endswith('mingw.map'): - create_mingw_map_file(args[1], args[2]) - return 0 - -# generate def file from map file. -# This works taking indented lines only which end with a ";" and which don't -# have a colon in them, i.e. the lines defining functions only. - else: - with open(args[1]) as f_in: - functions = [ln[:-2] + '\n' for ln in sorted(f_in.readlines()) - if is_function_line(ln)] - functions = ["EXPORTS\n"] + functions - - with open(args[2], 'w') as f_out: - f_out.writelines(functions) - return 0 - - -if __name__ == "__main__": - sys.exit(main(sys.argv)) diff --git a/buildtools/meson.build b/buildtools/meson.build index b745e9afa4..1cd1ce02fd 100644 --- a/buildtools/meson.build +++ b/buildtools/meson.build @@ -18,7 +18,6 @@ endif echo = py3 + ['-c', 'import sys; print(*sys.argv[1:])'] gen_version_map = py3 + files('gen-version-map.py') list_dir_globs = py3 + files('list-dir-globs.py') -map_to_win_cmd = py3 + files('map_to_win.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') get_cpu_count_cmd = py3 + files('get-cpu-count.py') get_numa_count_cmd = py3 + files('get-numa-count.py') diff --git a/devtools/check-symbol-change.sh b/devtools/check-symbol-change.sh deleted file mode 100755 index 8992214ac8..0000000000 --- a/devtools/check-symbol-change.sh +++ /dev/null @@ -1,186 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2018 Neil Horman - -build_map_changes() -{ - local fname="$1" - local mapdb="$2" - - cat "$fname" | awk ' - # Initialize our variables - BEGIN {map="";sym="";ar="";sec=""; in_sec=0; in_map=0} - - # Anything that starts with + or -, followed by an a - # and ends in the string .map is the name of our map file - # This may appear multiple times in a patch if multiple - # map files are altered, and all section/symbol names - # appearing between a triggering of this rule and the - # next trigger of this rule are associated with this file - /[-+] [ab]\/.*\.map/ {map=$2; in_map=1; next} - - # The previous rule catches all .map files, anything else - # indicates we left the map chunk. - /[-+] [ab]\// {in_map=0} - - # Triggering this rule, which starts a line and ends it - # with a { identifies a versioned section. The section name is - # the rest of the line with the + and { symbols removed. - # Triggering this rule sets in_sec to 1, which actives the - # symbol rule below - /^.*{/ { - gsub("+", ""); - if (in_map == 1) { - sec=$(NF-1); in_sec=1; - } - } - - # This rule identifies the end of a section, and disables the - # symbol rule - /.*}/ {in_sec=0} - - # This rule matches on a + followed by any characters except a : - # (which denotes a global vs local segment), and ends with a ;. - # The semicolon is removed and the symbol is printed with its - # association file name and version section, along with an - # indicator that the symbol is a new addition. Note this rule - # only works if we have found a version section in the rule - # above (hence the in_sec check) And found a map file (the - # in_map check). If we are not in a map chunk, do nothing. If - # we are in a map chunk but not a section chunk, record it as - # unknown. - /^+[^}].*[^:*];/ {gsub(";","");sym=$2; - if (in_map == 1) { - if (in_sec == 1) { - print map " " sym " " sec " add" - } else { - print map " " sym " unknown add" - } - } - } - - # This is the same rule as above, but the rule matches on a - # leading - rather than a +, denoting that the symbol is being - # removed. - /^-[^}].*[^:*];/ {gsub(";","");sym=$2; - if (in_map == 1) { - if (in_sec == 1) { - print map " " sym " " sec " del" - } else { - print map " " sym " unknown del" - } - } - }' > "$mapdb" - - sort -u "$mapdb" > "$mapdb.2" - mv -f "$mapdb.2" "$mapdb" - -} - -is_stable_section() { - [ "$1" != 'EXPERIMENTAL' ] && [ "$1" != 'INTERNAL' ] -} - -check_for_rule_violations() -{ - local mapdb="$1" - local mname - local symname - local secname - local ar - local ret=0 - - while read mname symname secname ar - do - if [ "$ar" = "add" ] - then - - if [ "$secname" = "unknown" ] - then - # Just inform the user of this occurrence, but - # don't flag it as an error - echo -n "INFO: symbol $symname is added but " - echo -n "patch has insufficient context " - echo -n "to determine the section name " - echo -n "please ensure the version is " - echo "EXPERIMENTAL" - continue - fi - - oldsecname=$(sed -n \ - "s#$mname $symname \(.*\) del#\1#p" "$mapdb") - - # A symbol can not enter a stable section directly - if [ -z "$oldsecname" ] - then - if ! is_stable_section $secname - then - echo -n "INFO: symbol $symname has " - echo -n "been added to the " - echo -n "$secname section of the " - echo "version map" - continue - else - echo -n "ERROR: symbol $symname " - echo -n "is added in the $secname " - echo -n "section, but is expected to " - echo -n "be added in the EXPERIMENTAL " - echo "section of the version map" - ret=1 - continue - fi - fi - - # This symbol is moving inside a section, nothing to do - if [ "$oldsecname" = "$secname" ] - then - continue - fi - - # This symbol is moving between two sections (the - # original section is a stable section). - # This can be legit, just warn. - if is_stable_section $oldsecname - then - echo -n "INFO: symbol $symname is being " - echo -n "moved from $oldsecname to $secname. " - echo -n "Ensure that it has gone through the " - echo "deprecation process" - continue - fi - else - - if ! grep -q "$mname $symname .* add" "$mapdb" && \ - is_stable_section $secname - then - # Just inform users that stable - # symbols need to go through a deprecation - # process - echo -n "INFO: symbol $symname is being " - echo -n "removed, ensure that it has " - echo "gone through the deprecation process" - fi - fi - done < "$mapdb" - - return $ret -} - -trap clean_and_exit_on_sig EXIT - -mapfile=`mktemp -t dpdk.mapdb.XXXXXX` -patch=$1 -exit_code=1 - -clean_and_exit_on_sig() -{ - rm -f "$mapfile" - exit $exit_code -} - -build_map_changes "$patch" "$mapfile" -check_for_rule_violations "$mapfile" -exit_code=$? -rm -f "$mapfile" - -exit $exit_code diff --git a/devtools/check-symbol-maps.sh b/devtools/check-symbol-maps.sh deleted file mode 100755 index fcd3931e5d..0000000000 --- a/devtools/check-symbol-maps.sh +++ /dev/null @@ -1,101 +0,0 @@ -#! /bin/sh -e -# SPDX-License-Identifier: BSD-3-Clause -# Copyright 2018 Mellanox Technologies, Ltd - -cd $(dirname $0)/.. - -# speed up by ignoring Unicode details -export LC_ALL=C - -if [ $# = 0 ] ; then - set -- $(find lib drivers -name '*.map' -a ! -path drivers/version.map) -fi - -ret=0 - -find_orphan_symbols () -{ - for map in $@ ; do - for sym in $(sed -rn 's,^([^}]*_.*);.*$,\1,p' $map) ; do - if echo $sym | grep -q '^per_lcore_' ; then - symsrc=${sym#per_lcore_} - elif echo $sym | grep -q '^__rte_.*_trace_' ; then - symsrc=${sym#__} - else - symsrc=$sym - fi - if [ -z "$(grep -rlw $symsrc $(dirname $map) | grep -v $map)" ] ; then - echo "$map: $sym" - fi - done - done -} - -orphan_symbols=$(find_orphan_symbols $@) -if [ -n "$orphan_symbols" ] ; then - echo "Found only in symbol map file:" - echo "$orphan_symbols" | sed 's,^,\t,' - ret=1 -fi - -find_duplicate_symbols () -{ - for map in $@ ; do - buildtools/map-list-symbol.sh $map | \ - sort | uniq -c | grep -v " 1 $map" || true - done -} - -duplicate_symbols=$(find_duplicate_symbols $@) -if [ -n "$duplicate_symbols" ] ; then - echo "Found duplicates in symbol map file:" - echo "$duplicate_symbols" - ret=1 -fi - -local_miss_maps=$(grep -L 'local: \*;' $@ || true) -if [ -n "$local_miss_maps" ] ; then - echo "Found maps without local catch-all:" - echo "$local_miss_maps" - ret=1 -fi - -find_bad_format_maps () -{ - abi_version=$(cut -d'.' -f 1 ABI_VERSION) - next_abi_version=$((abi_version + 1)) - for map in $@ ; do - cat $map | awk ' - /^(DPDK_('$abi_version'|'$next_abi_version')|EXPERIMENTAL|INTERNAL) \{$/ { next; } # start of a section - /^}( DPDK_'$abi_version')?;$/ { next; } # end of a section - /^$/ { next; } # empty line - /^\t(global:|local: \*;)$/ { next; } # qualifiers - /^\t[a-zA-Z_0-9]*;( # WINDOWS_NO_EXPORT)?$/ { next; } # symbols - /^\t# added in [0-9]*\.[0-9]*$/ { next; } # version comments - { print $0; }' || echo $map - done -} - -bad_format_maps=$(find_bad_format_maps $@) -if [ -n "$bad_format_maps" ] ; then - echo "Found badly formatted maps:" - echo "$bad_format_maps" - ret=1 -fi - -find_non_versioned_maps () -{ - for map in $@ ; do - [ $(buildtools/map-list-symbol.sh -S EXPERIMENTAL -V unset $map | wc -l) = '0' ] || - echo $map - done -} - -non_versioned_maps=$(find_non_versioned_maps $@) -if [ -n "$non_versioned_maps" ] ; then - echo "Found non versioned maps:" - echo "$non_versioned_maps" - ret=1 -fi - -exit $ret diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh index 7dcac7c8c9..1f3c551b31 100755 --- a/devtools/checkpatches.sh +++ b/devtools/checkpatches.sh @@ -9,7 +9,7 @@ # - DPDK_CHECKPATCH_OPTIONS . $(dirname $(readlink -f $0))/load-devel-config -VALIDATE_NEW_API=$(dirname $(readlink -f $0))/check-symbol-change.sh +VALIDATE_NEW_API=$(dirname $(readlink -f $0))/check-symbol-change.py # Enable codespell by default. This can be overwritten from a config file. # Codespell can also be enabled by setting DPDK_CHECKPATCH_CODESPELL to a valid path diff --git a/devtools/update-abi.sh b/devtools/update-abi.sh deleted file mode 100755 index 45437f3c3b..0000000000 --- a/devtools/update-abi.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/sh -e -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2019 Intel Corporation - -abi_version=$1 -abi_version_file="./ABI_VERSION" -update_path="lib drivers" - -# check ABI version format string -check_abi_version() { - echo $1 | grep -q -e "^[[:digit:]]\{1,2\}\.[[:digit:]]\{1,2\}$" -} - -if [ -z "$1" ]; then - # output to stderr - >&2 echo "Please provide ABI version" - exit 1 -fi - -# check version string format -if ! check_abi_version $abi_version ; then - # output to stderr - >&2 echo "ABI version must be formatted as MAJOR.MINOR version" - exit 1 -fi - -if [ -n "$2" ]; then - abi_version_file=$2 -fi - -if [ -n "$3" ]; then - # drop $1 and $2 - shift 2 - # assign all other arguments as update paths - update_path=$@ -fi - -echo "New ABI version:" $abi_version -echo "ABI_VERSION path:" $abi_version_file -echo "Path to update:" $update_path - -echo $abi_version > $abi_version_file - -find $update_path -name version.map -exec \ - devtools/update_version_map_abi.py {} \ - $abi_version \; -print diff --git a/devtools/update_version_map_abi.py b/devtools/update_version_map_abi.py deleted file mode 100755 index d17b02a327..0000000000 --- a/devtools/update_version_map_abi.py +++ /dev/null @@ -1,210 +0,0 @@ -#!/usr/bin/env python3 -# SPDX-License-Identifier: BSD-3-Clause -# Copyright(c) 2019 Intel Corporation - -""" -A Python program that updates and merges all available stable ABI versions into -one ABI version, while leaving experimental ABI exactly as it is. The intended -ABI version is supplied via command-line parameter. This script is to be called -from the devtools/update-abi.sh utility. -""" - -import argparse -import sys -import re - - -def __parse_map_file(f_in): - # match function name, followed by semicolon, followed by EOL or comments, - # optionally with whitespace in between each item - func_line_regex = re.compile(r"\s*" - r"(?P" - r"(?P[a-zA-Z_0-9]+)" - r"\s*" - r";" - r"\s*" - r"(?P#.+)?" - r")" - r"\s*" - r"$") - # match section name, followed by opening bracked, followed by EOL, - # optionally with whitespace in between each item - section_begin_regex = re.compile(r"\s*" - r"(?P[a-zA-Z0-9_\.]+)" - r"\s*" - r"{" - r"\s*" - r"$") - # match closing bracket, optionally followed by section name (for when we - # inherit from another ABI version), followed by semicolon, followed by - # EOL, optionally with whitespace in between each item - section_end_regex = re.compile(r"\s*" - r"}" - r"\s*" - r"(?P[a-zA-Z0-9_\.]+)?" - r"\s*" - r";" - r"\s*" - r"$") - - # for stable ABI, we don't care about which version introduced which - # function, we just flatten the list. there are dupes in certain files, so - # use a set instead of a list - stable_lines = set() - # copy experimental section as is - experimental_lines = [] - # copy internal section as is - internal_lines = [] - in_experimental = False - in_internal = False - has_stable = False - - # gather all functions - for line in f_in: - # clean up the line - line = line.strip('\n').strip() - - # is this an end of section? - match = section_end_regex.match(line) - if match: - # whatever section this was, it's not active any more - in_experimental = False - in_internal = False - continue - - # if we're in the middle of experimental section, we need to copy - # the section verbatim, so just add the line - if in_experimental: - experimental_lines += [line] - continue - - # if we're in the middle of internal section, we need to copy - # the section verbatim, so just add the line - if in_internal: - internal_lines += [line] - continue - - # skip empty lines - if not line: - continue - - # is this a beginning of a new section? - match = section_begin_regex.match(line) - if match: - cur_section = match.group("version") - # is it experimental? - in_experimental = cur_section == "EXPERIMENTAL" - # is it internal? - in_internal = cur_section == "INTERNAL" - if not in_experimental and not in_internal: - has_stable = True - continue - - # is this a function? - match = func_line_regex.match(line) - if match: - stable_lines.add(match.group("line")) - - return has_stable, stable_lines, experimental_lines, internal_lines - - -def __generate_stable_abi(f_out, abi_major, lines): - # print ABI version header - print("DPDK_{} {{".format(abi_major), file=f_out) - - # print global section if it exists - if lines: - print("\tglobal:", file=f_out) - # blank line - print(file=f_out) - - # print all stable lines, alphabetically sorted - for line in sorted(lines): - print("\t{}".format(line), file=f_out) - - # another blank line - print(file=f_out) - - # print local section - print("\tlocal: *;", file=f_out) - - # end stable version - print("};", file=f_out) - - -def __generate_experimental_abi(f_out, lines): - # start experimental section - print("EXPERIMENTAL {", file=f_out) - - # print all experimental lines as they were - for line in lines: - # don't print empty whitespace - if not line: - print("", file=f_out) - else: - print("\t{}".format(line), file=f_out) - - # end section - print("};", file=f_out) - -def __generate_internal_abi(f_out, lines): - # start internal section - print("INTERNAL {", file=f_out) - - # print all internal lines as they were - for line in lines: - # don't print empty whitespace - if not line: - print("", file=f_out) - else: - print("\t{}".format(line), file=f_out) - - # end section - print("};", file=f_out) - -def __main(): - arg_parser = argparse.ArgumentParser( - description='Merge versions in linker version script.') - - arg_parser.add_argument("map_file", type=str, - help='path to linker version script file ' - '(pattern: version.map)') - arg_parser.add_argument("abi_version", type=str, - help='target ABI version (pattern: MAJOR.MINOR)') - - parsed = arg_parser.parse_args() - - if not parsed.map_file.endswith('version.map'): - print("Invalid input file: {}".format(parsed.map_file), - file=sys.stderr) - arg_parser.print_help() - sys.exit(1) - - if not re.match(r"\d{1,2}\.\d{1,2}", parsed.abi_version): - print("Invalid ABI version: {}".format(parsed.abi_version), - file=sys.stderr) - arg_parser.print_help() - sys.exit(1) - abi_major = parsed.abi_version.split('.')[0] - - with open(parsed.map_file) as f_in: - has_stable, stable_lines, experimental_lines, internal_lines = __parse_map_file(f_in) - - with open(parsed.map_file, 'w') as f_out: - need_newline = has_stable and experimental_lines - if has_stable: - __generate_stable_abi(f_out, abi_major, stable_lines) - if need_newline: - # separate sections with a newline - print(file=f_out) - if experimental_lines: - __generate_experimental_abi(f_out, experimental_lines) - if internal_lines: - if has_stable or experimental_lines: - # separate sections with a newline - print(file=f_out) - __generate_internal_abi(f_out, internal_lines) - - -if __name__ == "__main__": - __main() diff --git a/doc/guides/contributing/abi_policy.rst b/doc/guides/contributing/abi_policy.rst index d96153c6b2..f03a7467ac 100644 --- a/doc/guides/contributing/abi_policy.rst +++ b/doc/guides/contributing/abi_policy.rst @@ -330,31 +330,14 @@ become part of a tracked ABI version. Note that marking an API as experimental is a multi step process. To mark an API as experimental, the symbols which are desired to be exported -must be placed in an EXPERIMENTAL version block in the corresponding libraries' -version map script. +must be annotated with a RTE_EXPORT_EXPERIMENTAL_SYMBOL call in the corresponding libraries' +sources. Experimental symbols must be commented so that it is clear in which DPDK version they were introduced. -.. code-block:: none - - EXPERIMENTAL { - global: - - # added in 20.11 - rte_foo_init; - rte_foo_configure; - - # added in 21.02 - rte_foo_cleanup; - ... - Secondly, the corresponding prototypes of those exported functions (in the development header files), must be marked with the ``__rte_experimental`` tag (see ``rte_compat.h``). -The DPDK build makefiles perform a check to ensure that the map file and the -C code reflect the same list of symbols. -This check can be circumvented by defining ``ALLOW_EXPERIMENTAL_API`` -during compilation in the corresponding library Makefile. In addition to tagging the code with ``__rte_experimental``, the doxygen markup must also contain the EXPERIMENTAL string, diff --git a/doc/guides/contributing/coding_style.rst b/doc/guides/contributing/coding_style.rst index 1ebc79ca3c..43e27bbd0a 100644 --- a/doc/guides/contributing/coding_style.rst +++ b/doc/guides/contributing/coding_style.rst @@ -1018,13 +1018,6 @@ name sources are stored in a directory ``lib/xyz``, this value should never be needed for new libraries. -.. note:: - - The name value also provides the name used to find the function version - map file, as part of the build process, so if the directory name and - library names differ, the ``version.map`` file should be named - consistently with the library, not the directory - objs **Default Value = []**. This variable can be used to pass to the library build some pre-built diff --git a/doc/guides/contributing/img/patch_cheatsheet.svg b/doc/guides/contributing/img/patch_cheatsheet.svg index 4debb07b98..a06d8a2a3b 100644 --- a/doc/guides/contributing/img/patch_cheatsheet.svg +++ b/doc/guides/contributing/img/patch_cheatsheet.svg @@ -1,18 +1,18 @@ + inkscape:version="1.4 (e7c3feb100, 2024-10-09)" + sodipodi:docname="patch_cheatsheet.svg" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns="http://www.w3.org/2000/svg" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:dc="http://purl.org/dc/elements/1.1/"> + inkscape:snap-grids="false" + inkscape:showpageshadow="2" + inkscape:pagecheckerboard="0" + inkscape:deskcolor="#d1d1d1" + inkscape:document-units="mm" /> **** + x="460.18405">*** * + + Rebase to git + + Checkpatch + + ABI breakage + + Maintainers file + + Release notes + + Documentation - The version.map function names must be in alphabetical order. - * - * - - + transform="matrix(1.0211743,0,0,1,25.427515,-31.583927)" + id="g3334"> + id="g3267" + transform="translate(-13.517932,3.1531035)"> + Rebase to git - + Checkpatch - + ABI breakage - + + + Maintainers file - + + + + Release notes + y="541.38928" + id="tspan4092-8-5-5-3-4-0-7" + style="font-style:normal;font-variant:normal;font-weight:300;font-stretch:normal;font-size:21px;line-height:125%;font-family:monospace;-inkscape-font-specification:'Monospace Bold';text-align:start;writing-mode:lr-tb;text-anchor:start">+ Doxygen + + Documentation - - - ** - - + Update version.map - - - + Doxygen - - *** - - + style="font-style:normal;font-weight:normal;font-size:25.6917px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none">** + * + style="font-style:normal;font-variant:normal;font-weight:300;font-stretch:normal;font-size:13px;line-height:125%;font-family:monospace;-inkscape-font-specification:'Monospace Bold';text-align:start;writing-mode:lr-tb;text-anchor:start" /> New header files must get a new page in the API docs. * + style="font-style:normal;font-variant:normal;font-weight:300;font-stretch:normal;font-size:13px;line-height:125%;font-family:monospace;-inkscape-font-specification:'Monospace Bold';text-align:start;writing-mode:lr-tb;text-anchor:start" /> ` and :ref:`ABI versioning ` - guides. New external functions should also be added in alphabetical order. +* New external functions should be exported. + See the :doc:`ABI policy ` and :ref:`ABI versioning ` + guides. * Any new API function should be used in ``/app`` test directory. diff --git a/drivers/meson.build b/drivers/meson.build index c8bc556f1a..6904d34eee 100644 --- a/drivers/meson.build +++ b/drivers/meson.build @@ -5,8 +5,6 @@ if is_ms_compiler subdir_done() endif -fs = import('fs') - # Defines the order of dependencies evaluation subdirs = [ 'common', @@ -260,59 +258,27 @@ foreach subpath:subdirs install: true) # now build the shared driver - version_map = '@0@/@1@/version.map'.format(meson.current_source_dir(), drv_path) - - if not fs.is_file(version_map) - if is_ms_linker - link_mode = 'msvc' - elif is_windows - link_mode = 'mingw' - else - link_mode = 'gnu' - endif - version_map = custom_target(lib_name + '_map', - command: [gen_version_map, link_mode, abi_version_file, '@OUTPUT@', '@INPUT@'], - input: sources, - output: 'lib@0@_exports.map'.format(lib_name)) - version_map_path = version_map.full_path() - version_map_dep = [version_map] - lk_deps = [version_map] - - if is_ms_linker - if is_ms_compiler - lk_args = ['/def:' + version_map.full_path()] - else - lk_args = ['-Wl,/def:' + version_map.full_path()] - endif - else - lk_args = ['-Wl,--version-script=' + version_map.full_path()] - endif + if is_ms_linker + link_mode = 'msvc' + elif is_windows + link_mode = 'mingw' else - version_map_path = version_map - version_map_dep = [] - lk_deps = [version_map] - - if is_windows - if is_ms_linker - def_file = custom_target(lib_name + '_def', - command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'], - input: version_map, - output: '@0@_exports.def'.format(lib_name)) - lk_deps += [def_file] - - lk_args = ['-Wl,/def:' + def_file.full_path()] - else - mingw_map = custom_target(lib_name + '_mingw', - command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'], - input: version_map, - output: '@0@_mingw.map'.format(lib_name)) - lk_deps += [mingw_map] - - lk_args = ['-Wl,--version-script=' + mingw_map.full_path()] - endif + link_mode = 'gnu' + endif + version_map = custom_target(lib_name + '_map', + command: [gen_version_map, link_mode, abi_version_file, '@OUTPUT@', '@INPUT@'], + input: sources, + output: 'lib@0@_exports.map'.format(lib_name)) + lk_deps = [version_map] + + if is_ms_linker + if is_ms_compiler + lk_args = ['/def:' + version_map.full_path()] else - lk_args = ['-Wl,--version-script=' + version_map] + lk_args = ['-Wl,/def:' + version_map.full_path()] endif + else + lk_args = ['-Wl,--version-script=' + version_map.full_path()] endif if not is_windows and developer_mode @@ -320,11 +286,11 @@ foreach subpath:subdirs # check-symbols.sh script, using it as a # dependency of the .so build lk_deps += custom_target(lib_name + '.sym_chk', - command: [check_symbols, version_map_path, '@INPUT@'], + command: [check_symbols, version_map.full_path(), '@INPUT@'], capture: true, input: static_lib, output: lib_name + '.sym_chk', - depends: version_map_dep) + depends: [version_map]) endif shared_lib = shared_library(lib_name, sources_pmd_info, diff --git a/lib/meson.build b/lib/meson.build index b6bac02b48..f143bc202b 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -1,7 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017-2019 Intel Corporation -fs = import('fs') # process all libraries equally, as far as possible # "core" libs first, then others alphabetically as far as possible @@ -255,61 +254,27 @@ foreach l:libraries include_directories: includes, dependencies: static_deps) - if not fs.is_file('@0@/@1@/version.map'.format(meson.current_source_dir(), l)) - if is_ms_linker - link_mode = 'msvc' - elif is_windows - link_mode = 'mingw' - else - link_mode = 'gnu' - endif - version_map = custom_target(libname + '_map', - command: [gen_version_map, link_mode, abi_version_file, '@OUTPUT@', '@INPUT@'], - input: sources, - output: 'lib@0@_exports.map'.format(libname)) - version_map_path = version_map.full_path() - version_map_dep = [version_map] - lk_deps = [version_map] - - if is_ms_linker - if is_ms_compiler - lk_args = ['/def:' + version_map.full_path()] - else - lk_args = ['-Wl,/def:' + version_map.full_path()] - endif - else - lk_args = ['-Wl,--version-script=' + version_map.full_path()] - endif + if is_ms_linker + link_mode = 'msvc' + elif is_windows + link_mode = 'mingw' else - version_map = '@0@/@1@/version.map'.format(meson.current_source_dir(), l) - version_map_path = version_map - version_map_dep = [] - lk_deps = [version_map] - if is_ms_linker - def_file = custom_target(libname + '_def', - command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'], - input: version_map, - output: '@0@_exports.def'.format(libname)) - lk_deps += [def_file] + link_mode = 'gnu' + endif + version_map = custom_target(libname + '_map', + command: [gen_version_map, link_mode, abi_version_file, '@OUTPUT@', '@INPUT@'], + input: sources, + output: 'lib@0@_exports.map'.format(libname)) + lk_deps = [version_map] - if is_ms_compiler - lk_args = ['/def:' + def_file.full_path()] - else - lk_args = ['-Wl,/def:' + def_file.full_path()] - endif + if is_ms_linker + if is_ms_compiler + lk_args = ['/def:' + version_map.full_path()] else - if is_windows - mingw_map = custom_target(libname + '_mingw', - command: [map_to_win_cmd, '@INPUT@', '@OUTPUT@'], - input: version_map, - output: '@0@_mingw.map'.format(libname)) - lk_deps += [mingw_map] - - lk_args = ['-Wl,--version-script=' + mingw_map.full_path()] - else - lk_args = ['-Wl,--version-script=' + version_map] - endif + lk_args = ['-Wl,/def:' + version_map.full_path()] endif + else + lk_args = ['-Wl,--version-script=' + version_map.full_path()] endif if developer_mode and not is_windows @@ -317,11 +282,11 @@ foreach l:libraries # check-symbols.sh script, using it as a # dependency of the .so build lk_deps += custom_target(name + '.sym_chk', - command: [check_symbols, version_map_path, '@INPUT@'], + command: [check_symbols, version_map.full_path(), '@INPUT@'], capture: true, input: static_lib, output: name + '.sym_chk', - depends: version_map_dep) + depends: [version_map]) endif if not use_function_versioning or is_windows -- 2.48.1