From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFB8FA058A; Fri, 17 Apr 2020 13:47:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 94D641E4B2; Fri, 17 Apr 2020 13:47:42 +0200 (CEST) Received: from relay0209.mxlogin.com (relay0209.mxlogin.com [199.181.239.209]) by dpdk.org (Postfix) with ESMTP id 2528A1E4AA for ; Fri, 17 Apr 2020 13:47:41 +0200 (CEST) Received: from filter003.mxroute.com ([168.235.111.26] 168-235-111-26.cloud.ramnode.com) (Authenticated sender: mN4UYu2MZsgR) by relay0209.mxlogin.com (ZoneMTA) with ESMTPSA id 17187f730090000766.001 for (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256); Fri, 17 Apr 2020 11:47:36 +0000 X-Zone-Loop: 84560d47950745e9c9b86d1f01e3651715b14cfdc1a9 X-Originating-IP: [168.235.111.26] Received: from galaxy.mxroute.com (unknown [23.92.70.113]) by filter003.mxroute.com (Postfix) with ESMTPS id F057C60039; Fri, 17 Apr 2020 11:47:34 +0000 (UTC) Received: from [192.198.151.43] by galaxy.mxroute.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1jPP3q-0006lO-4Z; Fri, 17 Apr 2020 07:21:30 -0400 To: Thomas Monjalon , Neil Horman Cc: dev@dpdk.org, david.marchand@redhat.com References: <20200416145414.262296-1-nhorman@tuxdriver.com> <311553a5-4c65-c312-cd4d-face87245f9a@ashroe.eu> <10824099.L8ug28u51p@thomas> From: Ray Kinsella Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: Date: Fri, 17 Apr 2020 12:47:31 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <10824099.L8ug28u51p@thomas> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCHv3] Remove validate-abi.sh from tree X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 17/04/2020 11:20, Thomas Monjalon wrote: > 17/04/2020 12:11, Ray Kinsella: >> >> On 16/04/2020 15:54, Neil Horman wrote: >>> Since we've moved away from our initial validate-abi.sh script, in >>> favor of check-abi.sh, which uses libabigail, remove the old script from >>> the tree, and update the docs accordingly >>> >>> Signed-off-by: Neil Horman >>> CC: thomas@monjalon.net >>> CC: david.marchand@redhat.com >>> CC: mdr@ashroe.eu >>> --- >>> MAINTAINERS | 1 - >>> devtools/validate-abi.sh | 251 --------------------- >>> doc/guides/contributing/abi_versioning.rst | 50 ++-- >>> 3 files changed, 18 insertions(+), 284 deletions(-) >>> delete mode 100755 devtools/validate-abi.sh >>> >>> diff --git a/MAINTAINERS b/MAINTAINERS >>> index 4800f6884..84b633431 100644 >>> --- a/MAINTAINERS >>> +++ b/MAINTAINERS >>> @@ -152,7 +152,6 @@ F: devtools/gen-abi.sh >>> F: devtools/libabigail.abignore >>> F: devtools/update-abi.sh >>> F: devtools/update_version_map_abi.py >>> -F: devtools/validate-abi.sh >>> F: buildtools/check-experimental-syms.sh >>> F: buildtools/map-list-symbol.sh >>> >>> diff --git a/devtools/validate-abi.sh b/devtools/validate-abi.sh >>> deleted file mode 100755 >>> index f64e19d38..000000000 >>> --- a/devtools/validate-abi.sh >>> +++ /dev/null >>> @@ -1,251 +0,0 @@ >>> -#!/usr/bin/env bash >>> -# SPDX-License-Identifier: BSD-3-Clause >>> -# Copyright(c) 2015 Neil Horman. All rights reserved. >>> -# Copyright(c) 2017 6WIND S.A. >>> -# All rights reserved >>> - >>> -set -e >>> - >>> -abicheck=abi-compliance-checker >>> -abidump=abi-dumper >>> -default_dst=abi-check >>> -default_target=x86_64-native-linuxapp-gcc >>> - >>> -# trap on error >>> -err_report() { >>> - echo "$0: error at line $1" >>> -} >>> -trap 'err_report $LINENO' ERR >>> - >>> -print_usage () { >>> - cat <<- END_OF_HELP >>> - $(basename $0) [options] >>> - >>> - This script compares the ABI of 2 git revisions of the current >>> - workspace. The output is a html report and a compilation log. >>> - >>> - The objective is to make sure that applications built against >>> - DSOs from the first revision can still run when executed using >>> - the DSOs built from the second revision. >>> - >>> - and are git commit id or tags. >>> - >>> - Options: >>> - -h show this help >>> - -j enable parallel compilation with threads >>> - -v show compilation logs on the console >>> - -d change working directory (default is ${default_dst}) >>> - -t the dpdk target to use (default is ${default_target}) >>> - -f overwrite existing files in destination directory >>> - >>> - The script returns 0 on success, or the value of last failing >>> - call of ${abicheck} (incompatible abi or the tool has run with errors). >>> - The errors returned by ${abidump} are ignored. >>> - >>> - END_OF_HELP >>> -} >>> - >>> -# log in the file, and on stdout if verbose >>> -# $1: level string >>> -# $2: string to be logged >>> -log() { >>> - echo "$1: $2" >>> - if [ "${verbose}" != "true" ]; then >>> - echo "$1: $2" >&3 >>> - fi >>> -} >>> - >>> -# launch a command and log it, taking care of surrounding spaces with quotes >>> -cmd() { >>> - local i s whitespace ret >>> - s="" >>> - whitespace="[[:space:]]" >>> - for i in "$@"; do >>> - if [[ $i =~ $whitespace ]]; then >>> - i=\"$i\" >>> - fi >>> - if [ -z "$s" ]; then >>> - s="$i" >>> - else >>> - s="$s $i" >>> - fi >>> - done >>> - >>> - ret=0 >>> - log "CMD" "$s" >>> - "$@" || ret=$? >>> - if [ "$ret" != "0" ]; then >>> - log "CMD" "previous command returned $ret" >>> - fi >>> - >>> - return $ret >>> -} >>> - >>> -# redirect or copy stderr/stdout to a file >>> -# the syntax is unfamiliar, but it makes the rest of the >>> -# code easier to read, avoiding the use of pipes >>> -set_log_file() { >>> - # save original stdout and stderr in fd 3 and 4 >>> - exec 3>&1 >>> - exec 4>&2 >>> - # create a new fd 5 that send to a file >>> - exec 5> >(cat > $1) >>> - # send stdout and stderr to fd 5 >>> - if [ "${verbose}" = "true" ]; then >>> - exec 1> >(tee /dev/fd/5 >&3) >>> - exec 2> >(tee /dev/fd/5 >&4) >>> - else >>> - exec 1>&5 >>> - exec 2>&5 >>> - fi >>> -} >>> - >>> -# Make sure we configure SHARED libraries >>> -# Also turn off IGB and KNI as those require kernel headers to build >>> -fixup_config() { >>> - local conf=config/defconfig_$target >>> - cmd sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" $conf >>> - cmd sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" $conf >>> - cmd sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" $conf >>> - cmd sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" $conf >>> - cmd sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" $conf >>> -} >>> - >>> -# build dpdk for the given tag and dump abi >>> -# $1: hash of the revision >>> -gen_abi() { >>> - local i >>> - >>> - cmd git clone ${dpdkroot} ${dst}/${1} >>> - cmd cd ${dst}/${1} >>> - >>> - log "INFO" "Checking out version ${1} of the dpdk" >>> - # Move to the old version of the tree >>> - cmd git checkout ${1} >>> - >>> - fixup_config >>> - >>> - # Now configure the build >>> - log "INFO" "Configuring DPDK ${1}" >>> - cmd make config T=$target O=$target >>> - >>> - # Checking abi compliance relies on using the dwarf information in >>> - # the shared objects. Build with -g to include them. >>> - log "INFO" "Building DPDK ${1}. This might take a moment" >>> - cmd make -j$parallel O=$target V=1 EXTRA_CFLAGS="-g -Og -Wno-error" \ >>> - EXTRA_LDFLAGS="-g" || log "INFO" "The build failed" >>> - >>> - # Move to the lib directory >>> - cmd cd ${PWD}/$target/lib >>> - log "INFO" "Collecting ABI information for ${1}" >>> - for i in *.so; do >>> - [ -e "$i" ] || break >>> - cmd $abidump ${i} -o $dst/${1}/${i}.dump -lver ${1} || true >>> - # hack to ignore empty SymbolsInfo section (no public ABI) >>> - if grep -q "'SymbolInfo' => {}," $dst/${1}/${i}.dump \ >>> - 2> /dev/null; then >>> - log "INFO" "${i} has no public ABI, remove dump file" >>> - cmd rm -f $dst/${1}/${i}.dump >>> - fi >>> - done >>> -} >>> - >>> -verbose=false >>> -parallel=1 >>> -dst=${default_dst} >>> -target=${default_target} >>> -force=0 >>> -while getopts j:vd:t:fh ARG ; do >>> - case $ARG in >>> - j ) parallel=$OPTARG ;; >>> - v ) verbose=true ;; >>> - d ) dst=$OPTARG ;; >>> - t ) target=$OPTARG ;; >>> - f ) force=1 ;; >>> - h ) print_usage ; exit 0 ;; >>> - ? ) print_usage ; exit 1 ;; >>> - esac >>> -done >>> -shift $(($OPTIND - 1)) >>> - >>> -if [ $# != 2 ]; then >>> - print_usage >>> - exit 1 >>> -fi >>> - >>> -tag1=$1 >>> -tag2=$2 >>> - >>> -# convert path to absolute >>> -case "${dst}" in >>> - /*) ;; >>> - *) dst=${PWD}/${dst} ;; >>> -esac >>> -dpdkroot=$(readlink -f $(dirname $0)/..) >>> - >>> -if [ -e "${dst}" -a "$force" = 0 ]; then >>> - echo "The ${dst} directory is not empty. Remove it, use another" >>> - echo "one (-d ), or force overriding (-f)" >>> - exit 1 >>> -fi >>> - >>> -rm -rf ${dst} >>> -mkdir -p ${dst} >>> -set_log_file ${dst}/abi-check.log >>> -log "INFO" "Logs available in ${dst}/abi-check.log" >>> - >>> -command -v ${abicheck} || log "INFO" "Can't find ${abicheck} utility" >>> -command -v ${abidump} || log "INFO" "Can't find ${abidump} utility" >>> - >>> -hash1=$(git show -s --format=%h "$tag1" -- 2> /dev/null | tail -1) >>> -hash2=$(git show -s --format=%h "$tag2" -- 2> /dev/null | tail -1) >>> - >>> -# Make hashes available in output for non-local reference >>> -tag1="$tag1 ($hash1)" >>> -tag2="$tag2 ($hash2)" >>> - >>> -if [ "$hash1" = "$hash2" ]; then >>> - log "ERROR" "$tag1 and $tag2 are the same revisions" >>> - exit 1 >>> -fi >>> - >>> -cmd mkdir -p ${dst} >>> - >>> -# dump abi for each revision >>> -gen_abi ${hash1} >>> -gen_abi ${hash2} >>> - >>> -# compare the abi dumps >>> -cmd cd ${dst} >>> -ret=0 >>> -list="" >>> -for i in ${hash2}/*.dump; do >>> - name=`basename $i` >>> - libname=${name%.dump} >>> - >>> - if [ ! -f ${hash1}/$name ]; then >>> - log "INFO" "$NAME does not exist in $tag1. skipping..." >>> - continue >>> - fi >>> - >>> - local_ret=0 >>> - cmd $abicheck -l $libname \ >>> - -old ${hash1}/$name -new ${hash2}/$name || local_ret=$? >>> - if [ $local_ret != 0 ]; then >>> - log "NOTICE" "$abicheck returned $local_ret" >>> - ret=$local_ret >>> - list="$list $libname" >>> - fi >>> -done >>> - >>> -if [ $ret != 0 ]; then >>> - log "NOTICE" "ABI may be incompatible, check reports/logs for details." >>> - log "NOTICE" "Incompatible list: $list" >>> -else >>> - log "NOTICE" "No error detected, ABI is compatible." >>> -fi >>> - >>> -log "INFO" "Logs are in ${dst}/abi-check.log" >>> -log "INFO" "HTML reports are in ${dst}/compat_reports directory" >>> - >>> -exit $ret >>> diff --git a/doc/guides/contributing/abi_versioning.rst b/doc/guides/contributing/abi_versioning.rst >>> index a21f4e7a4..219823923 100644 >>> --- a/doc/guides/contributing/abi_versioning.rst >>> +++ b/doc/guides/contributing/abi_versioning.rst >>> @@ -482,41 +482,27 @@ Running the ABI Validator >>> ------------------------- >>> >>> The ``devtools`` directory in the DPDK source tree contains a utility program, >>> -``validate-abi.sh``, for validating the DPDK ABI based on the Linux `ABI >>> -Compliance Checker >>> -`_. >>> +``check-abi.sh``, for validating the DPDK ABI based on the libabigail `abidiff >>> +utility: `_ >>> >>> -This has a dependency on the ``abi-compliance-checker`` and ``and abi-dumper`` >>> -utilities which can be installed via a package manager. For example:: >>> +The syntax of the ``check-abi.sh`` utility is:: >>> >>> - sudo yum install abi-compliance-checker >>> - sudo yum install abi-dumper >>> + ./devtools/check-abi.sh >>> >>> -The syntax of the ``validate-abi.sh`` utility is:: >>> +Where specifies the directory housing the reference build of dpdk, and >>> + specifies the dpdk build directory to check the abi of >>> >>> - ./devtools/validate-abi.sh >>> +Example: >>> +To compare your build branch to the ABI of the master branch >>> +after you have built your branch >>> >>> -Where ``REV1`` and ``REV2`` are valid gitrevisions(7) >>> -https://www.kernel.org/pub/software/scm/git/docs/gitrevisions.html >>> -on the local repo. >>> +#. $ cd >>> +#. $ mkdir ~/ref >>> +#. $ git clone --local --no-hardlinks --single-branch -b master . ~/ref >>> +#. $ cd ~/ref >>> +#. $ cp /.config ./.config >>> +#. $ make defconfig >>> +#. $ make >>> +#. $ /devtools/check-abi.sh ~/ref >>> >>> -For example:: >>> - >>> - # Check between the previous and latest commit: >>> - ./devtools/validate-abi.sh HEAD~1 HEAD >>> - >>> - # Check on a specific compilation target: >>> - ./devtools/validate-abi.sh -t x86_64-native-linux-gcc HEAD~1 HEAD >>> - >>> - # Check between two tags: >>> - ./devtools/validate-abi.sh v2.0.0 v2.1.0 >>> - >>> - # Check between git master and local topic-branch "vhost-hacking": >>> - ./devtools/validate-abi.sh master vhost-hacking >>> - >>> -After the validation script completes (it can take a while since it need to >>> -compile both tags) it will create compatibility reports in the >>> -``./abi-check/compat_report`` directory. Listed incompatibilities can be found >>> -as follows:: >>> - >>> - grep -lr Incompatible abi-check/compat_reports/ >>> + >>> >> >> check-abi.sh appears to be backward step in terms of usability. > > No, check-abi.sh benefits from a nice integration in build scripts. > See below. > >> With validate-abi.sh I do can do a "validate-abi.sh HEAD~1 HEAD". >> And it will do the build, install, dump and comparison for me. >> And it picked up my 20.0.2 - > 21.0 changes no problem. >> >> With check-abi on the other hand, I need to the build and install myself. >> check-abi requires dump files, but I see no reference in the documentation to how these are created. >> It silently fails when it doesn't find any ... >> >> Do I run abi-dumper on the so's myself, or how does it work? > > check-abi.sh is integrated in test-build.sh and test-meson-builds.sh. > Probably we should document usage in these scripts. > Looks like I need to set DPDK_ABI_REF_VERSION=master, not obvious. Any tips or tricks would be welcome.