From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2ECA1A0597; Thu, 9 Apr 2020 12:45:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 72F8C1C22B; Thu, 9 Apr 2020 12:45:40 +0200 (CEST) Received: from qrelay242.mxroute.com (qrelay242.mxroute.com [172.82.139.242]) by dpdk.org (Postfix) with ESMTP id 9E2971C22A for ; Thu, 9 Apr 2020 12:45:39 +0200 (CEST) Received: from filter003.mxroute.com ([168.235.111.26] 168-235-111-26.cloud.ramnode.com) (Authenticated sender: mN4UYu2MZsgR) by qrelay242.mxroute.com (ZoneMTA) with ESMTPA id 1715e8b90ea0000d83.001 for ; Thu, 09 Apr 2020 10:45:37 +0000 X-Zone-Loop: 9796db141b94146d8f10dd18d62eb847aa4d2111ea77 X-Originating-IP: [168.235.111.26] Received: from galaxy.mxroute.com (unknown [23.92.70.113]) by filter003.mxroute.com (Postfix) with ESMTPS id D2B1E60038; Thu, 9 Apr 2020 10:45:35 +0000 (UTC) Received: from [134.191.227.39] by galaxy.mxroute.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1jMUI7-0000QJ-6I; Thu, 09 Apr 2020 06:20:11 -0400 To: Bruce Richardson , Neil Horman Cc: dev@dpdk.org, thomas@monjalon.net, david.marchand@redhat.com References: <20200408195616.335004-1-nhorman@tuxdriver.com> <76a8e730-7b4c-0c10-d72e-9205760f06a9@ashroe.eu> <20200409103954.GA375185@hmswarspite.think-freely.org> <20200409104317.GB613@bricha3-MOBL.ger.corp.intel.com> From: Ray Kinsella Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: <926b5dce-1904-5fbf-39ce-8c08661b83d1@ashroe.eu> Date: Thu, 9 Apr 2020 11:45:32 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <20200409104317.GB613@bricha3-MOBL.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCHv2] Remove validate-abi.sh from tree X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 09/04/2020 11:43, Bruce Richardson wrote: > On Thu, Apr 09, 2020 at 06:39:54AM -0400, Neil Horman wrote: >> On Thu, Apr 09, 2020 at 08:57:34AM +0100, Ray Kinsella wrote: >>> >>> >>> On 08/04/2020 20:56, Neil Horman wrote: >>>> Since we've moved away from our initial validate-abi.sh script, in >>>> favor of check-abi.sh, which uses libabigail, remove the old script from >>>> the tree, and update the docs accordingly >>>> >>>> Signed-off-by: Neil Horman >>>> CC: thomas@monjalon.net >>>> CC: david.marchand@redhat.com >>>> CC: mdr@ashroe.eu >>>> --- >>>> MAINTAINERS | 1 - >>>> devtools/validate-abi.sh | 251 --------------------- >>>> doc/guides/contributing/abi_versioning.rst | 61 ++--- >>>> 3 files changed, 23 insertions(+), 290 deletions(-) >>>> delete mode 100755 devtools/validate-abi.sh >>>> >>>> diff --git a/MAINTAINERS b/MAINTAINERS >>>> index 4800f6884..84b633431 100644 >>>> --- a/MAINTAINERS >>>> +++ b/MAINTAINERS >>>> @@ -152,7 +152,6 @@ F: devtools/gen-abi.sh >>>> F: devtools/libabigail.abignore >>>> F: devtools/update-abi.sh >>>> F: devtools/update_version_map_abi.py >>>> -F: devtools/validate-abi.sh >>>> F: buildtools/check-experimental-syms.sh >>>> F: buildtools/map-list-symbol.sh >>>> >>>> diff --git a/devtools/validate-abi.sh b/devtools/validate-abi.sh >>>> deleted file mode 100755 >>>> index f64e19d38..000000000 >>>> --- a/devtools/validate-abi.sh >>>> +++ /dev/null >>>> @@ -1,251 +0,0 @@ >>>> -#!/usr/bin/env bash >>>> -# SPDX-License-Identifier: BSD-3-Clause >>>> -# Copyright(c) 2015 Neil Horman. All rights reserved. >>>> -# Copyright(c) 2017 6WIND S.A. >>>> -# All rights reserved >>>> - >>>> -set -e >>>> - >>>> -abicheck=abi-compliance-checker >>>> -abidump=abi-dumper >>>> -default_dst=abi-check >>>> -default_target=x86_64-native-linuxapp-gcc >>>> - >>>> -# trap on error >>>> -err_report() { >>>> - echo "$0: error at line $1" >>>> -} >>>> -trap 'err_report $LINENO' ERR >>>> - >>>> -print_usage () { >>>> - cat <<- END_OF_HELP >>>> - $(basename $0) [options] >>>> - >>>> - This script compares the ABI of 2 git revisions of the current >>>> - workspace. The output is a html report and a compilation log. >>>> - >>>> - The objective is to make sure that applications built against >>>> - DSOs from the first revision can still run when executed using >>>> - the DSOs built from the second revision. >>>> - >>>> - and are git commit id or tags. >>>> - >>>> - Options: >>>> - -h show this help >>>> - -j enable parallel compilation with threads >>>> - -v show compilation logs on the console >>>> - -d change working directory (default is ${default_dst}) >>>> - -t the dpdk target to use (default is ${default_target}) >>>> - -f overwrite existing files in destination directory >>>> - >>>> - The script returns 0 on success, or the value of last failing >>>> - call of ${abicheck} (incompatible abi or the tool has run with errors). >>>> - The errors returned by ${abidump} are ignored. >>>> - >>>> - END_OF_HELP >>>> -} >>>> - >>>> -# log in the file, and on stdout if verbose >>>> -# $1: level string >>>> -# $2: string to be logged >>>> -log() { >>>> - echo "$1: $2" >>>> - if [ "${verbose}" != "true" ]; then >>>> - echo "$1: $2" >&3 >>>> - fi >>>> -} >>>> - >>>> -# launch a command and log it, taking care of surrounding spaces with quotes >>>> -cmd() { >>>> - local i s whitespace ret >>>> - s="" >>>> - whitespace="[[:space:]]" >>>> - for i in "$@"; do >>>> - if [[ $i =~ $whitespace ]]; then >>>> - i=\"$i\" >>>> - fi >>>> - if [ -z "$s" ]; then >>>> - s="$i" >>>> - else >>>> - s="$s $i" >>>> - fi >>>> - done >>>> - >>>> - ret=0 >>>> - log "CMD" "$s" >>>> - "$@" || ret=$? >>>> - if [ "$ret" != "0" ]; then >>>> - log "CMD" "previous command returned $ret" >>>> - fi >>>> - >>>> - return $ret >>>> -} >>>> - >>>> -# redirect or copy stderr/stdout to a file >>>> -# the syntax is unfamiliar, but it makes the rest of the >>>> -# code easier to read, avoiding the use of pipes >>>> -set_log_file() { >>>> - # save original stdout and stderr in fd 3 and 4 >>>> - exec 3>&1 >>>> - exec 4>&2 >>>> - # create a new fd 5 that send to a file >>>> - exec 5> >(cat > $1) >>>> - # send stdout and stderr to fd 5 >>>> - if [ "${verbose}" = "true" ]; then >>>> - exec 1> >(tee /dev/fd/5 >&3) >>>> - exec 2> >(tee /dev/fd/5 >&4) >>>> - else >>>> - exec 1>&5 >>>> - exec 2>&5 >>>> - fi >>>> -} >>>> - >>>> -# Make sure we configure SHARED libraries >>>> -# Also turn off IGB and KNI as those require kernel headers to build >>>> -fixup_config() { >>>> - local conf=config/defconfig_$target >>>> - cmd sed -i -e"$ a\CONFIG_RTE_BUILD_SHARED_LIB=y" $conf >>>> - cmd sed -i -e"$ a\CONFIG_RTE_NEXT_ABI=n" $conf >>>> - cmd sed -i -e"$ a\CONFIG_RTE_EAL_IGB_UIO=n" $conf >>>> - cmd sed -i -e"$ a\CONFIG_RTE_LIBRTE_KNI=n" $conf >>>> - cmd sed -i -e"$ a\CONFIG_RTE_KNI_KMOD=n" $conf >>>> -} >>>> - >>>> -# build dpdk for the given tag and dump abi >>>> -# $1: hash of the revision >>>> -gen_abi() { >>>> - local i >>>> - >>>> - cmd git clone ${dpdkroot} ${dst}/${1} >>>> - cmd cd ${dst}/${1} >>>> - >>>> - log "INFO" "Checking out version ${1} of the dpdk" >>>> - # Move to the old version of the tree >>>> - cmd git checkout ${1} >>>> - >>>> - fixup_config >>>> - >>>> - # Now configure the build >>>> - log "INFO" "Configuring DPDK ${1}" >>>> - cmd make config T=$target O=$target >>>> - >>>> - # Checking abi compliance relies on using the dwarf information in >>>> - # the shared objects. Build with -g to include them. >>>> - log "INFO" "Building DPDK ${1}. This might take a moment" >>>> - cmd make -j$parallel O=$target V=1 EXTRA_CFLAGS="-g -Og -Wno-error" \ >>>> - EXTRA_LDFLAGS="-g" || log "INFO" "The build failed" >>>> - >>>> - # Move to the lib directory >>>> - cmd cd ${PWD}/$target/lib >>>> - log "INFO" "Collecting ABI information for ${1}" >>>> - for i in *.so; do >>>> - [ -e "$i" ] || break >>>> - cmd $abidump ${i} -o $dst/${1}/${i}.dump -lver ${1} || true >>>> - # hack to ignore empty SymbolsInfo section (no public ABI) >>>> - if grep -q "'SymbolInfo' => {}," $dst/${1}/${i}.dump \ >>>> - 2> /dev/null; then >>>> - log "INFO" "${i} has no public ABI, remove dump file" >>>> - cmd rm -f $dst/${1}/${i}.dump >>>> - fi >>>> - done >>>> -} >>>> - >>>> -verbose=false >>>> -parallel=1 >>>> -dst=${default_dst} >>>> -target=${default_target} >>>> -force=0 >>>> -while getopts j:vd:t:fh ARG ; do >>>> - case $ARG in >>>> - j ) parallel=$OPTARG ;; >>>> - v ) verbose=true ;; >>>> - d ) dst=$OPTARG ;; >>>> - t ) target=$OPTARG ;; >>>> - f ) force=1 ;; >>>> - h ) print_usage ; exit 0 ;; >>>> - ? ) print_usage ; exit 1 ;; >>>> - esac >>>> -done >>>> -shift $(($OPTIND - 1)) >>>> - >>>> -if [ $# != 2 ]; then >>>> - print_usage >>>> - exit 1 >>>> -fi >>>> - >>>> -tag1=$1 >>>> -tag2=$2 >>>> - >>>> -# convert path to absolute >>>> -case "${dst}" in >>>> - /*) ;; >>>> - *) dst=${PWD}/${dst} ;; >>>> -esac >>>> -dpdkroot=$(readlink -f $(dirname $0)/..) >>>> - >>>> -if [ -e "${dst}" -a "$force" = 0 ]; then >>>> - echo "The ${dst} directory is not empty. Remove it, use another" >>>> - echo "one (-d ), or force overriding (-f)" >>>> - exit 1 >>>> -fi >>>> - >>>> -rm -rf ${dst} >>>> -mkdir -p ${dst} >>>> -set_log_file ${dst}/abi-check.log >>>> -log "INFO" "Logs available in ${dst}/abi-check.log" >>>> - >>>> -command -v ${abicheck} || log "INFO" "Can't find ${abicheck} utility" >>>> -command -v ${abidump} || log "INFO" "Can't find ${abidump} utility" >>>> - >>>> -hash1=$(git show -s --format=%h "$tag1" -- 2> /dev/null | tail -1) >>>> -hash2=$(git show -s --format=%h "$tag2" -- 2> /dev/null | tail -1) >>>> - >>>> -# Make hashes available in output for non-local reference >>>> -tag1="$tag1 ($hash1)" >>>> -tag2="$tag2 ($hash2)" >>>> - >>>> -if [ "$hash1" = "$hash2" ]; then >>>> - log "ERROR" "$tag1 and $tag2 are the same revisions" >>>> - exit 1 >>>> -fi >>>> - >>>> -cmd mkdir -p ${dst} >>>> - >>>> -# dump abi for each revision >>>> -gen_abi ${hash1} >>>> -gen_abi ${hash2} >>>> - >>>> -# compare the abi dumps >>>> -cmd cd ${dst} >>>> -ret=0 >>>> -list="" >>>> -for i in ${hash2}/*.dump; do >>>> - name=`basename $i` >>>> - libname=${name%.dump} >>>> - >>>> - if [ ! -f ${hash1}/$name ]; then >>>> - log "INFO" "$NAME does not exist in $tag1. skipping..." >>>> - continue >>>> - fi >>>> - >>>> - local_ret=0 >>>> - cmd $abicheck -l $libname \ >>>> - -old ${hash1}/$name -new ${hash2}/$name || local_ret=$? >>>> - if [ $local_ret != 0 ]; then >>>> - log "NOTICE" "$abicheck returned $local_ret" >>>> - ret=$local_ret >>>> - list="$list $libname" >>>> - fi >>>> -done >>>> - >>>> -if [ $ret != 0 ]; then >>>> - log "NOTICE" "ABI may be incompatible, check reports/logs for details." >>>> - log "NOTICE" "Incompatible list: $list" >>>> -else >>>> - log "NOTICE" "No error detected, ABI is compatible." >>>> -fi >>>> - >>>> -log "INFO" "Logs are in ${dst}/abi-check.log" >>>> -log "INFO" "HTML reports are in ${dst}/compat_reports directory" >>>> - >>>> -exit $ret >>>> diff --git a/doc/guides/contributing/abi_versioning.rst b/doc/guides/contributing/abi_versioning.rst >>>> index a21f4e7a4..d32cf440a 100644 >>>> --- a/doc/guides/contributing/abi_versioning.rst >>>> +++ b/doc/guides/contributing/abi_versioning.rst >>>> @@ -482,41 +482,26 @@ Running the ABI Validator >>>> ------------------------- >>>> >>>> The ``devtools`` directory in the DPDK source tree contains a utility program, >>>> -``validate-abi.sh``, for validating the DPDK ABI based on the Linux `ABI >>>> -Compliance Checker >>>> -`_. >>>> - >>>> -This has a dependency on the ``abi-compliance-checker`` and ``and abi-dumper`` >>>> -utilities which can be installed via a package manager. For example:: >>>> - >>>> - sudo yum install abi-compliance-checker >>>> - sudo yum install abi-dumper >>>> - >>>> -The syntax of the ``validate-abi.sh`` utility is:: >>>> - >>>> - ./devtools/validate-abi.sh >>>> - >>>> -Where ``REV1`` and ``REV2`` are valid gitrevisions(7) >>>> -https://www.kernel.org/pub/software/scm/git/docs/gitrevisions.html >>>> -on the local repo. >>>> - >>>> -For example:: >>>> - >>>> - # Check between the previous and latest commit: >>>> - ./devtools/validate-abi.sh HEAD~1 HEAD >>>> - >>>> - # Check on a specific compilation target: >>>> - ./devtools/validate-abi.sh -t x86_64-native-linux-gcc HEAD~1 HEAD >>>> - >>>> - # Check between two tags: >>>> - ./devtools/validate-abi.sh v2.0.0 v2.1.0 >>>> - >>>> - # Check between git master and local topic-branch "vhost-hacking": >>>> - ./devtools/validate-abi.sh master vhost-hacking >>>> - >>>> -After the validation script completes (it can take a while since it need to >>>> -compile both tags) it will create compatibility reports in the >>>> -``./abi-check/compat_report`` directory. Listed incompatibilities can be found >>>> -as follows:: >>>> - >>>> - grep -lr Incompatible abi-check/compat_reports/ >>>> +``check-abi.sh``, for validating the DPDK ABI based on the libabigail abidiff >>>> +utility: >>>> +https://sourceware.org/libabigail/manual/abidiff.html >>> >>> (from v1 feedback) >>> Links should be in the format. >>> `PEP8 (Style Guide for Python Code) `_. >>> >> copy that, I'll fix it up >> >>>> + >>>> +The syntax of the ``check-abi.sh`` utility is:: >>>> + >>>> + ./devtools/check-abi.sh >>> >>> (from v1 feedback) >>> Could we simplify this all greatly, by telling people to use the meson/ninja build, >>> so they get this checking out of the box, without all the headache below? >>> >> I think bruce noted that was never merged, correct? >> > Yep, correct. :-( > apologies, was there a reason?