From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2453CA04F3; Fri, 20 Dec 2019 16:22:22 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CE34E2C19; Fri, 20 Dec 2019 16:22:20 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id F254EF90 for ; Fri, 20 Dec 2019 16:22:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576855338; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gcdTvBcgNVQkI98JCl1pmhjCj9lVL2zzor6LiKU9kp8=; b=doA951kBaizTwCoGLbQC4kSr3OFNmoRQ5+SIrph6mAcgDtdXMRE4S89c7r9Z3CDLCjQZ7y 5iD82dR+HkAt4XlOhcCiuNM/uyaXgkXjeSo/0FTu9XbvpKy0cj0y5p1RNwST7yZHyh80Xi kIslOgr4WK8/QI56gnk2Lobv1yfkIL0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-419-3lsL4V8VPrm_Rl77aALrSA-1; Fri, 20 Dec 2019 10:22:12 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0FB05801E72; Fri, 20 Dec 2019 15:22:10 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-206-38.brq.redhat.com [10.40.206.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2C1D360BF3; Fri, 20 Dec 2019 15:22:05 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, bruce.richardson@intel.com, kevin.laatz@intel.com, aconole@redhat.com, nhorman@tuxdriver.com, Michael Santana , John McNamara , Marko Kovacevic Date: Fri, 20 Dec 2019 16:20:58 +0100 Message-Id: <20191220152058.10739-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: 3lsL4V8VPrm_Rl77aALrSA-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Subject: [dpdk-dev] [PATCH] add ABI checks X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Starting from Kevin and Bruce idea of using libabigail, here is an alternate approach to implement ABI checks. By default, those checks are disabled and enabling them requires a manual step that generates the ABI dumps on a reference version for a set of configurations. Those checks are enabled in the CI by default for the default meson options on x86 and aarch64 so that proposed patches are validated. A cache of the ABI is stored in travis jobs. Checks can be only informational by setting ABI_CHECKS_WARN_ONLY when breaking the ABI in a future release. For advanced developers and maintainers, the contributing guide details the higher level scripts that are quite close to the existing devtools scripts. Signed-off-by: David Marchand --- .ci/linux-build.sh | 43 +++++++++++++++++++++++++++ .travis.yml | 20 +++++++++++-- devtools/check-abi-dump.sh | 46 +++++++++++++++++++++++++++++ devtools/check-abi-reference.sh | 27 +++++++++++++++++ devtools/dpdk.abignore | 2 ++ devtools/gen-abi-dump.sh | 29 ++++++++++++++++++ devtools/gen-abi-reference.sh | 24 +++++++++++++++ devtools/test-build.sh | 13 ++++++-- devtools/test-meson-builds.sh | 6 ++++ doc/guides/contributing/patches.rst | 25 ++++++++++++++++ 10 files changed, 230 insertions(+), 5 deletions(-) create mode 100755 devtools/check-abi-dump.sh create mode 100755 devtools/check-abi-reference.sh create mode 100644 devtools/dpdk.abignore create mode 100755 devtools/gen-abi-dump.sh create mode 100755 devtools/gen-abi-reference.sh diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index ccc3a7ccd..345dba264 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -30,8 +30,51 @@ fi =20 OPTS=3D"$OPTS --default-library=3D$DEF_LIB" meson build --werror -Dexamples=3Dall $OPTS + +if [ "$ABI_CHECKS" =3D "1" ]; then + git remote add ref ${REF_GIT_REPO:-https://dpdk.org/git/dpdk} + git fetch --tags ref ${REF_GIT_BRANCH:-master} + + head=3D$(git describe --all) + tag=3D$(git describe --abbrev=3D0) + + if [ "$(cat reference/VERSION 2>/dev/null)" !=3D "$tag" ]; then + rm -rf reference + fi + + if [ ! -d reference ]; then + gen_abi_dump=3D$(mktemp -t gen-abi-dump-XXX.sh) + cp -a devtools/gen-abi-dump.sh $gen_abi_dump + + git checkout -qf $tag + ninja -C build + $gen_abi_dump build reference + + if [ "$AARCH64" !=3D "1" ]; then + mkdir -p reference/app + cp -a build/app/dpdk-testpmd reference/app/ + + export LD_LIBRARY_PATH=3D$(pwd)/build/lib:$(pwd)/build/drivers + devtools/test-null.sh reference/app/dpdk-testpmd + unset LD_LIBRARY_PATH + fi + echo $tag > reference/VERSION + + git checkout -qf $head + fi +fi + ninja -C build =20 +if [ "$ABI_CHECKS" =3D "1" ]; then + devtools/check-abi-dump.sh build reference ${ABI_CHECKS_WARN_ONLY:-} + if [ "$AARCH64" !=3D "1" ]; then + export LD_LIBRARY_PATH=3D$(pwd)/build/lib:$(pwd)/build/drivers + devtools/test-null.sh reference/app/dpdk-testpmd + unset LD_LIBRARY_PATH + fi +fi + if [ "$AARCH64" !=3D "1" ]; then devtools/test-null.sh fi diff --git a/.travis.yml b/.travis.yml index 8f90d06f2..bbb060fa2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c -cache: ccache +cache: + ccache: true + directories: + - reference compiler: - gcc - clang @@ -21,7 +24,7 @@ aarch64_packages: &aarch64_packages =20 extra_packages: &extra_packages - *required_packages - - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4] + - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools] =20 build_32b_packages: &build_32b_packages - *required_packages @@ -59,6 +62,13 @@ matrix: apt: packages: - *aarch64_packages + - env: DEF_LIB=3D"shared" EXTRA_PACKAGES=3D1 ABI_CHECKS=3D1 AARCH64=3D1 + compiler: gcc + addons: + apt: + packages: + - *aarch64_packages + - *extra_packages - env: DEF_LIB=3D"static" EXTRA_PACKAGES=3D1 compiler: gcc addons: @@ -72,6 +82,12 @@ matrix: packages: - *extra_packages - *doc_packages + - env: DEF_LIB=3D"shared" EXTRA_PACKAGES=3D1 ABI_CHECKS=3D1 + compiler: gcc + addons: + apt: + packages: + - *extra_packages - env: DEF_LIB=3D"static" OPTS=3D"-Denable_kmods=3Dfalse" EXTRA_PACKAGES= =3D1 compiler: gcc addons: diff --git a/devtools/check-abi-dump.sh b/devtools/check-abi-dump.sh new file mode 100755 index 000000000..f48a2ae7e --- /dev/null +++ b/devtools/check-abi-dump.sh @@ -0,0 +1,46 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# !=3D 2 ] && [ $# !=3D 3 ]; then +=09echo "Usage: $0 builddir dumpdir [warnonly]" +=09exit 1 +fi + +builddir=3D$1 +dumpdir=3D$2 +warnonly=3D${3:-} +if [ ! -d $builddir ]; then +=09echo "Error: build directory '$builddir' does not exist." +=09exit 1 +fi +if [ ! -d $dumpdir ]; then +=09echo "Error: dump directory '$dumpdir' does not exist." +=09exit 1 +fi + +ABIDIFF_OPTIONS=3D"--suppr $(dirname $0)/dpdk.abignore" +for dump in $(find $dumpdir -name "*.dump"); do +=09libname=3D$(basename $dump) +=09libname=3D${libname%.dump} +=09result=3D +=09for f in $(find $builddir -name "$libname.so.*"); do +=09=09if test -L $f || [ "$f" !=3D "${f%%.symbols}" ]; then +=09=09=09continue +=09=09fi +=09=09result=3Dfound + +=09=09if ! abidiff $ABIDIFF_OPTIONS $dump $f; then +=09=09=09if [ -z "$warnonly" ]; then +=09=09=09=09echo "Error: ABI issue reported for $dump, $f" +=09=09=09=09exit 1 +=09=09=09fi +=09=09=09echo "Warning: ABI issue reported for $dump, $f" +=09=09fi +=09=09break +=09done +=09if [ "$result" !=3D "found" ]; then +=09=09echo "Error: can't find a library for dump file $dump" +=09=09exit 1 +=09fi +done diff --git a/devtools/check-abi-reference.sh b/devtools/check-abi-reference= .sh new file mode 100755 index 000000000..7addb094e --- /dev/null +++ b/devtools/check-abi-reference.sh @@ -0,0 +1,27 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +devtools_dir=3D$(dirname $(readlink -f $0)) +. $devtools_dir/load-devel-config + +abi_ref_build_dir=3D${DPDK_ABI_REF_BUILD_DIR:-reference} +builds_dir=3D${DPDK_BUILD_TEST_DIR:-.} + +for dir in $abi_ref_build_dir/*; do +=09if [ "$dir" =3D "$abi_ref_build_dir" ]; then +=09=09exit 1 +=09fi +=09if [ ! -d $dir/dump ]; then +=09=09echo "Skipping $dir" +=09=09continue +=09fi +=09target=3D$(basename $dir) +=09if [ -d $builds_dir/$target/install ]; then +=09=09libdir=3D$builds_dir/$target/install +=09else +=09=09libdir=3D$builds_dir/$target +=09fi +=09echo "Checking ABI between $libdir and $dir/dump" +=09$devtools_dir/check-abi-dump.sh $libdir $dir/dump +done diff --git a/devtools/dpdk.abignore b/devtools/dpdk.abignore new file mode 100644 index 000000000..b866b7f26 --- /dev/null +++ b/devtools/dpdk.abignore @@ -0,0 +1,2 @@ +[suppress_function] + symbol_version =3D EXPERIMENTAL diff --git a/devtools/gen-abi-dump.sh b/devtools/gen-abi-dump.sh new file mode 100755 index 000000000..4e38d751f --- /dev/null +++ b/devtools/gen-abi-dump.sh @@ -0,0 +1,29 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +if [ $# !=3D 2 ]; then +=09echo "Usage: $0 builddir dumpdir" +=09exit 1 +fi + +builddir=3D$1 +dumpdir=3D$2 +if [ ! -d $builddir ]; then +=09echo "Error: build directory '$builddir' does not exist." +=09exit 1 +fi +if [ -d $dumpdir ]; then +=09echo "Error: dump directory '$dumpdir' already exists." +=09exit 1 +fi + +mkdir -p $dumpdir +for f in $(find $builddir -name "*.so.*"); do +=09if test -L $f || [ "$f" !=3D "${f%%.symbols}" ]; then +=09=09continue +=09fi + +=09libname=3D$(basename $f) +=09abidw --out-file $dumpdir/${libname%.so.*}.dump $f +done diff --git a/devtools/gen-abi-reference.sh b/devtools/gen-abi-reference.sh new file mode 100755 index 000000000..f41d7fadc --- /dev/null +++ b/devtools/gen-abi-reference.sh @@ -0,0 +1,24 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Red Hat, Inc. + +devtools_dir=3D$(dirname $(readlink -f $0)) +. $devtools_dir/load-devel-config + +abi_ref_build_dir=3D${DPDK_ABI_REF_BUILD_DIR:-reference} +for dir in $abi_ref_build_dir/*; do +=09if [ "$dir" =3D "$abi_ref_build_dir" ]; then +=09=09exit 1 +=09fi +=09if [ -d $dir/dump ]; then +=09=09echo "Skipping $dir" +=09=09continue +=09fi +=09if [ -d $dir/install ]; then +=09=09libdir=3D$dir/install +=09else +=09=09libdir=3D$dir +=09fi +=09echo "Dumping libraries from $libdir in $dir/dump" +=09$devtools_dir/gen-abi-dump.sh $libdir $dir/dump +done diff --git a/devtools/test-build.sh b/devtools/test-build.sh index 52305fbb8..8cb5b56fb 100755 --- a/devtools/test-build.sh +++ b/devtools/test-build.sh @@ -30,7 +30,8 @@ default_path=3D$PATH # - LIBSSO_SNOW3G_PATH # - LIBSSO_KASUMI_PATH # - LIBSSO_ZUC_PATH -. $(dirname $(readlink -f $0))/load-devel-config +devtools_dir=3D$(dirname $(readlink -f $0)) +. $devtools_dir/load-devel-config =20 print_usage () { =09echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]" @@ -64,6 +65,7 @@ print_help () { [ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1 =20 J=3D$DPDK_MAKE_JOBS +abi_ref_build_dir=3D${DPDK_ABI_REF_BUILD_DIR:-reference} builds_dir=3D${DPDK_BUILD_TEST_DIR:-.} short=3Dfalse unset verbose @@ -97,7 +99,7 @@ trap "signal=3DINT ; trap - INT ; kill -INT $$" INT # notify result on exit trap on_exit EXIT =20 -cd $(dirname $(readlink -f $0))/.. +cd $devtools_dir/.. =20 reset_env () { @@ -233,7 +235,7 @@ for conf in $configs ; do =09# reload config with DPDK_TARGET set =09DPDK_TARGET=3D$target =09reset_env -=09. $(dirname $(readlink -f $0))/load-devel-config +=09. $devtools_dir/load-devel-config =20 =09options=3D$(echo $conf | sed 's,[^~+]*,,') =09dir=3D$builds_dir/$conf @@ -246,6 +248,11 @@ for conf in $configs ; do =09export RTE_TARGET=3D$target =09rm -rf $dir/install =09${MAKE} install O=3D$dir DESTDIR=3D$dir/install prefix=3D +=09if [ -d $abi_ref_build_dir/$conf/dump ]; then +=09=09echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Check A= BI $conf" +=09=09$devtools_dir/check-abi-dump.sh $dir/install \ +=09=09=09$abi_ref_build_dir/$conf/dump +=09fi =09echo "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Build exam= ples for $conf" =09export RTE_SDK=3D$(readlink -f $dir)/install/share/dpdk =09ln -sTf $(pwd)/lib $RTE_SDK/lib # workaround for vm_power_manager diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index 688567714..aaefa38a2 100755 --- a/devtools/test-meson-builds.sh +++ b/devtools/test-meson-builds.sh @@ -16,6 +16,7 @@ srcdir=3D$(dirname $(readlink -f $0))/.. =20 MESON=3D${MESON:-meson} use_shared=3D"--default-library=3Dshared" +abi_ref_build_dir=3D${DPDK_ABI_REF_BUILD_DIR:-reference} builds_dir=3D${DPDK_BUILD_TEST_DIR:-.} =20 if command -v gmake >/dev/null 2>&1 ; then @@ -88,6 +89,11 @@ build () # =09=09echo "$ninja_cmd -C $builddir" =09=09$ninja_cmd -C $builddir =09fi + +=09if [ -d $abi_ref_build_dir/$1/dump ]; then +=09=09$srcdir/devtools/check-abi-dump.sh $builddir +=09=09=09$abi_ref_build_dir/$1/dump +=09fi } =20 if [ "$1" =3D "-vv" ] ; then diff --git a/doc/guides/contributing/patches.rst b/doc/guides/contributing/= patches.rst index 0686450e4..de3dff145 100644 --- a/doc/guides/contributing/patches.rst +++ b/doc/guides/contributing/patches.rst @@ -513,6 +513,31 @@ in a single subfolder called "__builds" created in the= current directory. Setting ``DPDK_BUILD_TEST_DIR`` to an absolute directory path e.g. ``/tmp`= ` is also supported. =20 =20 +Checking ABI compatibility +-------------------------- + +The first thing is to build reference binaries for the latest release your +patches are built on top of. + +Either you are in a git tree and an easy way to identify this is to run:: + + git checkout $(git describe --abbrev=3D0) + +Or you use a tarball and you extract the sources in a director of your cho= ice. + +Next is building those sources, refer to the previous paragraph. +You can set ``DPDK_BUILD_TEST_DIR=3Dreference``, so that the builds occur = in this +directory. + +Finally, the ABI dump files are generated with the +``devtools/gen-abi-reference.sh`` script. This script will look for builds= in +the current sub directory ``reference``. But you can set the environment +variable ``DPDK_ABI_REF_BUILD_DIR`` to a different location. + +Once done, you can check your current binaries ABI with this reference wit= h the +``devtools/check-abi-reference.sh`` script. + + Sending Patches --------------- =20 --=20 2.23.0