[dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[David Marchand]

libabigail 1.2 (at least) reports changes in 'const' property as an ABI
breakage [1].
This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
in launchpad [3].

But for now, build and use the last version 1.6 so that the ABI checks
can be kept.

1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 .ci/linux-build.sh | 22 ++++++++++++++++++++++
 .travis.yml        | 15 ++++++++++-----
 2 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index c7c3840fc..0d4bc9a62 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -38,6 +38,28 @@ if [ "$AARCH64" != "1" ]; then
 fi
 
 if [ "$ABI_CHECKS" = "1" ]; then
+    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
+    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
+
+    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
+        rm -rf libabigail
+        # if we change libabigail, invalidate existing abi cache
+        rm -rf reference
+    fi
+
+    if [ ! -d libabigail ]; then
+        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
+        cd libabigail/src && autoconf -vfi && cd -
+        mkdir libabigail/src/build
+        cd libabigail/src/build && ../configure --prefix=$(pwd)/libabigail && cd -
+        make -C libabigail/src/build all install
+
+        rm -rf libabigail/src
+        echo $LIBABIGAIL_VERSION > libabigail/VERSION
+    fi
+
+    export PATH=$(pwd)/libabigail/bin:$PATH
+
     REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
     REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
 
diff --git a/.travis.yml b/.travis.yml
index 22539d823..d8253fdd4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,7 @@ language: c
 cache:
   ccache: true
   directories:
+    - libabigail
     - reference
 compiler:
   - gcc
@@ -24,7 +25,11 @@ aarch64_packages: &aarch64_packages
 
 extra_packages: &extra_packages
   - *required_packages
-  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
+  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+
+libabigail_build_packages: &libabigail_build_packages
+  - *required_packages
+  - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
 
 build_32b_packages: &build_32b_packages
   - *required_packages
@@ -154,18 +159,18 @@ matrix:
         packages:
           - *required_packages
           - *doc_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+  - env: DEF_LIB="shared" ABI_CHECKS=1
     compiler: gcc
     addons:
       apt:
         packages:
-          - *extra_packages
-  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
+          - *libabigail_build_packages
+  - env: DEF_LIB="shared" ABI_CHECKS=1
     arch: arm64
     compiler: gcc
     addons:
       apt:
         packages:
-          - *extra_packages
+          - *libabigail_build_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.23.0

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[Thomas Monjalon]

17/02/2020 14:59, David Marchand:
> libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> breakage [1].
> This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> in launchpad [3].
> 
> But for now, build and use the last version 1.6 so that the ABI checks
> can be kept.
> 
> 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
> 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
> +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
> +
> +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> +        rm -rf libabigail
> +        # if we change libabigail, invalidate existing abi cache
> +        rm -rf reference
> +    fi
> +
> +    if [ ! -d libabigail ]; then
> +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
> +        cd libabigail/src && autoconf -vfi && cd -
> +        mkdir libabigail/src/build
> +        cd libabigail/src/build && ../configure --prefix=$(pwd)/libabigail && cd -
> +        make -C libabigail/src/build all install
> +
> +        rm -rf libabigail/src
> +        echo $LIBABIGAIL_VERSION > libabigail/VERSION
> +    fi

Can we avoid compiling libabigail ourself?
Is there an up-to-date Ubuntu package somewhere?

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> breakage [1].
> This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> in launchpad [3].
>
> But for now, build and use the last version 1.6 so that the ABI checks
> can be kept.
>
> 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
> 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---

Does it make sense to base libabigail required ontop of extra packages?
Otherwise some libraries won't get built / checked, no?

>  .ci/linux-build.sh | 22 ++++++++++++++++++++++
>  .travis.yml        | 15 ++++++++++-----
>  2 files changed, 32 insertions(+), 5 deletions(-)
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index c7c3840fc..0d4bc9a62 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -38,6 +38,28 @@ if [ "$AARCH64" != "1" ]; then
>  fi
>  
>  if [ "$ABI_CHECKS" = "1" ]; then
> +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
> +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
> +
> +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> +        rm -rf libabigail
> +        # if we change libabigail, invalidate existing abi cache
> +        rm -rf reference
> +    fi
> +
> +    if [ ! -d libabigail ]; then
> +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
> +        cd libabigail/src && autoconf -vfi && cd -
> +        mkdir libabigail/src/build
> +        cd libabigail/src/build && ../configure --prefix=$(pwd)/libabigail && cd -
> +        make -C libabigail/src/build all install
> +
> +        rm -rf libabigail/src
> +        echo $LIBABIGAIL_VERSION > libabigail/VERSION
> +    fi
> +
> +    export PATH=$(pwd)/libabigail/bin:$PATH
> +
>      REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
>      REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
>  
> diff --git a/.travis.yml b/.travis.yml
> index 22539d823..d8253fdd4 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -2,6 +2,7 @@ language: c
>  cache:
>    ccache: true
>    directories:
> +    - libabigail
>      - reference
>  compiler:
>    - gcc
> @@ -24,7 +25,11 @@ aarch64_packages: &aarch64_packages
>  
>  extra_packages: &extra_packages
>    - *required_packages
> -  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
> +  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
> +
> +libabigail_build_packages: &libabigail_build_packages
> +  - *required_packages
> +  - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
>  
>  build_32b_packages: &build_32b_packages
>    - *required_packages
> @@ -154,18 +159,18 @@ matrix:
>          packages:
>            - *required_packages
>            - *doc_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> -  - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
> +          - *libabigail_build_packages
> +  - env: DEF_LIB="shared" ABI_CHECKS=1
>      arch: arm64
>      compiler: gcc
>      addons:
>        apt:
>          packages:
> -          - *extra_packages
> +          - *libabigail_build_packages
>  
>  script: ./.ci/${TRAVIS_OS_NAME}-build.sh

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[David Marchand]

On Mon, Feb 17, 2020 at 7:48 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> > breakage [1].
> > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> > in launchpad [3].
> >
> > But for now, build and use the last version 1.6 so that the ABI checks
> > can be kept.
> >
> > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> > 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
> > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
>
> Does it make sense to base libabigail required ontop of extra packages?
> Otherwise some libraries won't get built / checked, no?

The only change I see is the pcap driver being enabled.
On the principle, I agree that trying to build all possible
libraries/drivers is better when checking the ABI.
So I'll keep extra_packages yes.

I am currently testing that touching extra_packages (well, testing
Thomas patches) results in Travis treating the job as a new one (i.e.
with no cache).


>
> >  .ci/linux-build.sh | 22 ++++++++++++++++++++++
> >  .travis.yml        | 15 ++++++++++-----
> >  2 files changed, 32 insertions(+), 5 deletions(-)
> >
> > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> > index c7c3840fc..0d4bc9a62 100755
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> > @@ -38,6 +38,28 @@ if [ "$AARCH64" != "1" ]; then
> >  fi
> >
> >  if [ "$ABI_CHECKS" = "1" ]; then
> > +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
> > +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
> > +
> > +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> > +        rm -rf libabigail
> > +        # if we change libabigail, invalidate existing abi cache
> > +        rm -rf reference
> > +    fi
> > +
> > +    if [ ! -d libabigail ]; then
> > +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
> > +        cd libabigail/src && autoconf -vfi && cd -

And I managed to send a "oh yeah, seems better this way"/untested version...
Fixed.



--
David Marchand

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[David Marchand]

On Tue, Feb 18, 2020 at 10:40 AM David Marchand
<david.marchand@redhat.com> wrote:
>
> On Mon, Feb 17, 2020 at 7:48 PM Aaron Conole <aconole@redhat.com> wrote:
> >
> > David Marchand <david.marchand@redhat.com> writes:
> >
> > > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> > > breakage [1].
> > > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> > > in launchpad [3].
> > >
> > > But for now, build and use the last version 1.6 so that the ABI checks
> > > can be kept.
> > >
> > > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> > > 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
> > > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> > >
> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > > ---
> >
> > Does it make sense to base libabigail required ontop of extra packages?
> > Otherwise some libraries won't get built / checked, no?
>
> The only change I see is the pcap driver being enabled.
> On the principle, I agree that trying to build all possible
> libraries/drivers is better when checking the ABI.
> So I'll keep extra_packages yes.
>
> I am currently testing that touching extra_packages (well, testing
> Thomas patches) results in Travis treating the job as a new one (i.e.
> with no cache).

Travis bases each job cache on the job description:
https://docs.travis-ci.com/user/caching/

I tested Thomas change on extra_packages content, and the job used the
old cache.
My idea was to try to put *extra_packages in an env variable, but it
does not work (my yaml-fu is lacking).

If there is no easy way, I will invalidate the cache manually.


--
David Marchand

[dpdk-dev] [PATCH v2] ci: build and use libabigail 1.6

[David Marchand]

libabigail 1.2 (at least) reports changes in 'const' property as an ABI
breakage [1].
This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
in launchpad [3].

But for now, build and use the last version 1.6 so that the ABI checks
can be kept.

1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changelog since v1:
- keep extra_packages in ABI checks jobs,
- fix configure step,

---
 .ci/linux-build.sh | 23 +++++++++++++++++++++++
 .travis.yml        |  8 +++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index c7c3840fc..cb9627bd5 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -38,6 +38,29 @@ if [ "$AARCH64" != "1" ]; then
 fi
 
 if [ "$ABI_CHECKS" = "1" ]; then
+    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
+    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
+
+    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
+        rm -rf libabigail
+        # if we change libabigail, invalidate existing abi cache
+        rm -rf reference
+    fi
+
+    if [ ! -d libabigail ]; then
+        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
+        cd libabigail/src && autoreconf -i && cd -
+        instdir=$(pwd)/libabigail
+        mkdir libabigail/src/build
+        cd libabigail/src/build && ../configure --prefix=$instdir && cd -
+        make -C libabigail/src/build all install
+
+        rm -rf libabigail/src
+        echo $LIBABIGAIL_VERSION > libabigail/VERSION
+    fi
+
+    export PATH=$(pwd)/libabigail/bin:$PATH
+
     REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
     REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
 
diff --git a/.travis.yml b/.travis.yml
index 22539d823..030064c94 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,7 @@ language: c
 cache:
   ccache: true
   directories:
+    - libabigail
     - reference
 compiler:
   - gcc
@@ -24,7 +25,10 @@ aarch64_packages: &aarch64_packages
 
 extra_packages: &extra_packages
   - *required_packages
-  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
+  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+
+libabigail_build_packages: &libabigail_build_packages
+  - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
 
 build_32b_packages: &build_32b_packages
   - *required_packages
@@ -160,6 +164,7 @@ matrix:
       apt:
         packages:
           - *extra_packages
+          - *libabigail_build_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
     arch: arm64
     compiler: gcc
@@ -167,5 +172,6 @@ matrix:
       apt:
         packages:
           - *extra_packages
+          - *libabigail_build_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.23.0

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> On Tue, Feb 18, 2020 at 10:40 AM David Marchand
> <david.marchand@redhat.com> wrote:
>>
>> On Mon, Feb 17, 2020 at 7:48 PM Aaron Conole <aconole@redhat.com> wrote:
>> >
>> > David Marchand <david.marchand@redhat.com> writes:
>> >
>> > > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
>> > > breakage [1].
>> > > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
>> > > in launchpad [3].
>> > >
>> > > But for now, build and use the last version 1.6 so that the ABI checks
>> > > can be kept.
>> > >
>> > > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
>> > > 2:
>> > > https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
>> > > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
>> > >
>> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
>> > > ---
>> >
>> > Does it make sense to base libabigail required ontop of extra packages?
>> > Otherwise some libraries won't get built / checked, no?
>>
>> The only change I see is the pcap driver being enabled.
>> On the principle, I agree that trying to build all possible
>> libraries/drivers is better when checking the ABI.
>> So I'll keep extra_packages yes.
>>
>> I am currently testing that touching extra_packages (well, testing
>> Thomas patches) results in Travis treating the job as a new one (i.e.
>> with no cache).
>
> Travis bases each job cache on the job description:
> https://docs.travis-ci.com/user/caching/
>
> I tested Thomas change on extra_packages content, and the job used the
> old cache.
> My idea was to try to put *extra_packages in an env variable, but it
> does not work (my yaml-fu is lacking).
>
> If there is no easy way, I will invalidate the cache manually.

We don't actually use the EXTRA_PACKAGES variable for anything, so I
guess it's probably okay to change the value and that should invalidate
the cache.  Most of the variables, in fact, could be checked for
non-zero value rather than a specific positive value, and then it's easy
to invalidate the cache by just bumping them.  It's a thought (and
kindof a hack).  Or we can just use the travis CLI tool and delete the
caches (we'll have to do that for the ovsrobot as well, I think).

>
> --
> David Marchand

Re: [dpdk-dev] [PATCH] ci: build and use libabigail 1.6

[David Marchand]

On Tue, Feb 18, 2020 at 3:55 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > On Tue, Feb 18, 2020 at 10:40 AM David Marchand
> > <david.marchand@redhat.com> wrote:
> >>
> >> On Mon, Feb 17, 2020 at 7:48 PM Aaron Conole <aconole@redhat.com> wrote:
> >> >
> >> > David Marchand <david.marchand@redhat.com> writes:
> >> >
> >> > > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> >> > > breakage [1].
> >> > > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> >> > > in launchpad [3].
> >> > >
> >> > > But for now, build and use the last version 1.6 so that the ABI checks
> >> > > can be kept.
> >> > >
> >> > > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> >> > > 2:
> >> > > https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b986fe1cc87d9d8e7412998038392
> >> > > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> >> > >
> >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> >> > > ---
> >> >
> >> > Does it make sense to base libabigail required ontop of extra packages?
> >> > Otherwise some libraries won't get built / checked, no?
> >>
> >> The only change I see is the pcap driver being enabled.
> >> On the principle, I agree that trying to build all possible
> >> libraries/drivers is better when checking the ABI.
> >> So I'll keep extra_packages yes.
> >>
> >> I am currently testing that touching extra_packages (well, testing
> >> Thomas patches) results in Travis treating the job as a new one (i.e.
> >> with no cache).
> >
> > Travis bases each job cache on the job description:
> > https://docs.travis-ci.com/user/caching/
> >
> > I tested Thomas change on extra_packages content, and the job used the
> > old cache.
> > My idea was to try to put *extra_packages in an env variable, but it
> > does not work (my yaml-fu is lacking).
> >
> > If there is no easy way, I will invalidate the cache manually.
>
> We don't actually use the EXTRA_PACKAGES variable for anything, so I
> guess it's probably okay to change the value and that should invalidate
> the cache.  Most of the variables, in fact, could be checked for
> non-zero value rather than a specific positive value, and then it's easy
> to invalidate the cache by just bumping them.  It's a thought (and
> kindof a hack).  Or we can just use the travis CLI tool and delete the
> caches (we'll have to do that for the ovsrobot as well, I think).
>

What I had in mind was to convert the extra_packages yaml thing into a
string to pass into EXTRA_PACKAGES.
But I did not manage.

About bumping the value, users are likely to be unaware of this step
if they submit a patch touching .travis.yml.

Deleting the caches from ovsrobot if .travis.yml has been touched seems simpler.

On master, I will stick to manual cache invalidation.


-- 
David Marchand



--
David Marchand

Re: [dpdk-dev] [PATCH v2] ci: build and use libabigail 1.6

[Thomas Monjalon]

18/02/2020 15:29, David Marchand:
> libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> breakage [1].
> This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> in launchpad [3].
> 
> But for now, build and use the last version 1.6 so that the ABI checks
> can be kept.
> 
> 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
> 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

Acked-by: Thomas Monjalon <thomas@monjalon.net>

I suggest few improvements below:

> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
>  if [ "$ABI_CHECKS" = "1" ]; then

What do you think about moving the libabigail install in a function?

We could justify this with a comment about installing the latest version.

> +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
> +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
> +
> +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> +        rm -rf libabigail
> +        # if we change libabigail, invalidate existing abi cache
> +        rm -rf reference
> +    fi
> +
> +    if [ ! -d libabigail ]; then
> +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src

Why not using the tarball?
http://mirrors.kernel.org/sourceware/libabigail/libabigail-1.6.tar.gz

> +        cd libabigail/src && autoreconf -i && cd -
> +        instdir=$(pwd)/libabigail
> +        mkdir libabigail/src/build
> +        cd libabigail/src/build && ../configure --prefix=$instdir && cd -
> +        make -C libabigail/src/build all install
> +
> +        rm -rf libabigail/src
> +        echo $LIBABIGAIL_VERSION > libabigail/VERSION
> +    fi
> +
> +    export PATH=$(pwd)/libabigail/bin:$PATH

Re: [dpdk-dev] [PATCH v2] ci: build and use libabigail 1.6

[David Marchand]

On Tue, Feb 18, 2020 at 4:46 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 18/02/2020 15:29, David Marchand:
> > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> > breakage [1].
> > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> > in launchpad [3].
> >
> > But for now, build and use the last version 1.6 so that the ABI checks
> > can be kept.
> >
> > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> > 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
> > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
>
> I suggest few improvements below:
>
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> >  if [ "$ABI_CHECKS" = "1" ]; then
>
> What do you think about moving the libabigail install in a function?

No strong opinion, we had everything inline so far.


>
> We could justify this with a comment about installing the latest version.
>
> > +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
> > +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
> > +
> > +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
> > +        rm -rf libabigail
> > +        # if we change libabigail, invalidate existing abi cache
> > +        rm -rf reference
> > +    fi
> > +
> > +    if [ ! -d libabigail ]; then
> > +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
>
> Why not using the tarball?
> http://mirrors.kernel.org/sourceware/libabigail/libabigail-1.6.tar.gz

No good reason "now".

I was first bitten by a reference to redhat-hardened-ld in some
libtool script in the tarball (/me looks in Dodji direction).
I then considered switching to different versions of libabigail by
just setting the LIBABIGAIL_VERSION env variable from .travis.yml.
I ended up with the current latest version which is also what is in
Ubuntu latest releases.

The tarball is smaller than a git clone, so best to use it.
v3 incoming.


-- 
David Marchand

[dpdk-dev] [PATCH v3] ci: build and use libabigail 1.6

[David Marchand]

libabigail 1.2 (at least) reports changes in 'const' property as an ABI
breakage [1].
This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
in launchpad [3].

But for now, build and use the last version 1.6 so that the ABI checks
can be kept.

1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
Changelog since v2:
- used tarball instead of git,
- moved libabigail compilation/install in a function,

Changelog since v1:
- keep extra_packages in ABI checks jobs,
- fix configure step,

---
 .ci/linux-build.sh | 29 +++++++++++++++++++++++++++++
 .travis.yml        |  8 +++++++-
 2 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index c7c3840fc..d500c4c00 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -14,6 +14,20 @@ on_error() {
 }
 trap on_error EXIT
 
+install_libabigail() {
+    version=$1
+    instdir=$2
+
+    wget -q "http://mirrors.kernel.org/sourceware/libabigail/${version}.tar.gz"
+    tar -xf ${version}.tar.gz
+    cd $version && autoreconf -vfi && cd -
+    mkdir $version/build
+    cd $version/build && ../configure --prefix=$instdir && cd -
+    make -C $version/build all install
+    rm -rf $version
+    rm ${version}.tar.gz
+}
+
 if [ "$AARCH64" = "1" ]; then
     # convert the arch specifier
     OPTS="$OPTS --cross-file config/arm/arm64_armv8_linux_gcc"
@@ -38,6 +52,21 @@ if [ "$AARCH64" != "1" ]; then
 fi
 
 if [ "$ABI_CHECKS" = "1" ]; then
+    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
+
+    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
+        rm -rf libabigail
+        # if we change libabigail, invalidate existing abi cache
+        rm -rf reference
+    fi
+
+    if [ ! -d libabigail ]; then
+        install_libabigail $LIBABIGAIL_VERSION $(pwd)/libabigail
+        echo $LIBABIGAIL_VERSION > libabigail/VERSION
+    fi
+
+    export PATH=$(pwd)/libabigail/bin:$PATH
+
     REF_GIT_REPO=${REF_GIT_REPO:-https://dpdk.org/git/dpdk}
     REF_GIT_TAG=${REF_GIT_TAG:-v19.11}
 
diff --git a/.travis.yml b/.travis.yml
index 22539d823..030064c94 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,7 @@ language: c
 cache:
   ccache: true
   directories:
+    - libabigail
     - reference
 compiler:
   - gcc
@@ -24,7 +25,10 @@ aarch64_packages: &aarch64_packages
 
 extra_packages: &extra_packages
   - *required_packages
-  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4, abigail-tools]
+  - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+
+libabigail_build_packages: &libabigail_build_packages
+  - [autoconf, automake, libtool, pkg-config, libxml2-dev, libdw-dev]
 
 build_32b_packages: &build_32b_packages
   - *required_packages
@@ -160,6 +164,7 @@ matrix:
       apt:
         packages:
           - *extra_packages
+          - *libabigail_build_packages
   - env: DEF_LIB="shared" EXTRA_PACKAGES=1 ABI_CHECKS=1
     arch: arm64
     compiler: gcc
@@ -167,5 +172,6 @@ matrix:
       apt:
         packages:
           - *extra_packages
+          - *libabigail_build_packages
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
-- 
2.23.0

Re: [dpdk-dev] [PATCH v3] ci: build and use libabigail 1.6

[Aaron Conole]

David Marchand <david.marchand@redhat.com> writes:

> libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> breakage [1].
> This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> in launchpad [3].
>
> But for now, build and use the last version 1.6 so that the ABI checks
> can be kept.
>
> 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
> 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

Re: [dpdk-dev] [PATCH v3] ci: build and use libabigail 1.6

[David Marchand]

On Tue, Feb 18, 2020 at 9:31 PM Aaron Conole <aconole@redhat.com> wrote:
> David Marchand <david.marchand@redhat.com> writes:
>
> > libabigail 1.2 (at least) reports changes in 'const' property as an ABI
> > breakage [1].
> > This was fixed upstream in libabigail 1.4 [2], and a bug has been opened
> > in launchpad [3].
> >
> > But for now, build and use the last version 1.6 so that the ABI checks
> > can be kept.
> >
> > 1: https://travis-ci.com/DPDK/dpdk/jobs/287872118#L2242
> > 2: https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=215b7eb4fe8b
> > 3: https://bugs.launchpad.net/ubuntu/+source/libabigail/+bug/1863607
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
>
> Acked-by: Aaron Conole <aconole@redhat.com>

Applied.


-- 
David Marchand

Re: [dpdk-dev] [PATCH v2] ci: build and use libabigail 1.6

[Dodji Seketeli]

Hello,

David Marchand <david.marchand@redhat.com> writes:

>> > +    LIBABIGAIL_REPO=${LIBABIGAIL_REPO:-https://sourceware.org/git/libabigail.git}
>> > +    LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
>> > +
>> > +    if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ]; then
>> > +        rm -rf libabigail
>> > +        # if we change libabigail, invalidate existing abi cache
>> > +        rm -rf reference
>> > +    fi
>> > +
>> > +    if [ ! -d libabigail ]; then
>> > +        git clone --single-branch -b $LIBABIGAIL_VERSION $LIBABIGAIL_REPO libabigail/src
>>
>> Why not using the tarball?
>> http://mirrors.kernel.org/sourceware/libabigail/libabigail-1.6.tar.gz
>
> No good reason "now".
>
> I was first bitten by a reference to redhat-hardened-ld in some
> libtool script in the tarball (/me looks in Dodji direction).
> I then considered switching to different versions of libabigail by
> just setting the LIBABIGAIL_VERSION env variable from .travis.yml.
> I ended up with the current latest version which is also what is in
> Ubuntu latest releases.

I think either way (distro package, tarball and git) has some benefits
and drawbacks.

I think one benefit of /being able/ to use the code from git is for
cases where you guys need some new features (and we tend to continuously
add fixes/functionalities to the git repository) that is available in
git only for now.  Hopefully, with time, you'll only need to use the
distro package.

FWIW, I like the fact that your setup lets you choose between the distro
package, the tarball or the git code.  That's powerful.

Cheers,

-- 
		Dodji