DPDK patches and discussions
 help / color / mirror / Atom feed
From: Conor Walsh <conor.walsh@intel.com>
To: dev@dpdk.org
Cc: david.marchand@redhat.com, ray.kinsella@intel.com,
	nhorman@tuxdriver.com, aconole@redhat.com,
	maicolgabriel@hotmail.com, thomas@monjalon.net,
	bruce.richardson@intel.com, Conor Walsh <conor.walsh@intel.com>
Subject: [dpdk-dev] [PATCH v2 3/4] buildtools: add script to setup abi checks for meson
Date: Thu, 10 Sep 2020 14:21:20 +0000
Message-ID: <20200910142121.3995680-4-conor.walsh@intel.com> (raw)
In-Reply-To: <20200910142121.3995680-1-conor.walsh@intel.com>

This patch adds a script that is intended to be invoked by meson to
do the required setup for performing ABI breakage checks at build time.
The required ABI dump archives can come from several sources including
being generated at build time or prebuilt archives can be pulled from a
remote http location or local directory.
Invoke using "./abi-setup.py -t <tag> -d <dpdk_source_path>"
 - <tag>: dpdk tag e.g. "v20.11"
 - <dpdk_source_path>: path to dpdk source directory
E.g. "./abi-setup.py -t v20.08 -d /root/dpdk"
As this script is intended to be run by meson during a build
some options can be specified by environmental variables:
 - DPDK_ABI_DUMPS_PATH: Can be used to specify a custom directory for the
   systems dump directories.
 - CC: The required compiler will be determined using this
   environmental variable
 - DPDK_ABI_TAR_URI: Can be used to specify a location that the script
   can pull prebuilt or cached dump archives from. This can be a remote
   http location or a local directory
After the script has setup an appropriate ABI dump directory using one of
the multiple methods available to it, it will print the location of this
directory to the command line.

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
---
 buildtools/abi-setup.py | 104 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)
 create mode 100755 buildtools/abi-setup.py

diff --git a/buildtools/abi-setup.py b/buildtools/abi-setup.py
new file mode 100755
index 000000000..c156b3c73
--- /dev/null
+++ b/buildtools/abi-setup.py
@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Intel Corporation
+
+import sys
+import os
+import argparse
+
+# Get command line arguments
+parser = argparse.ArgumentParser(usage='\rThis script is intended to setup ABI dumps for meson to perform ABI checks\n'+
+                                       'Supported environmental variables\n'+
+                                       '\t- DPDK_ABI_DUMPS_PATH: Can be used to specify a custom directory for the systems dump directories.\n'+
+                                       '\t- CC: The required compiler will be determined using this environmental variable.\n'+
+                                       '\t- DPDK_ABI_TAR_URI: Can be used to specify a location that the script can pull prebuilt or cached dump archives from. This can be a remote http location or a local directory.\n')
+parser.add_argument('-t', '--tag', dest='tag', type=str, help='DPDK tag e.g. latest or v20.11')
+parser.add_argument('-d', '--dpdk', dest='dpdk', type=str, help='Path to DPDK source directory')
+args = parser.parse_args()
+
+# Get the DPDK tag if not supplied set as latest
+if args.tag:
+    user_tag = args.tag
+else:
+    user_tag = 'latest'
+
+tag = ''
+# If the user did not supply tag or wants latest then get latest tag
+if user_tag == 'latest':
+    # Get latest quarterly build tag from git repo
+    tag = os.popen('git ls-remote --tags http://dpdk.org/git/dpdk | grep -v "rc\|{}" | tail -n 1 | sed "s/.*\///"').read().strip()
+else:
+    tag = user_tag
+    # If the user supplied tag does not exist then fail
+    tag_check = int(os.popen('git ls-remote http://dpdk.org/git/dpdk refs/tags/'+tag+' | wc -l').read().strip())
+    if tag_check != 1:
+        print('ERROR supplied tag does not exist in DPDK repo')
+        exit()
+
+# Get the specified compiler from system
+comp_env = 'CC'
+if comp_env in os.environ:
+    comp = os.environ[comp_env]
+else:
+    comp = 'gcc'
+
+# Get the systems architecture
+arch = os.popen('uname -m').read().strip()
+
+# Get devtools path
+devtools_path = ''
+if args.dpdk:
+    devtools_path = os.path.abspath(os.path.join(args.dpdk,'devtools'))
+else:
+    print('ERROR DPDK source directory must be specified')
+    exit()
+
+# Get the abi dumps folder from args or env fail if none supplied
+abi_folder = ''
+abi_env = 'DPDK_ABI_DUMPS_PATH'
+if abi_env in os.environ:
+    abi_folder = os.path.abspath(os.environ[abi_env])
+else:
+    abi_folder = os.path.abspath(os.path.join(args.dpdk,'abi_dumps'))
+
+# If the directory doesn't exist create it and add a README to explain what it does
+if not os.path.exists(abi_folder):
+    os.makedirs(abi_folder)
+    f=open(abi_folder+'/README','w+')
+    f.write('This directory has been setup to contain the ABI dump folders needed to perform ABI checks\n')
+    f.write('Directories here must be in the format {DPDK Tag}-{Compiler ID}-{Architecture}-abi_dump\n')
+    f.write('e.g. v20.11-gcc-x86_64-abi_dump\n')
+    f.write('Directories that do not use this format will not be picked up by the meson ABI checks\n')
+    f.write('This directory is managed automatically unless desired by the user\n')
+    f.close()
+
+# Move to abi folder
+os.chdir(abi_folder)
+abi_dump=tag+'-'+comp+'-'+arch+'-abi_dump'
+# Download and untar abi dump if not present
+if not os.path.exists(abi_dump):
+    # Check DPDK_ABI_TAR_URI for the location of the tarballs local or web
+    tar_uri_env = 'DPDK_ABI_TAR_URI'
+    if tar_uri_env in os.environ:
+        abi_tar_uri = os.environ[tar_uri_env]
+        if abi_tar_uri.startswith('http'):
+            # Wget the required tarball
+            os.popen('wget '+os.path.join(abi_tar_uri,abi_dump)+'.tar.gz >/dev/null 2>&1').read()
+        else:
+            abi_tar_uri = os.path.abspath(abi_tar_uri)
+            os.popen('cp '+os.path.join(abi_tar_uri,abi_dump)+'.tar.gz . >/dev/null 2>&1').read()
+    # Check tarball was downloaded
+    if os.path.isfile(abi_dump+'.tar.gz'):
+        os.popen('tar -xzf '+abi_dump+'.tar.gz >/dev/null 2>&1').read()
+        os.popen('rm -rf '+abi_dump+'.tar.gz').read()
+    # If the tarball was not found then generate it
+    else:
+        os.popen(os.path.join(devtools_path,'gen-abi-tarball.py')+' -t '+tag+' -a '+arch+' >/dev/null 2>&1').read()
+        if not os.path.isfile(abi_dump+'.tar.gz'):
+            print('ERROR ABI check generation failed '+os.path.join(devtools_path,'gen-abi-tarball.py')+' -t '+tag+' -a '+arch)
+            exit()
+        os.popen('tar -xzf '+abi_dump+'.tar.gz >/dev/null 2>&1').read()
+        os.popen('rm -rf '+abi_dump+'.tar.gz').read()
+
+# Tell user where specified directory is
+print(os.path.abspath(abi_dump))
-- 
2.25.1


  parent reply	other threads:[~2020-09-10 14:21 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-10 14:01 [dpdk-dev] [PATCH 0/4] abi breakage " Conor Walsh
2020-09-10 14:01 ` [dpdk-dev] [PATCH 1/4] devtools: bug fix for gen-abi.sh Conor Walsh
2020-09-10 14:01 ` [dpdk-dev] [PATCH 2/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-09-10 14:01 ` [dpdk-dev] [PATCH 3/4] buildtools: add script to setup abi checks for meson Conor Walsh
2020-09-10 14:01 ` [dpdk-dev] [PATCH 4/4] build: add abi breakage checks to meson Conor Walsh
2020-09-10 14:21 ` [dpdk-dev] [PATCH v2 0/4] abi breakage checks for meson Conor Walsh
2020-09-10 14:21   ` [dpdk-dev] [PATCH v2 1/4] devtools: bug fix for gen-abi.sh Conor Walsh
2020-09-10 14:21   ` [dpdk-dev] [PATCH v2 2/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-09-10 14:21   ` Conor Walsh [this message]
2020-09-10 14:21   ` [dpdk-dev] [PATCH v2 4/4] build: add abi breakage checks to meson Conor Walsh
2020-09-11 16:03   ` [dpdk-dev] [PATCH v3 0/4] abi breakage checks for meson Conor Walsh
2020-09-11 16:03     ` [dpdk-dev] [PATCH v3 1/4] devtools: bug fix for gen-abi.sh Conor Walsh
2020-09-11 16:03     ` [dpdk-dev] [PATCH v3 2/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-09-14 12:50       ` Burakov, Anatoly
2020-09-15 14:35         ` Aaron Conole
2020-09-15 14:49           ` Walsh, Conor
2020-09-11 16:03     ` [dpdk-dev] [PATCH v3 3/4] buildtools: add script to setup abi checks for meson Conor Walsh
2020-09-11 16:03     ` [dpdk-dev] [PATCH v3 4/4] build: add abi breakage checks to meson Conor Walsh
2020-09-14  8:08     ` [dpdk-dev] [PATCH v3 0/4] abi breakage checks for meson Thomas Monjalon
2020-09-14  8:30       ` Kinsella, Ray
2020-09-14  9:34       ` Kinsella, Ray
2020-09-18 12:11     ` [dpdk-dev] [PATCH v4 " Conor Walsh
2020-09-18 12:11       ` [dpdk-dev] [PATCH v4 1/4] devtools: bug fix for gen-abi.sh Conor Walsh
2020-09-18 12:11       ` [dpdk-dev] [PATCH v4 2/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-09-18 12:11       ` [dpdk-dev] [PATCH v4 3/4] buildtools: add script to setup abi checks for meson Conor Walsh
2020-09-18 12:11       ` [dpdk-dev] [PATCH v4 4/4] build: add abi breakage checks to meson Conor Walsh
2020-10-12  8:08       ` [dpdk-dev] [PATCH v5 0/4] devtools: abi breakage checks Conor Walsh
2020-10-12  8:08         ` [dpdk-dev] [PATCH v5 1/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-10-12  8:08         ` [dpdk-dev] [PATCH v5 2/4] devtools: abi and UX changes for test-meson-builds.sh Conor Walsh
2020-10-12  8:08         ` [dpdk-dev] [PATCH v5 3/4] devtools: change dump file not found to warning in check-abi.sh Conor Walsh
2020-10-12  8:08         ` [dpdk-dev] [PATCH v5 4/4] doc: test-meson-builds.sh doc updates Conor Walsh
2020-10-12 13:03         ` [dpdk-dev] [PATCH v6 0/4] devtools: abi breakage checks Conor Walsh
2020-10-12 13:03           ` [dpdk-dev] [PATCH v6 1/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-10-14  9:38             ` Kinsella, Ray
2020-10-12 13:03           ` [dpdk-dev] [PATCH v6 2/4] devtools: abi and UX changes for test-meson-builds.sh Conor Walsh
2020-10-14  9:43             ` Kinsella, Ray
2020-10-12 13:03           ` [dpdk-dev] [PATCH v6 3/4] devtools: change dump file not found to warning in check-abi.sh Conor Walsh
2020-10-14  9:44             ` Kinsella, Ray
2020-10-12 13:03           ` [dpdk-dev] [PATCH v6 4/4] doc: test-meson-builds.sh doc updates Conor Walsh
2020-10-14  9:46             ` Kinsella, Ray
2020-10-14  9:37           ` [dpdk-dev] [PATCH v6 0/4] devtools: abi breakage checks Kinsella, Ray
2020-10-14 10:33             ` Walsh, Conor
2020-10-14 10:41           ` [dpdk-dev] [PATCH v7 " Conor Walsh
2020-10-14 10:41             ` [dpdk-dev] [PATCH v7 1/4] devtools: add generation of compressed abi dump archives Conor Walsh
2020-10-15 10:15               ` Kinsella, Ray
2020-10-14 10:41             ` [dpdk-dev] [PATCH v7 2/4] devtools: abi and UX changes for test-meson-builds.sh Conor Walsh
2020-10-15 10:16               ` Kinsella, Ray
2020-10-14 10:41             ` [dpdk-dev] [PATCH v7 3/4] devtools: change not found to warning check-abi.sh Conor Walsh
2020-10-14 10:41             ` [dpdk-dev] [PATCH v7 4/4] doc: test-meson-builds.sh doc updates Conor Walsh
     [not found]             ` <7206c209-ed4a-2aeb-12d8-ee162ef92596@ashroe.eu>
     [not found]               ` <CAJFAV8wmpft6XLRg1RAL+d4ibbJVrR9C0ghkE-kqyig_q_Meeg@mail.gmail.com>
2020-11-03 10:07                 ` [dpdk-dev] [PATCH v7 0/4] devtools: abi breakage checks Kinsella, Ray
2020-11-10 12:53                   ` David Marchand
2020-11-10 13:54                     ` Kinsella, Ray
2020-11-10 13:57                       ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200910142121.3995680-4-conor.walsh@intel.com \
    --to=conor.walsh@intel.com \
    --cc=aconole@redhat.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=maicolgabriel@hotmail.com \
    --cc=nhorman@tuxdriver.com \
    --cc=ray.kinsella@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git