From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8F53C42B83 for ; Tue, 23 May 2023 19:04:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C10640A80; Tue, 23 May 2023 19:04:43 +0200 (CEST) Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by mails.dpdk.org (Postfix) with ESMTP id A7E5D42D41 for ; Tue, 23 May 2023 19:04:42 +0200 (CEST) Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-3f6c0d651adso3402131cf.2 for ; Tue, 23 May 2023 10:04:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1684861482; x=1687453482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bO6ARftYqPfop+EdR1mJ3F7+CcJD16idalCT1MZv9kc=; b=QrtGuc35ye5gufRkx1zyIB4Y2PtiOC0qqBOqwx0g7KLut4xmC0ZIgPwL/E6PkudLo+ I7VEpv79FsTBJeNwVdgG9XlzBcYpKrM5ml7sViXFArCF2AyhveHBdlXq3+405Ooajk5L bG88/ZAMky9UfVhkWL920DeVobbhM2sapc6Fk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684861482; x=1687453482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bO6ARftYqPfop+EdR1mJ3F7+CcJD16idalCT1MZv9kc=; b=A8YogT3ij4zYUsLFbn37dpFaZR0/1SYkYXmXXhRbVrWVNjTSlnVXlgZSN9XU6QmmFg MqMNKdxuQcXt7zyuDQSFv9fCaN+i3H+7bX6/GBMtowwzWdz0NYPdAT1dz+WgAJ+vh4yH CGlXows/NdsNwXqDzjUruZRq4ibiSvcbEtSbO4TOCY82hmj2Lad40IgG3ywNxK1hQQlW bqmgimcw2XdddeGk7f9wS3KJl9L0336L/gtyihRu/QFAx47yA8QfnH1YeBDtfyYMyVA0 LiV0gWroONigdnyPmkuYsdznna3ju/eiL0Elfae8wPgJyyhL4yyz+m7NtuYtTtFAI0rK AL7A== X-Gm-Message-State: AC+VfDzYPlve+5X63VOWkGOccD6yNC89VIAvZwGwutZzQPyF4hpx02Oj zpLbwif1fSbsLEJphZJNhS4yOPJLOtPVM3jX20cTwqrlbEsrxME/jFp09/Dc95rw3eZWWxvMtIb 2dj7KRLYqsAjeKa/JfwUDMbK6hCHlYdtccTT3owV3fTEGst74DWVQDy0Mx9FYTg== X-Google-Smtp-Source: ACHHUZ6DwebvUszsZ78pg++EmBqYIbYT6dmQN532XOkOhcTz9TdjjrjEnwJtaex7mFqaDlG+huFs0A== X-Received: by 2002:a05:622a:1a90:b0:3f6:82e8:a09f with SMTP id s16-20020a05622a1a9000b003f682e8a09fmr26801440qtc.50.1684861481740; Tue, 23 May 2023 10:04:41 -0700 (PDT) Received: from ahassick-Desktop.iol.unh.edu ([2606:4100:3880:1220:7704:829d:adaa:3294]) by smtp.gmail.com with ESMTPSA id g14-20020ac8774e000000b003e38c9a2a22sm2996553qtu.92.2023.05.23.10.04.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 10:04:41 -0700 (PDT) From: Adam Hassick To: ci@dpdk.org Cc: Adam Hassick , aconole@redhat.com, alialnu@nvidia.com Subject: [PATCH v5 07/11] containers/inventory: Add ABI rev and coverity attribute Date: Tue, 23 May 2023 13:04:09 -0400 Message-Id: <20230523170413.812922-8-ahassick@iol.unh.edu> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523170413.812922-1-ahassick@iol.unh.edu> References: <20230523170413.812922-1-ahassick@iol.unh.edu> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Adds a new attribute that allows the version of libabigail to be checked out to a specific revision. This was needed because libabigail 2.3.0 does not build on Debian 10 or 11 at the time of writing. Adds another ABI "version" to the inventory to represent the XML comparison reference. Adds another new attribute to specify if a container requires Coverity Scan binaries. Updates the versions of Fedora in the inventory. Adds a new target for building the Coverity Scan container. Signed-off-by: Adam Hassick --- containers/template_engine/inventory.yaml | 240 ++++++++++++------ .../template_engine/inventory_schema.json | 100 ++++++-- 2 files changed, 247 insertions(+), 93 deletions(-) diff --git a/containers/template_engine/inventory.yaml b/containers/template_engine/inventory.yaml index eb299c0..2eb371b 100644 --- a/containers/template_engine/inventory.yaml +++ b/containers/template_engine/inventory.yaml @@ -4,15 +4,38 @@ abi: # versions should also include the abi versions for all stable ABIs. versions: - - major: 20 # oldest supported lts - minor: 11 - git_tag: v20.11 - - major: 21 # most recent lts - minor: 11 - git_tag: v21.11 - # - major: 22 # current abi - # minor: 07 - # git_tag: v22.07 + # - dpdk_version: # LTS - 3 + # major: 20 + # minor: 11 + # revision: 0 + # git_tag: "20.11" + # abi_version: + # major: 21 + # minor: 0 + - dpdk_version: # LTS - 2 + major: 21 + minor: 11 + revision: 0 + git_tag: "21.11" + abi_version: + major: 22 + minor: 0 + - dpdk_version: + major: 22 # LTS - 1 + minor: 11 + revision: 1 + git_tag: "tags/v22.11.1" + abi_version: + major: 23 + minor: 10 # .xml comparison + - dpdk_version: + major: 22 # LTS + minor: 11 + revision: 1 + git_tag: "tags/v22.11.1" + abi_version: + major: 23 + minor: 11 # .so comparison dockerfiles: groups: @@ -44,6 +67,7 @@ dockerfiles: - ccache - zip - autoconf + - libbpf-devel redhat_family: parent: "rpm" packages: [] @@ -64,6 +88,13 @@ dockerfiles: - elfutils-devel - libxml2-devel - libabigail + - openssl-devel + fedora_coverity: + parent: fedora + packages: [] + fedora_clang: + parent: fedora + packages: [] centos8: parent: "redhat_family" packages: @@ -73,6 +104,7 @@ dockerfiles: - libpcap-devel - numactl-devel - libabigail + - openssl-devel centos9: parent: "redhat_family" package_manager_command: "yum install --allowerasing -y" @@ -86,30 +118,31 @@ dockerfiles: - g++ - elfutils-devel - libxml2-devel + - openssl-devel rhel: parent: "redhat_family" - package_manager_command: "yum --disableplugin=subscription-manager install -y" + package_manager_command: "yum install -y" packages: - elfutils-libelf-devel - jansson-devel - libatomic - libpcap-devel - numactl-devel + - openssl-devel + - libabigail rhel7: parent: "rhel" packages: - - libabigail - ninja-build rhel8: parent: "rhel" - package_manager_command: "yum --allowerasing --disableplugin=subscription-manager install -y" + package_manager_command: "yum --allowerasing install -y" packages: - - libabigail - ninja-build - libxml2-devel rhel9: parent: "rhel" - package_manager_command: "yum --allowerasing --disableplugin=subscription-manager install -y" + package_manager_command: "yum --allowerasing install -y" packages: - libxml2-devel opensuse: @@ -126,34 +159,34 @@ dockerfiles: - libnuma-devel - libabigail-tools - tar + - openssl-devel alpine: - parent: 'all' - package_manager_command: 'apk add' + parent: "all" + package_manager_command: "apk add" packages: - - python3 - - py3-pip - - alpine-sdk + - bsd-compat-headers - ccache - - cmake - - bash - - autoconf - - automake - - libtool - - elfutils-dev - - libxml2-dev - - fts-dev - - fts - - numactl-dev + - jansson-dev + - libarchive-dev + - libbpf-dev - libc-dev - - bsd-compat-headers - - libexecinfo-dev + - libpcap-dev - linux-headers - - libatomic + - ninja + - numactl-dev + - openssl-dev + - py3-pip + - py3-elftools + - py3-setuptools + - py3-wheel + - zlib-dev + - python3 + - bash - zip - - pkgconfig + - meson arch: - parent: 'all' - package_manager_command: 'pacman -Syu --noconfirm' + parent: "all" + package_manager_command: "pacman -Syu --noconfirm" packages: - glibc - cmake @@ -171,9 +204,11 @@ dockerfiles: - diffutils - python-pip - libabigail + - openssl + - libbpf debian: - parent: 'all' - package_manager_command: 'apt-get update && apt-get install --no-install-recommends -y' + parent: "all" + package_manager_command: "apt-get update && apt-get install --no-install-recommends -y" packages: - libnuma-dev - zip @@ -203,20 +238,26 @@ dockerfiles: - parallel - python3-grpcio - lcov + - libbpf-dev + - libssl-dev debian10: - parent: 'debian' + parent: "debian" packages: [] debian11: - parent: 'debian' + parent: "debian" + packages: + - nasm + debian11_arm_ipsec: + parent: "debian11" packages: [] ubuntu: parent: 'debian' packages: [] ubuntu18.04: - parent: 'ubuntu' + parent: "ubuntu" packages: [] ubuntu20.04: - parent: 'ubuntu' + parent: "ubuntu" packages: - python3-grpcio - lcov @@ -224,14 +265,14 @@ dockerfiles: - default-jre - nodejs ubuntu22.04: - parent: 'ubuntu' + parent: "ubuntu" packages: - python3-grpcio - lcov - abigail-tools - jq ubuntu_cross: # x86 -> ARM - parent: 'ubuntu20.04' + parent: "ubuntu20.04" packages: - clang - llvm @@ -248,26 +289,30 @@ dockerfiles: - libc6-dev-armhf-cross - pkg-config-arm-linux-gnueabihf ubuntu_sve: - parent: 'ubuntu20.04' + parent: "ubuntu20.04" packages: - gcc-10 targets: - -# Alpine + # Alpine - name: alpine_compile - group: 'alpine' + group: "alpine" base_image: alpine - platforms: [linux/amd64] + platforms: [linux/amd64, linux/arm64] + force_disable_abi: true + minimum_dpdk_version: + major: 22 + minor: 0 + revision: 0 extra_information: environment_id: 45 -# Arch + # Arch - name: arch_compile - group: 'arch' + group: "arch" base_image: archlinux:base platforms: [linux/amd64] -# Fedora + # Fedora - name: fedora35_compile group: "fedora" base_image: fedora:35 @@ -275,13 +320,36 @@ dockerfiles: - name: fedora36_compile group: "fedora" base_image: fedora:36 - platforms: [linux/amd64] - - name: fedora36_clang + platforms: [linux/amd64, linux/arm64] + - name: fedora37_compile group: "fedora" - base_image: fedora:36 + base_image: fedora:37 + platforms: [linux/amd64, linux/arm64] + - name: fedora38_compile + group: "fedora" + base_image: fedora:38 + platforms: [linux/amd64, linux/arm64] + + - name: fedora38_clang + group: "fedora_clang" + base_image: fedora:38 + platforms: [linux/amd64, linux/arm64] + + - name: fedora37_coverity + group: "fedora_coverity" + base_image: fedora:37 + force_disable_abi: true + requires_coverity: true platforms: [linux/amd64] -# CentOs + - name: fedora37_coverity + group: "fedora_coverity" + base_image: fedora:37 + force_disable_abi: true + requires_coverity: true + platforms: [linux/amd64] + + # CentOs - name: centosstream8_compile group: "centos8" base_image: quay.io/centos/centos:stream8 @@ -290,9 +358,9 @@ dockerfiles: - name: centosstream9_compile group: "centos9" base_image: quay.io/centos/centos:stream9 - platforms: [linux/amd64] + platforms: [linux/amd64, linux/arm64] -# RHEL + # RHEL - name: rhel7_compile group: "rhel7" base_image: registry.access.redhat.com/ubi7/ubi @@ -306,30 +374,34 @@ dockerfiles: - name: rhel9_compile group: "rhel9" base_image: registry.access.redhat.com/ubi9/ubi - platforms: [linux/amd64] + platforms: [linux/amd64, linux/arm64] -# OpenSuse + # OpenSuse - name: opensuse-leap15_compile group: "opensuse" base_image: opensuse/leap:15 - platforms: [linux/amd64] + platforms: [linux/amd64, linux/arm64] -# Debian + # Debian - name: debian10_compile # debian 10 - group: 'debian10' - base_image: 'debian:buster' + group: "debian10" + base_image: "debian:buster" + libabigail_revision: libabigail-2.2 platforms: [linux/amd64] - name: debian11_compile # debian 11 - group: 'debian11' - base_image: 'debian:bullseye' - platforms: [linux/amd64] + group: "debian11" + base_image: "debian:bullseye" + libabigail_revision: libabigail-2.2 + platforms: + - linux/amd64 + - linux/arm64 -# Ubuntu + # Ubuntu - name: ubuntu20.04_compile - group: 'ubuntu20.04' - base_image: 'ubuntu:20.04' - platforms: [linux/amd64] + group: "ubuntu20.04" + base_image: "ubuntu:20.04" + platforms: [linux/amd64, linux/arm64] extra_tags: - $R/ubuntu20.04_libabigail:latest - $R/ubuntu20.04_libabigail:$T @@ -340,17 +412,31 @@ dockerfiles: # x86 -> ARM cross compile - name: ubuntu20.04_cross - group: 'ubuntu_cross' - base_image: 'ubuntu:20.04' + group: "ubuntu_cross" + base_image: "ubuntu:20.04" platforms: [linux/amd64] cross_file: "config/arm/arm64_armv8_linux_gcc" + # Also, x86 -> ARM cross compile (for SVE test) - name: ubuntu20.04_sve_compile - group: 'ubuntu_sve' - base_image: 'arm64v8/ubuntu:20.04' + group: "ubuntu_sve" + base_image: "arm64v8/ubuntu:20.04" platforms: [linux/arm64] + # TEMPORARY: This has the forked IPsec lib developed by ARM. This causes issues on older LTS versions of DPDK. + # Once all supported and tested LTS versions no longer break when this lib is installed, recombine with debian11_compile. + - name: debian11_arm_ipsec + group: "debian11_arm_ipsec" + base_image: "debian:bullseye" + platforms: [linux/arm64] + force_disable_abi: true + extra_tags: + - $R/debian11_zuc_compile:latest + - $R/debian11_snow3g_compile:latest + - $R/debian11_zuc_compile:$T + - $R/debian11_snow3g_compile:$T + - name: ubuntu22.04_compile - group: 'ubuntu22.04' - base_image: 'ubuntu:22.04' - platforms: [linux/amd64] \ No newline at end of file + group: "ubuntu22.04" + base_image: "ubuntu:22.04" + platforms: [linux/amd64] diff --git a/containers/template_engine/inventory_schema.json b/containers/template_engine/inventory_schema.json index 0e641b7..7b79e44 100644 --- a/containers/template_engine/inventory_schema.json +++ b/containers/template_engine/inventory_schema.json @@ -2,7 +2,6 @@ "$schema": "https://json-schema.org/draft-07/schema", "title": "dpdklab-ci container inventory schema", "$comment": "SPDX-License-Identifier: BSD-3-Clause\nCopyright (c) 2022 University of New Hampshire", - "definitions": {}, "type": "object", "properties": { "abi": { @@ -13,25 +12,57 @@ "items": { "type": "object", "properties": { - "major": { - "type": "integer", - "description": "The major version number (usually the year of release)" - }, - "minor": { - "type": "integer", - "description": "The minor version number (usually the month of release)" + "dpdk_version": { + "type": "object", + "properties": { + "major": { + "type": "integer", + "description": "The major version number (usually the year of release)" + }, + "minor": { + "type": "integer", + "description": "The minor version number (usually the month of release)" + }, + "revision": { + "type": "integer", + "description": "The revision version number. Starts at 0." + }, + "git_tag": { + "type": "string", + "description": "The git tag to check out to get this ABI version" + } + }, + "required": [ + "major", + "minor", + "revision", + "git_tag" + ], + "additionalProperties": false }, - "git_tag": { - "type": "string", - "description": "The git tag to check out to get this ABI version" + "abi_version": { + "type": "object", + "properties": { + "major": { + "type": "integer", + "description": "The major version number (usually the year of release)" + }, + "minor": { + "type": "integer", + "description": "The minor version number (usually the month of release)" + } + }, + "required": [ + "major", + "minor" + ], + "additionalProperties": false } }, "required": [ - "major", - "minor", - "git_tag" - ], - "additionalProperties": false + "dpdk_version", + "abi_version" + ] }, "minItems": 0 } @@ -113,6 +144,43 @@ "type": "string", "description": "A cross file to compile meson with. If not present, meson's automatic behavior will be used." }, + "libabigail_revision": { + "type": "string", + "description": "A revision or branch of libabigail to check out before building for a given target." + }, + "force_disable_abi": { + "type": "boolean", + "description": "Whether to forcibly disable ABI for this target. This should be used as a temporary measure until a fix can be created.", + "default": "false" + }, + "requires_coverity": { + "type": "boolean", + "description": "Whether this target requires the Coverity Scan binaries.", + "default": "false" + }, + "minimum_dpdk_version": { + "type": "object", + "properties": { + "major": { + "type": "integer", + "description": "The major version number (usually the year of release)" + }, + "minor": { + "type": "integer", + "description": "The minor version number (usually the month of release)" + }, + "revision": { + "type": "integer", + "description": "The revision version number. Starts at 0." + } + }, + "required": [ + "major", + "minor", + "revision" + ], + "additionalProperties": false + }, "extra_tags": { "type": "array", "description": "Tags to apply to this image", -- 2.34.1