From: ohilyard@iol.unh.edu
To: ci@dpdk.org
Cc: aconole@redhat.com, alialnu@nvidia.com,
Owen Hilyard <ohilyard@iol.unh.edu>
Subject: [PATCH v2 2/6] containers/inventory: Add inventory for container builder
Date: Wed, 19 Oct 2022 08:53:03 -0400 [thread overview]
Message-ID: <20221019125307.258582-3-ohilyard@iol.unh.edu> (raw)
In-Reply-To: <20221019125307.258582-1-ohilyard@iol.unh.edu>
From: Owen Hilyard <ohilyard@iol.unh.edu>
Adds a yaml file used to define what containers should be built, what
packages the containers should have, what platforms to build for and how they
should be tagged.
Also adds a JSON schema, which can be used to validate that the
inventory file is well-formed.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
containers/template_engine/inventory.yaml | 356 ++++++++++++++++++
.../template_engine/inventory_schema.json | 153 ++++++++
2 files changed, 509 insertions(+)
create mode 100644 containers/template_engine/inventory.yaml
create mode 100644 containers/template_engine/inventory_schema.json
diff --git a/containers/template_engine/inventory.yaml b/containers/template_engine/inventory.yaml
new file mode 100644
index 0000000..fc06641
--- /dev/null
+++ b/containers/template_engine/inventory.yaml
@@ -0,0 +1,356 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2022 University of New Hampshire
+---
+abi:
+ # versions should also include the abi versions for all stable ABIs.
+ versions:
+ - major: 20 # oldest supported lts
+ minor: 11
+ git_tag: v20.11
+ - major: 21 # most recent lts
+ minor: 11
+ git_tag: v21.11
+ # - major: 22 # current abi
+ # minor: 07
+ # git_tag: v22.07
+
+dockerfiles:
+ groups:
+ all:
+ package_manager_command: "NO_INSTALL_COMMAND_SET"
+ parent: "NONE"
+ packages:
+ - gcc
+ - make
+ - git
+ - sudo
+ - curl
+ - automake
+ - autoconf
+ - flex
+ - bison
+ - bc
+ rpm:
+ parent: "all"
+ package_manager_command: yum install -y
+ packages:
+ - diffutils
+ - pkg-config
+ - python3
+ - python3-pip
+ - librdmacm
+ - rdma-core-devel
+ - libmnl-devel
+ - ccache
+ - zip
+ - autoconf
+ redhat_family:
+ parent: "rpm"
+ packages: []
+ fedora:
+ parent: "redhat_family"
+ packages:
+ - python3-pyelftools
+ - zip
+ - clang
+ - ninja-build
+ - elfutils-libelf-devel
+ - jansson-devel
+ - libatomic
+ - libpcap-devel
+ - numactl-devel
+ - findutils
+ - libtool
+ - elfutils-devel
+ - libxml2-devel
+ - libabigail
+ centos8:
+ parent: "redhat_family"
+ packages:
+ - elfutils-libelf-devel
+ - jansson-devel
+ - libatomic
+ - libpcap-devel
+ - numactl-devel
+ - libabigail
+ centos9:
+ parent: "redhat_family"
+ package_manager_command: "yum install --allowerasing -y"
+ packages:
+ - elfutils-libelf-devel
+ - jansson-devel
+ - libatomic
+ - libpcap-devel
+ - numactl-devel
+ - libtool
+ - g++
+ - elfutils-devel
+ - libxml2-devel
+ rhel:
+ parent: "redhat_family"
+ package_manager_command: "yum --disableplugin=subscription-manager install -y"
+ packages:
+ - elfutils-libelf-devel
+ - jansson-devel
+ - libatomic
+ - libpcap-devel
+ - numactl-devel
+ rhel7:
+ parent: "rhel"
+ packages:
+ - libabigail
+ - ninja-build
+ rhel8:
+ parent: "rhel"
+ package_manager_command: "yum --allowerasing --disableplugin=subscription-manager install -y"
+ packages:
+ - libabigail
+ - ninja-build
+ - libxml2-devel
+ rhel9:
+ parent: "rhel"
+ package_manager_command: "yum --allowerasing --disableplugin=subscription-manager install -y"
+ packages:
+ - libxml2-devel
+ opensuse:
+ parent: "rpm"
+ package_manager_command: "zypper install -y"
+ packages:
+ - python3-pyelftools
+ - ninja
+ - gzip
+ - libelf-devel
+ - libjansson-devel
+ - librdmacm1
+ - libatomic1
+ - libnuma-devel
+ - libabigail-tools
+ - tar
+ alpine:
+ parent: 'all'
+ package_manager_command: 'apk add'
+ packages:
+ - python3
+ - py3-pip
+ - alpine-sdk
+ - ccache
+ - cmake
+ - bash
+ - autoconf
+ - automake
+ - libtool
+ - elfutils-dev
+ - libxml2-dev
+ - fts-dev
+ - fts
+ - numactl-dev
+ - libc-dev
+ - bsd-compat-headers
+ - libexecinfo-dev
+ - linux-headers
+ - libatomic
+ - zip
+ - pkgconfig
+ arch:
+ parent: 'all'
+ package_manager_command: 'pacman -Syu --noconfirm'
+ packages:
+ - glibc
+ - cmake
+ - ninja
+ - pandoc
+ - python
+ - python-docutils
+ - ethtool
+ - file
+ - fakeroot
+ - pkg-config
+ - numactl
+ - elfutils
+ - gawk
+ - diffutils
+ - python-pip
+ - libabigail
+ debian:
+ parent: 'all'
+ package_manager_command: 'apt-get update && apt-get install --no-install-recommends -y'
+ packages:
+ - libnuma-dev
+ - zip
+ - librdmacm1
+ - libcrypto++-dev
+ - python3-pyelftools
+ - libxml2-dev
+ - libelf-dev
+ - libpcap-dev
+ - libjansson-dev
+ - python3-setuptools
+ - autoconf
+ - pkg-config
+ - libtool
+ - ccache
+ - libatomic1
+ - python3-wheel
+ - libdw-dev
+ - rdma-core
+ - ninja-build
+ - automake
+ - librdmacm-dev
+ - libmnl-dev
+ - python3-pip
+ - build-essential
+ - aspell-en
+ - parallel
+ - python3-grpcio
+ - lcov
+ debian10:
+ parent: 'debian'
+ packages: []
+ debian11:
+ parent: 'debian'
+ packages: []
+ ubuntu:
+ parent: 'debian'
+ packages: []
+ ubuntu18.04:
+ parent: 'ubuntu'
+ packages: []
+ ubuntu20.04:
+ parent: 'ubuntu'
+ packages:
+ - python3-grpcio
+ - lcov
+ - jq
+ - default-jre
+ - nodejs
+ ubuntu22.04:
+ parent: 'ubuntu'
+ packages:
+ - python3-grpcio
+ - lcov
+ - abigail-tools
+ - jq
+ ubuntu_cross: # x86 -> ARM
+ parent: 'ubuntu20.04'
+ packages:
+ - clang
+ - llvm
+ - llvm-dev
+ - llvm-runtime
+ - lld
+ - gcc-aarch64-linux-gnu
+ - libgcc-9-dev-arm64-cross
+ - libatomic1-arm64-cross
+ - libc6-dev-arm64-cross
+ - pkg-config-aarch64-linux-gnu
+ - dpkg-dev
+ - gcc-arm-linux-gnueabihf
+ - libc6-dev-armhf-cross
+ - pkg-config-arm-linux-gnueabihf
+ ubuntu_sve:
+ parent: 'ubuntu20.04'
+ packages:
+ - gcc-10
+ targets:
+
+# Alpine
+ - name: alpine_compile
+ group: 'alpine'
+ base_image: alpine
+ platforms: [linux/amd64]
+ extra_information:
+ environment_id: 45
+
+# Arch
+ - name: arch_compile
+ group: 'arch'
+ base_image: archlinux:base
+ platforms: [linux/amd64]
+
+# Fedora
+ - name: fedora35_compile
+ group: "fedora"
+ base_image: fedora:35
+ platforms: [linux/amd64]
+ - name: fedora36_compile
+ group: "fedora"
+ base_image: fedora:36
+ platforms: [linux/amd64]
+ - name: fedora36_clang
+ group: "fedora"
+ base_image: fedora:36
+ platforms: [linux/amd64]
+
+# CentOs
+ - name: centosstream8_compile
+ group: "centos8"
+ base_image: quay.io/centos/centos:stream8
+ platforms: [linux/amd64]
+
+ - name: centosstream9_compile
+ group: "centos9"
+ base_image: quay.io/centos/centos:stream9
+ platforms: [linux/amd64]
+
+# RHEL
+ - name: rhel7_compile
+ group: "rhel7"
+ base_image: registry.access.redhat.com/ubi7/ubi
+ platforms: [linux/amd64]
+
+ - name: rhel8_compile
+ group: "rhel8"
+ base_image: registry.access.redhat.com/ubi8/ubi
+ platforms: [linux/amd64]
+
+ - name: rhel9_compile
+ group: "rhel9"
+ base_image: registry.access.redhat.com/ubi9/ubi
+ platforms: [linux/amd64]
+
+# OpenSuse
+ - name: opensuse-leap15_compile
+ group: "opensuse"
+ base_image: opensuse/leap:15
+ platforms: [linux/amd64]
+
+# Debian
+ - name: debian10_compile # debian 10
+ group: 'debian10'
+ base_image: 'debian:buster'
+ platforms: [linux/amd64]
+
+ - name: debian11_compile # debian 11
+ group: 'debian11'
+ base_image: 'debian:bullseye'
+ platforms: [linux/amd64]
+
+# Ubuntu
+ - name: ubuntu20.04_compile
+ group: 'ubuntu20.04'
+ base_image: 'ubuntu:20.04'
+ platforms: [linux/amd64]
+ extra_tags:
+ - $R/ubuntu20.04_libabigail:latest
+ - $R/ubuntu20.04_libabigail:$T
+ - $R/ubuntu20.04_spell-check:latest
+ - $R/ubuntu20.04_spell-check:$T
+ - $R/ubuntu20.04_sonarscanner:latest
+ - $R/ubuntu20.04_sonarscanner:$T
+
+ # x86 -> ARM cross compile
+ - name: ubuntu20.04_cross
+ group: 'ubuntu_cross'
+ base_image: 'ubuntu:20.04'
+ platforms: [linux/amd64]
+ cross_file: "config/arm/arm64_armv8_linux_gcc"
+
+ - name: ubuntu20.04_sve_compile
+ group: 'ubuntu_sve'
+ base_image: 'arm64v8/ubuntu:20.04'
+ platforms: [linux/arm64]
+
+ - name: ubuntu22.04_compile
+ group: 'ubuntu22.04'
+ base_image: 'ubuntu:22.04'
+ platforms: [linux/amd64]
\ No newline at end of file
diff --git a/containers/template_engine/inventory_schema.json b/containers/template_engine/inventory_schema.json
new file mode 100644
index 0000000..0e641b7
--- /dev/null
+++ b/containers/template_engine/inventory_schema.json
@@ -0,0 +1,153 @@
+{
+ "$schema": "https://json-schema.org/draft-07/schema",
+ "title": "dpdklab-ci container inventory schema",
+ "$comment": "SPDX-License-Identifier: BSD-3-Clause\nCopyright (c) 2022 University of New Hampshire",
+ "definitions": {},
+ "type": "object",
+ "properties": {
+ "abi": {
+ "type": "object",
+ "properties": {
+ "versions": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "major": {
+ "type": "integer",
+ "description": "The major version number (usually the year of release)"
+ },
+ "minor": {
+ "type": "integer",
+ "description": "The minor version number (usually the month of release)"
+ },
+ "git_tag": {
+ "type": "string",
+ "description": "The git tag to check out to get this ABI version"
+ }
+ },
+ "required": [
+ "major",
+ "minor",
+ "git_tag"
+ ],
+ "additionalProperties": false
+ },
+ "minItems": 0
+ }
+ },
+ "required": [
+ "versions"
+ ],
+ "additionalProperties": false
+ },
+ "dockerfiles": {
+ "type": "object",
+ "properties": {
+ "groups": {
+ "type": "object",
+ "patternProperties": {
+ "^[a-z][a-z_0-9\\.]+$": {
+ "type": "object",
+ "properties": {
+ "package_manager_command": {
+ "type": "string",
+ "description": "The command to install packages without user input."
+ },
+ "parent": {
+ "type": "string",
+ "description": "Which group to inherit from, or NONE if no parent exists"
+ },
+ "packages": {
+ "type": "array",
+ "description": "A list of package names to install",
+ "items": {
+ "type": "string"
+ },
+ "uniqueItems": true
+ }
+ },
+ "required": [
+ "parent",
+ "packages"
+ ],
+ "additionalProperties": false
+ }
+ },
+ "minProperties": 1
+ },
+ "targets": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "The name of the docker container"
+ },
+ "group": {
+ "type": "string",
+ "description": "Which group the target is a member of"
+ },
+ "base_image": {
+ "type": "string",
+ "description": "The base docker image."
+ },
+ "platforms": {
+ "type": "array",
+ "description": "What platforms to build for",
+ "items": {
+ "type": "string",
+ "enum": [
+ "linux/amd64",
+ "linux/arm64",
+ "linux/riscv64",
+ "linux/ppc64le",
+ "linux/arm/v7",
+ "linux/arm/v6"
+ ]
+ },
+ "minItems": 1
+ },
+ "cross_file": {
+ "type": "string",
+ "description": "A cross file to compile meson with. If not present, meson's automatic behavior will be used."
+ },
+ "extra_tags": {
+ "type": "array",
+ "description": "Tags to apply to this image",
+ "items": {
+ "type": "string",
+ "description": "$R will be replaced with the 'registry_hostname' variable in this file. $T will be replaced with a timestamp in the form of '%Y-%m-%d'. $N will be replaced by the name of the target.. The tags '$R/$N:latest' and '$R/$N:$T' will always be present"
+ }
+ },
+ "extra_information": {
+ "type": "object",
+ "description": "Extra information to be stored as a JSON file at /container_info.json",
+ "additionalProperties": true
+ }
+ },
+ "required": [
+ "name",
+ "group",
+ "base_image",
+ "platforms"
+ ],
+ "additionalProperties": false
+ },
+ "minItems": 1
+ }
+ },
+ "required": [
+ "groups",
+ "targets"
+ ],
+ "additionalProperties": false
+ }
+ },
+ "required": [
+ "abi",
+ "dockerfiles"
+ ],
+ "additionalProperties": false
+}
\ No newline at end of file
--
2.34.1
next prev parent reply other threads:[~2022-10-19 12:53 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-19 12:53 [PATCH v2 0/6] Community Lab Container Definitions ohilyard
2022-10-19 12:53 ` [PATCH v2 1/6] containers/docs: Add container builder start ohilyard
2022-10-19 12:53 ` ohilyard [this message]
2022-10-19 12:53 ` [PATCH v2 3/6] containers/builder: Dockerfile creation script ohilyard
2022-10-19 12:53 ` [PATCH v2 4/6] containers/templates: Templates for Dockerfiles ohilyard
2022-10-19 19:24 ` Ali Alnubani
2022-10-24 12:52 ` Owen Hilyard
2022-10-19 12:53 ` [PATCH v2 5/6] containers/container_builder: Container for python scripts ohilyard
2022-10-19 12:53 ` [PATCH v2 6/6] containers/Makefile: Makefile to automate builds ohilyard
2022-10-19 14:31 ` [PATCH v2 0/6] Community Lab Container Definitions Ali Alnubani
2022-10-19 14:55 ` Owen Hilyard
2022-10-20 8:25 ` Ali Alnubani
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221019125307.258582-3-ohilyard@iol.unh.edu \
--to=ohilyard@iol.unh.edu \
--cc=aconole@redhat.com \
--cc=alialnu@nvidia.com \
--cc=ci@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).