From: Adam Hassick <ahassick@iol.unh.edu>
To: ci@dpdk.org
Cc: Adam Hassick <ahassick@iol.unh.edu>,
aconole@redhat.com, alialnu@nvidia.com
Subject: [PATCH v5 00/11] Community Lab Containers and Builder Engine
Date: Tue, 23 May 2023 13:04:02 -0400 [thread overview]
Message-ID: <20230523170413.812922-1-ahassick@iol.unh.edu> (raw)
This patch series contains a new version of the DPDK CI containers. The old
version was tied very tightly to the Community Lab Infrastructure, so it was
not suitable for general use. This version is designed to make adding new
OSes or OS versions as easy as possible. The minimum functionality can easily
be built on any system that can compile DPDK. It includes support for
building containers for other non-native architectures (ex: arm containers
on x86) and for baking ABI references into the images. Support for
building the Coverity Scan container image has also been added.
The inventory file as added in this patch series defines what the community lab
currently supports.
If you want to build these yourself, don't try to do parallel Makefile builds
on non-server hardware. Libabigail is built into the containers, and since it
is not avaliable in all distros it is compiled from source for many targets.
If embedding the abi is enabled (DPDK_CI_CONTAINERS_BUILD_ABI=Y), with the
current settings, DPDK will be compiled thrice for every target in the
inventory file. Due to how the containers are constructed, building on
non-native architectures is especially expensive because DPDK is compiled with
an emulated compiler in a qemu vm that only has one thread.
Adam Hassick (5):
containers/inventory: Add ABI rev and coverity attribute
containers/builder: Add arguments to templating script
containers/templates: Update templates
containers/Makefile: Add new variables and target changes
containers/docs: Update README
Owen Hilyard (6):
containers/docs: Add container builder start
containers/inventory: Add inventory for container builder
containers/builder: Dockerfile creation script
containers/templates: Templates for Dockerfiles
containers/container_builder: Container for python scripts
containers/Makefile: Makefile to automate builds
containers/Makefile | 252 ++++++++++
containers/README.md | 169 +++++++
containers/container_builder.dockerfile | 26 ++
containers/template_engine/inventory.yaml | 442 ++++++++++++++++++
.../template_engine/inventory_schema.json | 221 +++++++++
containers/template_engine/make_dockerfile.py | 358 ++++++++++++++
containers/template_engine/poetry.lock | 220 +++++++++
containers/template_engine/pyproject.toml | 21 +
.../templates/containers.makefile.j2 | 73 +++
.../templates/containers/alpine.dockerfile.j2 | 4 +
.../templates/containers/arch.dockerfile.j2 | 37 ++
.../templates/containers/base.dockerfile.j2 | 121 +++++
.../containers/centos8.dockerfile.j2 | 21 +
.../containers/centos9.dockerfile.j2 | 17 +
.../templates/containers/debian.dockerfile.j2 | 7 +
.../containers/debian10.dockerfile.j2 | 3 +
.../containers/debian11.dockerfile.j2 | 3 +
.../debian11_arm_ipsec.dockerfile.j2 | 16 +
.../containers/debian_bullseye.dockerfile.j2 | 3 +
.../containers/debian_buster.dockerfile.j2 | 3 +
.../templates/containers/fedora.dockerfile.j2 | 11 +
.../containers/fedora36_clang.dockerfile.j2 | 7 +
.../containers/fedora_clang.dockerfile.j2 | 7 +
.../containers/fedora_coverity.dockerfile.j2 | 10 +
.../containers/opensuse.dockerfile.j2 | 10 +
.../containers/redhat_family.dockerfile.j2 | 5 +
.../templates/containers/rhel.dockerfile.j2 | 16 +
.../templates/containers/rhel7.dockerfile.j2 | 15 +
.../templates/containers/rhel8.dockerfile.j2 | 15 +
.../templates/containers/rhel9.dockerfile.j2 | 19 +
.../templates/containers/rpm.dockerfile.j2 | 3 +
.../templates/containers/ubuntu.dockerfile.j2 | 3 +
.../containers/ubuntu20.04.dockerfile.j2 | 12 +
.../containers/ubuntu22.04.dockerfile.j2 | 3 +
.../containers/ubuntu_cross.dockerfile.j2 | 12 +
.../containers/ubuntu_sve.dockerfile.j2 | 12 +
36 files changed, 2177 insertions(+)
create mode 100644 containers/Makefile
create mode 100644 containers/README.md
create mode 100644 containers/container_builder.dockerfile
create mode 100644 containers/template_engine/inventory.yaml
create mode 100644 containers/template_engine/inventory_schema.json
create mode 100755 containers/template_engine/make_dockerfile.py
create mode 100644 containers/template_engine/poetry.lock
create mode 100644 containers/template_engine/pyproject.toml
create mode 100644 containers/template_engine/templates/containers.makefile.j2
create mode 100644 containers/template_engine/templates/containers/alpine.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/arch.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/base.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/centos8.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/centos9.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian10.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian11.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian11_arm_ipsec.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian_bullseye.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/debian_buster.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/fedora.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/fedora36_clang.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/fedora_clang.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/fedora_coverity.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/opensuse.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/redhat_family.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/rhel.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/rhel7.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/rhel8.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/rhel9.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/rpm.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/ubuntu.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/ubuntu20.04.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/ubuntu22.04.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/ubuntu_cross.dockerfile.j2
create mode 100644 containers/template_engine/templates/containers/ubuntu_sve.dockerfile.j2
--
2.34.1
next reply other threads:[~2023-05-23 17:04 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-23 17:04 Adam Hassick [this message]
2023-05-23 17:04 ` [PATCH v5 01/11] containers/docs: Add container builder start Adam Hassick
2023-05-25 13:48 ` Aaron Conole
2023-05-23 17:04 ` [PATCH v5 02/11] containers/inventory: Add inventory for container builder Adam Hassick
2023-05-23 17:04 ` [PATCH v5 03/11] containers/builder: Dockerfile creation script Adam Hassick
2023-05-23 17:04 ` [PATCH v5 04/11] containers/templates: Templates for Dockerfiles Adam Hassick
2023-05-23 17:04 ` [PATCH v5 05/11] containers/container_builder: Container for python scripts Adam Hassick
2023-05-23 17:04 ` [PATCH v5 06/11] containers/Makefile: Makefile to automate builds Adam Hassick
2023-05-23 17:04 ` [PATCH v5 07/11] containers/inventory: Add ABI rev and coverity attribute Adam Hassick
2023-05-23 17:04 ` [PATCH v5 08/11] containers/builder: Add arguments to templating script Adam Hassick
2023-05-23 17:04 ` [PATCH v5 09/11] containers/templates: Update templates Adam Hassick
2023-05-23 17:04 ` [PATCH v5 10/11] containers/Makefile: Add new variables and target changes Adam Hassick
2023-05-23 17:04 ` [PATCH v5 11/11] containers/docs: Update README Adam Hassick
2023-05-25 13:45 ` Aaron Conole
2023-06-05 20:09 ` Adam Hassick
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230523170413.812922-1-ahassick@iol.unh.edu \
--to=ahassick@iol.unh.edu \
--cc=aconole@redhat.com \
--cc=alialnu@nvidia.com \
--cc=ci@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).