DPDK CI discussions
 help / color / mirror / Atom feed
* [PATCH v8 0/6] Community Lab Containers and Builder Engine
@ 2023-07-17 21:08 Adam Hassick
  2023-07-17 21:08 ` [PATCH v8 1/6] containers/docs: Add container builder start Adam Hassick
                   ` (6 more replies)
  0 siblings, 7 replies; 15+ messages in thread
From: Adam Hassick @ 2023-07-17 21:08 UTC (permalink / raw)
  To: ci; +Cc: aconole, alialnu, Adam Hassick

This patch series contains a new version of the DPDK CI containers. The old
version was tied very tightly to the Community Lab Infrastructure, so it was
not suitable for general use. This version is designed to make adding new
OSes or OS versions as easy as possible. The minimum functionality can easily
be built on any system that can compile DPDK. It includes support for
building containers for other non-native architectures (ex: arm containers
on x86) and for baking ABI references into the images.

The inventory file as added in this patch series defines what the community lab
currently supports.

If you want to build these yourself, don't try to do parallel Makefile builds
on non-server hardware. Libabigail is built into the containers, and since it
is not avaliable in all distros it is compiled from source for many targets.
If embedding the abi is enabled (DPDK_CI_CONTAINERS_BUILD_ABI=Y), with the
current settings, DPDK will be compiled twice for every target in the
inventory file. This will become three times when DPDK main has ABI stability
again. Due to how the containers are constructed, building on non-native
architectures is especially expensive because DPDK is compiled with an emulated
compiler in a qemu vm that only has one thread.

---

v2:
* Added poetry.lock
* Remove duplicates in inventory file
* Fix typo in base container template

v3:
* Libabigail is now not built by default unless DPDK_CI_CONTAINERS_BUILD_ABI=y
* Made minimum podman version 4.0.0 due to dependencies on cache mounts

v4:
* Fix whitespace errors

v5:
* Updated container templates and inventory
* Add support for OCI manifests
* Update README to reflect changes

v6:
* Fix spelling errors and whitespace errors

v7:
* Add workaround to Arch template to install Python packages as root
* Compare against v4 and polish the patchset

v8:
* Fix typos
* Correct the default for the dpdk-stable clone URL

Owen Hilyard (6):
  containers/docs: Add container builder start
  containers/inventory: Add inventory for container builder
  containers/builder: Dockerfile creation script
  containers/templates: Templates for Dockerfiles
  containers/container_builder: Container for python scripts
  containers/Makefile: Makefile to automate builds

 containers/Makefile                           | 250 ++++++++++
 containers/README.md                          | 178 +++++++
 containers/container_builder.dockerfile       |  26 +
 containers/template_engine/inventory.yaml     | 470 ++++++++++++++++++
 .../template_engine/inventory_schema.json     | 221 ++++++++
 containers/template_engine/make_dockerfile.py | 358 +++++++++++++
 containers/template_engine/poetry.lock        | 220 ++++++++
 containers/template_engine/pyproject.toml     |  21 +
 .../templates/containers.makefile.j2          |  73 +++
 .../templates/containers/alpine.dockerfile.j2 |   3 +
 .../templates/containers/arch.dockerfile.j2   |  39 ++
 .../templates/containers/base.dockerfile.j2   | 120 +++++
 .../containers/centos8.dockerfile.j2          |  21 +
 .../containers/centos9.dockerfile.j2          |  17 +
 .../templates/containers/debian.dockerfile.j2 |   7 +
 .../containers/debian10.dockerfile.j2         |   3 +
 .../containers/debian11.dockerfile.j2         |   3 +
 .../debian11_arm_ipsec.dockerfile.j2          |  16 +
 .../containers/debian_bullseye.dockerfile.j2  |   3 +
 .../containers/debian_buster.dockerfile.j2    |   3 +
 .../templates/containers/fedora.dockerfile.j2 |  11 +
 .../containers/fedora36_clang.dockerfile.j2   |   7 +
 .../containers/fedora_clang.dockerfile.j2     |   8 +
 .../containers/fedora_coverity.dockerfile.j2  |  10 +
 .../containers/opensuse.dockerfile.j2         |  10 +
 .../containers/redhat_family.dockerfile.j2    |   5 +
 .../templates/containers/rhel.dockerfile.j2   |  16 +
 .../templates/containers/rhel7.dockerfile.j2  |  15 +
 .../templates/containers/rhel8.dockerfile.j2  |  15 +
 .../templates/containers/rhel9.dockerfile.j2  |  19 +
 .../templates/containers/rpm.dockerfile.j2    |   3 +
 .../templates/containers/ubuntu.dockerfile.j2 |   3 +
 .../containers/ubuntu20.04.dockerfile.j2      |  12 +
 .../containers/ubuntu22.04.dockerfile.j2      |   3 +
 .../containers/ubuntu_cross.dockerfile.j2     |  11 +
 .../containers/ubuntu_sve.dockerfile.j2       |  12 +
 36 files changed, 2212 insertions(+)
 create mode 100644 containers/Makefile
 create mode 100644 containers/README.md
 create mode 100644 containers/container_builder.dockerfile
 create mode 100644 containers/template_engine/inventory.yaml
 create mode 100644 containers/template_engine/inventory_schema.json
 create mode 100755 containers/template_engine/make_dockerfile.py
 create mode 100644 containers/template_engine/poetry.lock
 create mode 100644 containers/template_engine/pyproject.toml
 create mode 100644 containers/template_engine/templates/containers.makefile.j2
 create mode 100644 containers/template_engine/templates/containers/alpine.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/arch.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/base.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/centos8.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/centos9.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian10.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian11.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian11_arm_ipsec.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian_bullseye.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/debian_buster.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/fedora.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/fedora36_clang.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/fedora_clang.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/fedora_coverity.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/opensuse.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/redhat_family.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/rhel.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/rhel7.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/rhel8.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/rhel9.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/rpm.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/ubuntu.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/ubuntu20.04.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/ubuntu22.04.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/ubuntu_cross.dockerfile.j2
 create mode 100644 containers/template_engine/templates/containers/ubuntu_sve.dockerfile.j2

-- 
2.41.0


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2023-08-04 19:35 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-17 21:08 [PATCH v8 0/6] Community Lab Containers and Builder Engine Adam Hassick
2023-07-17 21:08 ` [PATCH v8 1/6] containers/docs: Add container builder start Adam Hassick
2023-07-17 21:08 ` [PATCH v8 2/6] containers/inventory: Add inventory for container builder Adam Hassick
2023-08-04 13:58   ` Aaron Conole
2023-08-04 19:30     ` Adam Hassick
2023-07-17 21:08 ` [PATCH v8 3/6] containers/builder: Dockerfile creation script Adam Hassick
2023-08-04 13:59   ` Aaron Conole
2023-07-17 21:08 ` [PATCH v8 4/6] containers/templates: Templates for Dockerfiles Adam Hassick
2023-08-04 14:02   ` Aaron Conole
2023-08-04 19:34     ` Adam Hassick
2023-07-17 21:08 ` [PATCH v8 5/6] containers/container_builder: Container for python scripts Adam Hassick
2023-08-04 14:03   ` Aaron Conole
2023-07-17 21:08 ` [PATCH v8 6/6] containers/Makefile: Makefile to automate builds Adam Hassick
2023-08-04 14:06   ` Aaron Conole
2023-07-28 20:38 ` [PATCH v8 0/6] Community Lab Containers and Builder Engine Aaron Conole

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).