Re: [PATCH v2 0/6] Community Lab Container Definitions

[Owen Hilyard <ohilyard@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 3534 bytes --]

On Wed, Oct 19, 2022 at 10:31 AM Ali Alnubani <alialnu@nvidia.com> wrote:

> > -----Original Message-----
> > From: ohilyard@iol.unh.edu <ohilyard@iol.unh.edu>
> > Sent: Wednesday, October 19, 2022 3:53 PM
> > To: ci@dpdk.org
> > Cc: aconole@redhat.com; Ali Alnubani <alialnu@nvidia.com>; Owen Hilyard
> > <ohilyard@iol.unh.edu>
> > Subject: [PATCH v2 0/6] Community Lab Container Definitions
> >
> > From: Owen Hilyard <ohilyard@iol.unh.edu>
> >
> > This patch series contains a new version of the DPDK CI containers. The
> old
> > version was tied very tightly to the Community Lab Infrastructure, so it
> was
> > not suitable for general use. This version is designed to make adding new
> > OSes or OS versions as easy as possible. The minimum functionality can
> easily
> > be built on any system that can compile DPDK. It includes support for
> > building containers for other non-native architectures (ex: arm
> containers
> > on x86) and for baking ABI references into the images.
> >
> > The inventory file as added in this patch series defines what the
> community
> > lab
> > currently supports.
> >
> > If you want to build these yourself, don't try to do parallel Makefile
> builds
> > on non-server hardware. Libabigail is built into the containers, and
> since it
> > is not avaliable in all distros it is compiled from source for many
> targets.
> > If embedding the abi is enabled (DPDK_CI_CONTAINERS_BUILD_ABI=Y), with
> > the
> > current settings, DPDK will be compiled twice for every target in the
> > inventory file. This will become three times when DPDK main has ABI
> stability
> > again. Due to how the containers are constructed, building on non-native
> > architectures is especially expensive because DPDK is compiled with an
> > emulated
> > compiler in a qemu vm that only has one thread.
> >
> > Owen Hilyard (6):
> >   containers/docs: Add container builder start
> >   containers/inventory: Add inventory for container builder
> >   containers/builder: Dockerfile creation script
> >   containers/templates: Templates for Dockerfiles
> >   containers/container_builder: Container for python scripts
> >   containers/Makefile: Makefile to automate builds
> >
>
> Thanks for addressing my previous comments.
>
> The project doesn't specify a minimum version for podman/buildah.
> I'm using podman version 3.4.4 in Ubuntu 22.04 (installed from Ubuntu's
> official repos), and it seems to be missing support for mount=type=cache,
> causing the following errors when running make's build target:
> 199b42d26a291a4dcbb69bf64ddb1309b080ffa9aee12e4f6a6b533469": invalid mount
> type "cache"
> make[1]: *** [Makefile:19: build_alpine_compile] Error 125
> make[1]: Leaving directory '/root/dpdk-ci/containers/container_context'
> make: *** [Makefile:103: build] Error 2
>
> The support seems to have been added in Buildah v1.24.0 (see [1]), which
> is part of podman v4.0.0 (see [2]).
>
> [1]
> https://github.com/containers/buildah/commit/fc69aa68c23b32df5ab73726dcee88bf1add72c6
> [2]
> https://github.com/containers/podman/commit/4a4d86d40f9137144103147e7a0ba74fbf3aaca8
>
> Can the --mount RUN arg be enabled only for podman versions newer than
> v4.0.0 inside a Jinja2 block maybe?
> Or do you prefer to mention v4.0.0 as the minimum required version of
> podman in the documentation?
>
> Thanks,
> Ali
>

I think v4.0.0 as the minimum version is better, since not using that cache
makes an already long build process much longer. If no one else has any
comments, I'll submit that as a v3 next week.

Owen

[-- Attachment #2: Type: text/html, Size: 4912 bytes --]