From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14C00A09D3 for ; Wed, 19 Oct 2022 16:56:21 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9017742BFD; Wed, 19 Oct 2022 16:56:20 +0200 (CEST) Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by mails.dpdk.org (Postfix) with ESMTP id B544042BE7 for ; Wed, 19 Oct 2022 16:56:19 +0200 (CEST) Received: by mail-pf1-f179.google.com with SMTP id m6so17507072pfb.0 for ; Wed, 19 Oct 2022 07:56:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=T6Akdmrs4W+1mF+6t3VsULcmYIf/n7vUA85igw31jzE=; b=RnHRq5eqc7XFTPLNZaPhdnB3hTRGlkYCJ/klP3voez2r8hACeOBW5pPQjLcQIFfNGQ ZnWyhUAby1YyeLc5AdaE1kTN90oyOr5kWQHeUeXtSsVvY38nNo1Ol3Lz9LDDGIPPSFn3 ZRkIavwJpZXc3+iH9DV/8a5Q/B5HmZxw2OWJg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=T6Akdmrs4W+1mF+6t3VsULcmYIf/n7vUA85igw31jzE=; b=dq9E14ZzC6mskHi++rMRaB74pOI+7kTVGnGwzKkdGI3xf/7e+fZvIZm539KgN2CAHY onJfVN++vik62DTgMSH38Tgp9TFRNlMCD5iKM2PPuvYqS5mn9++1CMxyGfsFfiS2bJ40 KG/2g2nrH35zEWiUhKigLeOzTFJn3jZx5pOSLFv1mFhJv6yVlke6bZyVRPFg3ZjmcB3I WD8spif/x4Xu51cxzKD/ynOvtUdOkAyefFrXOVY+54TvSNVeMzJ3Yi/VbVG1bO4ic9J4 8Ok+TotiJYFq/m6zV8jHW6OQKpa6OwAE4zQqJZUrpizuAP+mHDXp+GiUl84xy9SqLVer 7ltA== X-Gm-Message-State: ACrzQf2nMIS8Gf6NJNuk8lqB/LglH1ht9qYbI548JHjQVmSrclxeuc5Z MA/eO0ulvm4Qi1wdEsT5fEW/RdFxqhxxbfIxihlMgw== X-Google-Smtp-Source: AMsMyM6EbtsK1Pm2BvL8Y9ips/F2Can6lB0ucgD1k9oro/JAiQHSSeyyTe3lM+39NFxbAbVXLE27XzUywX5PutmQeTw= X-Received: by 2002:a62:d448:0:b0:565:ef61:ee0d with SMTP id u8-20020a62d448000000b00565ef61ee0dmr8963851pfl.73.1666191378800; Wed, 19 Oct 2022 07:56:18 -0700 (PDT) MIME-Version: 1.0 References: <20221019125307.258582-1-ohilyard@iol.unh.edu> In-Reply-To: From: Owen Hilyard Date: Wed, 19 Oct 2022 10:55:42 -0400 Message-ID: Subject: Re: [PATCH v2 0/6] Community Lab Container Definitions To: Ali Alnubani Cc: "ci@dpdk.org" , "aconole@redhat.com" Content-Type: multipart/alternative; boundary="00000000000042fc5105eb6469fc" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --00000000000042fc5105eb6469fc Content-Type: text/plain; charset="UTF-8" On Wed, Oct 19, 2022 at 10:31 AM Ali Alnubani wrote: > > -----Original Message----- > > From: ohilyard@iol.unh.edu > > Sent: Wednesday, October 19, 2022 3:53 PM > > To: ci@dpdk.org > > Cc: aconole@redhat.com; Ali Alnubani ; Owen Hilyard > > > > Subject: [PATCH v2 0/6] Community Lab Container Definitions > > > > From: Owen Hilyard > > > > This patch series contains a new version of the DPDK CI containers. The > old > > version was tied very tightly to the Community Lab Infrastructure, so it > was > > not suitable for general use. This version is designed to make adding new > > OSes or OS versions as easy as possible. The minimum functionality can > easily > > be built on any system that can compile DPDK. It includes support for > > building containers for other non-native architectures (ex: arm > containers > > on x86) and for baking ABI references into the images. > > > > The inventory file as added in this patch series defines what the > community > > lab > > currently supports. > > > > If you want to build these yourself, don't try to do parallel Makefile > builds > > on non-server hardware. Libabigail is built into the containers, and > since it > > is not avaliable in all distros it is compiled from source for many > targets. > > If embedding the abi is enabled (DPDK_CI_CONTAINERS_BUILD_ABI=Y), with > > the > > current settings, DPDK will be compiled twice for every target in the > > inventory file. This will become three times when DPDK main has ABI > stability > > again. Due to how the containers are constructed, building on non-native > > architectures is especially expensive because DPDK is compiled with an > > emulated > > compiler in a qemu vm that only has one thread. > > > > Owen Hilyard (6): > > containers/docs: Add container builder start > > containers/inventory: Add inventory for container builder > > containers/builder: Dockerfile creation script > > containers/templates: Templates for Dockerfiles > > containers/container_builder: Container for python scripts > > containers/Makefile: Makefile to automate builds > > > > Thanks for addressing my previous comments. > > The project doesn't specify a minimum version for podman/buildah. > I'm using podman version 3.4.4 in Ubuntu 22.04 (installed from Ubuntu's > official repos), and it seems to be missing support for mount=type=cache, > causing the following errors when running make's build target: > 199b42d26a291a4dcbb69bf64ddb1309b080ffa9aee12e4f6a6b533469": invalid mount > type "cache" > make[1]: *** [Makefile:19: build_alpine_compile] Error 125 > make[1]: Leaving directory '/root/dpdk-ci/containers/container_context' > make: *** [Makefile:103: build] Error 2 > > The support seems to have been added in Buildah v1.24.0 (see [1]), which > is part of podman v4.0.0 (see [2]). > > [1] > https://github.com/containers/buildah/commit/fc69aa68c23b32df5ab73726dcee88bf1add72c6 > [2] > https://github.com/containers/podman/commit/4a4d86d40f9137144103147e7a0ba74fbf3aaca8 > > Can the --mount RUN arg be enabled only for podman versions newer than > v4.0.0 inside a Jinja2 block maybe? > Or do you prefer to mention v4.0.0 as the minimum required version of > podman in the documentation? > > Thanks, > Ali > I think v4.0.0 as the minimum version is better, since not using that cache makes an already long build process much longer. If no one else has any comments, I'll submit that as a v3 next week. Owen --00000000000042fc5105eb6469fc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Wed, Oct 19, 2022 at 10:31 AM Ali Alnu= bani <alialnu@nvidia.com> w= rote:
> -----= Original Message-----
> From: ohilya= rd@iol.unh.edu <ohilyard@iol.unh.edu>
> Sent: Wednesday, October 19, 2022 3:53 PM
> To: ci@dpdk.org > Cc: aconole@re= dhat.com; Ali Alnubani <alialnu@nvidia.com>; Owen Hilyard
> <ohilyard= @iol.unh.edu>
> Subject: [PATCH v2 0/6] Community Lab Container Definitions
>
> From: Owen Hilyard <ohilyard@iol.unh.edu>
>
> This patch series contains a new version of the DPDK CI containers. Th= e old
> version was tied very tightly to the Community Lab Infrastructure, so = it was
> not suitable for general use. This version is designed to make adding = new
> OSes or OS versions as easy as possible. The minimum functionality can= easily
> be built on any system that can compile DPDK. It includes support for<= br> > building containers for other non-native architectures (ex: arm contai= ners
> on x86) and for baking ABI references into the images.
>
> The inventory file as added in this patch series defines what the comm= unity
> lab
> currently supports.
>
> If you want to build these yourself, don't try to do parallel Make= file builds
> on non-server hardware. Libabigail is built into the containers, and s= ince it
> is not avaliable in all distros it is compiled from source for many ta= rgets.
> If embedding the abi is enabled (DPDK_CI_CONTAINERS_BUILD_ABI=3DY), wi= th
> the
> current settings, DPDK will be compiled twice for every target in the<= br> > inventory file. This will become three times when DPDK main has ABI st= ability
> again. Due to how the containers are constructed, building on non-nati= ve
> architectures is especially expensive because DPDK is compiled with an=
> emulated
> compiler in a qemu vm that only has one thread.
>
> Owen Hilyard (6):
>=C2=A0 =C2=A0containers/docs: Add container builder start
>=C2=A0 =C2=A0containers/inventory: Add inventory for container builder<= br> >=C2=A0 =C2=A0containers/builder: Dockerfile creation script
>=C2=A0 =C2=A0containers/templates: Templates for Dockerfiles
>=C2=A0 =C2=A0containers/container_builder: Container for python scripts=
>=C2=A0 =C2=A0containers/Makefile: Makefile to automate builds
>

Thanks for addressing my previous comments.

The project doesn't specify a minimum version for podman/buildah.
I'm using podman version 3.4.4 in Ubuntu 22.04 (installed from Ubuntu&#= 39;s official repos), and it seems to be missing support for mount=3Dtype= =3Dcache, causing the following errors when running make's build target= :
199b42d26a291a4dcbb69bf64ddb1309b080ffa9aee12e4f6a6b533469": invalid m= ount type "cache"
make[1]: *** [Makefile:19: build_alpine_compile] Error 125
make[1]: Leaving directory '/root/dpdk-ci/containers/container_context&= #39;
make: *** [Makefile:103: build] Error 2

The support seems to have been added in Buildah v1.24.0 (see [1]), which is= part of podman v4.0.0 (see [2]).

[1] https://git= hub.com/containers/buildah/commit/fc69aa68c23b32df5ab73726dcee88bf1add72c6<= /a>
[2] https://gith= ub.com/containers/podman/commit/4a4d86d40f9137144103147e7a0ba74fbf3aaca8

Can the --mount RUN arg be enabled only for podman versions newer than v4.0= .0 inside a Jinja2 block maybe?
Or do you prefer to mention v4.0.0 as the minimum required version of podma= n in the documentation?

Thanks,
Ali


--00000000000042fc5105eb6469fc--