From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C5BE42B9B for ; Thu, 25 May 2023 15:45:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D7AA40DF8; Thu, 25 May 2023 15:45:57 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 6061A40DDB for ; Thu, 25 May 2023 15:45:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685022355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nRUY7wPBG7HNb7ki3FuWLzreDQGnrSpE/dO/Tby8UOQ=; b=NzgT9WJTWCZEpe5YtiG3SZYE4So4J5f4o4Edg9PTPHFF4xmwu8gxIMEqzyNUEWhDXiHp0O SCpHu+oRXTEevaO+SfGRcLs/zx+AnaJpBNOifqEFW7PSrPMyMY0oVAyYbqyEhsNt3uYBKx LxmQTV97lg87jVOxW1acFcpDIB354L8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-301-FT5_3cJ-NoOdm5ycVDYA5Q-1; Thu, 25 May 2023 09:45:54 -0400 X-MC-Unique: FT5_3cJ-NoOdm5ycVDYA5Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3046728237C1; Thu, 25 May 2023 13:45:54 +0000 (UTC) Received: from RHTPC1VM0NT (unknown [10.22.17.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CF3651121314; Thu, 25 May 2023 13:45:53 +0000 (UTC) From: Aaron Conole To: Adam Hassick Cc: ci@dpdk.org, alialnu@nvidia.com Subject: Re: [PATCH v5 11/11] containers/docs: Update README References: <20230523170413.812922-1-ahassick@iol.unh.edu> <20230523170413.812922-12-ahassick@iol.unh.edu> Date: Thu, 25 May 2023 09:45:53 -0400 In-Reply-To: <20230523170413.812922-12-ahassick@iol.unh.edu> (Adam Hassick's message of "Tue, 23 May 2023 13:04:13 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Adam Hassick writes: > Updates the README with information on the added arguments > and support for building OCI manifests. > > Signed-off-by: Adam Hassick > --- I stopped reviewing. The changes here introduce spelling mistakes and spaces at the end of lines. Please proofread it before making changes. > containers/README.md | 123 +++++++++++++++++++++++++++---------------- > 1 file changed, 77 insertions(+), 46 deletions(-) > > diff --git a/containers/README.md b/containers/README.md > index 5d01caf..59edfcc 100644 > --- a/containers/README.md > +++ b/containers/README.md > @@ -14,73 +14,80 @@ nearly mandatory for this task. > > 1. Licensed RHEL containers need podman > > -The build system MUST be able to handle creating properly licensed RHEL > -containers, so that the RHEL CI testing is as accurate as possible. > +The build system MUST be able to handle creating properly licensed RHEL > +containers, so that the RHEL CI testing is as accurate as possbile. Why these lines changed? Rather, I see that a space was added at the end, and a spelling mistake was introduced s/possbile/possible/. That can't be intentional - please undo this change. > 2. "Developer Laptop Friendliness" > > -Another goal of the build system was to enable anyone to easily build the > +Another goal of the build system was to enable anyone to easily build the > containers. Not all developers are able to use Linux as the main OS on their > main development machine. Podman runs on MacOS via podman-machine and Windows > -either by podman-machine or WSL. > +either by podman-machine or WSL. Why were spaces added at the end in this section? Please don't do that. > 3. OCI Containers > > -OCI containers are more portable than some other container solutions. Much of > +OCI containers are more portable than some other container solutions. Much of > the progress on getting containers running on top of FreeBSD jails targets OCI > -containers specifically. The tracking issue for this is > +containers specifically. The tracking issue for this is > [https://reviews.freebsd.org/D21570](https://reviews.freebsd.org/D21570). > -Once upstream support happens, there should be a relatively simple path to > +Once upstream support happens, there should be a relatively simple path to > supporting containers in FreeBSD once podman/docker APIs are better supported. > -At the moment, lack up upstream support means no support in this project for > -FreeBSD. > +At the moment, lack up upstream support means no support in this project for > +FreeBSD. > > ### Python and Makefiles instead of Buildah as a library > > -The next question someone might have is why a combination of Python and > +The next question someone might have is why a combination of Python and > makefiles were used instead of using buildah as a library. The largest > reason is that every DPDK developer is going to need to have some > -level of familiarity with Python due to DTS. Buildah is only available > -as a library via Go, and would tie DPDK to a particular container > +level of familarity with Python due to DTS. Buildah is only avaliable familiarity was correct. available was correct. > +as a library via Go, and would tie DPDK to a particular container > implementation. Go, while not difficult to learn, is a compiled language, > meaning that the build system would require a build system. > > -The other reason is that most of the logic that needs to be performed is very > -simple, and python has a few libraries that do most of the work. If it weren't > -for the desire to have an inventory file (inventory.yaml) with a schema > -(inventory_schema.json), this probably could have been an AWK script. After the > -container images are produced, it is very easy to use the same template > -engine to produce a makefile that can be used to both build and push the > -containers. This makefile can be run with multiple jobs for parallel building > -of containers, something not supported by all compose implementations. > - > -Meson was considered instead of Makefiles, however, Meson does not handle new > -Meson being generated during the build very well, and Meson wants most commands > -to have an output file, which is not true of many of the commands. Meson is > -also more difficult to generate using a templating library than Makefile > -targets. > +The other reason is that most of the logic that needs to be performed is very > +simple, and python has a few libraries that do most of the work. If it weren't > +for the desire to have an inventory file (inventory.yaml) with a schema > +(inventory_schema.json), this probably could have been an AWK script. After the > +container images are produced, it is very easy to use the same templating > +engine to produce a makefile that can be used to both build and push the > +containers. This makefile can be run with multiple jobs for parallel building > +of containers, something not supported by all compose implementations. > + > +Meson was considered instead of Makefiles, however, Meson does not handle new > +Meson being generated during the build very well, and Meson wants most commands > +to have an output file, which is not true of many of the commands. Meson is > +also more difficult to generate using a templating library than Makefile > +targets. > > ## Building > > ### Environment Variables > > -All environment variables are namespaced to DPDK_CI_CONTAINERS to avoid any > -issues. > +All environment variables are namespaced to DPDK_CI_CONTAINERS to avoid any > +issues. > > | Variable | Description | Default | Valid Values | > | -------------------------- | ----------------------------------------------- | ------- | ------------ | > -DPDK_CI_CONTAINERS_ON_RHEL | Whether you are building on licensed RHEL. RHEL containers must be built on licensed RHEL, this can be used to forcibly enable/disable RHEL containers if automatic detection fails. | (grep -q 'Red Hat Enterprise Linux' /etc/redhat-release && echo 'Y') \|\| echo 'N' | 'Y' or 'N' > +| DPDK_CI_CONTAINERS_ON_RHEL | Whether you are building on licensed RHEL. RHEL containers must be built on licensed RHEL, this can be used to forcibly enable/disable RHEL containers if automatic detection fails. | (grep -q 'Red Hat Enterprise Linux' /etc/redhat-release && echo 'Y') \|\| echo 'N' | 'Y' or 'N' > DPDK_CI_CONTAINERS_FAIL_ON_UNBUILDABLE | Fail during dockerfile generation if any container in the inventory is not buildable. Currently will cause a failure if you are not on RHEL and try to build RHEL containers. | 'N' | 'Y' or 'N' > +DPDK_CI_CONTAINERS_ONLY_HOST_ARCH | If set to 'Y', only images for the local system architecture will be built. | 'N' | 'Y' or 'N' > +DPDK_CI_CONTAINERS_IS_BUILDER | If set to 'Y', disables the manifest features, and only builds images for the local system architecture. Intended to be set when used inside another orchestration sofware. | 'N' | 'Y' or 'N' > +DPDK_CI_CONTAINERS_NINJA_WORKERS | The number of Ninja workers to use to build ABI images. Variable setting is benign if ABI is disabled. | unset | Any positive integer greater than zero. > DPDK_CI_CONTAINERS_BUILD_ABI | Whether to bake ABI images into the containers. | 'N' | 'Y' or 'N' > -DPDK_CI_CONTAINERS_BUILD_LIBABIGAIL | Whether to build libabigail from source on distros that do not package it. DPDK_CI_CONTAINERS_BUILD_ABI=Y overrides this to 'Y' | 'N' | 'Y' or 'N' > +DPDK_CI_CONTAINERS_NO_LATEST_TAG | Disables tagging the final manifests as "latest" in the local store and remote registry. | 'N' | 'Y' or 'N' > +DPDK_CI_CONTAINERS_COVERITY | Enable building Coverity images. Setting this flag will make the Coverity binaries required. | 'N' | 'Y' or 'N' > DPDK_CI_CONTAINER_BUILDER_PROGRAM | What container builder program to use. | 'podman' | Any container builder that exposes the same interface and provides the same behavior as podman. > -DPDK_CI_CONTAINERS_LIBABIGAIL_CLONE_URL | What URL to clone libabigail from, since some distros need to compile it from source. | 'git://sourceware.org/git/libabigail.git' | A repository containing libabigail which shares history with the main repository. > +DPDK_CI_CONTAINERS_LIBABIGAIL_CLONE_URL | What URL to clone libabigail from, since some distros need to compile it from source. | 'git://sourceware.org/git/libabigail.git' | A repository containing libabigail which shares history with the main repository. > DPDK_CI_CONTAINERS_DPDK_CLONE_URL | What URL to clone DPDK from. | 'https://dpdk.org/git/dpdk' | Any DPDK mirror. > +DPDK_CI_CONTAINERS_DPDK_STABLE_CLONE_URL | What URL to clone DPDK stable form. | http://dpdk.org/git/dpdk-stable | Any DPDK stable mirror. > DPDK_CI_CONTAINERS_CONTAINER_BUILDER_TAG | What tag to give to the container which creates the dockerfiles. The default should be fine unless you have issues with collisions. | 'dpdk_ci_container_builder' | Any valid OCI container tag (A valid C function name will work) > DPDK_CI_CONTAINERS_EXTRA_PUSH_ARGS | Extra arguments to add to the push command, can be used for credentials if 'podman login' won't work. | '' | [https://docs.podman.io/en/latest/markdown/podman-push.1.html#options](https://docs.podman.io/en/latest/markdown/podman-push.1.html#options) > DPDK_CI_CONTAINERS_REGISTRY_HOSTNAME | The hostname of the registry to push to. | 'localhost' | The hostname of any system exposing an OCI container registry or localhost to push to local storage. > DPDK_CI_CONTAINERS_EXTRA_SCRIPTS_PATH | The path to a directory to copy into all of the containers at /scripts | unset | The path to any local file directory. > +DPDK_CI_CONTAINERS_COVERITY_PATH | The path to Coverity Scan binaries. Only required of the Coverity flag is set. | unset | The path to any local file directory. > DPDK_CI_CONTAINERS_CONTEXT_DIRECTORY | Set the directory to build the containers in. All generated files will be placed in this directory or one of it's children | '$(CURDIR)/container_context' | Any absolute directory path > +DPDK_CI_CONTAINERS_DATE_TAG_OVERRIDE | Uses a provided string instead of generating a new date tag. Intended for development use. | unset | Any string that is a valid OCI manifest tag. > > ### Builder System Requirements > > @@ -91,16 +98,15 @@ DPDK_CI_CONTAINERS_CONTEXT_DIRECTORY | Set the directory to build the containers > * find > * posix utilities (GNU coreutils will work) > * bash > -* podman >= 4.0.0 (docker or other container builder programs may work, but are > -unsupported) > - * podman 4.0.0 allows run mounts, which allow mounting a directory into the build context of a container. This is used to persist ccache directories for each container. > -* qemu-$ARCH-static for any non-native architecture/revision you want to build for. > +* podman (docker is unsupported, and will NOT work for making manifests) > +* qemu-$ARCH-static for any non-native architecture/revision you want to build > +for. > > #### Hardware > > -| Hardware Type | Requirement | Reason | > -| ------------- | ---------------------------- | ----------------------------------- | > -| Disk space | 5 GB of disk space per image | Some images are 4 GB at the moment, and as DPDK's API grows, so will the ABI references. > +| Hardware Type | Requirement | Reason | > +| ------------- | ----------------------------- | ----------------------------------- | > +| Disk space | 10 GB of disk space per image | Many of the final images are 4 GB at the moment, and as DPDK's API grows, so will the ABI references. Intermediate images generated by the builds will consume some additional space that is recoverable after the build. > Memory | Either 1.5x or 2x the memory needed to compile DPDK per makefile job | 1.5x is enough for the container overhead and caching when compiling natively, 2x is for builds under emulation (ARM container on x86, etc). > > > @@ -111,28 +117,53 @@ RHEL container images must be built on RHEL. > ### Build containers locally > > ```bash > +# Build using the default arguments > make build > ``` > > +The resulting images will be tagged based on the date tag and platform. > +Image generated tags follow this format: `image-{{ platform }}-{{ date_tag }}` > +Where `platform` denotes the platform of the image, and `date_tag` is the generated date tag or the override string provided > +through the environment variable. > + > +They should appear in the local image store on your system. > + > ### Push containers to registry > > -This will probably involve following prompts in your terminal, but if you have > -other authentication set up, (LDAP, Kerberos, etc), it may not prompt you. > +This will probably involve following prompts in your terminal, but if you have > +other authentication set up, (LDAP, Kerberos, etc), it may not prompt you. > Logging into a registry is what allows you to upload containers to a remote > -system for others to pull down. > +system for others to pull down. > > If you are working alone, you probably can ignore this and keep the containers > -locally. If you are in an enterprise setting, ask your DevOps or Systems > -Administration team where the preferred location for hosting containers is. > +locally. If you are in an enterprise setting, ask your DevOps or Systems > +Administration team where the preferred location for hosting containers is. > > -Since these images take so long to build, it is recommended to use a container > +Since these images take so long to build, it is recommended to use a container > registry and have any CI systems pull from that registry. > > -Redhat guide to setting up a podman container registry: > +Redhat guide to setting up a podman container registry: > [https://www.redhat.com/sysadmin/simple-container-registry](https://www.redhat.com/sysadmin/simple-container-registry) > > ```bash > $DPDK_CI_CONTAINER_BUILDER_PROGRAM login $DPDK_CI_CONTAINERS_REGISTRY_HOSTNAME > # < Complete login process > > make push > -``` > \ No newline at end of file > +``` > + > +#### Manifests > + > +OCI manifests allow the grouping of images for different platforms under the same tag in a repository on a registry. > +The use of OCI manifests over tagged images reduces the amount of system platform related branching in CI scripting. > + > +The Makefile provides the option to push only the images, only the manifests, or push the images and make manifests. > +The default "push" target will perform the last case. > +If you choose to create the manifests, then these will be created with the "final" tags like "latest" and the date timestamp. > + > +Manifest creation is known to not be compatible with Docker. > +This feature is known to work when using Podman to post content to a Docker v2 registry. > + > +The manifests may be created on the registry independently of the image builds using the `push_manifests` target in place of the `push` target. > +In contrast, the `push_images` target will only push the images and not create the manifests. > + > +If the `DPDK_CI_CONTAINERS_IS_BUILDER` variable is set to 'Y', then the `push_manifests` target will be disabled. > \ No newline at end of file