From: Nicholas Pratte <npratte@iol.unh.edu>
To: dev@dpdk.org, Jeremy Spewock <jspewock@iol.unh.edu>
Subject: Re: [PATCH v4] dts: add Dockerfile
Date: Thu, 29 Feb 2024 11:14:18 -0500 [thread overview]
Message-ID: <CAKXZ7ehBuBhoBr_LhBixFWtRvW11iDaw18mKNLHAXW-wi4FuGg@mail.gmail.com> (raw)
In-Reply-To: <CAJvnSUDDpGdxe8D-GmtbVixrrrB_GRfvS985Cw1RLx79aLcXnA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 9103 bytes --]
Tested-by: Nicholas Pratte <npratte@iol.unh.edu>
On Thu, Feb 29, 2024 at 10:48 AM Patrick Robb <probb@iol.unh.edu> wrote:
>
>
> ---------- Forwarded message ---------
> From: <jspewock@iol.unh.edu>
> Date: Tue, Jan 16, 2024 at 2:18 PM
> Subject: [PATCH v4] dts: add Dockerfile
> To: <Honnappa.Nagarahalli@arm.com>, <juraj.linkes@pantheon.tech>, <
> thomas@monjalon.net>, <wathsala.vithanage@arm.com>, <probb@iol.unh.edu>, <
> paul.szczepanek@arm.com>, <yoan.picchi@foss.arm.com>
> Cc: <dev@dpdk.org>, Jeremy Spewock <jspewock@iol.unh.edu>
>
>
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
>
> The Dockerfile defines development and CI runner images.
>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
> v4:
>
> Remove an example from and updated a comment in the devcontainer.json
> and added the --no-root flag to the README to comply with the warning
> message and the discussion on slack.
>
> v3:
>
> Remove pexpect.
>
> v2:
>
> This verson updates the dockerfile to instead install poetry using pipx
> due to the version of poetry installed using the package repositories of
> the distro being out of date, and to conform to documentation on
> installing poetry.
>
> This version also adds extra information to the README about the
> preference of using SSH keys, and added a way to inject them into the
> devcontainer for vscode.
>
> dts/.devcontainer/devcontainer.json | 30 +++++++++++++
> dts/Dockerfile | 38 ++++++++++++++++
> dts/README.md | 70 +++++++++++++++++++++++++++++
> 3 files changed, 138 insertions(+)
> create mode 100644 dts/.devcontainer/devcontainer.json
> create mode 100644 dts/Dockerfile
> create mode 100644 dts/README.md
>
> diff --git a/dts/.devcontainer/devcontainer.json
> b/dts/.devcontainer/devcontainer.json
> new file mode 100644
> index 0000000000..4d737f1b40
> --- /dev/null
> +++ b/dts/.devcontainer/devcontainer.json
> @@ -0,0 +1,30 @@
> +// For format details, see https://aka.ms/devcontainer.json. For config
> options, see the README at:
> +//
> https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/containers/docker-existing-dockerfile
> +{
> + "name": "Existing Dockerfile",
> +
> + // Sets the run context to one level up instead of the
> .devcontainer folder.
> + "context": "..",
> +
> + // Update the 'dockerFile' property if you aren't using the
> standard 'Dockerfile' filename.
> + "dockerFile": "../Dockerfile",
> +
> + // Use 'forwardPorts' to make a list of ports inside the container
> available locally.
> + // "forwardPorts": [],
> +
> + // The next line runs commands after the container is created - in
> our case, installing dependencies.
> + "postCreateCommand": "poetry install --no-root",
> +
> + "extensions": [
> + "ms-python.vscode-pylance",
> + ]
> +
> + // Uncomment when using a ptrace-based debugger like C++, Go, and
> Rust
> + // "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt",
> "seccomp=unconfined" ],
> +
> + // Uncomment to use the Docker CLI from inside the container. See
> https://aka.ms/vscode-remote/samples/docker-from-docker.
> + // "mounts": [
> "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ],
> +
> + // Uncomment to mount your SSH keys into the devcontainer used by
> vscode.
> + // "mounts":
> ["source=${localEnv:HOME}/.ssh,destination=/root/.ssh,type=bind,readonly"]
> +}
> diff --git a/dts/Dockerfile b/dts/Dockerfile
> new file mode 100644
> index 0000000000..fa4c1af10e
> --- /dev/null
> +++ b/dts/Dockerfile
> @@ -0,0 +1,38 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 University of New Hampshire
> +
> +# There are two Docker images defined in this Dockerfile.
> +# One is to be used in CI for automated testing.
> +# The other provides a DTS development environment, simplifying Python
> dependency management.
> +
> +FROM ubuntu:22.04 AS base
> +
> +RUN apt-get -y update && apt-get -y upgrade && \
> + apt-get -y install --no-install-recommends \
> + python3 \
> + python3-pip \
> + pipx \
> + python3-cachecontrol \
> + openssh-client && \
> + pipx install poetry>=1.5.1 && pipx ensurepath
> +WORKDIR /dpdk/dts
> +
> +
> +FROM base AS runner
> +
> +# This image is intended to be used as the base for automated systems.
> +# It bakes DTS into the image during the build.
> +
> +COPY . /dpdk/dts
> +RUN poetry install --no-dev
> +
> +CMD ["poetry", "run", "python", "main.py"]
> +
> +FROM base AS dev
> +
> +# This image is intended to be used as DTS development environment. It
> doesn't need C compilation
> +# capabilities, only Python dependencies. Once a container mounting DTS
> using this image is running,
> +# the dependencies should be installed using Poetry.
> +
> +RUN apt-get -y install --no-install-recommends \
> + vim emacs git
> diff --git a/dts/README.md b/dts/README.md
> new file mode 100644
> index 0000000000..36c8cc9a0c
> --- /dev/null
> +++ b/dts/README.md
> @@ -0,0 +1,70 @@
> +# DTS Environment
> +The execution and development environments for DTS are the same, a
> +[Docker](https://docs.docker.com/) container defined by our
> [Dockerfile](./Dockerfile).
> +Using a container for the development environment helps with a few things.
> +
> +1. It helps enforce the boundary between the DTS environment and the
> TG/SUT, something
> + which caused issues in the past.
> +2. It makes creating containers to run DTS inside automated tooling much
> easier, since
> + they can be based off of a known-working environment that will be
> updated as DTS is.
> +3. It abstracts DTS from the server it is running on. This means that the
> bare-metal os
> + can be whatever corporate policy or your personal preferences dictate,
> and DTS does
> + not have to try to support all distros that are supported by DPDK CI.
> +4. It makes automated testing for DTS easier, since new dependencies can
> be sent in with
> + the patches.
> +5. It fixes the issue of undocumented dependencies, where some test
> suites require
> + python libraries that are not installed.
> +6. Allows everyone to use the same python version easily, even if they
> are using a
> + distribution or Windows with out-of-date packages.
> +7. Allows you to run the tester on Windows while developing via Docker
> for Windows.
> +
> +## Tips for setting up a development environment
> +
> +### Getting a docker shell
> +These commands will give you a bash shell inside the container with all
> the python
> +dependencies installed. This will place you inside a python virtual
> environment. DTS is
> +mounted via a volume, which is essentially a symlink from the host to the
> container.
> +This enables you to edit and run inside the container and then delete the
> container when
> +you are done, keeping your work. It is also strongly recommended that you
> mount your SSH
> +keys into the container to allow you to connect to hosts without
> specifying a password.
> +
> +#### Start docker container with SSH keys
> +
> +```shell
> +docker build --target dev -t dpdk-dts .
> +docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it
> dpdk-dts bash
> +$ poetry install --no-root
> +$ poetry shell
> +```
> +
> +#### Start docker container without SSH keys
> +
> +```shell
> +docker build --target dev -t dpdk-dts .
> +docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
> +$ poetry install --no-root
> +$ poetry shell
> +```
> +
> +### Vim/Emacs
> +Any editor in the ubuntu repos should be easy to use, with vim and emacs
> already
> +installed. You can add your normal config files as a volume, enabling you
> to use your
> +preferred settings.
> +
> +```shell
> +docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it dpdk-dts
> bash
> +```
> +
> +### Visual Studio Code
> +VSCode has first-class support for developing with containers. You may
> need to run the
> +non-docker setup commands in the integrated terminal. DTS contains a
> .devcontainer
> +config, so if you open the folder in vscode it should prompt you to use
> the dev
> +container assuming you have the plugin installed. Please refer to
> +[VS Development Containers Docs](
> https://code.visualstudio.com/docs/remote/containers)
> +to set it all up. Additionally, there is a line in
> `.devcontainer/devcontainer.json`
> +that, when included, will mount the SSH keys of the user currently
> running vscode into
> +the container for you. The `source` on this line can be altered to mount
> any SSH keys on
> +the local machine into the container at the correct location.
> +
> +### Other
> +Searching for '$IDE dev containers' will probably lead you in the right
> direction.
> --
> 2.43.0
>
>
>
[-- Attachment #2: Type: text/html, Size: 11498 bytes --]
prev parent reply other threads:[~2024-02-29 16:14 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-03 13:46 [PATCH v1] " Juraj Linkeš
2022-11-04 9:16 ` Juraj Linkeš
2023-02-09 16:49 ` Patrick Robb
2023-04-28 19:34 ` Jeremy Spewock
2023-10-17 13:52 ` Paul Szczepanek
2023-10-26 21:56 ` Jeremy Spewock
2023-10-27 9:19 ` Juraj Linkeš
2024-01-11 21:35 ` [PATCH v2] " jspewock
2024-01-11 22:26 ` [PATCH v3] " jspewock
2024-01-12 10:23 ` Juraj Linkeš
2024-01-12 14:59 ` Jeremy Spewock
2024-01-15 9:22 ` Juraj Linkeš
2024-01-16 19:18 ` [PATCH v4] " jspewock
2024-01-17 9:00 ` Juraj Linkeš
2024-02-21 3:40 ` Patrick Robb
2024-03-07 10:51 ` Thomas Monjalon
[not found] ` <CAJvnSUDDpGdxe8D-GmtbVixrrrB_GRfvS985Cw1RLx79aLcXnA@mail.gmail.com>
2024-02-29 16:14 ` Nicholas Pratte [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKXZ7ehBuBhoBr_LhBixFWtRvW11iDaw18mKNLHAXW-wi4FuGg@mail.gmail.com \
--to=npratte@iol.unh.edu \
--cc=dev@dpdk.org \
--cc=jspewock@iol.unh.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).