From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF4EE429C4; Fri, 28 Apr 2023 21:34:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DA236410DD; Fri, 28 Apr 2023 21:34:53 +0200 (CEST) Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by mails.dpdk.org (Postfix) with ESMTP id 4D65C406B5 for ; Fri, 28 Apr 2023 21:34:53 +0200 (CEST) Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-63b5c4c76aaso301866b3a.2 for ; Fri, 28 Apr 2023 12:34:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1682710492; x=1685302492; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=wKjGOrFYFoKnoKs97AtNd0TOsSmvg6jmN88uf0HUHHQ=; b=RIGYFkETiEnXFokrSPTVoTHbZ2QKBb/ssHZoawx+jk2+NIwSVl+X1cvvWspa9xXfK0 sPdTOaMraLZyjO1shvy3i+CSV4w6nH/+L/Mgj1INVLhZUCmdS4HVVFR4auVjiZv2+0L5 H+/pfcViUcjfjpstZxa7JPnxJ2fkpPa9DMBiU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682710492; x=1685302492; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wKjGOrFYFoKnoKs97AtNd0TOsSmvg6jmN88uf0HUHHQ=; b=CFfisaID6j8K+R2J7hK6BjVJkmurWfsS99PORGRT6DnMs8yTtq6yWM35ieLX9c/vXM vsdp8rd8IMP1YnLrlw7CwR9l3cnVMi/pVRxrhSvJZ/X9CR8uE7PnkISPkZK3+tnb3R+8 s/6BLG4Qv0CTa5GMNEGBBIzzeWFJ0LA8a62BkL7eRpD1KW1QBLLKICTqbeuBSuA1WooQ AubdBkD5IAYrFsFuKdYtcT6ij82nRSvMRbbVI90xrFATi29NCae3p0L9asADN7Djks/C RSbq3kqf3/nQw2TNB+pJt2VTBuI5o/leaCDUagcTFnFOps7bLdHCB7H5jvyLf/OfPGTb em2A== X-Gm-Message-State: AC+VfDwvG9wBW+hgpqey9W7zshtxd48wVG+kT45+lxOdpn8Cc0fqRR95 6csV9rQ3rWdrk+xA1EzEedlRKUI8cmIesPrfAXN7Sg== X-Google-Smtp-Source: ACHHUZ4bCs2QwFjCkH/VWBbgT5HEioF1dt94EiqROYXK34YdI9/bCRsoQWEYGWvuw9CmB0ZlAURZMmvoy9K3JLUaZZM= X-Received: by 2002:a17:90a:cc08:b0:23f:29a:5554 with SMTP id b8-20020a17090acc0800b0023f029a5554mr5679983pju.48.1682710492398; Fri, 28 Apr 2023 12:34:52 -0700 (PDT) MIME-Version: 1.0 References: <20221103134633.446646-1-juraj.linkes@pantheon.tech> <681d961a4ff141b58255a22aa15b054c@pantheon.tech> In-Reply-To: From: Jeremy Spewock Date: Fri, 28 Apr 2023 15:34:41 -0400 Message-ID: Subject: Re: [PATCH v1] dts: add Dockerfile To: Patrick Robb Cc: =?UTF-8?Q?Juraj_Linke=C5=A1?= , Honnappa Nagarahalli , Owen Hilyard , "thomas@monjalon.net" , Lijuan Tu , "Richardson, Bruce" , "dev@dpdk.org" Content-Type: multipart/alternative; boundary="00000000000028e01005fa6a91d6" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000028e01005fa6a91d6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Acked-by: Jeremy Spweock On Thu, Feb 9, 2023 at 11:49=E2=80=AFAM Patrick Robb wr= ote: > > > On Fri, Nov 4, 2022 at 5:16 AM Juraj Linke=C5=A1 > wrote: > >> Adding folks I forgot to add. >> >> > -----Original Message----- >> > From: Juraj Linke=C5=A1 >> > Sent: Thursday, November 3, 2022 2:47 PM >> > Cc: dev@dpdk.org; Juraj Linke=C5=A1 >> > Subject: [PATCH v1] dts: add Dockerfile >> > >> > The Dockerfile defines development and CI runner images. >> > >> > Signed-off-by: Juraj Linke=C5=A1 >> > --- >> > dts/.devcontainer/devcontainer.json | 30 ++++++++++++++++ >> > dts/Dockerfile | 38 ++++++++++++++++++++ >> > dts/README.md | 55 ++++++++++++++++++++++++++++= + >> > 3 files changed, 123 insertions(+) >> > create mode 100644 dts/.devcontainer/devcontainer.json >> > create mode 100644 dts/Dockerfile >> > create mode 100644 dts/README.md >> > >> > diff --git a/dts/.devcontainer/devcontainer.json >> > b/dts/.devcontainer/devcontainer.json >> > new file mode 100644 >> > index 0000000000..41ca28fc17 >> > --- /dev/null >> > +++ b/dts/.devcontainer/devcontainer.json >> > @@ -0,0 +1,30 @@ >> > +// For format details, see https://aka.ms/devcontainer.json. For >> config >> > options, see the README at: >> > +// >> > + >> https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/contai >> > +ners/docker-existing-dockerfile >> > +{ >> > + "name": "Existing Dockerfile", >> > + >> > + // Sets the run context to one level up instead of the >> .devcontainer >> > folder. >> > + "context": "..", >> > + >> > + // Update the 'dockerFile' property if you aren't using the >> standard >> > 'Dockerfile' filename. >> > + "dockerFile": "../Dockerfile", >> > + >> > + // Use 'forwardPorts' to make a list of ports inside the contain= er >> > available locally. >> > + // "forwardPorts": [], >> > + >> > + // Uncomment the next line to run commands after the container i= s >> > created - for example installing curl. >> > + "postCreateCommand": "poetry install", >> > + >> > + "extensions": [ >> > + "ms-python.vscode-pylance", >> > + ] >> > + >> > + // Uncomment when using a ptrace-based debugger like C++, Go, >> > and Rust >> > + // "runArgs": [ "--cap-add=3DSYS_PTRACE", "--security-opt", >> > +"seccomp=3Dunconfined" ], >> > + >> > + // Uncomment to use the Docker CLI from inside the container. Se= e >> > https://aka.ms/vscode-remote/samples/docker-from-docker. >> > + // "mounts": [ >> > +"source=3D/var/run/docker.sock,target=3D/var/run/docker.sock,type=3Db= ind" ], >> > + >> > + // Uncomment to connect as a non-root user if you've added one. >> > See https://aka.ms/vscode-remote/containers/non-root. >> > + // "remoteUser": "vscode" >> > +} >> > diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644 inde= x >> > 0000000000..9a91949eeb >> > --- /dev/null >> > +++ b/dts/Dockerfile >> > @@ -0,0 +1,38 @@ >> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022 Universit= y >> > +of New Hampshire >> > + >> > +# There are two Docker images defined in this Dockerfile. >> > +# One is to be used in CI for automated testing. >> > +# The other provides a DTS development environment, simplifying Pytho= n >> > dependency management. >> > + >> > +FROM ubuntu:22.04 AS base >> > + >> > +RUN apt-get -y update && apt-get -y upgrade && \ >> > + apt-get -y install --no-install-recommends \ >> > + python3 \ >> > + python3-pip \ >> > + python3-pexpect \ >> > + python3-poetry \ >> > + python3-cachecontrol \ >> > + openssh-client >> > +WORKDIR /dpdk/dts >> > + >> > + >> > +FROM base AS runner >> > + >> > +# This image is intended to be used as the base for automated systems= . >> > +# It bakes DTS into the image during the build. >> > + >> > +COPY . /dpdk/dts >> > +RUN poetry install --no-dev >> > + >> > +CMD ["poetry", "run", "python", "main.py"] >> > + >> > +FROM base AS dev >> > + >> > +# This image is intended to be used as DTS development environment. I= t >> > +doesn't need C compilation # capabilities, only Python dependencies. >> > +Once a container mounting DTS using this image is running, # the >> > dependencies should be installed using Poetry. >> > + >> > +RUN apt-get -y install --no-install-recommends \ >> > + vim emacs git >> > diff --git a/dts/README.md b/dts/README.md new file mode 100644 index >> > 0000000000..fd9f32595c >> > --- /dev/null >> > +++ b/dts/README.md >> > @@ -0,0 +1,55 @@ >> > +# DTS Environment >> > +The execution and development environments for DTS are the same, a >> > +[Docker](https://docs.docker.com/) container defined by our >> > [Dockerfile](./Dockerfile). >> > +Using a container for the development environment helps with a few >> > things. >> > + >> > +1. It helps enforce the boundary between the DTS environment and the >> > TG/SUT, something >> > + which caused issues in the past. >> > +2. It makes creating containers to run DTS inside automated tooling >> much >> > easier, since >> > + they can be based off of a known-working environment that will be >> > updated as DTS is. >> > +3. It abstracts DTS from the server it is running on. This means that >> the bare- >> > metal os >> > + can be whatever corporate policy or your personal preferences >> dictate, >> > and DTS does >> > + not have to try to support all distros that are supported by DPDK >> CI. >> > +4. It makes automated testing for DTS easier, since new dependencies >> > +can be sent in with >> > + the patches. >> > +5. It fixes the issue of undocumented dependencies, where some test >> > suites require >> > + python libraries that are not installed. >> > +6. Allows everyone to use the same python version easily, even if the= y >> are >> > using a >> > + distribution or Windows with out-of-date packages. >> > +7. Allows you to run the tester on Windows while developing via Docke= r >> for >> > Windows. >> > + >> > +## Tips for setting up a development environment >> > + >> > +### Getting a docker shell >> > +These commands will give you a bash shell inside the container with a= ll >> > +the python dependencies installed. This will place you inside a pytho= n >> > +virtual environment. DTS is mounted via a volume, which is essentiall= y >> a >> > symlink from the host to the container. >> > +This enables you to edit and run inside the container and then delete >> > +the container when you are done, keeping your work. >> > + >> > +```shell >> > +docker build --target dev -t dpdk-dts . >> > +docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash $ poetry install $ >> > +poetry shell ``` >> > + >> > +### Vim/Emacs >> > +Any editor in the ubuntu repos should be easy to use, with vim and >> > +emacs already installed. You can add your normal config files as a >> > +volume, enabling you to use your preferred settings. >> > + >> > +```shell >> > +docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it >> > +dpdk-dts bash ``` >> > + >> > +### Visual Studio Code >> > +VSCode has first-class support for developing with containers. You ma= y >> > +need to run the non-docker setup commands in the integrated terminal. >> > +DTS contains a .devcontainer config, so if you open the folder in >> > +vscode it should prompt you to use the dev container assuming you hav= e >> > +the plugin installed. Please refer to [VS Development Containers >> > +Docs](https://code.visualstudio.com/docs/remote/containers) >> > +to set it all up. >> > + >> > +### Other >> > +Searching for '$IDE dev containers' will probably lead you in the rig= ht >> > direction. >> > -- >> > 2.30.2 >> >> Tested-by: Patrick Robb > > -- > > Patrick Robb > > Technical Service Manager > > UNH InterOperability Laboratory > > 21 Madbury Rd, Suite 100, Durham, NH 03824 > > www.iol.unh.edu > > > --00000000000028e01005fa6a91d6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Acked-by: Jeremy Spweock <jspweock@iol.unh.edu>

On Thu, Feb 9,= 2023 at 11:49=E2=80=AFAM Patrick Robb <probb@iol.unh.edu> wrote:


On Fri, Nov 4, = 2022 at 5:16 AM Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech> wrote:=
Adding folks I = forgot to add.

> -----Original Message-----
> From: Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech>
> Sent: Thursday, November 3, 2022 2:47 PM
> Cc: dev@dpdk.org= ; Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech>
> Subject: [PATCH v1] dts: add Dockerfile
>
> The Dockerfile defines development and CI runner images.
>
> Signed-off-by: Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech> > ---
>=C2=A0 dts/.devcontainer/devcontainer.json | 30 ++++++++++++++++
>=C2=A0 dts/Dockerfile=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 | 38 ++++++++++++++++++++
>=C2=A0 dts/README.md=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 55 +++++++++++++++++++++++++++++
>=C2=A0 3 files changed, 123 insertions(+)
>=C2=A0 create mode 100644 dts/.devcontainer/devcontainer.json
>=C2=A0 create mode 100644 dts/Dockerfile
>=C2=A0 create mode 100644 dts/README.md
>
> diff --git a/dts/.devcontainer/devcontainer.json
> b/dts/.devcontainer/devcontainer.json
> new file mode 100644
> index 0000000000..41ca28fc17
> --- /dev/null
> +++ b/dts/.devcontainer/devcontainer.json
> @@ -0,0 +1,30 @@
> +// For format details, see https://aka.ms/devcontainer.json. For config
> options, see the README at:
> +//
> +https://github.com/micr= osoft/vscode-dev-containers/tree/v0.241.1/contai
> +ners/docker-existing-dockerfile
> +{
> +=C2=A0 =C2=A0 =C2=A0"name": "Existing Dockerfile"= ,
> +
> +=C2=A0 =C2=A0 =C2=A0// Sets the run context to one level up instead o= f the .devcontainer
> folder.
> +=C2=A0 =C2=A0 =C2=A0"context": "..",
> +
> +=C2=A0 =C2=A0 =C2=A0// Update the 'dockerFile' property if yo= u aren't using the standard
> 'Dockerfile' filename.
> +=C2=A0 =C2=A0 =C2=A0"dockerFile": "../Dockerfile"= ,
> +
> +=C2=A0 =C2=A0 =C2=A0// Use 'forwardPorts' to make a list of p= orts inside the container
> available locally.
> +=C2=A0 =C2=A0 =C2=A0// "forwardPorts": [],
> +
> +=C2=A0 =C2=A0 =C2=A0// Uncomment the next line to run commands after = the container is
> created - for example installing curl.
> +=C2=A0 =C2=A0 =C2=A0"postCreateCommand": "poetry insta= ll",
> +
> +=C2=A0 =C2=A0 =C2=A0"extensions": [
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"ms-python.vscod= e-pylance",
> +=C2=A0 =C2=A0 =C2=A0]
> +
> +=C2=A0 =C2=A0 =C2=A0// Uncomment when using a ptrace-based debugger l= ike C++, Go,
> and Rust
> +=C2=A0 =C2=A0 =C2=A0// "runArgs": [ "--cap-add=3DSYS_P= TRACE", "--security-opt",
> +"seccomp=3Dunconfined" ],
> +
> +=C2=A0 =C2=A0 =C2=A0// Uncomment to use the Docker CLI from inside th= e container. See
> https://aka.ms/vscode-remote/samples/doc= ker-from-docker.
> +=C2=A0 =C2=A0 =C2=A0// "mounts": [
> +"source=3D/var/run/docker.sock,target=3D/var/run/docker.sock,typ= e=3Dbind" ],
> +
> +=C2=A0 =C2=A0 =C2=A0// Uncomment to connect as a non-root user if you= 've added one.
> See https://aka.ms/vscode-remote/containers/n= on-root.
> +=C2=A0 =C2=A0 =C2=A0// "remoteUser": "vscode"
> +}
> diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644 inde= x
> 0000000000..9a91949eeb
> --- /dev/null
> +++ b/dts/Dockerfile
> @@ -0,0 +1,38 @@
> +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022 Universit= y
> +of New Hampshire
> +
> +# There are two Docker images defined in this Dockerfile.
> +# One is to be used in CI for automated testing.
> +# The other provides a DTS development environment, simplifying Pytho= n
> dependency management.
> +
> +FROM ubuntu:22.04 AS base
> +
> +RUN apt-get -y update && apt-get -y upgrade && \
> +=C2=A0 =C2=A0 apt-get -y install --no-install-recommends \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 python3 \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 python3-pip \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 python3-pexpect \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 python3-poetry \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 python3-cachecontrol \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 openssh-client
> +WORKDIR /dpdk/dts
> +
> +
> +FROM base AS runner
> +
> +# This image is intended to be used as the base for automated systems= .
> +# It bakes DTS into the image during the build.
> +
> +COPY . /dpdk/dts
> +RUN poetry install --no-dev
> +
> +CMD ["poetry", "run", "python", "m= ain.py"]
> +
> +FROM base AS dev
> +
> +# This image is intended to be used as DTS development environment. I= t
> +doesn't need C compilation # capabilities, only Python dependenci= es.
> +Once a container mounting DTS using this image is running, # the
> dependencies should be installed using Poetry.
> +
> +RUN apt-get -y install --no-install-recommends \
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 vim emacs git
> diff --git a/dts/README.md b/dts/README.md new file mode 100644 index<= br> > 0000000000..fd9f32595c
> --- /dev/null
> +++ b/dts/README.md
> @@ -0,0 +1,55 @@
> +# DTS Environment
> +The execution and development environments for DTS are the same, a > +[Docker](https://docs.docker.com/) container defined by our
> [Dockerfile](./Dockerfile).
> +Using a container for the development environment helps with a few > things.
> +
> +1. It helps enforce the boundary between the DTS environment and the<= br> > TG/SUT, something
> +=C2=A0 =C2=A0which caused issues in the past.
> +2. It makes creating containers to run DTS inside automated tooling m= uch
> easier, since
> +=C2=A0 =C2=A0they can be based off of a known-working environment tha= t will be
> updated as DTS is.
> +3. It abstracts DTS from the server it is running on. This means that= the bare-
> metal os
> +=C2=A0 =C2=A0can be whatever corporate policy or your personal prefer= ences dictate,
> and DTS does
> +=C2=A0 =C2=A0not have to try to support all distros that are supporte= d by DPDK CI.
> +4. It makes automated testing for DTS easier, since new dependencies<= br> > +can be sent in with
> +=C2=A0 the patches.
> +5. It fixes the issue of undocumented dependencies, where some test > suites require
> +=C2=A0 =C2=A0python libraries that are not installed.
> +6. Allows everyone to use the same python version easily, even if the= y are
> using a
> +=C2=A0 =C2=A0distribution or Windows with out-of-date packages.
> +7. Allows you to run the tester on Windows while developing via Docke= r for
> Windows.
> +
> +## Tips for setting up a development environment
> +
> +### Getting a docker shell
> +These commands will give you a bash shell inside the container with a= ll
> +the python dependencies installed. This will place you inside a pytho= n
> +virtual environment. DTS is mounted via a volume, which is essentiall= y a
> symlink from the host to the container.
> +This enables you to edit and run inside the container and then delete=
> +the container when you are done, keeping your work.
> +
> +```shell
> +docker build --target dev -t dpdk-dts .
> +docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash $ poetry install $ > +poetry shell ```
> +
> +### Vim/Emacs
> +Any editor in the ubuntu repos should be easy to use, with vim and > +emacs already installed. You can add your normal config files as a > +volume, enabling you to use your preferred settings.
> +
> +```shell
> +docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it
> +dpdk-dts bash ```
> +
> +### Visual Studio Code
> +VSCode has first-class support for developing with containers. You ma= y
> +need to run the non-docker setup commands in the integrated terminal.=
> +DTS contains a .devcontainer config, so if you open the folder in
> +vscode it should prompt you to use the dev container assuming you hav= e
> +the plugin installed. Please refer to [VS Development Containers
> +Docs](https://code.visualstudio.com/docs/r= emote/containers)
> +to set it all up.
> +
> +### Other
> +Searching for '$IDE dev containers' will probably lead you in= the right
> direction.
> --
> 2.30.2

Tested-by: Patrick Robb <probb@iol.unh.edu>
=
--

Patr= ick Robb

Technical Service Manager

UNH Int= erOperability Laboratory

21 Madbury Rd, Suite 100, Durha= m, NH 03824

www.iol.unh.edu

=


--00000000000028e01005fa6a91d6--