From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 189FDA0548; Tue, 11 Oct 2022 17:41:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2E7442DE1; Tue, 11 Oct 2022 17:41:29 +0200 (CEST) Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by mails.dpdk.org (Postfix) with ESMTP id 77D3F42B7D for ; Tue, 11 Oct 2022 17:41:28 +0200 (CEST) Received: by mail-pl1-f173.google.com with SMTP id h10so13571469plb.2 for ; Tue, 11 Oct 2022 08:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ykXqZSJ+jthiYFbY47X/28V360bUsMHy8itdJjC0KHI=; b=XFX5Xootx9W+fwywJaVp/YcXwbhUqZ/j3QECjkUfE33KQODxbNBocyXWxdkr4xDC4R Dq9+BIv7S7UQUi7LwuUsjwQ1bbjxo+EhbwEZY6jc3FH/KSPKnH1zR3y0g7ouWeM2gUFN eM8ZKYk2VOk70U4gH6F+OPzCvc40GOzbqRGXc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ykXqZSJ+jthiYFbY47X/28V360bUsMHy8itdJjC0KHI=; b=zzUe4lm88NrPS1lixu6R/GyhIL5JRk9iSu+tTRV5maJxL78vNExcI1czruza2Joajt 9034PPNNc3k3kHv6BblavLWTceDBXNK37ghHlwpvhyQzzFZ8QSboxMU74hdekL5rsWof zMKaBbcHRzxLC5QrTn487zRmo7iX9UonmPT5K/RX8oRB7wgWLERVJdRgAalEZgFLZ0Hi zj1c1fryCLlYEXOKxBBpfk7cTevVKrgu2z7oDj1FAfwyiG8cznBrgLBIL8uaUAUhF1/N R97Zp5fczsx5397KIamZhz6krMqa0jcmEH1BhTWgruM3j7/LwKLRA/SqdUs9XUpEouzO noPg== X-Gm-Message-State: ACrzQf3SrP8RFKgxCGd1nPfJPz0o/2VOUJMrdVBYx0VTAbP5mzlBwSfv hD+a2QPhglRRceY8SatI6C8YiN/ry+EFbsgofipg8Q== X-Google-Smtp-Source: AMsMyM7c+AR08RRJtDY6mdBeVq/fr/Zs+n+gG/0vyFIFcC1TZ9ToW+AvUKN9GvMLwxU/o/zuVgckV9MVoHrlXvDrgQM= X-Received: by 2002:a17:902:b944:b0:179:fdb0:1c39 with SMTP id h4-20020a170902b94400b00179fdb01c39mr25063100pls.98.1665502887646; Tue, 11 Oct 2022 08:41:27 -0700 (PDT) MIME-Version: 1.0 References: <20220926141713.2415010-1-juraj.linkes@pantheon.tech> In-Reply-To: <20220926141713.2415010-1-juraj.linkes@pantheon.tech> From: Owen Hilyard Date: Tue, 11 Oct 2022 11:40:51 -0400 Message-ID: Subject: Re: [PATCH v5 00/10] dts: ssh connection to a node To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: thomas@monjalon.net, david.marchand@redhat.com, Honnappa.Nagarahalli@arm.com, lijuan.tu@intel.com, kda@semihalf.com, bruce.richardson@intel.com, dev@dpdk.org Content-Type: multipart/alternative; boundary="000000000000fdb2f405eac41b40" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000fdb2f405eac41b40 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Sep 26, 2022 at 10:17 AM Juraj Linke=C5=A1 wrote: > All the necessary code needed to connect to a node in a topology with > a bit more, such as basic logging and some extra useful methods. > > To run the code, modify the config file, conf.yaml and execute ./main.py > from the root dts folder. Here's an example config: > executions: > - system_under_test: "SUT 1" > nodes: > - name: "SUT 1" > hostname: 127.0.0.1 > user: root > password: mypw.change.me > > There are configuration files with a README that help with setting up > the execution/development environment. > > The code only connects to a node. You'll see logs emitted to console > saying where DTS connected. > > There's only a bit of documentation, as there's not much to document. > We'll add some real docs when there's enough functionality to document, > when the HelloWorld testcases is in (point 4 in our roadmap below). What > will be documented later is runtime dependencies and how to set up the DT= S > control node environment. > > This is our current roadmap: > 1. Review this patchset and do the rest of the items in parallel, if > possible. > 2. We have extracted the code needed to run the most basic testcase, > HelloWorld, which runs the DPDK Hello World application. We'll split > this along logical/functional boundaries and send after 1 is done. > 3. Once we have 2 applied, we're planning on adding a basic functional > testcase - pf_smoke. This send a bit of traffic, so the big addition is > the software traffic generator, Scapy. There's some work already done on > Traffic generators we'll be sending as a dependence on this patch > series. > 4. After 3, we'll add a basic performance testcase which doesn't use > Scapy, but Trex or Ixia instead. > 5. This is far in the future, but at this point we should have all of > the core functionality in place. What then remains is adding the rest of > the testcases. > > We're already working on items 2-4 and we may send more patches even > before this patch series is accepted if that's beneficial. The new > patches would then depend on this patch. > > This patch, as well as all others in the pipeline, are the result of > extensive DTS workgroup review which happens internally. If you'd like > us to make it more public we'd have no problem with that. > > v3: > Added project config files and developer tools. > Removed locks for parallel nodes, which are not needed now and will be > implemented much later (in a different patch). > > v4: > Minor fixes - added missing Exception and utils function. > > v5: > Reordered commits because the dependencies between commits changed. > Added more developer tools. > Added definitions of DTS testbed elements. > Reworked SSH implementation - split it so that the split between an > abstraction and the actual implementation is clearer. > Modified the directory structure to better organize the current and the > future code. > > Juraj Linke=C5=A1 (9): > dts: add project tools config > dts: add developer tools > dts: add basic logging facility > dts: add remote session abstraction > dts: add ssh connection module > dts: add node base class > dts: add dts workflow module > dts: add dts executable script > maintainers: add dts maintainers > > Owen Hilyard (1): > dts: add config parser module > > .editorconfig | 2 +- > .gitignore | 9 +- > MAINTAINERS | 5 + > devtools/python-checkpatch.sh | 39 ++ > devtools/python-format.sh | 54 +++ > devtools/python-lint.sh | 26 ++ > doc/guides/contributing/coding_style.rst | 4 +- > dts/.devcontainer/devcontainer.json | 30 ++ > dts/Dockerfile | 39 ++ > dts/README.md | 154 ++++++++ > dts/conf.yaml | 6 + > dts/framework/__init__.py | 4 + > dts/framework/config/__init__.py | 99 +++++ > dts/framework/config/conf_yaml_schema.json | 73 ++++ > dts/framework/dts.py | 69 ++++ > dts/framework/exception.py | 71 ++++ > dts/framework/logger.py | 115 ++++++ > dts/framework/remote_session/__init__.py | 5 + > .../remote_session/remote_session.py | 100 +++++ > .../remote_session/session_factory.py | 16 + > dts/framework/remote_session/ssh_session.py | 189 ++++++++++ > dts/framework/settings.py | 108 ++++++ > dts/framework/testbed_model/__init__.py | 8 + > dts/framework/testbed_model/node.py | 83 +++++ > dts/framework/utils.py | 31 ++ > dts/main.py | 24 ++ > dts/poetry.lock | 351 ++++++++++++++++++ > dts/pyproject.toml | 55 +++ > 28 files changed, 1765 insertions(+), 4 deletions(-) > create mode 100755 devtools/python-checkpatch.sh > create mode 100755 devtools/python-format.sh > create mode 100755 devtools/python-lint.sh > create mode 100644 dts/.devcontainer/devcontainer.json > create mode 100644 dts/Dockerfile > create mode 100644 dts/README.md > create mode 100644 dts/conf.yaml > create mode 100644 dts/framework/__init__.py > create mode 100644 dts/framework/config/__init__.py > create mode 100644 dts/framework/config/conf_yaml_schema.json > create mode 100644 dts/framework/dts.py > create mode 100644 dts/framework/exception.py > create mode 100644 dts/framework/logger.py > create mode 100644 dts/framework/remote_session/__init__.py > create mode 100644 dts/framework/remote_session/remote_session.py > create mode 100644 dts/framework/remote_session/session_factory.py > create mode 100644 dts/framework/remote_session/ssh_session.py > create mode 100644 dts/framework/settings.py > create mode 100644 dts/framework/testbed_model/__init__.py > create mode 100644 dts/framework/testbed_model/node.py > create mode 100644 dts/framework/utils.py > create mode 100755 dts/main.py > create mode 100644 dts/poetry.lock > create mode 100644 dts/pyproject.toml > > -- > 2.30.2 > > Everything looks good from my perspective. --000000000000fdb2f405eac41b40 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Mon, Sep 26, 2022 at 10:17 AM Juraj Linke= =C5=A1 <juraj.linkes@pantheon.tech> wrote:
All the necessary code needed to connect t= o a node in a topology with
a bit more, such as basic logging and some extra useful methods.

To run the code, modify the config file, conf.yaml and execute ./main.py from the root dts folder. Here's an example config:
executions:
=C2=A0 - system_under_test: "SUT 1"
nodes:
=C2=A0 - name: "SUT 1"
=C2=A0 =C2=A0 hostname: 127.0.0.1
=C2=A0 =C2=A0 user: root
=C2=A0 =C2=A0 password: mypw.change.me

There are configuration files with a README that help with setting up
the execution/development environment.

The code only connects to a node. You'll see logs emitted to console saying where DTS connected.

There's only a bit of documentation, as there's not much to documen= t.
We'll add some real docs when there's enough functionality to docum= ent,
when the HelloWorld testcases is in (point 4 in our roadmap below). What will be documented later is runtime dependencies and how to set up the DTS<= br> control node environment.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional<= br> testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done o= n
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't us= e
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of the testcases.

We're already working on items 2-4 and we may send more patches even before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.

This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like<= br> us to make it more public we'd have no problem with that.

v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).

v4:
Minor fixes - added missing Exception and utils function.

v5:
Reordered commits because the dependencies between commits changed.
Added more developer tools.
Added definitions of DTS testbed elements.
Reworked SSH implementation - split it so that the split between an
abstraction and the actual implementation is clearer.
Modified the directory structure to better organize the current and the
future code.

Juraj Linke=C5=A1 (9):
=C2=A0 dts: add project tools config
=C2=A0 dts: add developer tools
=C2=A0 dts: add basic logging facility
=C2=A0 dts: add remote session abstraction
=C2=A0 dts: add ssh connection module
=C2=A0 dts: add node base class
=C2=A0 dts: add dts workflow module
=C2=A0 dts: add dts executable script
=C2=A0 maintainers: add dts maintainers

Owen Hilyard (1):
=C2=A0 dts: add config parser module

=C2=A0.editorconfig=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2= =A02 +-
=C2=A0.gitignore=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 = =C2=A09 +-
=C2=A0MAINTAINERS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2= =A0 =C2=A05 +
=C2=A0devtools/python-checkpatch.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 39 ++
=C2=A0devtools/python-format.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 54 +++
=C2=A0devtools/python-lint.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 26 ++
=C2=A0doc/guides/contributing/coding_style.rst=C2=A0 =C2=A0 =C2=A0 |=C2=A0 = =C2=A04 +-
=C2=A0dts/.devcontainer/devcontainer.json=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0|=C2=A0 30 ++
=C2=A0dts/Dockerfile=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 39 ++
=C2=A0dts/README.md=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 154 +++++++= +
=C2=A0dts/conf.yaml=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2= =A06 +
=C2=A0dts/framework/__init__.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A04 +
=C2=A0dts/framework/config/__init__.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 |=C2=A0 99 +++++
=C2=A0dts/framework/config/conf_yaml_schema.json=C2=A0 =C2=A0 |=C2=A0 73 ++= ++
=C2=A0dts/framework/dts.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 69 ++++
=C2=A0dts/framework/exception.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 71 ++++
=C2=A0dts/framework/logger.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 115 ++++++
=C2=A0dts/framework/remote_session/__init__.py=C2=A0 =C2=A0 =C2=A0 |=C2=A0 = =C2=A05 +
=C2=A0.../remote_session/remote_session.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 | 100 +++++
=C2=A0.../remote_session/session_factory.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0|=C2=A0 16 +
=C2=A0dts/framework/remote_session/ssh_session.py=C2=A0 =C2=A0| 189 +++++++= +++
=C2=A0dts/framework/settings.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 108 ++++++
=C2=A0dts/framework/testbed_model/__init__.py=C2=A0 =C2=A0 =C2=A0 =C2=A0|= =C2=A0 =C2=A08 +
=C2=A0dts/framework/testbed_model/node.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0|=C2=A0 83 +++++
=C2=A0dts/framework/utils.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 31 ++
=C2=A0dts/main.py=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2= =A0 24 ++
=C2=A0dts/poetry.lock=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| 351 ++++++++++= ++++++++
=C2=A0dts/pyproject.toml=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 55 +++
=C2=A028 files changed, 1765 insertions(+), 4 deletions(-)
=C2=A0create mode 100755 devtools/python-checkpatch.sh
=C2=A0create mode 100755 devtools/python-format.sh
=C2=A0create mode 100755 devtools/python-lint.sh
=C2=A0create mode 100644 dts/.devcontainer/devcontainer.json
=C2=A0create mode 100644 dts/Dockerfile
=C2=A0create mode 100644 dts/README.md
=C2=A0create mode 100644 dts/conf.yaml
=C2=A0create mode 100644 dts/framework/__init__.py
=C2=A0create mode 100644 dts/framework/config/__init__.py
=C2=A0create mode 100644 dts/framework/config/conf_yaml_schema.json
=C2=A0create mode 100644 dts/framework/dts.py
=C2=A0create mode 100644 dts/framework/exception.py
=C2=A0create mode 100644 dts/framework/logger.py
=C2=A0create mode 100644 dts/framework/remote_session/__init__.py
=C2=A0create mode 100644 dts/framework/remote_session/remote_session.py
=C2=A0create mode 100644 dts/framework/remote_session/session_factory.py =C2=A0create mode 100644 dts/framework/remote_session/ssh_session.py
=C2=A0create mode 100644 dts/framework/settings.py
=C2=A0create mode 100644 dts/framework/testbed_model/__init__.py
=C2=A0create mode 100644 dts/framework/testbed_model/node.py
=C2=A0create mode 100644 dts/framework/utils.py
=C2=A0create mode 100755 dts/main.py
=C2=A0create mode 100644 dts/poetry.lock
=C2=A0create mode 100644 dts/pyproject.toml

--
2.30.2


Everything looks good from my perspect= ive.=C2=A0=C2=A0
--000000000000fdb2f405eac41b40--