test suite reviews and discussions
 help / color / mirror / Atom feed
From: Owen Hilyard <ohilyard@iol.unh.edu>
To: dts@dpdk.org
Subject: [dts] [RFC] Reorganization of DTS
Date: Thu, 5 Aug 2021 14:49:16 -0400	[thread overview]
Message-ID: <CAHx6DYA86Rt4ncuMwyPPDWyEW_QRTFsdgg6z_4mWFjnky=KOOQ@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 3040 bytes --]

Hello everyone,

The DTS working group has decided that, conditional upon no issues being
discovered, DTS should be re-organized into a more standard python
project format. We have a few reasons for this.

How DTS currently is structured is only possible due to modifications of
the python interpreter search path. The specific modifications are adding
all of the directories with python code in them, such as framework, nic,
and tests, to the search path. This enables an import statement like
"import logging" to work in a test suite, as opposed to the "import
framework.logging as logging" that would otherwise be needed. However, this
change is not detected by most python tooling. As a result, features like
autocompletion and jump-to-definition are broken in many IDEs, and static
analysis tools can only do surface-level inspections, which also produce a
lot of false positives due to undefined variables. There are two core
reasons the DTS working group has decided to move in this direction.

1. Developer Convenience

There has been a long-running discussion around making DTS submissions for
DPDK patches that introduce new features mandatory. We feel that making DTS
look more like a standard python project would help ease the transition
period. In addition, as mentioned before, IDE features like autocomplete
and jump-to-definition would be functional, and we feel that those are very
useful for learning a codebase. This will also enable any future changes to
be made with the aid of automated refactoring tools, reducing the risk of
breaking changes.

2. Automated Tooling

Most automated tooling also cannot process the additions to the python
path. Some tools, like formatters, are file-oriented and work perfectly
fine right now, but others, such as flake8, do not. Due to python's nature
as a duck typed language, not knowing the base type of something (because
it can't be imported inside of the tool), means that everything it
interacts with also has an unknown type. This results in some tooling
choosing to treat it as always the correct type, and some tooling to treat
it as though it could be any type in the project. Both of these approaches
reduce the accuracy of the tooling.


What will change:

* The main file, currently located at dts/framework/main.py, will be moved
to dts/main.py.
* __init__.py files will be added to all modules to allow for exporting the
types the modules needs to expose.
* Most import paths will be updated, it is expected that most imports will
change from "import logging" to "import framework.logging as logging", to
avoid needing to change any source code during the reorganization.

What we need from the community:

What the DTS working group needs from the community is to let us know if
there are other changes that should be made, or if this change would cause
issues for you. We anticipate that there will not be many issues with this
reorganization unless there is external tooling attached to DTS that is
incompatible with a standard python module format.

Owen Hilyard

[-- Attachment #2: Type: text/html, Size: 3404 bytes --]

             reply	other threads:[~2021-08-05 18:49 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-05 18:49 Owen Hilyard [this message]
2021-08-11 14:08 ` Owen Hilyard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHx6DYA86Rt4ncuMwyPPDWyEW_QRTFsdgg6z_4mWFjnky=KOOQ@mail.gmail.com' \
    --to=ohilyard@iol.unh.edu \
    --cc=dts@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).