The DTS working group has decided that, conditional upon no issues being discovered, DTS should be re-organized into a more standard python project format. We have a few reasons for this.

How DTS currently is structured is only possible due to modifications of the python interpreter search path. The specific modifications are adding all of the directories with python code in them, such as framework, nic, and tests, to the search path. This enables an import statement like "import logging" to work in a test suite, as opposed to the "import framework.logging as logging" that would otherwise be needed. However, this change is not detected by most python tooling. As a result, features like autocompletion and jump-to-definition are broken in many IDEs, and static analysis tools can only do surface-level inspections, which also produce a lot of false positives due to undefined variables. There are two core reasons the DTS working group has decided to move in this direction.

1. Developer Convenience 

There has been a long-running discussion around making DTS submissions for DPDK patches that introduce new features mandatory. We feel that making DTS look more like a standard python project would help ease the transition period. In addition, as mentioned before, IDE features like autocomplete and jump-to-definition would be functional, and we feel that those are very useful for learning a codebase. This will also enable any future changes to be made with the aid of automated refactoring tools, reducing the risk of breaking changes.

2. Automated Tooling

Most automated tooling also cannot process the additions to the python path. Some tools, like formatters, are file-oriented and work perfectly fine right now, but others, such as flake8, do not. Due to python's nature as a duck typed language, not knowing the base type of something (because it can't be imported inside of the tool), means that everything it interacts with also has an unknown type. This results in some tooling choosing to treat it as always the correct type, and some tooling to treat it as though it could be any type in the project. Both of these approaches reduce the accuracy of the tooling. 

What will change:

* The main file, currently located at dts/framework/main.py, will be moved to dts/main.py. 

* __init__.py files will be added to all modules to allow for exporting the types the modules needs to expose.

* Most import paths will be updated, it is expected that most imports will change from "import logging" to "import framework.logging as logging", to avoid needing to change any source code during the reorganization. 

What we need from the community:

What the DTS working group needs from the community is to let us know if there are other changes that should be made, or if this change would cause issues for you. We anticipate that there will not be many issues with this reorganization unless there is external tooling attached to DTS that is incompatible with a standard python module format. 

Owen Hilyard