From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 37A8BA00C5; Fri, 4 Nov 2022 12:05:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6083742D1A; Fri, 4 Nov 2022 12:05:33 +0100 (CET) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by mails.dpdk.org (Postfix) with ESMTP id CEA0A42D10 for ; Fri, 4 Nov 2022 12:05:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id 226E51BA5B6; Fri, 4 Nov 2022 12:05:29 +0100 (CET) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KG7ZTI3Iekoo; Fri, 4 Nov 2022 12:05:26 +0100 (CET) Received: from entguard.lab.pantheon.local (unknown [46.229.239.141]) by lb.pantheon.sk (Postfix) with ESMTP id 7D6551BA5A5; Fri, 4 Nov 2022 12:05:26 +0100 (CET) From: =?UTF-8?q?Juraj=20Linke=C5=A1?= To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, ohilyard@iol.unh.edu, lijuan.tu@intel.com, kda@semihalf.com, bruce.richardson@intel.com Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= Subject: [PATCH v8 1/9] dts: add project tools config Date: Fri, 4 Nov 2022 11:05:15 +0000 Message-Id: <20221104110523.511367-2-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221104110523.511367-1-juraj.linkes@pantheon.tech> References: <20221103151934.450887-1-juraj.linkes@pantheon.tech> <20221104110523.511367-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add configuration for Python tools used in DTS: Poetry, dependency and package manager Black, formatter Pylama, static analysis Isort, import sorting Signed-off-by: Owen Hilyard Signed-off-by: Juraj Linkeš --- MAINTAINERS | 5 + doc/guides/tools/dts.rst | 56 ++++++ doc/guides/tools/index.rst | 1 + dts/poetry.lock | 337 +++++++++++++++++++++++++++++++++++++ dts/pyproject.toml | 46 +++++ 5 files changed, 445 insertions(+) create mode 100644 doc/guides/tools/dts.rst create mode 100644 dts/poetry.lock create mode 100644 dts/pyproject.toml diff --git a/MAINTAINERS b/MAINTAINERS index 51d77460ec..e018e396d0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -114,6 +114,11 @@ F: buildtools/symlink-drivers-solibs.py F: devtools/test-meson-builds.sh F: devtools/check-meson.py +DTS +M: Lijuan Tu +M: Owen Hilyard +F: dts/ + Public CI M: Aaron Conole M: Michael Santana diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst new file mode 100644 index 0000000000..0a88901119 --- /dev/null +++ b/doc/guides/tools/dts.rst @@ -0,0 +1,56 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2022 PANTHEON.tech s.r.o. + +DPDK Test Suite +=============== + +The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suites +implementing functional and performance tests used to test DPDK. + + +DTS Environment +--------------- + +DTS is written entirely in Python using a variety of dependencies. +DTS uses Poetry as its Python dependency management. +Python build/development and runtime environments are the same and DTS development environment, +DTS runtime environment or just plain DTS environment are used interchangeably. + + +Setting up DTS environment +-------------------------- + +#. **Python Version** + + The Python Version required by DTS is specified in ``dts/pyproject.toml`` in the + **[tool.poetry.dependencies]** section: + + .. literalinclude:: ../../../dts/pyproject.toml + :language: cfg + :start-at: [tool.poetry.dependencies] + :end-at: python + + The Python dependency manager DTS uses, Poetry, doesn't install Python, so you may need + to satisfy this requirement by other means if your Python is not up-to-date. + A tool such as `Pyenv `_ is a good way to get Python, + though not the only one. + +#. **Poetry** + + The typical style of python dependency management, pip with ``requirements.txt``, + has a few issues. + The advantages of Poetry include specifying what Python version is required and forcing you + to specify versions, enforced by a lockfile, both of which help prevent broken dependencies. + Another benefit is the usage of ``pyproject.toml``, which has become the standard config file + for python projects, improving project organization. + To install Poetry, visit their `doc pages `_. + +#. **Getting a Poetry shell** + + Once you have Poetry along with the proper Python version all set up, it's just a matter + of installing dependencies via Poetry and using the virtual environment Poetry provides: + + .. code-block:: console + + poetry install + poetry shell diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst index 0e5041a3f0..f21ef0aac9 100644 --- a/doc/guides/tools/index.rst +++ b/doc/guides/tools/index.rst @@ -20,3 +20,4 @@ DPDK Tools User Guides comp_perf testeventdev testregex + dts diff --git a/dts/poetry.lock b/dts/poetry.lock new file mode 100644 index 0000000000..0b2a007d4d --- /dev/null +++ b/dts/poetry.lock @@ -0,0 +1,337 @@ +[[package]] +name = "attrs" +version = "22.1.0" +description = "Classes Without Boilerplate" +category = "main" +optional = false +python-versions = ">=3.5" + +[package.extras] +dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] +docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] +tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] +tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"] + +[[package]] +name = "black" +version = "22.10.0" +description = "The uncompromising code formatter." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +click = ">=8.0.0" +mypy-extensions = ">=0.4.3" +pathspec = ">=0.9.0" +platformdirs = ">=2" +tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""} + +[package.extras] +colorama = ["colorama (>=0.4.3)"] +d = ["aiohttp (>=3.7.4)"] +jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] +uvloop = ["uvloop (>=0.15.2)"] + +[[package]] +name = "click" +version = "8.1.3" +description = "Composable command line interface toolkit" +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} + +[[package]] +name = "colorama" +version = "0.4.6" +description = "Cross-platform colored terminal text." +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" + +[[package]] +name = "isort" +version = "5.10.1" +description = "A Python utility / library to sort Python imports." +category = "dev" +optional = false +python-versions = ">=3.6.1,<4.0" + +[package.extras] +pipfile_deprecated_finder = ["pipreqs", "requirementslib"] +requirements_deprecated_finder = ["pipreqs", "pip-api"] +colors = ["colorama (>=0.4.3,<0.5.0)"] +plugins = ["setuptools"] + +[[package]] +name = "jsonpatch" +version = "1.32" +description = "Apply JSON-Patches (RFC 6902)" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[package.dependencies] +jsonpointer = ">=1.9" + +[[package]] +name = "jsonpointer" +version = "2.3" +description = "Identify specific nodes in a JSON document (RFC 6901)" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "jsonschema" +version = "4.17.0" +description = "An implementation of JSON Schema validation for Python" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +attrs = ">=17.4.0" +pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2" + +[package.extras] +format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"] +format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"] + +[[package]] +name = "mccabe" +version = "0.7.0" +description = "McCabe checker, plugin for flake8" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "mypy" +version = "0.961" +description = "Optional static typing for Python" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +mypy-extensions = ">=0.4.3" +tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""} +typing-extensions = ">=3.10" + +[package.extras] +dmypy = ["psutil (>=4.0)"] +python2 = ["typed-ast (>=1.4.0,<2)"] +reports = ["lxml"] + +[[package]] +name = "mypy-extensions" +version = "0.4.3" +description = "Experimental type system extensions for programs checked with the mypy typechecker." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "pathspec" +version = "0.10.1" +description = "Utility library for gitignore style pattern matching of file paths." +category = "dev" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "pexpect" +version = "4.8.0" +description = "Pexpect allows easy control of interactive console applications." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +ptyprocess = ">=0.5" + +[[package]] +name = "platformdirs" +version = "2.5.2" +description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"] +test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"] + +[[package]] +name = "ptyprocess" +version = "0.7.0" +description = "Run a subprocess in a pseudo terminal" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pycodestyle" +version = "2.9.1" +description = "Python style guide checker" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "pydocstyle" +version = "6.1.1" +description = "Python docstring style checker" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +snowballstemmer = "*" + +[package.extras] +toml = ["toml"] + +[[package]] +name = "pyflakes" +version = "2.5.0" +description = "passive checker of Python programs" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "pylama" +version = "8.4.1" +description = "Code audit tool for python" +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +mccabe = ">=0.7.0" +pycodestyle = ">=2.9.1" +pydocstyle = ">=6.1.1" +pyflakes = ">=2.5.0" + +[package.extras] +all = ["pylint", "eradicate", "radon", "mypy", "vulture"] +eradicate = ["eradicate"] +mypy = ["mypy"] +pylint = ["pylint"] +radon = ["radon"] +tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "toml", "vulture", "types-setuptools", "types-toml"] +toml = ["toml (>=0.10.2)"] +vulture = ["vulture"] + +[[package]] +name = "pyrsistent" +version = "0.19.1" +description = "Persistent/Functional/Immutable data structures" +category = "main" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "pyyaml" +version = "6.0" +description = "YAML parser and emitter for Python" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "snowballstemmer" +version = "2.2.0" +description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "toml" +version = "0.10.2" +description = "Python Library for Tom's Obvious, Minimal Language" +category = "dev" +optional = false +python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" + +[[package]] +name = "tomli" +version = "2.0.1" +description = "A lil' TOML parser" +category = "dev" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "types-pyyaml" +version = "6.0.12.1" +description = "Typing stubs for PyYAML" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "typing-extensions" +version = "4.4.0" +description = "Backported and Experimental Type Hints for Python 3.7+" +category = "dev" +optional = false +python-versions = ">=3.7" + +[[package]] +name = "warlock" +version = "2.0.1" +description = "Python object model built on JSON schema and JSON patch." +category = "main" +optional = false +python-versions = ">=3.7,<4.0" + +[package.dependencies] +jsonpatch = ">=1,<2" +jsonschema = ">=4,<5" + +[metadata] +lock-version = "1.1" +python-versions = "^3.10" +content-hash = "a0f040b07fc6ce4deb0be078b9a88c2a465cb6bccb9e260a67e92c2403e2319f" + +[metadata.files] +attrs = [] +black = [] +click = [] +colorama = [] +isort = [] +jsonpatch = [] +jsonpointer = [] +jsonschema = [] +mccabe = [] +mypy = [] +mypy-extensions = [] +pathspec = [] +pexpect = [ + {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"}, + {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"}, +] +platformdirs = [ + {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"}, + {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"}, +] +ptyprocess = [] +pycodestyle = [] +pydocstyle = [] +pyflakes = [] +pylama = [] +pyrsistent = [] +pyyaml = [] +snowballstemmer = [] +toml = [] +tomli = [] +types-pyyaml = [] +typing-extensions = [] +warlock = [] diff --git a/dts/pyproject.toml b/dts/pyproject.toml new file mode 100644 index 0000000000..a136c91e5e --- /dev/null +++ b/dts/pyproject.toml @@ -0,0 +1,46 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2022 University of New Hampshire + +[tool.poetry] +name = "dts" +version = "0.1.0" +description = "" +authors = ["Owen Hilyard ", "dts@dpdk.org"] + +[tool.poetry.dependencies] +python = "^3.10" +pexpect = "^4.8.0" +warlock = "^2.0.1" +PyYAML = "^6.0" +types-PyYAML = "^6.0.8" + +[tool.poetry.dev-dependencies] +mypy = "^0.961" +black = "^22.6.0" +isort = "^5.10.1" +pylama = "^8.4.1" +pyflakes = "2.5.0" +toml = "^0.10.2" + +[tool.poetry.scripts] +dts = "main:main" + +[build-system] +requires = ["poetry-core>=1.0.0"] +build-backend = "poetry.core.masonry.api" + +[tool.pylama] +linters = "pep8,pylint,mccabe,mypy,pycodestyle,pyflakes" +format = "pylint" +max_line_length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length + +[tool.mypy] +python_version = "3.10" + +[tool.isort] +profile = "black" + +[tool.black] +target-version = ['py310'] +include = '\.pyi?$' +line-length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length -- 2.30.2