From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4704AA00C2; Thu, 3 Nov 2022 16:19:54 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7539D42D1C; Thu, 3 Nov 2022 16:19:42 +0100 (CET) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by mails.dpdk.org (Postfix) with ESMTP id 8884E42D12 for ; Thu, 3 Nov 2022 16:19:41 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id 34B621B68FA; Thu, 3 Nov 2022 16:19:40 +0100 (CET) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f2awPOl0idC9; Thu, 3 Nov 2022 16:19:38 +0100 (CET) Received: from entguard.lab.pantheon.local (unknown [46.229.239.141]) by lb.pantheon.sk (Postfix) with ESMTP id AA6221B68FE; Thu, 3 Nov 2022 16:19:36 +0100 (CET) From: =?UTF-8?q?Juraj=20Linke=C5=A1?= To: Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= Subject: [PATCH v7 2/9] dts: add developer tools Date: Thu, 3 Nov 2022 15:19:27 +0000 Message-Id: <20221103151934.450887-3-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221103151934.450887-1-juraj.linkes@pantheon.tech> References: <20221013103517.3443997-1-juraj.linkes@pantheon.tech> <20221103151934.450887-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The devtools that check Python code are Black and Isort to format the code and Pylama to do static analysis. Signed-off-by: Owen Hilyard Signed-off-by: Juraj Linkeš --- devtools/dts-check-format.sh | 87 ++++++++++++++++++++++++++++++++++++ doc/guides/tools/dts.rst | 67 +++++++++++++++++++++++++++ 2 files changed, 154 insertions(+) create mode 100755 devtools/dts-check-format.sh diff --git a/devtools/dts-check-format.sh b/devtools/dts-check-format.sh new file mode 100755 index 0000000000..5a1e495634 --- /dev/null +++ b/devtools/dts-check-format.sh @@ -0,0 +1,87 @@ +#!/bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2022 University of New Hampshire + +usage() { + echo "Run formatting and linting programs for DTS. Usage:" + + # Get source code comments after getopts arguments and print them both + grep -E '[a-zA-Z]+\) +#' "$0" | tr -d '#' + exit 0 +} + +format=true +lint=true + +# Comments after args serve as documentation; must be present +while getopts "hfl" arg; do + case $arg in + h) # Display this message + usage + ;; + f) # Don't run formatters + format=false + ;; + l) # Don't run linter + lint=false + ;; + *) + esac +done + + +errors=0 + +if $format; then + if command -v git > /dev/null; then + if git rev-parse --is-inside-work-tree >&-; then + echo "Formatting:" + if command -v black > /dev/null; then + echo "Formatting code with black:" + black . + else + echo "black is not installed, not formatting" + errors=$((errors + 1)) + fi + if command -v isort > /dev/null; then + echo "Sorting imports with isort:" + isort . + else + echo "isort is not installed, not sorting imports" + errors=$((errors + 1)) + fi + + git update-index --refresh + retval=$? + if [ $retval -ne 0 ]; then + echo 'The "needs update" files have been reformatted.' + echo 'Please update your commit.' + fi + errors=$((errors + retval)) + else + echo ".git directory not found, not formatting code" + errors=$((errors + 1)) + fi + else + echo "git command not found, not formatting code" + errors=$((errors + 1)) + fi +fi + +if $lint; then + if $format; then + echo + fi + echo "Linting:" + if command -v pylama > /dev/null; then + pylama . + errors=$((errors + $?)) + else + echo "pylama not found, unable to run linter" + errors=$((errors + 1)) + fi +fi + +echo +echo "Found $errors errors" +exit $errors diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 7913693a1c..0b9bcbdaac 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -8,6 +8,44 @@ The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suite implementing functional and performance tests used to test DPDK. +DTS Terminology +--------------- + +DTS node + A generic description of any host/server DTS connects to. + +DTS runtime environment + An environment containing Python with packages needed to run DTS. + +DTS runtime environment node + A node where at least one DTS runtime environment is present. + This is the node where we run DTS and from which DTS connects to other nodes. + +System under test + An SUT is the combination of DPDK and the hardware we're testing + in conjunction with DPDK (NICs, crypto and other devices). + +System under test node + A node where at least one SUT is present. + +Traffic generator + A TG is either software or hardware capable of sending packets. + +Traffic generator node + A node where at least one TG is present. + In case of hardware traffic generators, the TG and the node are literally the same. + + +In most cases, interchangeably referring to a runtime environment, SUT, TG or the node +they're running on (e.g. using SUT and SUT node interchangeably) doesn't cause confusion. +There could theoretically be more than of these running on the same node and in that case +it's useful to have stricter definitions. +An example would be two different traffic generators (such as Trex and Scapy) +running on the same node. +A different example would be a node containing both a DTS runtime environment +and a traffic generator, in which case it's both a DTS runtime environment node and a TG node. + + DTS Environment --------------- @@ -54,3 +92,32 @@ Setting up DTS environment poetry install poetry shell + + +DTS Developer Tools +------------------- + +There are three tools used in DTS to help with code checking, style and formatting: + +* `isort `_ + + Alphabetically sorts python imports within blocks. + +* `black `_ + + Does most of the actual formatting (whitespaces, comments, line length etc.) + and works similarly to clang-format. + +* `pylama `_ + + Runs a collection of python linters and aggregates output. + It will run these tools over the repository: + + .. literalinclude:: ../../../dts/pyproject.toml + :language: toml + :start-after: [tool.pylama] + :end-at: linters + +These three tools are all used in ``devtools/dts-check-format.sh``, +the DTS code check and format script. +Refer to the script for usage: ``devtools/dts-check-format.sh -h`` -- 2.30.2