From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C1F2459FA; Sat, 28 Sep 2024 00:16:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8655E40C35; Sat, 28 Sep 2024 00:13:56 +0200 (CEST) Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by mails.dpdk.org (Postfix) with ESMTP id F1240402CC for ; Fri, 27 Sep 2024 18:09:04 +0200 (CEST) Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-42cb5b3c57eso21226585e9.2 for ; Fri, 27 Sep 2024 09:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1727453344; x=1728058144; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2BvLkRxVATmXkcdwftDs5jSkNFg3esjxGLEP04+pjaI=; b=Vp7AG+qtq7FR/6TibFW6nohIUg7/58z0dFWVEiqbm+pLwKS+KYA25r4D2Zxo08Pai3 oLu+5CsLYHLPI5LJs6FTYdyvcpTkWOfBqZ2pj6hix4H2ewHmEZfmU8Ml8XWkyp4k9HN3 hMtbExZTrxK+WDuzM6beifqubhhxMRxywPXhmA24K8uAWq/4dHsLQGslpUMUXMloa4RT BGJj06wLZVR1sHKz7xk75Ugufh3yg/2Cy3K7yIXy0nsubzoh/vu/Slv5YOypmA0QuKyi cpMyjflQtKyzsVP2VK2p4DFrQC/U4cdzTyukxHygdgRIXd66Gg+PUDGxRXNMTb11HTvv zdKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727453344; x=1728058144; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2BvLkRxVATmXkcdwftDs5jSkNFg3esjxGLEP04+pjaI=; b=g3e6TuReLj4ZoACKstD4kaehfOXNVWabD8x1CZ0Y215emh4ivt98LsWI+6vPmc2OEa c/wAD6pBN6yD0Xf4oBTvB/QuzMLpf8ueaVgz3tvAz+PgFxOyv3M8qcRAY7DL+UMk6Je/ BaeIXxqA+urSxdim8ih3v/gkzys55SD+TiJr5iPy3luFfcydRwJhIbpwMYjpjfgwD01w vqVSIuBLOc7CsnHRv7WQ+ys9ajzTsxskwTv+xo4UGBOk9RsN+qp6YXjwRKHZUBKKrXZ2 Yzrtdvd9E2WafLqhG/3txzX++sqH6FXbUBckvombW2qnc2Jhaa+ROhZWK4YEn2AaG6h6 93zw== X-Gm-Message-State: AOJu0Yyr9kqkvkH9D+R/faSdJ2eLJMTatRB/y/5Lcx8TkFGDXXIpLZkr fiEgAd2pmiBhKR1rs9q4Wfrp3kgqq3bxbtUvWIpqTFP5TeGRPDFE1ccL+y/Ru4jP2xA9qEofmmZ 8hV0= X-Google-Smtp-Source: AGHT+IGGGTnc174JrEc3YIYap01z6BcclOgWLAa1DNp/mgdfnstg+wIyJ3xC8CebIeJEEa05IfxXPg== X-Received: by 2002:a5d:54c2:0:b0:377:9867:9028 with SMTP id ffacd0b85a97d-37cd5ad1321mr2541804f8f.44.1727453344207; Fri, 27 Sep 2024 09:09:04 -0700 (PDT) Received: from fedora.. ([185.242.191.25]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37cd564bf8bsm2943471f8f.20.2024.09.27.09.09.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Sep 2024 09:09:03 -0700 (PDT) From: =?UTF-8?q?Tom=C3=A1=C5=A1=20=C4=8Eurovec?= To: dev@dpdk.org, Luca.Vizzarro@arm.com, probb@iol.unh.edu, npratte@iol.unh.edu, dmarx@iol.unh.edu Cc: =?UTF-8?q?Tom=C3=A1=C5=A1=20=C4=8Eurovec?= Subject: [PATCH 7/7] dts: remove git ref option Date: Fri, 27 Sep 2024 18:08:54 +0200 Message-ID: <20240927160854.279253-8-tomas.durovec@pantheon.tech> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240927160854.279253-1-tomas.durovec@pantheon.tech> References: <20240927160854.279253-1-tomas.durovec@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Sat, 28 Sep 2024 00:13:29 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In the previous commits we're adding the support for copying the whole local DPDK tree directory and git-ref option was meant to do the same thing. Signed-off-by: Tomáš Ďurovec --- doc/guides/tools/dts.rst | 9 --- dts/framework/settings.py | 51 +++------------- dts/framework/utils.py | 119 +------------------------------------- 3 files changed, 8 insertions(+), 171 deletions(-) diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 20d4d18b18..d806dce2ae 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -239,9 +239,6 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet --dpdk-tree DIR_PATH [DTS_DPDK_TREE] Path to DPDK source code tree to test. (default: None) --tarball FILE_PATH, --snapshot FILE_PATH [DTS_DPDK_TARBALL] Path to DPDK source code tarball to test. (default: None) - --revision ID, --rev ID, --git-ref ID - [DTS_DPDK_REVISION_ID] Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, first - commit them, then use their commit ID. (default: None) --remote-source [DTS_REMOTE_SOURCE] Set when the DPDK source tree or tarball is located on the SUT node. (default: False) --build-dir DIR_NAME [DTS_BUILD_DIR] A directory name, which would be located in the `dpdk tree` or `tarball`. (default: None) --compile-timeout SECONDS @@ -263,12 +260,6 @@ The minimum DTS needs is a config file and a pre-built DPDK or DPDK sources location which can be specified in said config file or on the command line or environment variables. -Example command for running DTS with the template configuration and DPDK tag v23.11: - -.. code-block:: console - - (dts-py3.10) $ ./main.py --git-ref v23.11 - DTS Results ~~~~~~~~~~~ diff --git a/dts/framework/settings.py b/dts/framework/settings.py index 17594ecb15..a4ab674189 100644 --- a/dts/framework/settings.py +++ b/dts/framework/settings.py @@ -42,20 +42,12 @@ .. option:: --dpdk-tree .. envvar:: DTS_DPDK_TREE - The path to DPDK source tree directory to test. Only this or tarball or revision can be - provided. + The path to DPDK source tree directory to test. Only this or tarball can be provided. .. option:: --tarball, --snapshot .. envvar:: DTS_DPDK_TARBALL - The path to DPDK source tarball to test. Only this or DPDK tree or revision can be provided. - -.. option:: --revision, --rev, --git-ref -.. envvar:: DTS_DPDK_REVISION_ID - - Git revision ID to test. Could be commit, tag, tree ID etc. - To test local changes, first commit them, then use their commit ID. - Only this or DPDK tree or tarball can be provided. + The path to DPDK source tarball to test. Only this or DPDK tree can be provided. .. option:: --remote-source .. envvar:: DTS_REMOTE_SOURCE @@ -108,8 +100,6 @@ from typing import Callable from .config import DPDKLocation, TestSuiteConfig -from .exception import ConfigurationError -from .utils import DPDKGitTarball, get_commit_id @dataclass(slots=True) @@ -256,14 +246,6 @@ def _get_help_string(self, action): return help -def _parse_revision_id(rev_id: str) -> str: - """Validate revision ID and retrieve corresponding commit ID.""" - try: - return get_commit_id(rev_id) - except ConfigurationError: - raise argparse.ArgumentTypeError("The Git revision ID supplied is invalid or ambiguous") - - def _required_with_one_of(parser: _DTSArgumentParser, action: Action, *required_dests: str) -> None: """Verify that `action` is listed together with at least one of `required_dests`. @@ -362,15 +344,15 @@ def _get_parser() -> _DTSArgumentParser: dpdk_build = parser.add_argument_group( "DPDK Build Options", description="Arguments in this group (and subgroup) will be applied to a " - ":class:`DPDKLocation` when the DPDK tree, tarball or revision will be provided, " - "other arguments like remote source and build dir are optional. A :class:`DPDKLocation` " + ":class:`DPDKLocation` when the DPDK tree, tarball will be provided, other " + "arguments like remote source and build dir are optional. A :class:`DPDKLocation` " "from settings are used instead of from config if construct successful.", ) dpdk_source = dpdk_build.add_mutually_exclusive_group() action = dpdk_source.add_argument( "--dpdk-tree", - help="The path to DPDK source tree directory to test. Only this or tarball or revision " + help="The path to DPDK source tree directory to test. Only this or tarball " "can be provided.", metavar="DIR_PATH", dest="dpdk_tree_path", @@ -380,26 +362,12 @@ def _get_parser() -> _DTSArgumentParser: action = dpdk_source.add_argument( "--tarball", "--snapshot", - help="The path to DPDK source tarball to test. Only this or DPDK tree or revision " - "can be provided.", + help="The path to DPDK source tarball to test. Only this or DPDK tree " "can be provided.", metavar="FILE_PATH", dest="dpdk_tarball_path", ) _add_env_var_to_action(action, "DPDK_TARBALL") - action = dpdk_source.add_argument( - "--revision", - "--rev", - "--git-ref", - type=_parse_revision_id, - help="Git revision ID to test. Could be commit, tag, tree ID etc. " - "To test local changes, first commit them, then use their commit ID." - "Only this or DPDK tree or tarball can be provided.", - metavar="ID", - dest="dpdk_revision_id", - ) - _add_env_var_to_action(action) - action = dpdk_build.add_argument( "--remote-source", action="store_true", @@ -408,9 +376,7 @@ def _get_parser() -> _DTSArgumentParser: "instead of the execution host. This can be provided only with DPDK tree or tarball.", ) _add_env_var_to_action(action) - _required_with_one_of( - parser, action, "dpdk_tarball_path", "dpdk_tree_path" - ) # ignored if passed with git-ref + _required_with_one_of(parser, action, "dpdk_tarball_path", "dpdk_tree_path") action = dpdk_build.add_argument( "--build-dir", @@ -567,9 +533,6 @@ def get_settings() -> Settings: args = parser.parse_args() - if args.dpdk_revision_id: - args.dpdk_tarball_path = Path(DPDKGitTarball(args.dpdk_revision_id, args.output_dir)) - args.dpdk_location = _process_dpdk_location( args.dpdk_tree_path, args.dpdk_tarball_path, args.remote_source, args.build_dir ) diff --git a/dts/framework/utils.py b/dts/framework/utils.py index 382357ffe8..4b8843bf20 100644 --- a/dts/framework/utils.py +++ b/dts/framework/utils.py @@ -14,22 +14,19 @@ REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. ``0000:00:08.0``. """ -import atexit import fnmatch import json import os import random -import subprocess import tarfile from enum import Enum, Flag from pathlib import Path -from subprocess import SubprocessError from typing import Any, Callable from scapy.layers.inet import IP, TCP, UDP, Ether # type: ignore[import-untyped] from scapy.packet import Packet # type: ignore[import-untyped] -from .exception import ConfigurationError, InternalError +from .exception import InternalError REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/" @@ -76,31 +73,6 @@ def get_packet_summaries(packets: list[Packet]) -> str: return f"Packet contents: \n{packet_summaries}" -def get_commit_id(rev_id: str) -> str: - """Given a Git revision ID, return the corresponding commit ID. - - Args: - rev_id: The Git revision ID. - - Raises: - ConfigurationError: The ``git rev-parse`` command failed, suggesting - an invalid or ambiguous revision ID was supplied. - """ - result = subprocess.run( - ["git", "rev-parse", "--verify", rev_id], - text=True, - capture_output=True, - ) - if result.returncode != 0: - raise ConfigurationError( - f"{rev_id} is not a valid git reference.\n" - f"Command: {result.args}\n" - f"Stdout: {result.stdout}\n" - f"Stderr: {result.stderr}" - ) - return result.stdout.strip() - - class StrEnum(Enum): """Enum with members stored as strings.""" @@ -176,95 +148,6 @@ def extension(self): return f"{self.value}" if self == self.none else f"{self.none.value}.{self.value}" -class DPDKGitTarball: - """Compressed tarball of DPDK from the repository. - - The class supports the :class:`os.PathLike` protocol, - which is used to get the Path of the tarball:: - - from pathlib import Path - tarball = DPDKGitTarball("HEAD", "output") - tarball_path = Path(tarball) - """ - - _git_ref: str - _tar_compression_format: TarCompressionFormat - _tarball_dir: Path - _tarball_name: str - _tarball_path: Path | None - - def __init__( - self, - git_ref: str, - output_dir: str, - tar_compression_format: TarCompressionFormat = TarCompressionFormat.xz, - ): - """Create the tarball during initialization. - - The DPDK version is specified with `git_ref`. The tarball will be compressed with - `tar_compression_format`, which must be supported by the DTS execution environment. - The resulting tarball will be put into `output_dir`. - - Args: - git_ref: A git commit ID, tag ID or tree ID. - output_dir: The directory where to put the resulting tarball. - tar_compression_format: The compression format to use. - """ - self._git_ref = git_ref - self._tar_compression_format = tar_compression_format - - self._tarball_dir = Path(output_dir, "tarball") - - self._create_tarball_dir() - - self._tarball_name = ( - f"dpdk-tarball-{self._git_ref}.{self._tar_compression_format.extension}" - ) - self._tarball_path = self._check_tarball_path() - if not self._tarball_path: - self._create_tarball() - - def _create_tarball_dir(self) -> None: - os.makedirs(self._tarball_dir, exist_ok=True) - - def _check_tarball_path(self) -> Path | None: - if self._tarball_name in os.listdir(self._tarball_dir): - return Path(self._tarball_dir, self._tarball_name) - return None - - def _create_tarball(self) -> None: - self._tarball_path = Path(self._tarball_dir, self._tarball_name) - - atexit.register(self._delete_tarball) - - result = subprocess.run( - 'git -C "$(git rev-parse --show-toplevel)" archive ' - f'{self._git_ref} --prefix="dpdk-tarball-{self._git_ref + os.sep}" | ' - f"{self._tar_compression_format} > {Path(self._tarball_path.absolute())}", - shell=True, - text=True, - capture_output=True, - ) - - if result.returncode != 0: - raise SubprocessError( - f"Git archive creation failed with exit code {result.returncode}.\n" - f"Command: {result.args}\n" - f"Stdout: {result.stdout}\n" - f"Stderr: {result.stderr}" - ) - - atexit.unregister(self._delete_tarball) - - def _delete_tarball(self) -> None: - if self._tarball_path and os.path.exists(self._tarball_path): - os.remove(self._tarball_path) - - def __fspath__(self) -> str: - """The os.PathLike protocol implementation.""" - return str(self._tarball_path) - - def convert_to_list_of_string(value: Any | list[Any]) -> list[str]: """Convert the input to the list of strings.""" return list(map(str, value) if isinstance(value, list) else str(value)) -- 2.46.1