From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10F0C45A6E; Mon, 30 Sep 2024 18:19:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 567B540684; Mon, 30 Sep 2024 18:18:34 +0200 (CEST) Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by mails.dpdk.org (Postfix) with ESMTP id 090974065C for ; Mon, 30 Sep 2024 18:18:27 +0200 (CEST) Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a8d2b4a5bf1so629749766b.2 for ; Mon, 30 Sep 2024 09:18:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1727713106; x=1728317906; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2BvLkRxVATmXkcdwftDs5jSkNFg3esjxGLEP04+pjaI=; b=R50LCFirKzqZg9JKextwCBkL58oZ7m/BMfYo6l1FpdiGzrFEo33Yvyk3Mqyc62qj46 0YqaWhPrJOkzKWiojp6jGpm8SUBwCYne0X3yteWS4AGA5NoX63n0lIQqAIV0R2xuOPuw aRbr5eHakKjhpE7DucxC3yabSwyIMXWI7ZMn2X89iK9KSTyg7IjZC2KTR7ahEldMOJr0 ahZjCOiD+Gtphea0iM6XbPlfKr6zuCeATKw8FtfIbOJVUy8wMuJbX9m+UQw0VWYbsXOK 5L9CPMS+C/kUh5AFwHFC23IAURoaa41Xq1xdtoHrXML4gkcU3Xk1vRSGNYng4etN7pvf DrEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727713106; x=1728317906; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2BvLkRxVATmXkcdwftDs5jSkNFg3esjxGLEP04+pjaI=; b=SZF3rTeMC2NrvE7fwCzGPKdmOkfFnnWHCbtJfFj0bomZNoRsAT7cEipsPy5A0D3uj3 /pWryCXGgdqLNfVXSkx2X+Dl/ah+8N2Ccb8//MBwuG5SHkjNUgf0LgKzOgAmTJeDeQyM ihwfLatqm+aIlryF7Nw5Q32IT7HH6vxFP5j2IE/p9ZG6yS0gCsSctdzglxmcOrsKMRfA WWa8U/TNg9gpTsfT92ue9FY60/CDzlXUiHjvXxWjGp5BK8uizzPa88lsMLm6DznDmqP4 FHJKw9YDmqwBiC1hk+G49YcKDQ7V7vnzq7JQq2GgYM3MgEO1v02iYWS1igxmVkYco32s DVmQ== X-Gm-Message-State: AOJu0YzpP3xRI1nv6qbho1cbf7bZoXvvRudWZ6AtoEeFtYqLjkmCMuUR 3zIwfb0ZspU6Lcc4FJG9/Sy747hntlyH1upk6Lu+bdYLVhnwfaavfdDGxDRYVpJFLJzIsS56ly0 Us1Y= X-Google-Smtp-Source: AGHT+IHJ8kaXZugLOGK1yosV83D+3HMZx3reTUnDyzBB9udy4D/y7FvJXwM8UVIXr7eiNeBrUjT/WQ== X-Received: by 2002:a17:907:3e1c:b0:a88:b93b:cdcb with SMTP id a640c23a62f3a-a93c4a68305mr1151339966b.47.1727713106112; Mon, 30 Sep 2024 09:18:26 -0700 (PDT) Received: from fedora.. (ip-46.34.236.94.o2inet.sk. [46.34.236.94]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c2775d0bsm564923666b.11.2024.09.30.09.18.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 09:18:25 -0700 (PDT) From: =?UTF-8?q?Tom=C3=A1=C5=A1=20=C4=8Eurovec?= To: dev@dpdk.org Cc: =?UTF-8?q?Tom=C3=A1=C5=A1=20=C4=8Eurovec?= Subject: [PATCH 7/7] dts: remove git ref option Date: Mon, 30 Sep 2024 18:18:14 +0200 Message-ID: <20240930161814.26070-8-tomas.durovec@pantheon.tech> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930161814.26070-1-tomas.durovec@pantheon.tech> References: <20240930161814.26070-1-tomas.durovec@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In the previous commits we're adding the support for copying the whole local DPDK tree directory and git-ref option was meant to do the same thing. Signed-off-by: Tomáš Ďurovec --- doc/guides/tools/dts.rst | 9 --- dts/framework/settings.py | 51 +++------------- dts/framework/utils.py | 119 +------------------------------------- 3 files changed, 8 insertions(+), 171 deletions(-) diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst index 20d4d18b18..d806dce2ae 100644 --- a/doc/guides/tools/dts.rst +++ b/doc/guides/tools/dts.rst @@ -239,9 +239,6 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet --dpdk-tree DIR_PATH [DTS_DPDK_TREE] Path to DPDK source code tree to test. (default: None) --tarball FILE_PATH, --snapshot FILE_PATH [DTS_DPDK_TARBALL] Path to DPDK source code tarball to test. (default: None) - --revision ID, --rev ID, --git-ref ID - [DTS_DPDK_REVISION_ID] Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, first - commit them, then use their commit ID. (default: None) --remote-source [DTS_REMOTE_SOURCE] Set when the DPDK source tree or tarball is located on the SUT node. (default: False) --build-dir DIR_NAME [DTS_BUILD_DIR] A directory name, which would be located in the `dpdk tree` or `tarball`. (default: None) --compile-timeout SECONDS @@ -263,12 +260,6 @@ The minimum DTS needs is a config file and a pre-built DPDK or DPDK sources location which can be specified in said config file or on the command line or environment variables. -Example command for running DTS with the template configuration and DPDK tag v23.11: - -.. code-block:: console - - (dts-py3.10) $ ./main.py --git-ref v23.11 - DTS Results ~~~~~~~~~~~ diff --git a/dts/framework/settings.py b/dts/framework/settings.py index 17594ecb15..a4ab674189 100644 --- a/dts/framework/settings.py +++ b/dts/framework/settings.py @@ -42,20 +42,12 @@ .. option:: --dpdk-tree .. envvar:: DTS_DPDK_TREE - The path to DPDK source tree directory to test. Only this or tarball or revision can be - provided. + The path to DPDK source tree directory to test. Only this or tarball can be provided. .. option:: --tarball, --snapshot .. envvar:: DTS_DPDK_TARBALL - The path to DPDK source tarball to test. Only this or DPDK tree or revision can be provided. - -.. option:: --revision, --rev, --git-ref -.. envvar:: DTS_DPDK_REVISION_ID - - Git revision ID to test. Could be commit, tag, tree ID etc. - To test local changes, first commit them, then use their commit ID. - Only this or DPDK tree or tarball can be provided. + The path to DPDK source tarball to test. Only this or DPDK tree can be provided. .. option:: --remote-source .. envvar:: DTS_REMOTE_SOURCE @@ -108,8 +100,6 @@ from typing import Callable from .config import DPDKLocation, TestSuiteConfig -from .exception import ConfigurationError -from .utils import DPDKGitTarball, get_commit_id @dataclass(slots=True) @@ -256,14 +246,6 @@ def _get_help_string(self, action): return help -def _parse_revision_id(rev_id: str) -> str: - """Validate revision ID and retrieve corresponding commit ID.""" - try: - return get_commit_id(rev_id) - except ConfigurationError: - raise argparse.ArgumentTypeError("The Git revision ID supplied is invalid or ambiguous") - - def _required_with_one_of(parser: _DTSArgumentParser, action: Action, *required_dests: str) -> None: """Verify that `action` is listed together with at least one of `required_dests`. @@ -362,15 +344,15 @@ def _get_parser() -> _DTSArgumentParser: dpdk_build = parser.add_argument_group( "DPDK Build Options", description="Arguments in this group (and subgroup) will be applied to a " - ":class:`DPDKLocation` when the DPDK tree, tarball or revision will be provided, " - "other arguments like remote source and build dir are optional. A :class:`DPDKLocation` " + ":class:`DPDKLocation` when the DPDK tree, tarball will be provided, other " + "arguments like remote source and build dir are optional. A :class:`DPDKLocation` " "from settings are used instead of from config if construct successful.", ) dpdk_source = dpdk_build.add_mutually_exclusive_group() action = dpdk_source.add_argument( "--dpdk-tree", - help="The path to DPDK source tree directory to test. Only this or tarball or revision " + help="The path to DPDK source tree directory to test. Only this or tarball " "can be provided.", metavar="DIR_PATH", dest="dpdk_tree_path", @@ -380,26 +362,12 @@ def _get_parser() -> _DTSArgumentParser: action = dpdk_source.add_argument( "--tarball", "--snapshot", - help="The path to DPDK source tarball to test. Only this or DPDK tree or revision " - "can be provided.", + help="The path to DPDK source tarball to test. Only this or DPDK tree " "can be provided.", metavar="FILE_PATH", dest="dpdk_tarball_path", ) _add_env_var_to_action(action, "DPDK_TARBALL") - action = dpdk_source.add_argument( - "--revision", - "--rev", - "--git-ref", - type=_parse_revision_id, - help="Git revision ID to test. Could be commit, tag, tree ID etc. " - "To test local changes, first commit them, then use their commit ID." - "Only this or DPDK tree or tarball can be provided.", - metavar="ID", - dest="dpdk_revision_id", - ) - _add_env_var_to_action(action) - action = dpdk_build.add_argument( "--remote-source", action="store_true", @@ -408,9 +376,7 @@ def _get_parser() -> _DTSArgumentParser: "instead of the execution host. This can be provided only with DPDK tree or tarball.", ) _add_env_var_to_action(action) - _required_with_one_of( - parser, action, "dpdk_tarball_path", "dpdk_tree_path" - ) # ignored if passed with git-ref + _required_with_one_of(parser, action, "dpdk_tarball_path", "dpdk_tree_path") action = dpdk_build.add_argument( "--build-dir", @@ -567,9 +533,6 @@ def get_settings() -> Settings: args = parser.parse_args() - if args.dpdk_revision_id: - args.dpdk_tarball_path = Path(DPDKGitTarball(args.dpdk_revision_id, args.output_dir)) - args.dpdk_location = _process_dpdk_location( args.dpdk_tree_path, args.dpdk_tarball_path, args.remote_source, args.build_dir ) diff --git a/dts/framework/utils.py b/dts/framework/utils.py index 382357ffe8..4b8843bf20 100644 --- a/dts/framework/utils.py +++ b/dts/framework/utils.py @@ -14,22 +14,19 @@ REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. ``0000:00:08.0``. """ -import atexit import fnmatch import json import os import random -import subprocess import tarfile from enum import Enum, Flag from pathlib import Path -from subprocess import SubprocessError from typing import Any, Callable from scapy.layers.inet import IP, TCP, UDP, Ether # type: ignore[import-untyped] from scapy.packet import Packet # type: ignore[import-untyped] -from .exception import ConfigurationError, InternalError +from .exception import InternalError REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/" @@ -76,31 +73,6 @@ def get_packet_summaries(packets: list[Packet]) -> str: return f"Packet contents: \n{packet_summaries}" -def get_commit_id(rev_id: str) -> str: - """Given a Git revision ID, return the corresponding commit ID. - - Args: - rev_id: The Git revision ID. - - Raises: - ConfigurationError: The ``git rev-parse`` command failed, suggesting - an invalid or ambiguous revision ID was supplied. - """ - result = subprocess.run( - ["git", "rev-parse", "--verify", rev_id], - text=True, - capture_output=True, - ) - if result.returncode != 0: - raise ConfigurationError( - f"{rev_id} is not a valid git reference.\n" - f"Command: {result.args}\n" - f"Stdout: {result.stdout}\n" - f"Stderr: {result.stderr}" - ) - return result.stdout.strip() - - class StrEnum(Enum): """Enum with members stored as strings.""" @@ -176,95 +148,6 @@ def extension(self): return f"{self.value}" if self == self.none else f"{self.none.value}.{self.value}" -class DPDKGitTarball: - """Compressed tarball of DPDK from the repository. - - The class supports the :class:`os.PathLike` protocol, - which is used to get the Path of the tarball:: - - from pathlib import Path - tarball = DPDKGitTarball("HEAD", "output") - tarball_path = Path(tarball) - """ - - _git_ref: str - _tar_compression_format: TarCompressionFormat - _tarball_dir: Path - _tarball_name: str - _tarball_path: Path | None - - def __init__( - self, - git_ref: str, - output_dir: str, - tar_compression_format: TarCompressionFormat = TarCompressionFormat.xz, - ): - """Create the tarball during initialization. - - The DPDK version is specified with `git_ref`. The tarball will be compressed with - `tar_compression_format`, which must be supported by the DTS execution environment. - The resulting tarball will be put into `output_dir`. - - Args: - git_ref: A git commit ID, tag ID or tree ID. - output_dir: The directory where to put the resulting tarball. - tar_compression_format: The compression format to use. - """ - self._git_ref = git_ref - self._tar_compression_format = tar_compression_format - - self._tarball_dir = Path(output_dir, "tarball") - - self._create_tarball_dir() - - self._tarball_name = ( - f"dpdk-tarball-{self._git_ref}.{self._tar_compression_format.extension}" - ) - self._tarball_path = self._check_tarball_path() - if not self._tarball_path: - self._create_tarball() - - def _create_tarball_dir(self) -> None: - os.makedirs(self._tarball_dir, exist_ok=True) - - def _check_tarball_path(self) -> Path | None: - if self._tarball_name in os.listdir(self._tarball_dir): - return Path(self._tarball_dir, self._tarball_name) - return None - - def _create_tarball(self) -> None: - self._tarball_path = Path(self._tarball_dir, self._tarball_name) - - atexit.register(self._delete_tarball) - - result = subprocess.run( - 'git -C "$(git rev-parse --show-toplevel)" archive ' - f'{self._git_ref} --prefix="dpdk-tarball-{self._git_ref + os.sep}" | ' - f"{self._tar_compression_format} > {Path(self._tarball_path.absolute())}", - shell=True, - text=True, - capture_output=True, - ) - - if result.returncode != 0: - raise SubprocessError( - f"Git archive creation failed with exit code {result.returncode}.\n" - f"Command: {result.args}\n" - f"Stdout: {result.stdout}\n" - f"Stderr: {result.stderr}" - ) - - atexit.unregister(self._delete_tarball) - - def _delete_tarball(self) -> None: - if self._tarball_path and os.path.exists(self._tarball_path): - os.remove(self._tarball_path) - - def __fspath__(self) -> str: - """The os.PathLike protocol implementation.""" - return str(self._tarball_path) - - def convert_to_list_of_string(value: Any | list[Any]) -> list[str]: """Convert the input to the list of strings.""" return list(map(str, value) if isinstance(value, list) else str(value)) -- 2.46.1