DPDK patches and discussions
 help / color / mirror / Atom feed
From: Luca Vizzarro <luca.vizzarro@arm.com>
To: dev@dpdk.org
Cc: "Paul Szczepanek" <paul.szczepanek@arm.com>,
	"Patrick Robb" <probb@iol.unh.edu>,
	"Tomáš Ďurovec" <tomas.durovec@pantheon.tech>,
	"Luca Vizzarro" <luca.vizzarro@arm.com>
Subject: [PATCH v2 7/7] dts: remove git ref option
Date: Mon, 21 Oct 2024 14:49:35 +0100	[thread overview]
Message-ID: <20241021134935.1210500-8-luca.vizzarro@arm.com> (raw)
In-Reply-To: <20241021134935.1210500-1-luca.vizzarro@arm.com>

From: Tomáš Ďurovec <tomas.durovec@pantheon.tech>

Given the whole DPDK tree directory can now be copied to the nodes,
there is no more need to use the git ref option, as the tree can be
controlled directly by the user.

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
---
 doc/guides/tools/dts.rst  |   9 ---
 dts/framework/settings.py |  48 ++-------------
 dts/framework/utils.py    | 119 +-------------------------------------
 3 files changed, 7 insertions(+), 169 deletions(-)

diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 7b90c4856e..a00d987ece 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -237,9 +237,6 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet
    -t SECONDS, --timeout SECONDS
                            [DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK. (default: 15)
    -v, --verbose           [DTS_VERBOSE] Specify to enable verbose output, logging all messages to the console. (default: False)
-   --revision ID, --rev ID, --git-ref ID
-                           [DTS_DPDK_REVISION_ID] Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, first
-                           commit them, then use their commit ID. (default: None)
    --compile-timeout SECONDS
                            [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200)
    --test-suite TEST_SUITE [TEST_CASES ...]
@@ -275,12 +272,6 @@ The minimum DTS needs is a config file and a pre-built DPDK or DPDK
 sources location which can be specified in said config file or on the
 command line or environment variables.
 
-Example command for running DTS with the template configuration and DPDK tag v23.11:
-
-.. code-block:: console
-
-   (dts-py3.10) $ ./main.py --git-ref v23.11
-
 
 DTS Results
 ~~~~~~~~~~~
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 08529805da..a452319b90 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -42,21 +42,14 @@
 .. option:: --dpdk-tree
 .. envvar:: DTS_DPDK_TREE
 
-    The path to the DPDK source tree directory to test. Cannot be used in conjunction with --tarball
-    and --revision.
+    The path to the DPDK source tree directory to test. Cannot be used in conjunction with
+    --tarball.
 
 .. option:: --tarball, --snapshot
 .. envvar:: DTS_DPDK_TARBALL
 
     The path to the DPDK source tarball to test. DPDK must be contained in a folder with the same
-    name as the tarball file. Cannot be used in conjunction with --dpdk-tree and
-    --revision.
-
-.. option:: --revision, --rev, --git-ref
-.. envvar:: DTS_DPDK_REVISION_ID
-
-    Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, first commit
-    them, then use their commit ID. Cannot be used in conjunction with --dpdk-tree and --tarball.
+    name as the tarball file. Cannot be used in conjunction with --dpdk-tree.
 
 .. option:: --remote-source
 .. envvar:: DTS_REMOTE_SOURCE
@@ -109,8 +102,6 @@
 from typing import Callable
 
 from .config import DPDKLocation, TestSuiteConfig
-from .exception import ConfigurationError
-from .utils import DPDKGitTarball, get_commit_id
 
 
 @dataclass(slots=True)
@@ -257,14 +248,6 @@ def _get_help_string(self, action):
         return help
 
 
-def _parse_revision_id(rev_id: str) -> str:
-    """Validate revision ID and retrieve corresponding commit ID."""
-    try:
-        return get_commit_id(rev_id)
-    except ConfigurationError:
-        raise argparse.ArgumentTypeError("The Git revision ID supplied is invalid or ambiguous")
-
-
 def _required_with_one_of(parser: _DTSArgumentParser, action: Action, *required_dests: str) -> None:
     """Verify that `action` is listed together with at least one of `required_dests`.
 
@@ -372,7 +355,7 @@ def _get_parser() -> _DTSArgumentParser:
     action = dpdk_source.add_argument(
         "--dpdk-tree",
         help="The path to the DPDK source tree directory to test. Cannot be used in conjunction "
-        "with --tarball and --revision.",
+        "with --tarball.",
         metavar="DIR_PATH",
         dest="dpdk_tree_path",
     )
@@ -382,26 +365,12 @@ def _get_parser() -> _DTSArgumentParser:
         "--tarball",
         "--snapshot",
         help="The path to the DPDK source tarball to test. DPDK must be contained in a folder with "
-        "the same name as the tarball file. Cannot be used in conjunction with --dpdk-tree and "
-        "--revision.",
+        "the same name as the tarball file. Cannot be used in conjunction with --dpdk-tree.",
         metavar="FILE_PATH",
         dest="dpdk_tarball_path",
     )
     _add_env_var_to_action(action, "DPDK_TARBALL")
 
-    action = dpdk_source.add_argument(
-        "--revision",
-        "--rev",
-        "--git-ref",
-        type=_parse_revision_id,
-        help="Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, "
-        "first commit them, then use their commit ID. Cannot be used in conjunction with "
-        "--dpdk-tree and --tarball.",
-        metavar="ID",
-        dest="dpdk_revision_id",
-    )
-    _add_env_var_to_action(action)
-
     action = dpdk_build.add_argument(
         "--remote-source",
         action="store_true",
@@ -410,9 +379,7 @@ def _get_parser() -> _DTSArgumentParser:
         "the SUT node. Can only be used with --dpdk-tree or --tarball.",
     )
     _add_env_var_to_action(action)
-    _required_with_one_of(
-        parser, action, "dpdk_tarball_path", "dpdk_tree_path"
-    )  # ignored if passed with git-ref
+    _required_with_one_of(parser, action, "dpdk_tarball_path", "dpdk_tree_path")
 
     action = dpdk_build.add_argument(
         "--precompiled-build-dir",
@@ -568,9 +535,6 @@ def get_settings() -> Settings:
 
     args = parser.parse_args()
 
-    if args.dpdk_revision_id:
-        args.dpdk_tarball_path = Path(DPDKGitTarball(args.dpdk_revision_id, args.output_dir))
-
     args.dpdk_location = _process_dpdk_location(
         args.dpdk_tree_path, args.dpdk_tarball_path, args.remote_source, args.precompiled_build_dir
     )
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 04b5813613..78a39e32c7 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -14,22 +14,19 @@
     REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. ``0000:00:08.0``.
 """
 
-import atexit
 import fnmatch
 import json
 import os
 import random
-import subprocess
 import tarfile
 from enum import Enum, Flag
 from pathlib import Path
-from subprocess import SubprocessError
 from typing import Any, Callable
 
 from scapy.layers.inet import IP, TCP, UDP, Ether  # type: ignore[import-untyped]
 from scapy.packet import Packet  # type: ignore[import-untyped]
 
-from .exception import ConfigurationError, InternalError
+from .exception import InternalError
 
 REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/"
 _REGEX_FOR_COLON_OR_HYPHEN_SEP_MAC: str = r"(?:[\da-fA-F]{2}[:-]){5}[\da-fA-F]{2}"
@@ -80,31 +77,6 @@ def get_packet_summaries(packets: list[Packet]) -> str:
     return f"Packet contents: \n{packet_summaries}"
 
 
-def get_commit_id(rev_id: str) -> str:
-    """Given a Git revision ID, return the corresponding commit ID.
-
-    Args:
-        rev_id: The Git revision ID.
-
-    Raises:
-        ConfigurationError: The ``git rev-parse`` command failed, suggesting
-            an invalid or ambiguous revision ID was supplied.
-    """
-    result = subprocess.run(
-        ["git", "rev-parse", "--verify", rev_id],
-        text=True,
-        capture_output=True,
-    )
-    if result.returncode != 0:
-        raise ConfigurationError(
-            f"{rev_id} is not a valid git reference.\n"
-            f"Command: {result.args}\n"
-            f"Stdout: {result.stdout}\n"
-            f"Stderr: {result.stderr}"
-        )
-    return result.stdout.strip()
-
-
 class StrEnum(Enum):
     """Enum with members stored as strings."""
 
@@ -180,95 +152,6 @@ def extension(self):
         return f"{self.value}" if self == self.none else f"{self.none.value}.{self.value}"
 
 
-class DPDKGitTarball:
-    """Compressed tarball of DPDK from the repository.
-
-    The class supports the :class:`os.PathLike` protocol,
-    which is used to get the Path of the tarball::
-
-        from pathlib import Path
-        tarball = DPDKGitTarball("HEAD", "output")
-        tarball_path = Path(tarball)
-    """
-
-    _git_ref: str
-    _tar_compression_format: TarCompressionFormat
-    _tarball_dir: Path
-    _tarball_name: str
-    _tarball_path: Path | None
-
-    def __init__(
-        self,
-        git_ref: str,
-        output_dir: str,
-        tar_compression_format: TarCompressionFormat = TarCompressionFormat.xz,
-    ):
-        """Create the tarball during initialization.
-
-        The DPDK version is specified with `git_ref`. The tarball will be compressed with
-        `tar_compression_format`, which must be supported by the DTS execution environment.
-        The resulting tarball will be put into `output_dir`.
-
-        Args:
-            git_ref: A git commit ID, tag ID or tree ID.
-            output_dir: The directory where to put the resulting tarball.
-            tar_compression_format: The compression format to use.
-        """
-        self._git_ref = git_ref
-        self._tar_compression_format = tar_compression_format
-
-        self._tarball_dir = Path(output_dir, "tarball")
-
-        self._create_tarball_dir()
-
-        self._tarball_name = (
-            f"dpdk-tarball-{self._git_ref}.{self._tar_compression_format.extension}"
-        )
-        self._tarball_path = self._check_tarball_path()
-        if not self._tarball_path:
-            self._create_tarball()
-
-    def _create_tarball_dir(self) -> None:
-        os.makedirs(self._tarball_dir, exist_ok=True)
-
-    def _check_tarball_path(self) -> Path | None:
-        if self._tarball_name in os.listdir(self._tarball_dir):
-            return Path(self._tarball_dir, self._tarball_name)
-        return None
-
-    def _create_tarball(self) -> None:
-        self._tarball_path = Path(self._tarball_dir, self._tarball_name)
-
-        atexit.register(self._delete_tarball)
-
-        result = subprocess.run(
-            'git -C "$(git rev-parse --show-toplevel)" archive '
-            f'{self._git_ref} --prefix="dpdk-tarball-{self._git_ref + os.sep}" | '
-            f"{self._tar_compression_format} > {Path(self._tarball_path.absolute())}",
-            shell=True,
-            text=True,
-            capture_output=True,
-        )
-
-        if result.returncode != 0:
-            raise SubprocessError(
-                f"Git archive creation failed with exit code {result.returncode}.\n"
-                f"Command: {result.args}\n"
-                f"Stdout: {result.stdout}\n"
-                f"Stderr: {result.stderr}"
-            )
-
-        atexit.unregister(self._delete_tarball)
-
-    def _delete_tarball(self) -> None:
-        if self._tarball_path and os.path.exists(self._tarball_path):
-            os.remove(self._tarball_path)
-
-    def __fspath__(self) -> str:
-        """The os.PathLike protocol implementation."""
-        return str(self._tarball_path)
-
-
 def convert_to_list_of_string(value: Any | list[Any]) -> list[str]:
     """Convert the input to the list of strings."""
     return list(map(str, value) if isinstance(value, list) else str(value))
-- 
2.43.0


  parent reply	other threads:[~2024-10-21 13:50 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-30 16:01 [PATCH 0/7] DTS external DPDK build Tomáš Ďurovec
2024-09-30 16:01 ` [PATCH 1/7] dts: rename build target to " Tomáš Ďurovec
2024-09-30 16:01 ` [PATCH 2/7] dts: one dpdk build per test run Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 3/7] dts: fix remote session file transfer vars Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 4/7] dts: add the ability to copy directories via remote Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 5/7] dts: add support for externally compiled DPDK Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 6/7] doc: update argument options for external DPDK build Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 7/7] dts: remove git ref option Tomáš Ďurovec
2024-10-21 13:49 ` [PATCH v2 0/7] DTS external DPDK build Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 1/7] dts: rename build target to " Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 2/7] dts: enforce one dpdk build per test run Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 3/7] dts: fix remote session file transfer vars Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 4/7] dts: enable copying directories to and from nodes Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 5/7] dts: add support for externally compiled DPDK Luca Vizzarro
2024-10-21 13:49   ` [PATCH v2 6/7] doc: update argument options for external DPDK build Luca Vizzarro
2024-10-21 13:49   ` Luca Vizzarro [this message]
2024-10-21 22:39   ` [PATCH v2 0/7] DTS " Dean Marx
2024-10-22  7:27     ` Paul Szczepanek
2024-10-22 14:56     ` Luca Vizzarro
2024-10-21 22:46 ` [PATCH v3 " Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 1/7] dts: rename build target to " Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 2/7] dts: enforce one dpdk build per test run Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 3/7] dts: change remote and local paths objects Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 4/7] dts: enable copying directories to and from nodes Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 5/7] dts: add support for externally compiled DPDK Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 6/7] doc: update argument options for external DPDK build Luca Vizzarro
2024-10-21 22:46   ` [PATCH v3 7/7] dts: remove git ref option Luca Vizzarro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241021134935.1210500-8-luca.vizzarro@arm.com \
    --to=luca.vizzarro@arm.com \
    --cc=dev@dpdk.org \
    --cc=paul.szczepanek@arm.com \
    --cc=probb@iol.unh.edu \
    --cc=tomas.durovec@pantheon.tech \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).