DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Tomáš Ďurovec" <tomas.durovec@pantheon.tech>
To: dev@dpdk.org
Cc: "Tomáš Ďurovec" <tomas.durovec@pantheon.tech>
Subject: [PATCH 7/7] dts: remove git ref option
Date: Fri, 27 Sep 2024 17:38:29 +0200	[thread overview]
Message-ID: <20240927153830.266075-7-tomas.durovec@pantheon.tech> (raw)
In-Reply-To: <20240927153830.266075-1-tomas.durovec@pantheon.tech>

In the previous commits we're adding the support for copying
the whole local DPDK tree directory and git-ref option was
meant to do the same thing.

Signed-off-by: Tomáš Ďurovec <tomas.durovec@pantheon.tech>
---
 doc/guides/tools/dts.rst  |   9 ---
 dts/framework/settings.py |  51 +++-------------
 dts/framework/utils.py    | 119 +-------------------------------------
 3 files changed, 8 insertions(+), 171 deletions(-)

diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 20d4d18b18..d806dce2ae 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -239,9 +239,6 @@ DTS is run with ``main.py`` located in the ``dts`` directory after entering Poet
      --dpdk-tree DIR_PATH  [DTS_DPDK_TREE] Path to DPDK source code tree to test. (default: None)
      --tarball FILE_PATH, --snapshot FILE_PATH
                            [DTS_DPDK_TARBALL] Path to DPDK source code tarball to test. (default: None)
-     --revision ID, --rev ID, --git-ref ID
-                           [DTS_DPDK_REVISION_ID] Git revision ID to test. Could be commit, tag, tree ID etc. To test local changes, first
-                           commit them, then use their commit ID. (default: None)
      --remote-source       [DTS_REMOTE_SOURCE] Set when the DPDK source tree or tarball is located on the SUT node. (default: False)
      --build-dir DIR_NAME  [DTS_BUILD_DIR] A directory name, which would be located in the `dpdk tree` or `tarball`. (default: None)
      --compile-timeout SECONDS
@@ -263,12 +260,6 @@ The minimum DTS needs is a config file and a pre-built DPDK or DPDK
 sources location which can be specified in said config file or on the
 command line or environment variables.
 
-Example command for running DTS with the template configuration and DPDK tag v23.11:
-
-.. code-block:: console
-
-   (dts-py3.10) $ ./main.py --git-ref v23.11
-
 
 DTS Results
 ~~~~~~~~~~~
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 17594ecb15..a4ab674189 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -42,20 +42,12 @@
 .. option:: --dpdk-tree
 .. envvar:: DTS_DPDK_TREE
 
-    The path to DPDK source tree directory to test. Only this or tarball or revision can be
-    provided.
+    The path to DPDK source tree directory to test. Only this or tarball can be provided.
 
 .. option:: --tarball, --snapshot
 .. envvar:: DTS_DPDK_TARBALL
 
-    The path to DPDK source tarball to test. Only this or DPDK tree or revision can be provided.
-
-.. option:: --revision, --rev, --git-ref
-.. envvar:: DTS_DPDK_REVISION_ID
-
-    Git revision ID to test. Could be commit, tag, tree ID etc.
-    To test local changes, first commit them, then use their commit ID.
-    Only this or DPDK tree or tarball can be provided.
+    The path to DPDK source tarball to test. Only this or DPDK tree can be provided.
 
 .. option:: --remote-source
 .. envvar:: DTS_REMOTE_SOURCE
@@ -108,8 +100,6 @@
 from typing import Callable
 
 from .config import DPDKLocation, TestSuiteConfig
-from .exception import ConfigurationError
-from .utils import DPDKGitTarball, get_commit_id
 
 
 @dataclass(slots=True)
@@ -256,14 +246,6 @@ def _get_help_string(self, action):
         return help
 
 
-def _parse_revision_id(rev_id: str) -> str:
-    """Validate revision ID and retrieve corresponding commit ID."""
-    try:
-        return get_commit_id(rev_id)
-    except ConfigurationError:
-        raise argparse.ArgumentTypeError("The Git revision ID supplied is invalid or ambiguous")
-
-
 def _required_with_one_of(parser: _DTSArgumentParser, action: Action, *required_dests: str) -> None:
     """Verify that `action` is listed together with at least one of `required_dests`.
 
@@ -362,15 +344,15 @@ def _get_parser() -> _DTSArgumentParser:
     dpdk_build = parser.add_argument_group(
         "DPDK Build Options",
         description="Arguments in this group (and subgroup) will be applied to a "
-        ":class:`DPDKLocation` when the DPDK tree, tarball or revision will be provided, "
-        "other arguments like remote source and build dir are optional. A :class:`DPDKLocation` "
+        ":class:`DPDKLocation` when the DPDK tree, tarball will be provided, other "
+        "arguments like remote source and build dir are optional. A :class:`DPDKLocation` "
         "from settings are used instead of from config if construct successful.",
     )
 
     dpdk_source = dpdk_build.add_mutually_exclusive_group()
     action = dpdk_source.add_argument(
         "--dpdk-tree",
-        help="The path to DPDK source tree directory to test. Only this or tarball or revision "
+        help="The path to DPDK source tree directory to test. Only this or tarball "
         "can be provided.",
         metavar="DIR_PATH",
         dest="dpdk_tree_path",
@@ -380,26 +362,12 @@ def _get_parser() -> _DTSArgumentParser:
     action = dpdk_source.add_argument(
         "--tarball",
         "--snapshot",
-        help="The path to DPDK source tarball to test. Only this or DPDK tree or revision "
-        "can be provided.",
+        help="The path to DPDK source tarball to test. Only this or DPDK tree " "can be provided.",
         metavar="FILE_PATH",
         dest="dpdk_tarball_path",
     )
     _add_env_var_to_action(action, "DPDK_TARBALL")
 
-    action = dpdk_source.add_argument(
-        "--revision",
-        "--rev",
-        "--git-ref",
-        type=_parse_revision_id,
-        help="Git revision ID to test. Could be commit, tag, tree ID etc. "
-        "To test local changes, first commit them, then use their commit ID."
-        "Only this or DPDK tree or tarball can be provided.",
-        metavar="ID",
-        dest="dpdk_revision_id",
-    )
-    _add_env_var_to_action(action)
-
     action = dpdk_build.add_argument(
         "--remote-source",
         action="store_true",
@@ -408,9 +376,7 @@ def _get_parser() -> _DTSArgumentParser:
         "instead of the execution host. This can be provided only with DPDK tree or tarball.",
     )
     _add_env_var_to_action(action)
-    _required_with_one_of(
-        parser, action, "dpdk_tarball_path", "dpdk_tree_path"
-    )  # ignored if passed with git-ref
+    _required_with_one_of(parser, action, "dpdk_tarball_path", "dpdk_tree_path")
 
     action = dpdk_build.add_argument(
         "--build-dir",
@@ -567,9 +533,6 @@ def get_settings() -> Settings:
 
     args = parser.parse_args()
 
-    if args.dpdk_revision_id:
-        args.dpdk_tarball_path = Path(DPDKGitTarball(args.dpdk_revision_id, args.output_dir))
-
     args.dpdk_location = _process_dpdk_location(
         args.dpdk_tree_path, args.dpdk_tarball_path, args.remote_source, args.build_dir
     )
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 382357ffe8..4b8843bf20 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -14,22 +14,19 @@
     REGEX_FOR_PCI_ADDRESS: The regex representing a PCI address, e.g. ``0000:00:08.0``.
 """
 
-import atexit
 import fnmatch
 import json
 import os
 import random
-import subprocess
 import tarfile
 from enum import Enum, Flag
 from pathlib import Path
-from subprocess import SubprocessError
 from typing import Any, Callable
 
 from scapy.layers.inet import IP, TCP, UDP, Ether  # type: ignore[import-untyped]
 from scapy.packet import Packet  # type: ignore[import-untyped]
 
-from .exception import ConfigurationError, InternalError
+from .exception import InternalError
 
 REGEX_FOR_PCI_ADDRESS: str = "/[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}.[0-9]{1}/"
 
@@ -76,31 +73,6 @@ def get_packet_summaries(packets: list[Packet]) -> str:
     return f"Packet contents: \n{packet_summaries}"
 
 
-def get_commit_id(rev_id: str) -> str:
-    """Given a Git revision ID, return the corresponding commit ID.
-
-    Args:
-        rev_id: The Git revision ID.
-
-    Raises:
-        ConfigurationError: The ``git rev-parse`` command failed, suggesting
-            an invalid or ambiguous revision ID was supplied.
-    """
-    result = subprocess.run(
-        ["git", "rev-parse", "--verify", rev_id],
-        text=True,
-        capture_output=True,
-    )
-    if result.returncode != 0:
-        raise ConfigurationError(
-            f"{rev_id} is not a valid git reference.\n"
-            f"Command: {result.args}\n"
-            f"Stdout: {result.stdout}\n"
-            f"Stderr: {result.stderr}"
-        )
-    return result.stdout.strip()
-
-
 class StrEnum(Enum):
     """Enum with members stored as strings."""
 
@@ -176,95 +148,6 @@ def extension(self):
         return f"{self.value}" if self == self.none else f"{self.none.value}.{self.value}"
 
 
-class DPDKGitTarball:
-    """Compressed tarball of DPDK from the repository.
-
-    The class supports the :class:`os.PathLike` protocol,
-    which is used to get the Path of the tarball::
-
-        from pathlib import Path
-        tarball = DPDKGitTarball("HEAD", "output")
-        tarball_path = Path(tarball)
-    """
-
-    _git_ref: str
-    _tar_compression_format: TarCompressionFormat
-    _tarball_dir: Path
-    _tarball_name: str
-    _tarball_path: Path | None
-
-    def __init__(
-        self,
-        git_ref: str,
-        output_dir: str,
-        tar_compression_format: TarCompressionFormat = TarCompressionFormat.xz,
-    ):
-        """Create the tarball during initialization.
-
-        The DPDK version is specified with `git_ref`. The tarball will be compressed with
-        `tar_compression_format`, which must be supported by the DTS execution environment.
-        The resulting tarball will be put into `output_dir`.
-
-        Args:
-            git_ref: A git commit ID, tag ID or tree ID.
-            output_dir: The directory where to put the resulting tarball.
-            tar_compression_format: The compression format to use.
-        """
-        self._git_ref = git_ref
-        self._tar_compression_format = tar_compression_format
-
-        self._tarball_dir = Path(output_dir, "tarball")
-
-        self._create_tarball_dir()
-
-        self._tarball_name = (
-            f"dpdk-tarball-{self._git_ref}.{self._tar_compression_format.extension}"
-        )
-        self._tarball_path = self._check_tarball_path()
-        if not self._tarball_path:
-            self._create_tarball()
-
-    def _create_tarball_dir(self) -> None:
-        os.makedirs(self._tarball_dir, exist_ok=True)
-
-    def _check_tarball_path(self) -> Path | None:
-        if self._tarball_name in os.listdir(self._tarball_dir):
-            return Path(self._tarball_dir, self._tarball_name)
-        return None
-
-    def _create_tarball(self) -> None:
-        self._tarball_path = Path(self._tarball_dir, self._tarball_name)
-
-        atexit.register(self._delete_tarball)
-
-        result = subprocess.run(
-            'git -C "$(git rev-parse --show-toplevel)" archive '
-            f'{self._git_ref} --prefix="dpdk-tarball-{self._git_ref + os.sep}" | '
-            f"{self._tar_compression_format} > {Path(self._tarball_path.absolute())}",
-            shell=True,
-            text=True,
-            capture_output=True,
-        )
-
-        if result.returncode != 0:
-            raise SubprocessError(
-                f"Git archive creation failed with exit code {result.returncode}.\n"
-                f"Command: {result.args}\n"
-                f"Stdout: {result.stdout}\n"
-                f"Stderr: {result.stderr}"
-            )
-
-        atexit.unregister(self._delete_tarball)
-
-    def _delete_tarball(self) -> None:
-        if self._tarball_path and os.path.exists(self._tarball_path):
-            os.remove(self._tarball_path)
-
-    def __fspath__(self) -> str:
-        """The os.PathLike protocol implementation."""
-        return str(self._tarball_path)
-
-
 def convert_to_list_of_string(value: Any | list[Any]) -> list[str]:
     """Convert the input to the list of strings."""
     return list(map(str, value) if isinstance(value, list) else str(value))
-- 
2.46.1


  parent reply	other threads:[~2024-09-27 22:14 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-27 15:38 [PATCH 1/7] dts: rename build target to DPDK build Tomáš Ďurovec
2024-09-27 15:38 ` [PATCH 2/7] dts: one dpdk build per test run Tomáš Ďurovec
2024-09-27 15:38 ` [PATCH 3/7] dts: fix remote session file transfer vars Tomáš Ďurovec
2024-09-27 15:38 ` [PATCH 4/7] dts: add the ability to copy directories via remote Tomáš Ďurovec
2024-09-27 15:38 ` [PATCH 5/7] dts: add support for externally compiled DPDK Tomáš Ďurovec
2024-09-27 15:38 ` [PATCH 6/7] doc: update argument options for external DPDK build Tomáš Ďurovec
2024-09-27 15:38 ` Tomáš Ďurovec [this message]
2024-09-27 16:08 [PATCH 0/7] DTS " Tomáš Ďurovec
2024-09-27 16:08 ` [PATCH 7/7] dts: remove git ref option Tomáš Ďurovec
2024-09-30 16:01 [PATCH 0/7] DTS external DPDK build Tomáš Ďurovec
2024-09-30 16:02 ` [PATCH 7/7] dts: remove git ref option Tomáš Ďurovec
2024-09-30 16:18 [PATCH 0/7] DTS external DPDK build Tomáš Ďurovec
2024-09-30 16:18 ` [PATCH 7/7] dts: remove git ref option Tomáš Ďurovec

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240927153830.266075-7-tomas.durovec@pantheon.tech \
    --to=tomas.durovec@pantheon.tech \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).