[PATCH v1 0/1] Add DPDK build directory configuration script

[PATCH v1 0/1] Add DPDK build directory configuration script

[Anatoly Burakov]

Note: this patch depends upon Bruce's v3 patchset:

https://patches.dpdk.org/project/dpdk/list/?series=32891

This patch is based on initial script for VSCode configuration:

https://patches.dpdk.org/project/dpdk/patch/6a6b20c037cffcc5f68a341c4b4e4f21990ae991.1721997016.git.anatoly.burakov@intel.com/

This is basically a TUI frontend for Meson. It is by no means meant to be used as
a replacement for using Meson proper, it is merely a shortcut for those who constantly
deal with creating new Meson build directories but doesn't want to type out all components
each time.

It relies on dependency graphs from the above Bruce's patchset (v3 introduced support
for optional dependencies, which this script requires) to work. It'll create a Meson build
directory in the background, enabling all options, and then using both dependency graph and
meson introspection to figure out what can be built, and what dependencies it has.

With this script it is possible to produce very minimal builds - the script is not only able
to track dependencies between components to enable them, but it can also (with a command line
switch) specify which libraries we want to enable (omitting those not required by currently
selected components). This can be useful for users who frequently reconfigure their tree with
e.g. debug/release, shared/static etc. builds while keeping the reconfiguration time fairly
small.

We used to have a "setup.sh" script to "set up" DPDK. This is not that, but it's a good start.

Anatoly Burakov (1):
  usertools: add DPDK build directory setup script

 usertools/dpdk-setup.py | 669 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 669 insertions(+)
 create mode 100755 usertools/dpdk-setup.py

-- 
2.43.5

[PATCH v1 1/1] usertools: add DPDK build directory setup script

[Anatoly Burakov]

Currently, the only way to set up a build directory for DPDK development
is through running Meson directly. This has a number of drawbacks.

For one, the default configuration is very "fat", meaning everything gets
enabled and built (aside from examples, which have to be enabled
manually), so while Meson is very good at minimizing work needed to rebuild
DPDK, for any change that affects a lot of components (such as editing an
EAL header), there's a lot of rebuilding to do.

It is of course possible to reduce the number of components built through
meson options, but this mechanism isn't perfect, as the user needs to
remember exact spelling of all the options and components, and currently
it doesn't handle inter-component dependencies very well (e.g. if net/ice
is enabled, common/iavf is not automatically enabled, so net/ice can't be
built unless user also doesn't forget to specify common/iavf).

Enter this script. It relies on Meson's introspection capabilities as well
as the dependency graphs generated by our build system to display all
available components, and handle any dependencies for them automatically,
while also not forcing user to remember any command-line options and lists
of drivers, and instead relying on interactive TUI to display list of
available options. It can also produce builds that are as minimal as
possible (including cutting down libraries being built) by utilizing the
fact that our dependency graphs report which dependency is mandatory and
which one is optional.

Because it is not meant to replace native Meson build configuration but
is rather targeted at users who are not intimately familiar wtih DPDK's
build system, it is run in interactive mode by default. However, it is
also possible to run it without interaction, in which case it will pass
all its parameters to Meson directly, with added benefit of dependency
tracking and producing minimal builds if desired.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 usertools/dpdk-setup.py | 669 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 669 insertions(+)
 create mode 100755 usertools/dpdk-setup.py

diff --git a/usertools/dpdk-setup.py b/usertools/dpdk-setup.py
new file mode 100755
index 0000000000..a1ac247e4c
--- /dev/null
+++ b/usertools/dpdk-setup.py
@@ -0,0 +1,669 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Intel Corporation
+
+"""
+Displays an interactive TUI-based menu for configuring a DPDK build directory.
+"""
+
+# This is an interactive script that allows the user to configure a DPDK build directory using a
+# text-based user interface (TUI). The script will prompt the user to select various configuration
+# options, and will then call `meson setup` to configure the build directory with the selected
+# options.
+#
+# To be more user-friendly, the script will also run `meson setup` into a temporary directory in
+# the background, which will generate both the list of available options, and any dependencies
+# between them, so whenever the user selects an option, we automatically enable its dependencies.
+# This will also allow us to use meson introspection to get list of things we are capable of
+# building, and warn the user if they selected something that can't be built.
+
+import argparse
+import collections
+import fnmatch
+import json
+import os
+import subprocess
+import sys
+import textwrap
+import typing as T
+from tempfile import TemporaryDirectory
+
+
+# cut off dpdk- prefix
+def _rename_app(app: str) -> str:
+    return app[5:]
+
+
+# replace underscore with slash
+def _rename_driver(driver: str) -> str:
+    return driver.replace("_", "/", 1)
+
+
+def wrap_text(message: str, cols: int) -> T.Tuple[int, int, str]:
+    """Wrap text to N columns and calculate resulting dimensions."""
+    wrapped_lines = textwrap.wrap(message.strip(), cols)
+    h = len(wrapped_lines)
+    w = max(len(line) for line in wrapped_lines)
+    return h, w, "\n".join(wrapped_lines)
+
+
+def calc_opt_width(option: T.Any) -> int:
+    """Calculate the width of an option."""
+    if isinstance(option, str):
+        return len(option)
+    return sum(calc_opt_width(opt) for opt in option) + len(option)  # padding
+
+
+def calc_list_width(options: T.List[T.Any], checkbox: bool) -> int:
+    """Calculate the width of a list."""
+    pad = 5
+    # add 4 for the checkbox
+    if checkbox:
+        pad += 4
+    return max(calc_opt_width(opt) for opt in options) + pad
+
+
+def whiptail_msgbox(message: str) -> None:
+    """Display a message box."""
+    # set max width to 60
+    h, w, message = wrap_text(message, 60)
+    # add some padding
+    w += 10
+    h += 6
+    args = ["whiptail", "--msgbox", message, str(h), str(w)]
+    subprocess.run(args, check=True)
+
+
+def whiptail_checklist(
+    title: str, prompt: str, options: T.List[T.Tuple[str, str]], checked: T.List[str]
+) -> T.List[str]:
+    """Display a checklist and get user input."""
+    # at least two free spaces, but no more than 10 in total
+    lh = min(len(options) + 2, 10)
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # width was set to prompt width, but we need to account for the list
+    lw = calc_list_width(options, True)
+    # adjust width to account for list width as well
+    w = max(w, lw)
+    # add some padding and list height
+    w += 10
+    h += 6 + lh
+
+    # build whiptail checklist
+    checklist = [
+        (label, desc, "on" if label in checked else "off") for label, desc in options
+    ]
+    # flatten the list
+    flat = [item for tup in checklist for item in tup]
+    # build whiptail arguments
+    args = [
+        "whiptail",
+        "--notags",
+        "--separate-output",
+        "--title",
+        title,
+        "--checklist",
+        prompt,
+        str(h),
+        str(w),
+        str(lh),
+    ] + flat
+
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    # capture selected options
+    return result.stderr.decode().strip().split()
+
+
+def whiptail_menu(title: str, prompt: str, options: T.List[T.Tuple[str, str]]) -> str:
+    """Display a menu and get user input."""
+    # at least two free spaces, but no more than 10 in total
+    lh = min(len(options) + 2, 10)
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # width was set to prompt width, but we need to account for the list
+    lw = calc_list_width(options, False)
+    # adjust width to account for list width as well
+    w = max(w, lw)
+    # add some padding
+    w += 10
+    h += 6 + lh
+    # flatten the list
+    flat = [item for tup in options for item in tup]
+    args = [
+        "whiptail",
+        "--notags",
+        "--title",
+        title,
+        "--menu",
+        prompt,
+        str(h),
+        str(w),
+        str(lh),
+    ] + flat
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    return result.stderr.decode().strip()
+
+
+def whiptail_inputbox(title: str, prompt: str, default: str = "") -> str:
+    """Display an input box and get user input."""
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # add some padding
+    w += 10
+    h += 6
+    args = ["whiptail", "--inputbox", "--title", title, prompt, str(h), str(w), default]
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    return result.stderr.decode().strip()
+
+
+class DepGraph:
+    """A dependency graph for Meson options."""
+
+    def __init__(self, src_dir: str):
+        self.dst_dir = TemporaryDirectory()
+        self.src_dir = src_dir
+        # start the meson setup process in the background - user needs to call parse_deps() before
+        # this class's data is usable
+        self._proc = self._create_dep_tree()
+        self._deps_parsed = False
+        # components that can be built according to meson's introspection
+        self.can_be_built: T.Set[str] = set()
+        # components that were read from dependency graph
+        self.required_deps: T.Dict[str, T.Set[str]] = {}
+        self.optional_deps: T.Dict[str, T.Set[str]] = {}
+        # separate component list into libs, drivers, apps, and examples
+        self.libs: T.Set[str] = set()
+        self.drivers: T.Set[str] = set()
+        self.apps: T.Set[str] = set()
+        self.examples: T.Set[str] = set()
+
+    def _create_dep_tree(self) -> subprocess.Popen[bytes]:
+        # we want all examples as well
+        args = ["meson", "setup", self.dst_dir.name, "-Dexamples=all"]
+        return subprocess.Popen(
+            args, cwd=self.src_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE
+        )
+
+    def _parse_dep_line(self, line: str) -> T.Tuple[str, T.Set[str], str, bool]:
+        """Parse digraph line into (component, {dependencies}, type, optional)."""
+        # extract attributes first
+        first, last = line.index("["), line.rindex("]")
+        edge_str, attr_str = line[:first], line[first + 1 : last]
+        # key=value, key=value, ...
+        attrs = {
+            key.strip('" '): value.strip('" ')
+            for attr_kv in attr_str.split(",")
+            for key, value in [attr_kv.strip().split("=", 1)]
+        }
+        # check if edge is defined as dotted line, meaning it's optional
+        optional = "dotted" in attrs.get("style", "")
+        try:
+            component_type = attrs["dpdk_componentType"]
+        except KeyError as _e:
+            raise ValueError(f"Error: missing component type: {line}") from _e
+
+        # now, extract component name and any of its dependencies
+        deps: T.Set[str] = set()
+        try:
+            component, deps_str = edge_str.strip('" ').split("->", 1)
+            component = component.strip().strip('" ')
+            deps_str = deps_str.strip().strip("{}")
+            deps = {d.strip('" ') for d in deps_str.split(",")}
+        except ValueError as _e:
+            component = edge_str.strip('" ')
+
+        return component, deps, component_type, optional
+
+    def parse_deps(self) -> None:
+        """Parse the dependencies generated by meson."""
+        if self._deps_parsed:
+            return
+        self._proc.wait()
+
+        # first, read the dep graph
+        dep_graph_path = os.path.join(self.dst_dir.name, "deps.dot")
+        with open(dep_graph_path, encoding="utf-8") as f:
+            for line in f:
+                # skip lines that aren't edges
+                if line.strip() == "digraph {" or line.strip() == "}":
+                    continue
+
+                component, deps, c_type, optional = self._parse_dep_line(line)
+
+                # record component type
+                type_to_set = {
+                    "lib": self.libs,
+                    "drivers": self.drivers,
+                    "app": self.apps,
+                    "examples": self.examples,
+                }
+                type_to_set[c_type].add(component)
+
+                # store dependencies
+                if optional:
+                    self.optional_deps[component] = deps
+                else:
+                    self.required_deps[component] = deps
+
+        # now, use Meson introspection to read the list of components that can be built
+        args = ["meson", "introspect", "--targets"]
+        output = subprocess.check_output(args, cwd=self.dst_dir.name, encoding="utf-8")
+        # parse output as JSON
+        introspected = json.loads(output)
+
+        # we want to filter out certain things from the introspection output
+        def _filter_target(target: T.Dict[str, T.Any]) -> bool:
+            t_name: str = target["name"]
+            t_type: str = target["type"]
+
+            # if target is a library, we only want those that start with "rte_"
+            if t_type in ["static library", "shared library"]:
+                return t_name.startswith("rte_")
+
+            # if target is an executable, we only want those that start with "dpdk-"
+            if t_type == "executable":
+                return t_name.startswith("dpdk-")
+
+            return False
+
+        for target in filter(_filter_target, introspected):
+            t_name: str = target["name"]
+            t_type: str = target["type"]
+
+            # for libraries, cut off rte_ prefix
+            if t_type in ["static library", "shared library"]:
+                t_name = t_name[4:]
+
+            # there may be duplicate targets because of shared/static libraries
+            if t_name in self.can_be_built:
+                continue
+
+            self.can_be_built.add(t_name)
+
+        self._deps_parsed = True
+
+
+class SetupCtx:
+    """POD class to hold context for the setup script."""
+
+    def __init__(self) -> None:
+        self.dg: DepGraph
+        self.use_ui = False
+        self.minimal = False
+        self.input_parsed = False
+        self.dpdk_dir = ""
+        self.build_dir = ""
+
+        # what did user specify on the command-line?
+        self.enabled_apps_str = ""
+        self.enabled_drivers_str = ""
+        self.enabled_examples_str = ""
+        self.enabled_libs_str = ""
+        self.meson_args_str = ""
+
+        # what did we end up with after parsing user's input?
+        self.enabled_apps: T.List[str] = []
+        self.enabled_drivers: T.List[str] = []
+        self.enabled_examples: T.List[str] = []
+        self.enabled_libs: T.List[str] = []
+
+        # some apps have different names in Meson
+        self.rename_map = {
+            "testpmd": "test-pmd",
+        }
+
+    def _create_meson_option_cmd(
+        self,
+        meson_option_cmd: str,
+        entries: T.Set[str],
+        rename_func: T.Optional[T.Callable[[str], str]] = None,
+    ) -> str:
+        """Create a Meson option command from a set of entries."""
+        opt_list = [
+            entry if rename_func is None else rename_func(entry)
+            for entry in sorted(entries)
+        ]
+        return f"-D{meson_option_cmd}={','.join(opt_list)}"
+
+    def _wildcard_match(
+        self,
+        components: T.Set[str],
+        pattern: str,
+        pattern_func: T.Optional[T.Callable[[str], str]] = None,
+    ) -> T.Set[str]:
+        """Match a pattern against a set of components."""
+        if not pattern:
+            return set()
+        if pattern_func is not None:
+            pattern = pattern_func(pattern)
+        # if this is not a wildcard, return component explicitly
+        if "*" not in pattern:
+            return {pattern}
+        # this is a wildcard match, so use wildcard matching
+        match = {c for c in components if fnmatch.fnmatch(c, pattern)}
+        # filter out anything that isn't buildable
+        return match & self.dg.can_be_built
+
+    def parse_input(self) -> None:
+        """Parse user input."""
+        if self.input_parsed:
+            return
+        self.dg.parse_deps()
+
+        # when parsing user input, we expect to see a list of components separated by commas, as
+        # well as maybe wildcards. We will expand wildcards into a list of components, but by
+        # default we won't enable anything that can't be built even if it matches wildcard. also,
+        # component named used by Meson user-facing code and component names used in the backend
+        # are not exactly the same. for example, apps and examples will not have "dpdk-" prefixes,
+        # while drivers will have underscores instead of slashes. we need to take all of that into
+        # account when matching user input to actual components.
+
+        def _app_pattern_func(app: str) -> str:
+            return f"dpdk-{app}"
+
+        def _driver_pattern_func(driver: str) -> str:
+            return driver.replace("/", "_", 1)
+
+        self.enabled_apps = [
+            app
+            for pattern in self.enabled_apps_str.split(",")
+            for app in self._wildcard_match(self.dg.apps, pattern, _app_pattern_func)
+        ]
+        self.enabled_examples = [
+            example
+            for pattern in self.enabled_examples_str.split(",")
+            for example in self._wildcard_match(
+                self.dg.examples, pattern, _app_pattern_func
+            )
+        ]
+        self.enabled_drivers = [
+            driver
+            for pattern in self.enabled_drivers_str.split(",")
+            for driver in self._wildcard_match(
+                self.dg.drivers, pattern, _driver_pattern_func
+            )
+        ]
+        self.enabled_libs = [
+            lib
+            for pattern in self.enabled_libs_str.split(",")
+            for lib in self._wildcard_match(self.dg.libs, pattern)
+        ]
+
+        self.input_parsed = True
+
+    def create_meson_cmdline(self) -> T.List[str]:
+        """Dump all configuration into Meson command-line string."""
+        # ensure input was parsed before we got here
+        self.parse_input()
+
+        args: T.List[str] = []
+        enabled_apps: T.Set[str] = set()
+        enabled_drivers: T.Set[str] = set()
+        enabled_examples: T.Set[str] = set()
+        enabled_libs: T.Set[str] = set()
+
+        # gather everything
+        enabled_apps = set(self.enabled_apps)
+        enabled_drivers = set(self.enabled_drivers)
+        enabled_examples = set(self.enabled_examples)
+        enabled_libs = set(self.enabled_libs)
+
+        enabled_all = enabled_examples | enabled_apps | enabled_drivers | enabled_libs
+
+        # gather all dependencies
+        new_deps: T.Set[str] = set()
+        for component in enabled_all:
+            deps = self.dg.required_deps[component]
+            new_deps.add(component)
+            # deps do not include complete list, so walk through all dependencies
+            dep_stack = collections.deque(deps)
+            while dep_stack:
+                dc = dep_stack.pop()
+                if dc in new_deps:
+                    continue
+                new_deps.add(dc)
+
+                # get dependencies for this dependency
+                deps = self.dg.required_deps[dc]
+                # recurse deeper
+                dep_stack.extend(deps)
+
+        # extend all lists with new dependencies
+        enabled_apps |= new_deps & self.dg.apps
+        enabled_drivers |= new_deps & self.dg.drivers
+        enabled_examples |= new_deps & self.dg.examples
+        enabled_libs |= new_deps & self.dg.libs
+        enabled_all |= new_deps
+
+        # check if everything can be built
+        diff = enabled_all - self.dg.can_be_built
+        if diff:
+            print(
+                f"Warning: {', '.join(diff)} requested but cannot be built",
+                file=sys.stderr,
+            )
+
+        # we've resolved all dependencies, time to dump it all out
+
+        if enabled_apps:
+            # special case: some apps are renamed
+            enabled_apps = {self.rename_map.get(app, app) for app in enabled_apps}
+            args += [
+                self._create_meson_option_cmd("enable_apps", enabled_apps, _rename_app)
+            ]
+
+        if enabled_examples:
+            args += [
+                self._create_meson_option_cmd("examples", enabled_examples, _rename_app)
+            ]
+
+        if enabled_drivers:
+            args += [
+                self._create_meson_option_cmd(
+                    "enable_drivers", enabled_drivers, _rename_driver
+                )
+            ]
+
+        # if we have specified any other components, this will not be empty.
+        # however, we only want to specify enabled libs if we want to have a
+        # minimal build. so, before only enabling libs we depend on, check if
+        # user actually wanted a minimal build.
+        if (self.minimal or self.enabled_libs) and enabled_libs:
+            args += [self._create_meson_option_cmd("enable_libs", enabled_libs)]
+
+        # did user specify any extra Meson arguments?
+        if self.meson_args_str:
+            args += self.meson_args_str.split()
+
+        return args
+
+
+def select_items(
+    app_list: T.List[str],
+    rename_func: T.Optional[T.Callable[[str], str]],
+    checked_list: T.List[str],
+) -> None:
+    """Select apps to enable."""
+    # create a dialog selection for apps
+    options = [
+        (app, rename_func(app) if rename_func is not None else app)
+        for app in sorted(app_list)
+    ]
+
+    try:
+        selected = whiptail_checklist(
+            "DPDK standard applications",
+            "Select apps to enable",
+            options,
+            checked_list,
+        )
+        checked_list.clear()
+        checked_list.extend(selected)
+    except subprocess.CalledProcessError:
+        # user pressed cancel, don't do anything
+        pass
+
+
+def main_menu(ctx: SetupCtx) -> None:
+    """Display main menu."""
+    while True:
+        options = {
+            "apps": "Enable apps",
+            "examples": "Enable examples",
+            "libs": "Enable libraries",
+            "drivers": "Enable drivers",
+            "meson": "Enter custom Meson options",
+            "exit": "Save & exit",
+        }
+        ret = whiptail_menu(
+            "Setup DPDK build directory",
+            "Select an option",
+            list(options.items()),
+        )
+
+        if ret in ["apps", "examples", "libs", "drivers"]:
+            # before we're able to select apps, we need to parse input
+            if not ctx.input_parsed:
+                print("Parsing dependency tree, please wait...")
+                ctx.parse_input()
+            selection_screens: T.Dict[str, T.Any] = {
+                "apps": (list(ctx.dg.apps), _rename_app, ctx.enabled_apps),
+                "examples": (list(ctx.dg.examples), _rename_app, ctx.enabled_examples),
+                "libs": (list(ctx.dg.libs), None, ctx.enabled_libs),
+                "drivers": (list(ctx.dg.drivers), _rename_driver, ctx.enabled_drivers),
+            }
+            try:
+                items_list, rename_func, enabled_list = selection_screens[ret]
+                select_items(items_list, rename_func, enabled_list)
+
+                # did user select something that cannot be built?
+                diff = set(enabled_list) - ctx.dg.can_be_built
+                if diff:
+                    comp_str = ", ".join(diff)
+                    whiptail_msgbox(
+                        f"Warning: selected component(s) {comp_str} cannot be built."
+                    )
+            except subprocess.CalledProcessError:
+                # user pressed cancel, don't do anything
+                pass
+        elif ret == "meson":
+            try:
+                ctx.meson_args_str = whiptail_inputbox(
+                    "Custom Meson options",
+                    "Enter custom Meson options",
+                    ctx.meson_args_str,
+                )
+            except subprocess.CalledProcessError:
+                # user pressed cancel, don't do anything
+                pass
+        elif ret == "exit":
+            break
+
+
+def parse_args() -> SetupCtx:
+    """Parse command-line arguments and return a context."""
+    # find out where we are
+    self_path = os.path.abspath(__file__)
+    # go one level up to get to DPDK source directory
+    dpdk_dir = os.path.dirname(os.path.dirname(self_path))
+
+    parser = argparse.ArgumentParser(description="Configure a DPDK build directory.")
+    parser.add_argument(
+        "--dpdk-dir", "-S", default=dpdk_dir, help="Path to the DPDK source directory."
+    )
+    parser.add_argument(
+        "--build-dir", "-B", default="build", help="Path to the DPDK build directory."
+    )
+    parser.add_argument(
+        "--no-ui",
+        action="store_true",
+        help="Disable the TUI and use command-line arguments directly.",
+    )
+    parser.add_argument(
+        "--minimal",
+        action="store_true",
+        help="Try to remove unneeded libraries from build.",
+    )
+    parser.add_argument(
+        "--apps",
+        "-a",
+        default="",
+        help="Comma-separated list of apps to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--drivers",
+        "-d",
+        default="",
+        help="Comma-separated list of drivers to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--examples",
+        "-e",
+        default="",
+        help="Comma-separated list of examples to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--libs",
+        "-l",
+        default="",
+        help="Comma-separated list of libraries to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--meson-args", "-m", default="", help="Extra arguments to pass to Meson setup."
+    )
+    args = parser.parse_args()
+
+    ctx = SetupCtx()
+    ctx.build_dir = args.build_dir
+    ctx.dpdk_dir = args.dpdk_dir
+    ctx.use_ui = not args.no_ui
+    ctx.minimal = args.minimal
+    ctx.enabled_apps_str = args.apps
+    ctx.enabled_drivers_str = args.drivers
+    ctx.enabled_examples_str = args.examples
+    ctx.enabled_libs_str = args.libs
+    ctx.meson_args_str = args.meson_args
+
+    return ctx
+
+
+def _main() -> int:
+    # parse command-line arguments
+    ctx = parse_args()
+
+    # did we discover a valid DPDK directory?
+    if not os.path.exists(os.path.join(ctx.dpdk_dir, "meson.build")):
+        raise FileNotFoundError("DPDK source directory not found.")
+
+    # parse deps in background
+    ctx.dg = DepGraph(ctx.dpdk_dir)
+
+    # if we're not using the UI, parse input and exit
+    if not ctx.use_ui:
+        print("UI is disabled, using command-line arguments directly")
+        print("Parsing dependency tree...")
+        ctx.parse_input()
+    else:
+        # we're using menu-driven UI, so wait until user tells us to exit
+        try:
+            main_menu(ctx)
+        except subprocess.CalledProcessError:
+            # user pressed cancel, exit
+            print("Operation cancelled")
+            return 1
+
+    # run meson
+    meson_cmdline = ctx.create_meson_cmdline()
+    run_args = ["meson", "setup", ctx.build_dir, *meson_cmdline]
+    print("Running: ", *run_args, sep=" ")
+    runret = subprocess.run(run_args, check=True)
+    return runret.returncode
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(_main())
+    except FileNotFoundError as e:
+        print(e, file=sys.stderr)
+        sys.exit(1)
-- 
2.43.5

RE: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[Morten Brørup]

> From: Anatoly Burakov [mailto:anatoly.burakov@intel.com]
> 
> Currently, the only way to set up a build directory for DPDK development
> is through running Meson directly. This has a number of drawbacks.
> 
> For one, the default configuration is very "fat", meaning everything gets
> enabled and built (aside from examples, which have to be enabled
> manually), so while Meson is very good at minimizing work needed to rebuild
> DPDK, for any change that affects a lot of components (such as editing an
> EAL header), there's a lot of rebuilding to do.
> 
> It is of course possible to reduce the number of components built through
> meson options, but this mechanism isn't perfect, as the user needs to
> remember exact spelling of all the options and components, and currently
> it doesn't handle inter-component dependencies very well (e.g. if net/ice
> is enabled, common/iavf is not automatically enabled, so net/ice can't be
> built unless user also doesn't forget to specify common/iavf).
> 
> Enter this script. It relies on Meson's introspection capabilities as well
> as the dependency graphs generated by our build system to display all
> available components, and handle any dependencies for them automatically,
> while also not forcing user to remember any command-line options and lists
> of drivers, and instead relying on interactive TUI to display list of
> available options. It can also produce builds that are as minimal as
> possible (including cutting down libraries being built) by utilizing the
> fact that our dependency graphs report which dependency is mandatory and
> which one is optional.
> 
> Because it is not meant to replace native Meson build configuration but
> is rather targeted at users who are not intimately familiar wtih DPDK's
> build system, it is run in interactive mode by default. However, it is
> also possible to run it without interaction, in which case it will pass
> all its parameters to Meson directly, with added benefit of dependency
> tracking and producing minimal builds if desired.
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

Acked-by: Morten Brørup <mb@smartsharesystems.com>

Re: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[David Marchand]

On Wed, Sep 4, 2024 at 5:17 PM Anatoly Burakov
<anatoly.burakov@intel.com> wrote:
>
> Currently, the only way to set up a build directory for DPDK development
> is through running Meson directly. This has a number of drawbacks.
>
> For one, the default configuration is very "fat", meaning everything gets
> enabled and built (aside from examples, which have to be enabled
> manually), so while Meson is very good at minimizing work needed to rebuild
> DPDK, for any change that affects a lot of components (such as editing an
> EAL header), there's a lot of rebuilding to do.
>
> It is of course possible to reduce the number of components built through
> meson options, but this mechanism isn't perfect, as the user needs to
> remember exact spelling of all the options and components, and currently
> it doesn't handle inter-component dependencies very well (e.g. if net/ice
> is enabled, common/iavf is not automatically enabled, so net/ice can't be
> built unless user also doesn't forget to specify common/iavf).

There should be an explicit error explaining why the driver is not enabled.
Is it not the case?


>
> Enter this script. It relies on Meson's introspection capabilities as well
> as the dependency graphs generated by our build system to display all
> available components, and handle any dependencies for them automatically,
> while also not forcing user to remember any command-line options and lists
> of drivers, and instead relying on interactive TUI to display list of
> available options. It can also produce builds that are as minimal as
> possible (including cutting down libraries being built) by utilizing the
> fact that our dependency graphs report which dependency is mandatory and
> which one is optional.
>
> Because it is not meant to replace native Meson build configuration but
> is rather targeted at users who are not intimately familiar wtih DPDK's
> build system, it is run in interactive mode by default. However, it is
> also possible to run it without interaction, in which case it will pass
> all its parameters to Meson directly, with added benefit of dependency
> tracking and producing minimal builds if desired.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

There is no documentation.
And it is a devtools script and not a usertools, iow, no point in
installing this along a built dpdk.

I don't see a lot of value in such script.
In my opinion, people who really want to tune their dpdk build should
enter the details carefully and understand the implications.
But other than that, I have no strong objection.


-- 
David Marchand

Re: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[Burakov, Anatoly]

On 9/5/2024 9:29 AM, David Marchand wrote:
> On Wed, Sep 4, 2024 at 5:17 PM Anatoly Burakov
> <anatoly.burakov@intel.com> wrote:
>>
>> Currently, the only way to set up a build directory for DPDK development
>> is through running Meson directly. This has a number of drawbacks.
>>
>> For one, the default configuration is very "fat", meaning everything gets
>> enabled and built (aside from examples, which have to be enabled
>> manually), so while Meson is very good at minimizing work needed to rebuild
>> DPDK, for any change that affects a lot of components (such as editing an
>> EAL header), there's a lot of rebuilding to do.
>>
>> It is of course possible to reduce the number of components built through
>> meson options, but this mechanism isn't perfect, as the user needs to
>> remember exact spelling of all the options and components, and currently
>> it doesn't handle inter-component dependencies very well (e.g. if net/ice
>> is enabled, common/iavf is not automatically enabled, so net/ice can't be
>> built unless user also doesn't forget to specify common/iavf).
> 
> There should be an explicit error explaining why the driver is not enabled.
> Is it not the case?

It is there alright, however

1) the error message is not perfect because e.g. in case of net/ice it 
asks to enable "common_iavf" and doesn't say whether it's a driver or 
something else, which can be confusing (and it was for me!)

2) I would still prefer this happening automatically (requires much more 
effort to fix it in the build system itself and arguably isn't worth 
it), and

3) preferably without typing much while still allowing me to be flexible

> 
> 
>>
>> Enter this script. It relies on Meson's introspection capabilities as well
>> as the dependency graphs generated by our build system to display all
>> available components, and handle any dependencies for them automatically,
>> while also not forcing user to remember any command-line options and lists
>> of drivers, and instead relying on interactive TUI to display list of
>> available options. It can also produce builds that are as minimal as
>> possible (including cutting down libraries being built) by utilizing the
>> fact that our dependency graphs report which dependency is mandatory and
>> which one is optional.
>>
>> Because it is not meant to replace native Meson build configuration but
>> is rather targeted at users who are not intimately familiar wtih DPDK's
>> build system, it is run in interactive mode by default. However, it is
>> also possible to run it without interaction, in which case it will pass
>> all its parameters to Meson directly, with added benefit of dependency
>> tracking and producing minimal builds if desired.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> There is no documentation.

I will add some in later revisions. This is just me planting the flag 
before the v1 deadline :)

> And it is a devtools script and not a usertools, iow, no point in
> installing this along a built dpdk.

Agreed, I will move it to devtools.

> 
> I don't see a lot of value in such script.
> In my opinion, people who really want to tune their dpdk build should
> enter the details carefully and understand the implications.
> But other than that, I have no strong objection.
> 

I understand your perspective, but I respectfully disagree about 
"entering details carefully and understanding the implications" for 
basic things like enabling example applications or drivers. This may 
apply for more advanced meson options like enabling IOVA as PA or 
setting mbuf headroom or whatever, but these aren't provided in this 
script - for those options one indeed would want to run meson manually 
or at least enter additional meson arguments (implying the person has 
already looked at the documentation, found the option, and more-or-less 
understands what will happen when they change its value).

However, for your "standard" options such as which apps to enable, I 
don't think it's a stretch to suggest it's pretty obvious what will 
happen when this or that option is enabled or disabled, the difference 
would be simply in the fact that one doesn't have to remember exact 
syntax or spelling (e.g. is it "test-pmd"? "testpmd"? "dpdk-testpmd"?) 
to work with basic options. The dependency tracking will also protect 
the user from obvious mistakes such as not building mempool library or 
whatever, so there are arguably no far reaching "implications" that have 
to be considered here.

-- 
Thanks,
Anatoly

Re: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[fengchengwen]

On 2024/9/5 15:29, David Marchand wrote:
> On Wed, Sep 4, 2024 at 5:17 PM Anatoly Burakov
> <anatoly.burakov@intel.com> wrote:
>>
>> Currently, the only way to set up a build directory for DPDK development
>> is through running Meson directly. This has a number of drawbacks.
>>
>> For one, the default configuration is very "fat", meaning everything gets
>> enabled and built (aside from examples, which have to be enabled
>> manually), so while Meson is very good at minimizing work needed to rebuild
>> DPDK, for any change that affects a lot of components (such as editing an
>> EAL header), there's a lot of rebuilding to do.
>>
>> It is of course possible to reduce the number of components built through
>> meson options, but this mechanism isn't perfect, as the user needs to
>> remember exact spelling of all the options and components, and currently
>> it doesn't handle inter-component dependencies very well (e.g. if net/ice
>> is enabled, common/iavf is not automatically enabled, so net/ice can't be
>> built unless user also doesn't forget to specify common/iavf).
> 
> There should be an explicit error explaining why the driver is not enabled.
> Is it not the case?
> 
> 
>>
>> Enter this script. It relies on Meson's introspection capabilities as well
>> as the dependency graphs generated by our build system to display all
>> available components, and handle any dependencies for them automatically,
>> while also not forcing user to remember any command-line options and lists
>> of drivers, and instead relying on interactive TUI to display list of
>> available options. It can also produce builds that are as minimal as
>> possible (including cutting down libraries being built) by utilizing the
>> fact that our dependency graphs report which dependency is mandatory and
>> which one is optional.
>>
>> Because it is not meant to replace native Meson build configuration but
>> is rather targeted at users who are not intimately familiar wtih DPDK's
>> build system, it is run in interactive mode by default. However, it is
>> also possible to run it without interaction, in which case it will pass
>> all its parameters to Meson directly, with added benefit of dependency
>> tracking and producing minimal builds if desired.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> There is no documentation.
> And it is a devtools script and not a usertools, iow, no point in
> installing this along a built dpdk.
> 
> I don't see a lot of value in such script.

+1
I just run this script, and it provide UI just like Linux kernel "make menuconfig",
but I think DPDK is not complicated enough to have to use such menuconfig.

> In my opinion, people who really want to tune their dpdk build should
> enter the details carefully and understand the implications.
> But other than that, I have no strong objection.
> 
> 

RE: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[Morten Brørup]

> From: fengchengwen [mailto:fengchengwen@huawei.com]
> Sent: Friday, 6 September 2024 09.41
> 
> On 2024/9/5 15:29, David Marchand wrote:
> > On Wed, Sep 4, 2024 at 5:17 PM Anatoly Burakov
> > <anatoly.burakov@intel.com> wrote:
> >>
> >> Enter this script. It relies on Meson's introspection capabilities as well
> >> as the dependency graphs generated by our build system to display all
> >> available components, and handle any dependencies for them automatically,
> >> while also not forcing user to remember any command-line options and lists
> >> of drivers, and instead relying on interactive TUI to display list of
> >> available options. It can also produce builds that are as minimal as
> >> possible (including cutting down libraries being built) by utilizing the
> >> fact that our dependency graphs report which dependency is mandatory and
> >> which one is optional.
> >>
> >> Because it is not meant to replace native Meson build configuration but
> >> is rather targeted at users who are not intimately familiar wtih DPDK's
> >> build system, it is run in interactive mode by default. However, it is
> >> also possible to run it without interaction, in which case it will pass
> >> all its parameters to Meson directly, with added benefit of dependency
> >> tracking and producing minimal builds if desired.
> >>
> >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> >
> > There is no documentation.

+1

> > And it is a devtools script and not a usertools, iow, no point in
> > installing this along a built dpdk.

+1

> >
> > I don't see a lot of value in such script.
> 
> +1
> I just run this script, and it provide UI just like Linux kernel "make
> menuconfig",
> but I think DPDK is not complicated enough to have to use such menuconfig.
> 
> > In my opinion, people who really want to tune their dpdk build should
> > enter the details carefully and understand the implications.
> > But other than that, I have no strong objection.

I think this script is a good step on the roadmap towards making DPDK build time configuration more developer friendly.

The idea of making DPDK 100 % runtime configurable and 0 % build time configurable has failed.

DPDK should be buildable by distros with a lot of features and drivers enabled, and projects using it for special use cases should have the ability to build a purpose-specific variant. Just like the kernel.

Re: [PATCH v1 1/1] usertools: add DPDK build directory setup script

[Burakov, Anatoly]

On 9/6/2024 10:28 AM, Morten Brørup wrote:
>> From: fengchengwen [mailto:fengchengwen@huawei.com]
>> Sent: Friday, 6 September 2024 09.41
>>
>> On 2024/9/5 15:29, David Marchand wrote:
>>> On Wed, Sep 4, 2024 at 5:17 PM Anatoly Burakov
>>> <anatoly.burakov@intel.com> wrote:
>>>>
>>>> Enter this script. It relies on Meson's introspection capabilities as well
>>>> as the dependency graphs generated by our build system to display all
>>>> available components, and handle any dependencies for them automatically,
>>>> while also not forcing user to remember any command-line options and lists
>>>> of drivers, and instead relying on interactive TUI to display list of
>>>> available options. It can also produce builds that are as minimal as
>>>> possible (including cutting down libraries being built) by utilizing the
>>>> fact that our dependency graphs report which dependency is mandatory and
>>>> which one is optional.
>>>>
>>>> Because it is not meant to replace native Meson build configuration but
>>>> is rather targeted at users who are not intimately familiar wtih DPDK's
>>>> build system, it is run in interactive mode by default. However, it is
>>>> also possible to run it without interaction, in which case it will pass
>>>> all its parameters to Meson directly, with added benefit of dependency
>>>> tracking and producing minimal builds if desired.
>>>>
>>>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>
>>> There is no documentation.
> 
> +1
> 
>>> And it is a devtools script and not a usertools, iow, no point in
>>> installing this along a built dpdk.
> 
> +1
> 
>>>
>>> I don't see a lot of value in such script.
>>
>> +1
>> I just run this script, and it provide UI just like Linux kernel "make
>> menuconfig",
>> but I think DPDK is not complicated enough to have to use such menuconfig.
>>
>>> In my opinion, people who really want to tune their dpdk build should
>>> enter the details carefully and understand the implications.
>>> But other than that, I have no strong objection.
> 
> I think this script is a good step on the roadmap towards making DPDK build time configuration more developer friendly.
> 
> The idea of making DPDK 100 % runtime configurable and 0 % build time configurable has failed.
> 
> DPDK should be buildable by distros with a lot of features and drivers enabled, and projects using it for special use cases should have the ability to build a purpose-specific variant. Just like the kernel.
> 

Well, technically, this doesn't enable this use case any more than it is 
already enabled by Meson, it's just a more friendly frontend for doing 
that sort of thing. menuconfig is a good analogy, although the script is 
way more limited in scope than menuconfig, and doesn't cover nearly as 
many DPDK options as a proper menuconfig-like script would, as for 
example it doesn't cover things like CPU instruction sets or other 
build-time configuration that we have in Meson. (I did have this in my 
internal prototype, but I decided to remove this feature because the 
script was getting positively giant, it's pushing 800 lines as it is)

Still, I think it'll be easier to use for people unfamiliar with DPDK 
(or people who don't like typing a lot, of which I am one).

-- 
Thanks,
Anatoly

[PATCH v2 0/1] Add DPDK build directory configuration script

[Anatoly Burakov]

Note: this patch depends upon Bruce's v4 patchset:

https://patches.dpdk.org/project/dpdk/list/?series=34036

This patch is based on initial script for VSCode configuration:

https://patches.dpdk.org/project/dpdk/patch/6a6b20c037cffcc5f68a341c4b4e4f21990ae991.1721997016.git.anatoly.burakov@intel.com/

This is a TUI frontend for Meson. It is by no means meant to be used as a
replacement for using Meson proper, it is merely a shortcut for those who
constantly deal with creating new Meson build directories but don't want to type
out all components each time.

It relies on dependency graphs from the above Bruce's patchset (v3 introduced support
for optional dependencies, which this script requires) to work. It'll create a Meson build
directory in the background, enabling all options, and then using both dependency graph and
meson introspection to figure out what can be built, and what dependencies it has.

With this script it is possible to produce very minimal builds - the script is not only able
to track dependencies between components to enable them, but it can also (with a command line
switch) specify which libraries we want to enable (omitting those not required by currently
selected components). This can be useful for users who frequently reconfigure their tree with
e.g. debug/release, shared/static etc. builds while keeping the reconfiguration time fairly
small.

We used to have a "setup.sh" script to "set up" DPDK. This is not that, but it's a good start.

Anatoly Burakov (1):
  devtools: add DPDK build directory setup script

 devtools/dpdk-setup.py              | 784 ++++++++++++++++++++++++++++
 doc/guides/linux_gsg/build_dpdk.rst |  26 +
 2 files changed, 810 insertions(+)
 create mode 100755 devtools/dpdk-setup.py

-- 
2.43.5

[PATCH v2 1/1] devtools: add DPDK build directory setup script

[Anatoly Burakov]

Currently, the only way to set up a build directory for DPDK development
is through running Meson directly. This has a number of drawbacks.

For one, the default configuration is very "fat", meaning everything gets
enabled and built (aside from examples, which have to be enabled
manually), so while Meson is very good at minimizing work needed to
rebuild DPDK, for any change that affects a lot of components (such as
editing an EAL header), there's a lot of rebuilding to do, which may not
be needed.

It is of course possible to reduce the number of components built through
meson options, but this mechanism isn't perfect, as the user needs to
remember exact spelling of all the options and components, and currently
it doesn't handle inter-component dependencies very well (e.g. if net/ice
is enabled, common/iavf is not automatically enabled, so net/ice can't be
built unless user also doesn't forget to specify common/iavf). Error
messages are displayed, but to an untrained eye it is not always clear
what the user has to do for them to go away.

Enter this script. It relies on Meson's introspection capabilities as
well as the dependency graphs generated by our build system to display
all available components, and handle any dependencies for them
automatically, while also not forcing user to remember any command-line
options and lists of drivers, and instead relying on interactive TUI to
display list of available options. It can also produce builds that are as
minimal as possible (including cutting down libraries being built) by
utilizing the fact that our dependency graphs report which dependency is
mandatory and which one is optional.

Because it is not meant to replace native Meson build configuration but
is rather targeted at developers who are not intimately familiar with
DPDK's build system or want to quickly enable this or that without
thinking about dependencies, it is run in interactive mode by default.
However, it is also possible to run it without interaction, in which case
it will pass all its parameters to Meson directly, with added benefit of
dependency tracking and producing minimal builds if desired.
Reconfiguring existing build directory is also supported, in which case
existing configuration for the flags managed by the script will be kept
updated (for other options we can rely on the fact that Meson keeps track
of all of the specified options and thus we don't have to re-specify them
when we reconfigure).

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    v2 -> v1:
    - Moved to devtools
    - Added a --dry-run mode
    - Improved support for reconfiguring existing directory
    - Some refactoring and code improvements
    - Different menus now display different prompts
    - Added documentation

 devtools/dpdk-setup.py              | 784 ++++++++++++++++++++++++++++
 doc/guides/linux_gsg/build_dpdk.rst |  26 +
 2 files changed, 810 insertions(+)
 create mode 100755 devtools/dpdk-setup.py

diff --git a/devtools/dpdk-setup.py b/devtools/dpdk-setup.py
new file mode 100755
index 0000000000..b0e28f3d36
--- /dev/null
+++ b/devtools/dpdk-setup.py
@@ -0,0 +1,784 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Intel Corporation
+
+"""
+Displays an interactive TUI-based menu for configuring a DPDK build directory.
+"""
+
+# This is an interactive script that allows the user to configure a DPDK build directory using a
+# text-based user interface (TUI). The script will prompt the user to select various configuration
+# options, and will then call `meson setup|configure` to configure the build directory with the
+# selected options.
+#
+# To be more user-friendly, the script will also run `meson setup` into a temporary directory in
+# the background, which will generate both the list of available options, and any dependencies
+# between them, so whenever the user selects an option, we automatically enable its dependencies.
+# This will also allow us to use meson introspection to get list of things we are capable of
+# building, and warn the user if they selected something that can't be built.
+
+import argparse
+import collections
+import fnmatch
+import json
+import os
+import subprocess
+import sys
+import textwrap
+import typing as T
+from tempfile import TemporaryDirectory
+
+
+# some apps have different names in the Meson build system
+APP_RENAME_MAP = {
+    "testpmd": "test-pmd",
+}
+
+
+# cut off dpdk- prefix
+def _unprefix_app(app: str) -> str:
+    return app[5:]
+
+
+def _prefix_app(app: str) -> str:
+    return f"dpdk-{app}"
+
+
+def _slash_driver(driver: str) -> str:
+    return driver.replace("/", "_", 1)
+
+
+def _unslash_driver(driver: str) -> str:
+    return driver.replace("_", "/", 1)
+
+
+def create_meson_build(src_dir: str, build_dir: str) -> subprocess.Popen[bytes]:
+    """Create a Meson build directory in the background."""
+    # we want all examples
+    args = ["meson", "setup", build_dir, "-Dexamples=all"]
+    return subprocess.Popen(
+        args, cwd=src_dir, stdout=subprocess.PIPE, stderr=subprocess.PIPE
+    )
+
+
+def wrap_text(message: str, cols: int) -> T.Tuple[int, int, str]:
+    """Wrap text to N columns and calculate resulting dimensions."""
+    wrapped_lines = textwrap.wrap(message.strip(), cols)
+    h = len(wrapped_lines)
+    w = max(len(line) for line in wrapped_lines)
+    return h, w, "\n".join(wrapped_lines)
+
+
+def calc_opt_width(option: T.Any) -> int:
+    """Calculate the width of an option."""
+    if isinstance(option, str):
+        return len(option)
+    return sum(calc_opt_width(opt) for opt in option) + len(option)  # padding
+
+
+def calc_list_width(options: T.List[T.Any], checkbox: bool) -> int:
+    """Calculate the width of a list."""
+    pad = 5
+    # add 4 for the checkbox
+    if checkbox:
+        pad += 4
+    return max(calc_opt_width(opt) for opt in options) + pad
+
+
+def whiptail_msgbox(message: str) -> None:
+    """Display a message box."""
+    # set max width to 60
+    h, w, message = wrap_text(message, 60)
+    # add some padding
+    w += 10
+    h += 6
+    args = ["whiptail", "--msgbox", message, str(h), str(w)]
+    subprocess.run(args, check=True)
+
+
+def whiptail_checklist(
+    title: str, prompt: str, options: T.List[T.Tuple[str, str]], checked: T.List[str]
+) -> T.List[str]:
+    """Display a checklist and get user input."""
+    # at least two free spaces, but no more than 10 in total
+    lh = min(len(options) + 2, 10)
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # width was set to prompt width, but we need to account for the list
+    lw = calc_list_width(options, True)
+    # adjust width to account for list width as well
+    w = max(w, lw)
+    # add some padding and list height
+    w += 10
+    h += 6 + lh
+
+    # build whiptail checklist
+    checklist = [
+        (label, desc, "on" if label in checked else "off") for label, desc in options
+    ]
+    # flatten the list
+    flat = [item for tup in checklist for item in tup]
+    # build whiptail arguments
+    args = [
+        "whiptail",
+        "--notags",
+        "--separate-output",
+        "--title",
+        title,
+        "--checklist",
+        prompt,
+        str(h),
+        str(w),
+        str(lh),
+    ] + flat
+
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    # capture selected options
+    return result.stderr.decode().strip().split()
+
+
+def whiptail_menu(title: str, prompt: str, options: T.List[T.Tuple[str, str]]) -> str:
+    """Display a menu and get user input."""
+    # at least two free spaces, but no more than 10 in total
+    lh = min(len(options) + 2, 10)
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # width was set to prompt width, but we need to account for the list
+    lw = calc_list_width(options, False)
+    # adjust width to account for list width as well
+    w = max(w, lw)
+    # add some padding
+    w += 10
+    h += 6 + lh
+    # flatten the list
+    flat = [item for tup in options for item in tup]
+    args = [
+        "whiptail",
+        "--notags",
+        "--title",
+        title,
+        "--menu",
+        prompt,
+        str(h),
+        str(w),
+        str(lh),
+    ] + flat
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    return result.stderr.decode().strip()
+
+
+def whiptail_inputbox(title: str, prompt: str, default: str = "") -> str:
+    """Display an input box and get user input."""
+    # set max width to 60
+    h, w, prompt = wrap_text(prompt, 60)
+    # add some padding
+    w += 10
+    h += 6
+    args = ["whiptail", "--inputbox", "--title", title, prompt, str(h), str(w), default]
+    result = subprocess.run(args, stderr=subprocess.PIPE, check=True)
+    return result.stderr.decode().strip()
+
+
+class DPDKBuildInfo:
+    """Encapsulate all information about a DPDK build directory."""
+
+    def __init__(self, build_dir: str) -> None:
+        self.build_dir = build_dir
+        # components that can be built according to meson's introspection
+        self.can_be_built: T.Set[str] = set()
+        # components that were read from dependency graph
+        self.required_deps: T.Dict[str, T.Set[str]] = {}
+        self.optional_deps: T.Dict[str, T.Set[str]] = {}
+        # separate component list into libs, drivers, apps, and examples
+        self.libs: T.Set[str] = set()
+        self.drivers: T.Set[str] = set()
+        self.apps: T.Set[str] = set()
+        self.examples: T.Set[str] = set()
+
+        # store all meson configuration options
+        self.meson_flags: T.Dict[str, T.Any] = {}
+
+        self._parse()
+
+    def _parse_dep_line(self, line: str) -> T.Tuple[str, T.Set[str], str, bool]:
+        """Parse digraph line into (component, {dependencies}, type, optional)."""
+        # extract attributes first
+        first, last = line.index("["), line.rindex("]")
+        edge_str, attr_str = line[:first], line[first + 1 : last]
+        # key=value, key=value, ...
+        attrs = {
+            key.strip('" '): value.strip('" ')
+            for attr_kv in attr_str.split(",")
+            for key, value in [attr_kv.strip().split("=", 1)]
+        }
+        # check if edge is defined as dotted line, meaning it's optional
+        optional = "dotted" in attrs.get("style", "")
+        try:
+            component_type = attrs["dpdk_componentType"]
+        except KeyError as _e:
+            raise ValueError(f"Error: missing component type: {line}") from _e
+
+        # now, extract component name and any of its dependencies
+        deps: T.Set[str] = set()
+        try:
+            component, deps_str = edge_str.strip('" ').split("->", 1)
+            component = component.strip().strip('" ')
+            deps_str = deps_str.strip().strip("{}")
+            deps = {d.strip('" ') for d in deps_str.split(",")}
+        except ValueError as _e:
+            component = edge_str.strip('" ')
+
+        return component, deps, component_type, optional
+
+    def _parse(self) -> None:
+        """Parse information from DPDK build directory."""
+
+        # first, read the dep graph
+        dep_graph_path = os.path.join(self.build_dir, "deps.dot")
+        with open(dep_graph_path, encoding="utf-8") as f:
+            for line in f:
+                # skip lines that aren't edges
+                if line.strip() == "digraph {" or line.strip() == "}":
+                    continue
+
+                component, deps, c_type, optional = self._parse_dep_line(line)
+
+                # record component type
+                type_to_set = {
+                    "lib": self.libs,
+                    "drivers": self.drivers,
+                    "app": self.apps,
+                    "examples": self.examples,
+                }
+                type_to_set[c_type].add(component)
+
+                # store dependencies
+                if optional:
+                    self.optional_deps[component] = deps
+                else:
+                    self.required_deps[component] = deps
+
+        # now, use Meson introspection to read the list of components that can be built
+        args = ["meson", "introspect", "--targets"]
+        output = subprocess.check_output(args, cwd=self.build_dir, encoding="utf-8")
+        # parse output as JSON
+        introspected_targets = json.loads(output)
+
+        # we want to filter out certain things from the introspection output
+        def _filter_target(target: T.Dict[str, T.Any]) -> bool:
+            t_name: str = target["name"]
+            t_type: str = target["type"]
+
+            # if target is a library, we only want those that start with "rte_"
+            if t_type in ["static library", "shared library"]:
+                return t_name.startswith("rte_")
+
+            # if target is an executable, we only want those that start with "dpdk-"
+            if t_type == "executable":
+                return t_name.startswith("dpdk-")
+
+            return False
+
+        for target in filter(_filter_target, introspected_targets):
+            t_name: str = target["name"]
+            t_type: str = target["type"]
+
+            # for libraries, cut off rte_ prefix
+            if t_type in ["static library", "shared library"]:
+                t_name = t_name[4:]
+
+            # there may be duplicate targets because of shared/static libraries
+            if t_name in self.can_be_built:
+                continue
+
+            self.can_be_built.add(t_name)
+
+        # now, use Meson introspection to read build options and their values
+        args = ["meson", "introspect", "--buildoptions"]
+        output = subprocess.check_output(args, cwd=self.build_dir, encoding="utf-8")
+        # parse output as JSON
+        introspected_options = json.loads(output)
+
+        # populate available options values from introspection
+        for option in introspected_options:
+            name = option["name"]
+            value = option["value"]
+
+            self.meson_flags[name] = value
+
+
+class SetupCtx:
+    """POD class to hold context for the setup script."""
+
+    def __init__(self) -> None:
+        self.complete_dg: DPDKBuildInfo
+        # when reconfiguring existing directory, we want to pick up options from existing
+        # directory, but pick up everything else from the big dg
+        self.configure_dg: DPDKBuildInfo
+
+        # for delayed creation of dependency graph
+        self.tmp_build_dir: str
+        self.tmp_build_proc: subprocess.Popen[bytes]
+
+        self.use_ui = False
+        self.minimal = False
+        self.configure = False
+        self.dry_run = False
+        self.src_dir = ""
+        self.build_dir = ""
+
+        self.parsed_input = False
+
+        # what did user specify on the command-line?
+        self.enabled_apps_str = ""
+        self.enabled_drivers_str = ""
+        self.enabled_examples_str = ""
+        self.enabled_libs_str = ""
+        self.meson_args_str = ""
+
+        # what did we end up with after parsing user's input?
+        self.enabled_apps: T.List[str] = []
+        self.enabled_drivers: T.List[str] = []
+        self.enabled_examples: T.List[str] = []
+        self.enabled_libs: T.List[str] = []
+
+    def _create_meson_option_cmd(
+        self,
+        meson_option_cmd: str,
+        entries: T.Set[str],
+        rename_func: T.Optional[T.Callable[[str], str]] = None,
+    ) -> str:
+        """Create a Meson option command from a set of entries."""
+        opt_list = [
+            entry if rename_func is None else rename_func(entry)
+            for entry in sorted(entries)
+        ]
+        return f"-D{meson_option_cmd}={','.join(opt_list)}"
+
+    def _resolve_wildcard(
+        self,
+        components: T.Set[str],
+        pattern: str,
+        pattern_func: T.Optional[T.Callable[[str], str]] = None,
+    ) -> T.Set[str]:
+        """Match a pattern against a set of components."""
+        if not pattern:
+            return set()
+        if pattern_func is not None:
+            pattern = pattern_func(pattern)
+        # if this is not a wildcard, return component explicitly - that's what user requested
+        if "*" not in pattern:
+            return {pattern}
+        # this is a wildcard match, so use wildcard matching
+        match = {c for c in components if fnmatch.fnmatch(c, pattern)}
+        # filter out anything that isn't buildable
+        return match & self.complete_dg.can_be_built
+
+    def _parse_list(
+        self,
+        dst: T.List[str],
+        pattern: str,
+        rename_func: T.Optional[T.Callable[[str], str]],
+        src_set: T.Set[str],
+    ) -> None:
+        """Populate list from wildcard matches, optionally with rename on the fly."""
+        dst.clear()
+        res_lst = [
+            entry
+            for p in pattern.split(",")
+            for entry in self._resolve_wildcard(src_set, p, rename_func)
+        ]
+        dst.extend(res_lst)
+
+    def parse(self) -> None:
+        """Parse user input."""
+        # when parsing user input, we expect to see a list of components separated by commas, as
+        # well as maybe wildcards. We will expand wildcards into a list of components, but by
+        # default we won't enable anything that can't be built even if it matches wildcard. also,
+        # component named used by Meson user-facing code and component names used in the backend
+        # are not exactly the same. for example, apps and examples will not have "dpdk-" prefixes,
+        # while drivers will have underscores instead of slashes. we need to take all of that into
+        # account when matching user input to actual components.
+
+        enabled_apps_str = self.enabled_apps_str
+        enabled_examples_str = self.enabled_examples_str
+        enabled_drivers_str = self.enabled_drivers_str
+        enabled_libs_str = self.enabled_libs_str
+        if self.configure:
+            flags = self.configure_dg.meson_flags
+            # on configure, override existing build if user input is specified
+            enabled_apps_str = enabled_apps_str or flags["enable_apps"]
+            enabled_examples_str = enabled_examples_str or flags["examples"]
+            enabled_drivers_str = enabled_drivers_str or flags["enable_drivers"]
+            enabled_libs_str = enabled_libs_str or flags["enable_libs"]
+
+        # now, parse specified configuration
+        self._parse_list(
+            self.enabled_apps,
+            enabled_apps_str,
+            _prefix_app,
+            self.complete_dg.apps,
+        )
+        self._parse_list(
+            self.enabled_examples,
+            enabled_examples_str,
+            _prefix_app,
+            self.complete_dg.examples,
+        )
+        self._parse_list(
+            self.enabled_drivers,
+            enabled_drivers_str,
+            _slash_driver,
+            self.complete_dg.drivers,
+        )
+        self._parse_list(
+            self.enabled_libs, enabled_libs_str, None, self.complete_dg.libs
+        )
+        self.parsed_input = True
+
+    def create_meson_cmdline(self) -> T.List[str]:
+        """Dump all configuration into Meson command-line string."""
+        assert self.parsed_input, "parse() must be called before create_meson_cmdline()"
+
+        args: T.List[str] = []
+        enabled_apps: T.Set[str] = set()
+        enabled_drivers: T.Set[str] = set()
+        enabled_examples: T.Set[str] = set()
+        enabled_libs: T.Set[str] = set()
+
+        # gather everything
+        enabled_apps = set(self.enabled_apps)
+        enabled_drivers = set(self.enabled_drivers)
+        enabled_examples = set(self.enabled_examples)
+        enabled_libs = set(self.enabled_libs)
+
+        enabled_all = enabled_examples | enabled_apps | enabled_drivers | enabled_libs
+
+        # gather all dependencies
+        new_deps: T.Set[str] = set()
+        for component in enabled_all:
+            deps = self.complete_dg.required_deps[component]
+            new_deps.add(component)
+            # deps do not include complete list, so walk through all dependencies
+            dep_stack = collections.deque(deps)
+            while dep_stack:
+                dc = dep_stack.pop()
+                if dc in new_deps:
+                    continue
+                new_deps.add(dc)
+
+                # get dependencies for this dependency
+                deps = self.complete_dg.required_deps[dc]
+                # recurse deeper
+                dep_stack.extend(deps)
+
+        # extend all lists with new dependencies
+        enabled_apps |= new_deps & self.complete_dg.apps
+        enabled_drivers |= new_deps & self.complete_dg.drivers
+        enabled_examples |= new_deps & self.complete_dg.examples
+        enabled_libs |= new_deps & self.complete_dg.libs
+        enabled_all |= new_deps
+
+        # check if everything can be built
+        diff = enabled_all - self.complete_dg.can_be_built
+        if diff:
+            print(
+                f"Warning: {', '.join(diff)} requested but cannot be built",
+                file=sys.stderr,
+            )
+
+        # we've resolved all dependencies, time to dump it all out
+
+        if enabled_apps:
+            # special case: some apps are renamed
+            enabled_apps = {APP_RENAME_MAP.get(app, app) for app in enabled_apps}
+            args += [
+                self._create_meson_option_cmd(
+                    "enable_apps", enabled_apps, _unprefix_app
+                )
+            ]
+
+        if enabled_examples:
+            args += [
+                self._create_meson_option_cmd(
+                    "examples", enabled_examples, _unprefix_app
+                )
+            ]
+
+        if enabled_drivers:
+            args += [
+                self._create_meson_option_cmd(
+                    "enable_drivers", enabled_drivers, _slash_driver
+                )
+            ]
+
+        # if we have specified any other components, enabled_libs will not be empty. however, we
+        # only want to specify enabled libs if we want to have a minimal build. so, before only
+        # enabling libs we depend on, check if user actually wanted a minimal build.
+        if (self.minimal or self.enabled_libs) and enabled_libs:
+            args += [self._create_meson_option_cmd("enable_libs", enabled_libs)]
+
+        # if minimal build is enabled and tests are not, disable tests as well
+        if self.minimal and "dpdk-test" not in enabled_apps:
+            args.append("-Dtests=false")
+
+        # did user specify any extra Meson arguments?
+        if self.meson_args_str:
+            args += self.meson_args_str.split()
+
+        return args
+
+
+def select_items(
+    title: str,
+    prompt: str,
+    item_list: T.List[str],
+    rename_func: T.Optional[T.Callable[[str], str]],
+    checked_list: T.List[str],
+) -> None:
+    """Select items to enable."""
+    # create a dialog selection for items
+    options = [
+        (app, rename_func(app) if rename_func is not None else app)
+        for app in sorted(item_list)
+    ]
+
+    try:
+        selected = whiptail_checklist(
+            title,
+            prompt,
+            options,
+            checked_list,
+        )
+        checked_list.clear()
+        checked_list.extend(selected)
+    except subprocess.CalledProcessError:
+        # user pressed cancel, don't do anything
+        pass
+
+
+def main_menu(ctx: SetupCtx) -> None:
+    """Display main menu."""
+    while True:
+        options = {
+            "apps": "Select applications",
+            "examples": "Select examples",
+            "drivers": "Select drivers",
+            "libs": "Select libraries",
+            "meson": "Enter custom Meson options",
+            "exit": "Save & exit",
+        }
+        ret = whiptail_menu(
+            "Setup DPDK build directory",
+            "Select an option",
+            list(options.items()),
+        )
+
+        if ret not in ["meson", "exit"]:
+            # before we're able to use selection dialogs, we need to parse input
+            if not ctx.parsed_input:
+                # we need to wait for the background process to finish
+                print("Parsing dependency tree, please wait...")
+                ctx.tmp_build_proc.wait()
+                ctx.complete_dg = DPDKBuildInfo(ctx.tmp_build_dir)
+                ctx.parse()
+
+        # selector dialogs are pretty similar
+        if ret in ["apps", "examples", "libs", "drivers"]:
+            selection_screens: T.Dict[str, T.Any] = {
+                "apps": (
+                    "Applications",
+                    "Select applications to enable:",
+                    list(ctx.complete_dg.apps),
+                    _unprefix_app,
+                    ctx.enabled_apps,
+                ),
+                "examples": (
+                    "Examples",
+                    "Select example applications to enable:",
+                    list(ctx.complete_dg.examples),
+                    _unprefix_app,
+                    ctx.enabled_examples,
+                ),
+                "libs": (
+                    "Libraries",
+                    "Select libraries to enable:",
+                    list(ctx.complete_dg.libs),
+                    None,
+                    ctx.enabled_libs,
+                ),
+                "drivers": (
+                    "Drivers",
+                    "Select drivers to enable:",
+                    list(ctx.complete_dg.drivers),
+                    _unslash_driver,
+                    ctx.enabled_drivers,
+                ),
+            }
+            try:
+                t, p, il, rf, el = selection_screens[ret]
+                select_items(t, p, il, rf, el)
+
+                # did user select something that cannot be built?
+                diff = set(el) - ctx.complete_dg.can_be_built
+                if diff:
+                    comp_str = ", ".join(diff)
+                    whiptail_msgbox(
+                        f"Warning: selected component(s) {comp_str} cannot be built."
+                    )
+            except subprocess.CalledProcessError:
+                # user pressed cancel, don't do anything
+                pass
+        elif ret == "meson":
+            try:
+                ctx.meson_args_str = whiptail_inputbox(
+                    "Custom Meson options",
+                    "Enter custom options to pass to Meson setup:",
+                    ctx.meson_args_str,
+                )
+            except subprocess.CalledProcessError:
+                # user pressed cancel, don't do anything
+                pass
+        elif ret == "exit":
+            break
+
+
+def parse_args() -> SetupCtx:
+    """Parse command-line arguments and return a context."""
+    # find out where we are
+    self_path = os.path.abspath(__file__)
+    # go one level up to get to DPDK source directory
+    src_dir = os.path.dirname(os.path.dirname(self_path))
+
+    parser = argparse.ArgumentParser(description="Configure a DPDK build directory.")
+    parser.add_argument(
+        "--src-dir", "-S", default=src_dir, help="Path to the DPDK source directory."
+    )
+    parser.add_argument(
+        "--build-dir", "-B", default="build", help="Path to the DPDK build directory."
+    )
+    parser.add_argument(
+        "--no-ui",
+        action="store_true",
+        help="Disable the TUI and use command-line arguments directly.",
+    )
+    parser.add_argument(
+        "--minimal",
+        action="store_true",
+        help="Try to remove unneeded libraries from build.",
+    )
+    parser.add_argument(
+        "--configure",
+        action="store_true",
+        help="Reconfigure existing build directory instead of creating new one.",
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Print resulting Meson command-line arguments but do not run Meson.",
+    )
+    parser.add_argument(
+        "--apps",
+        "-a",
+        default="",
+        help="Comma-separated list of apps to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--drivers",
+        "-d",
+        default="",
+        help="Comma-separated list of drivers to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--examples",
+        "-e",
+        default="",
+        help="Comma-separated list of examples to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--libs",
+        "-l",
+        default="",
+        help="Comma-separated list of libraries to enable (wildcards are accepted).",
+    )
+    parser.add_argument(
+        "--meson-args", "-m", default="", help="Extra arguments to pass to Meson setup."
+    )
+    args = parser.parse_args()
+
+    ctx = SetupCtx()
+    ctx.build_dir = args.build_dir
+    ctx.src_dir = args.src_dir
+    ctx.use_ui = not args.no_ui
+    ctx.minimal = args.minimal
+    ctx.dry_run = args.dry_run
+    ctx.configure = args.configure
+    ctx.enabled_apps_str = args.apps
+    ctx.enabled_drivers_str = args.drivers
+    ctx.enabled_examples_str = args.examples
+    ctx.enabled_libs_str = args.libs
+    ctx.meson_args_str = args.meson_args
+
+    return ctx
+
+
+def _run_setup(ctx: SetupCtx) -> int:
+    # we want the big graph unconditionally
+    ctx.tmp_build_proc = create_meson_build(ctx.src_dir, ctx.tmp_build_dir)
+
+    # we want the small graph only if we're reconfiguring
+    if ctx.configure:
+        # build directory already created, so we can parse the graph directly
+        ctx.configure_dg = DPDKBuildInfo(ctx.build_dir)
+
+    # if we're not using the UI, parse input and exit
+    if not ctx.use_ui:
+        print("UI is disabled, using command-line arguments directly")
+        print("Parsing dependency tree...")
+        ctx.tmp_build_proc.wait()
+        ctx.complete_dg = DPDKBuildInfo(ctx.tmp_build_dir)
+        ctx.parse()
+    else:
+        # we're using menu-driven UI, so wait until user tells us to exit
+        try:
+            main_menu(ctx)
+        except subprocess.CalledProcessError:
+            # user pressed cancel, exit
+            print("Operation cancelled")
+            return 1
+
+    # user may not have selected anything, so graph may still be unparsed
+    if not ctx.parsed_input:
+        print("Parsing dependency tree...")
+        ctx.tmp_build_proc.wait()
+        ctx.complete_dg = DPDKBuildInfo(ctx.tmp_build_dir)
+        ctx.parse()
+
+    # run meson
+    meson_cmd = ["meson", "setup"] if not ctx.configure else ["meson", "configure"]
+    meson_cmdline = ctx.create_meson_cmdline()
+    run_args = [*meson_cmd, ctx.build_dir, *meson_cmdline]
+    print("The following command will be run:")
+    print(*run_args, sep=" ")
+    if ctx.dry_run:
+        return 0
+    runret = subprocess.run(run_args, check=False)
+    return runret.returncode
+
+
+def _main() -> int:
+    # parse command-line arguments
+    try:
+        ctx = parse_args()
+
+        with TemporaryDirectory() as tmp_build_dir:
+            ctx.tmp_build_dir = tmp_build_dir
+            return _run_setup(ctx)
+    # any uncaught CalledProcessError is from graph parser
+    except (OSError, ValueError, subprocess.CalledProcessError) as e:
+        print(f"Error: {e}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(_main())
diff --git a/doc/guides/linux_gsg/build_dpdk.rst b/doc/guides/linux_gsg/build_dpdk.rst
index 9c0dd9daf6..f67bd7401b 100644
--- a/doc/guides/linux_gsg/build_dpdk.rst
+++ b/doc/guides/linux_gsg/build_dpdk.rst
@@ -149,6 +149,32 @@ When `-Dexamples=all` is set as a meson option, meson will check each example ap
 and add all which can be built to the list of tasks in the ninja build configuration file.
 
 
+Text Interface for DPDK Build Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is also possible to use a Text User Interface (TUI) to configure the DPDK build. To run TUI setup script, the following
+command may be used::
+
+  devtools/dpdk-setup.py
+
+This will show a TUI dialog which will allow the user to pick which applications, drivers, example apps, and libraries they with to build.
+Additionally, there are command-line options for the script that can be useful, such as:
+
+* ``-B <build dir>`` - specify the build directory to use (defaults to `./build`)
+* ``-S <src dir>`` - specify the source directory to use (defaults to wherever the script is run from)
+* ``--no-ui`` - non-interactive mode, useful for automation
+* ``--minimal`` - attempt to produce the most minimal build possible (i.e. don't build unnecessary libraries)
+* ``--dry-run`` - show which Meson command will be run as a result, but do not run it
+* ``--configure`` - run the Meson configure instead of Meson setup (useful for existing build directories)
+* ``-a <app1,app2,...>`` - specify which applications to build
+* ``-e <example1,example2,...>`` - specify which example applications to build
+* ``-d <driver1,driver2,...>`` - specify which drivers to build
+* ``-l <lib1,lib2,...>`` - specify which libraries to build
+* ``--meson-args <args>`` - specify additional arguments to pass to Meson (e.g. debug build etc.)
+
+This script will also track all dependencies between components automatically. Note that command-line options for enabling drivers, apps, examples, and libraries
+are used as *default* selections, and the user will still be able to change them in the TUI dialog (unless ``--no-ui`` is specified).
+
 Building 32-bit DPDK on 64-bit Systems
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-- 
2.43.5