DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Subject: [PATCH v5 3/8] devtools: add script to flag unneeded dependencies
Date: Wed, 27 Nov 2024 14:56:51 +0000	[thread overview]
Message-ID: <fd7718804f6c9d8da8d936fd14c8635c6babc063.1732719309.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1732719309.git.anatoly.burakov@intel.com>

From: Bruce Richardson <bruce.richardson@intel.com>

While not a serious problem, DPDK components often list more
dependencies than are actually necessary to build, due to the use of
recursive dependencies. In extreme cases, such as with core libraries,
this can lead to longer configuration times due to meson having to
deduplicate long lists of dependencies. Therefore we can add a script to
identify when a component has got unnecessary dependencies listed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 devtools/find-duplicate-deps.py | 62 +++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100755 devtools/find-duplicate-deps.py

diff --git a/devtools/find-duplicate-deps.py b/devtools/find-duplicate-deps.py
new file mode 100755
index 0000000000..cbd0ca7196
--- /dev/null
+++ b/devtools/find-duplicate-deps.py
@@ -0,0 +1,62 @@
+#! /usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Intel Corporation
+
+"""Identify any superfluous dependencies listed in DPDK deps graph."""
+
+import sys
+
+all_deps = {}
+
+
+class dep:
+    """Holds a component and its dependencies."""
+
+    def __init__(self, name, dep_names):
+        """Create and process a component and its deps."""
+        self.name = name.strip('" ')
+        self.base_deps = [all_deps[dn.strip('" ')] for dn in dep_names]
+        self.recursive_deps = []
+        for d in self.base_deps:
+            self.recursive_deps.extend(d.base_deps)
+            self.recursive_deps.extend(d.recursive_deps)
+        self.extra_deps = []
+        for d in self.base_deps:
+            if d in self.recursive_deps:
+                self.extra_deps.append(d.name)
+        if self.extra_deps:
+            print(f'{self.name}: extra deps {self.extra_deps}')
+
+    def dict_add(self, d):
+        """Add this object to a dictionary by name."""
+        d[self.name] = self
+
+
+def remove_attrs(ln):
+    """Remove attributes from a line."""
+    while ln.find("[") != -1:
+        start = ln.find("[")
+        end = ln.find("]")
+        ln = ln[:start] + ln[end + 1 :]
+    return ln.strip()
+
+
+def main(argv):
+    """Read the dependency tree from a dot file and process it."""
+    if len(argv) != 2:
+        print(f'Usage: {argv[0]} <build-directory>/deps.dot', file=sys.stderr)
+        sys.exit(1)
+
+    with open(argv[1]) as f:
+        for ln in f.readlines():
+            ln = remove_attrs(ln.strip())
+            if '->' in ln:
+                name, deps = ln.split('->')
+                deps = deps.strip(' {}')
+                dep(name, deps.split(',')).dict_add(all_deps)
+            elif ln.startswith('"') and ln.endswith('"'):
+                dep(ln.strip('"'), []).dict_add(all_deps)
+
+
+if __name__ == '__main__':
+    main(sys.argv)
-- 
2.43.5


  parent reply	other threads:[~2024-11-27 14:57 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-30 14:55 [PATCH] build: output a dependency log in build directory Bruce Richardson
2024-07-31  9:07 ` Konstantin Ananyev
2024-07-31 10:17 ` Ferruh Yigit
2024-07-31 10:27   ` Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 0/7] record and rework component dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 1/7] build: output a dependency log in build directory Bruce Richardson
2024-09-02 14:34     ` Burakov, Anatoly
2024-09-03  8:31       ` Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 2/7] devtools: add script to flag unneeded dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 3/7] build: remove kvargs from driver class dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 4/7] build: reduce library dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 5/7] build: reduce driver dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 6/7] build: reduce app dependencies Bruce Richardson
2024-08-02 12:44   ` [PATCH v2 7/7] devtools: add script to generate DPDK dependency graphs Bruce Richardson
2024-08-02 13:29   ` [PATCH v2 0/7] record and rework component dependencies Morten Brørup
2024-08-02 15:05   ` Patrick Robb
2024-08-02 15:11     ` Bruce Richardson
2024-08-02 17:18   ` Ferruh Yigit
2024-08-06  8:35     ` Bruce Richardson
2024-09-04 15:08 ` [PATCH v3 0/8] " Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 1/8] build: split dependencies into mandatory and optional Anatoly Burakov
2024-09-06 14:51     ` Bruce Richardson
2024-09-09  8:41       ` Burakov, Anatoly
2024-09-09  9:01         ` Bruce Richardson
2024-09-04 15:08   ` [PATCH v3 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 3/8] devtools: add script to flag unneeded dependencies Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 5/8] build: reduce library dependencies Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 6/8] build: reduce driver dependencies Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 7/8] build: reduce app dependencies Anatoly Burakov
2024-09-04 15:08   ` [PATCH v3 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov
2024-09-05  6:05   ` [PATCH v3 0/8] record and rework component dependencies Morten Brørup
2024-11-26 14:39   ` [PATCH v4 0/8] Record " Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 1/8] build: split dependencies into mandatory and optional Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 3/8] devtools: add script to flag unneeded dependencies Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 5/8] build: reduce library dependencies Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 6/8] build: reduce driver dependencies Anatoly Burakov
2024-11-27  2:17       ` Xu, Rosen
2024-11-26 14:39     ` [PATCH v4 7/8] build: reduce app dependencies Anatoly Burakov
2024-11-26 14:39     ` [PATCH v4 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov
2024-11-26 15:11     ` [PATCH v4 0/8] Record and rework component dependencies Burakov, Anatoly
2024-11-27 14:56   ` [PATCH v5 " Anatoly Burakov
2024-11-27 14:56     ` [PATCH v5 1/8] build: introduce optional internal dependencies Anatoly Burakov
2024-11-27 14:56     ` [PATCH v5 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-11-27 14:56     ` Anatoly Burakov [this message]
2024-11-27 14:56     ` [PATCH v5 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-11-27 14:56     ` [PATCH v5 5/8] build: reduce library dependencies Anatoly Burakov
2024-11-27 14:56     ` [PATCH v5 6/8] build: reduce driver dependencies Anatoly Burakov
2024-11-27 16:19       ` Burakov, Anatoly
2024-11-27 14:56     ` [PATCH v5 7/8] build: reduce app dependencies Anatoly Burakov
2024-11-27 14:56     ` [PATCH v5 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fd7718804f6c9d8da8d936fd14c8635c6babc063.1732719309.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).