From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Subject: [PATCH v5 0/8] Record and rework component dependencies
Date: Wed, 27 Nov 2024 14:56:48 +0000 [thread overview]
Message-ID: <cover.1732719309.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1725462264.git.anatoly.burakov@intel.com>
As part of the meson build, we can record the dependencies for each
component as we process it, logging them to a file. This file can be
used as input to a number of other scripts and tools, for example, to
graph the dependencies, or to allow higher-level build-config tools to
automatically enable component requirements, etc.
The first patch of this set separates dependencies inside meson into
optional or mandatory. The second patch of this set generates the basic
dependency tree. The third patch does some processing of that dependency
tree to identify cases where dependencies are being unnecessarily
specified. Reducing these makes it easier to have readable dependency
graphs in future, without affecting the build.
The following 4 patches are based on the output of the second patch, and
greatly cut down the number of direct dependency links between
components. Even with the cut-down dependencies, the full dependency
graph is nigh-unreadable, so the final patch adds a new script to
generate dependency tree subgraphs, creating dot files for e.g. the
dependencies of a particular component, or a component class such as
mempool drivers.
v4 -> v5:
- Improve optional dependency handling by making it (mostly) automatic
- Fix CI issue from accidentally changing testpmd binary filename
- Explicitly encode display name vs binary name information in the graph
- Reduce some more dependencies
- Update all scripts to handle new graph output format
v3 -> v4:
- Update to latest main
v2 -> v3:
- Split dependencies into optional and mandatory
- Fixup graph scripts to read and generate graphs that encode optional
dependencies into the graph
- Python version fixes to avoid using features not available in minimum
supported Python version
- Formatting with Ruff, and PEP-484 compliance
Anatoly Burakov (1):
build: introduce optional internal dependencies
Bruce Richardson (7):
build: output a dependency log in build directory
devtools: add script to flag unneeded dependencies
build: remove kvargs from driver class dependencies
build: reduce library dependencies
build: reduce driver dependencies
build: reduce app dependencies
devtools: add script to generate DPDK dependency graphs
app/dumpcap/meson.build | 2 +-
app/graph/meson.build | 2 +-
app/meson.build | 31 ++-
app/pdump/meson.build | 2 +-
app/proc-info/meson.build | 6 +-
app/test-bbdev/meson.build | 18 +-
app/test-crypto-perf/meson.build | 6 +-
app/test-fib/meson.build | 2 +-
app/test-pmd/meson.build | 56 ++----
app/test-sad/meson.build | 2 +-
app/test/meson.build | 14 +-
buildtools/log-deps.py | 94 +++++++++
buildtools/meson.build | 2 +
devtools/draw-dependency-graphs.py | 223 +++++++++++++++++++++
devtools/find-duplicate-deps.py | 62 ++++++
drivers/baseband/fpga_5gnr_fec/meson.build | 2 +-
drivers/baseband/fpga_lte_fec/meson.build | 2 +-
drivers/baseband/la12xx/meson.build | 2 +-
drivers/baseband/null/meson.build | 2 +-
drivers/baseband/turbo_sw/meson.build | 2 +-
drivers/bus/auxiliary/meson.build | 2 -
drivers/bus/dpaa/meson.build | 2 +-
drivers/bus/fslmc/meson.build | 2 +-
drivers/bus/ifpga/meson.build | 2 +-
drivers/bus/pci/meson.build | 4 +-
drivers/bus/platform/meson.build | 1 -
drivers/bus/uacce/meson.build | 2 -
drivers/bus/vdev/meson.build | 2 -
drivers/common/cnxk/meson.build | 4 +-
drivers/common/cpt/meson.build | 2 +-
drivers/common/idpf/meson.build | 2 +-
drivers/common/mlx5/meson.build | 2 +-
drivers/common/qat/meson.build | 3 +-
drivers/compress/mlx5/meson.build | 2 +-
drivers/compress/nitrox/meson.build | 2 +-
drivers/compress/octeontx/meson.build | 2 +-
drivers/crypto/bcmfs/meson.build | 2 +-
drivers/crypto/cnxk/meson.build | 2 +-
drivers/crypto/dpaa_sec/meson.build | 2 +-
drivers/crypto/ipsec_mb/meson.build | 2 +-
drivers/crypto/mlx5/meson.build | 2 +-
drivers/crypto/nitrox/meson.build | 2 +-
drivers/dma/cnxk/meson.build | 2 +-
drivers/dma/dpaa/meson.build | 2 +-
drivers/dma/dpaa2/meson.build | 2 +-
drivers/dma/odm/meson.build | 2 +-
drivers/dma/skeleton/meson.build | 2 +-
drivers/event/cnxk/meson.build | 2 +-
drivers/event/dlb2/meson.build | 2 +-
drivers/event/dpaa2/meson.build | 2 +-
drivers/event/meson.build | 2 +-
drivers/event/octeontx/meson.build | 3 +-
drivers/event/sw/meson.build | 2 +-
drivers/mempool/cnxk/meson.build | 2 +-
drivers/mempool/dpaa/meson.build | 2 +-
drivers/mempool/dpaa2/meson.build | 2 +-
drivers/mempool/octeontx/meson.build | 2 +-
drivers/meson.build | 27 ++-
drivers/ml/cnxk/meson.build | 2 +-
drivers/net/cnxk/meson.build | 4 +-
drivers/net/iavf/meson.build | 2 +-
drivers/net/ice/meson.build | 2 +-
drivers/net/mana/meson.build | 2 +-
drivers/net/meson.build | 2 +-
drivers/net/mlx5/meson.build | 2 +-
drivers/net/sfc/meson.build | 2 +-
drivers/net/softnic/meson.build | 2 +-
drivers/net/virtio/meson.build | 2 +-
drivers/raw/cnxk_bphy/meson.build | 2 +-
drivers/raw/cnxk_gpio/meson.build | 2 +-
drivers/raw/gdtc/meson.build | 2 +-
drivers/raw/ntb/meson.build | 2 +-
drivers/raw/skeleton/meson.build | 2 +-
drivers/regex/cn9k/meson.build | 2 +-
drivers/regex/meson.build | 2 +-
drivers/regex/mlx5/meson.build | 2 +-
drivers/vdpa/ifc/meson.build | 2 +-
drivers/vdpa/meson.build | 3 +-
drivers/vdpa/mlx5/meson.build | 2 +-
drivers/vdpa/sfc/meson.build | 2 +-
examples/ethtool/meson.build | 4 +-
examples/l2fwd-crypto/meson.build | 4 +-
examples/l3fwd/meson.build | 4 +-
examples/meson.build | 31 ++-
examples/vm_power_manager/meson.build | 16 +-
lib/argparse/meson.build | 2 +-
lib/bbdev/meson.build | 2 +-
lib/bitratestats/meson.build | 2 +-
lib/bpf/meson.build | 2 +-
lib/cmdline/meson.build | 2 +-
lib/compressdev/meson.build | 2 +-
lib/cryptodev/meson.build | 2 +-
lib/dispatcher/meson.build | 2 +-
lib/distributor/meson.build | 2 +-
lib/dmadev/meson.build | 2 -
lib/eal/meson.build | 5 +-
lib/efd/meson.build | 2 +-
lib/ethdev/meson.build | 2 +-
lib/eventdev/meson.build | 3 +-
lib/fib/meson.build | 4 +-
lib/gpudev/meson.build | 2 +-
lib/graph/meson.build | 2 +-
lib/gro/meson.build | 2 +-
lib/gso/meson.build | 2 +-
lib/hash/meson.build | 4 +-
lib/ip_frag/meson.build | 2 +-
lib/ipsec/meson.build | 2 +-
lib/kvargs/meson.build | 2 +-
lib/latencystats/meson.build | 2 +-
lib/lpm/meson.build | 4 +-
lib/mbuf/meson.build | 2 +-
lib/member/meson.build | 2 +-
lib/mempool/meson.build | 2 +-
lib/meson.build | 27 ++-
lib/metrics/meson.build | 2 +-
lib/mldev/meson.build | 2 +-
lib/net/meson.build | 2 +-
lib/node/meson.build | 2 +-
lib/pcapng/meson.build | 2 +-
lib/pdcp/meson.build | 2 +-
lib/pdump/meson.build | 2 +-
lib/pipeline/meson.build | 2 +-
lib/port/meson.build | 2 +-
lib/power/meson.build | 3 +-
lib/rawdev/meson.build | 2 -
lib/rcu/meson.build | 2 +-
lib/regexdev/meson.build | 2 +-
lib/reorder/meson.build | 2 +-
lib/rib/meson.build | 2 +-
lib/ring/meson.build | 1 -
lib/sched/meson.build | 2 +-
lib/security/meson.build | 2 +-
lib/table/meson.build | 2 +-
lib/telemetry/meson.build | 2 +-
lib/vhost/meson.build | 2 +-
135 files changed, 647 insertions(+), 232 deletions(-)
create mode 100644 buildtools/log-deps.py
create mode 100755 devtools/draw-dependency-graphs.py
create mode 100755 devtools/find-duplicate-deps.py
--
2.43.5
next prev parent reply other threads:[~2024-11-27 14:57 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-07-30 14:55 [PATCH] build: output a dependency log in build directory Bruce Richardson
2024-07-31 9:07 ` Konstantin Ananyev
2024-07-31 10:17 ` Ferruh Yigit
2024-07-31 10:27 ` Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 0/7] record and rework component dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 1/7] build: output a dependency log in build directory Bruce Richardson
2024-09-02 14:34 ` Burakov, Anatoly
2024-09-03 8:31 ` Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 2/7] devtools: add script to flag unneeded dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 3/7] build: remove kvargs from driver class dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 4/7] build: reduce library dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 5/7] build: reduce driver dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 6/7] build: reduce app dependencies Bruce Richardson
2024-08-02 12:44 ` [PATCH v2 7/7] devtools: add script to generate DPDK dependency graphs Bruce Richardson
2024-08-02 13:29 ` [PATCH v2 0/7] record and rework component dependencies Morten Brørup
2024-08-02 15:05 ` Patrick Robb
2024-08-02 15:11 ` Bruce Richardson
2024-08-02 17:18 ` Ferruh Yigit
2024-08-06 8:35 ` Bruce Richardson
2024-09-04 15:08 ` [PATCH v3 0/8] " Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 1/8] build: split dependencies into mandatory and optional Anatoly Burakov
2024-09-06 14:51 ` Bruce Richardson
2024-09-09 8:41 ` Burakov, Anatoly
2024-09-09 9:01 ` Bruce Richardson
2024-09-04 15:08 ` [PATCH v3 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 3/8] devtools: add script to flag unneeded dependencies Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 5/8] build: reduce library dependencies Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 6/8] build: reduce driver dependencies Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 7/8] build: reduce app dependencies Anatoly Burakov
2024-09-04 15:08 ` [PATCH v3 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov
2024-09-05 6:05 ` [PATCH v3 0/8] record and rework component dependencies Morten Brørup
2024-11-26 14:39 ` [PATCH v4 0/8] Record " Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 1/8] build: split dependencies into mandatory and optional Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 3/8] devtools: add script to flag unneeded dependencies Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 5/8] build: reduce library dependencies Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 6/8] build: reduce driver dependencies Anatoly Burakov
2024-11-27 2:17 ` Xu, Rosen
2024-11-26 14:39 ` [PATCH v4 7/8] build: reduce app dependencies Anatoly Burakov
2024-11-26 14:39 ` [PATCH v4 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov
2024-11-26 15:11 ` [PATCH v4 0/8] Record and rework component dependencies Burakov, Anatoly
2024-11-27 14:56 ` Anatoly Burakov [this message]
2024-11-27 14:56 ` [PATCH v5 1/8] build: introduce optional internal dependencies Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 2/8] build: output a dependency log in build directory Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 3/8] devtools: add script to flag unneeded dependencies Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 4/8] build: remove kvargs from driver class dependencies Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 5/8] build: reduce library dependencies Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 6/8] build: reduce driver dependencies Anatoly Burakov
2024-11-27 16:19 ` Burakov, Anatoly
2024-11-27 14:56 ` [PATCH v5 7/8] build: reduce app dependencies Anatoly Burakov
2024-11-27 14:56 ` [PATCH v5 8/8] devtools: add script to generate DPDK dependency graphs Anatoly Burakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1732719309.git.anatoly.burakov@intel.com \
--to=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).