Re: [PATCH v5 0/4] add support for self monitoring

[Bruce Richardson <bruce.richardson@intel.com>]

On Wed, Jan 11, 2023 at 02:24:28PM +0000, Tomasz Duszynski wrote:
> 
> 
> >-----Original Message-----
> >From: Morten Brørup <mb@smartsharesystems.com>
> >Sent: Wednesday, January 11, 2023 10:31 AM
> >To: Tyler Retzlaff <roretzla@linux.microsoft.com>; Tomasz Duszynski <tduszynski@marvell.com>;
> >bruce.richardson@intel.com
> >Cc: dev@dpdk.org; thomas@monjalon.net; Jerin Jacob Kollanukkaran <jerinj@marvell.com>;
> >Ruifeng.Wang@arm.com; mattias.ronnblom@ericsson.com; zhoumin@loongson.cn
> >Subject: [EXT] RE: [PATCH v5 0/4] add support for self monitoring
> >
> >External Email
> >
> >----------------------------------------------------------------------
> >> From: Tyler Retzlaff [mailto:roretzla@linux.microsoft.com]
> >> Sent: Wednesday, 11 January 2023 01.32
> >>
> >> On Wed, Jan 11, 2023 at 12:46:38AM +0100, Tomasz Duszynski wrote:
> >> > This series adds self monitoring support i.e allows to configure and
> >> > read performance measurement unit (PMU) counters in runtime without
> >> > using perf utility. This has certain adventages when application
> >> > runs
> >> on
> >> > isolated cores with nohz_full kernel parameter.
> >> >
> >> > Events can be read directly using rte_pmu_read() or using dedicated
> >> > tracepoint rte_eal_trace_pmu_read(). The latter will cause events to
> >> be
> >> > stored inside CTF file.
> >> >
> >> > By design, all enabled events are grouped together and the same
> >> > group is attached to lcores that use self monitoring funtionality.
> >> >
> >> > Events are enabled by names, which need to be read from standard
> >> > location under sysfs i.e
> >> >
> >> > /sys/bus/event_source/devices/PMU/events
> >> >
> >> > where PMU is a core pmu i.e one measuring cpu events. As of today
> >> > raw events are not supported.
> >> >
> >> > v5:
> >> > - address review comments
> >> > - fix sign extension while reading pmu on x86
> >> > - fix regex mentioned in doc
> >> > - various minor changes/improvements here and there
> >> > v4:
> >> > - fix freeing mem detected by debug_autotest
> >> > v3:
> >> > - fix shared build
> >> > v2:
> >> > - fix problems reported by test build infra
> >> >
> >> > Tomasz Duszynski (4):
> >> >   eal: add generic support for reading PMU events
> >> >   eal/arm: support reading ARM PMU events in runtime
> >> >   eal/x86: support reading Intel PMU events in runtime
> >> >   eal: add PMU support to tracing library
> >> >
> >> >  app/test/meson.build                     |   1 +
> >> >  app/test/test_pmu.c                      |  47 +++
> >> >  app/test/test_trace_perf.c               |   4 +
> >> >  doc/guides/prog_guide/profile_app.rst    |  13 +
> >> >  doc/guides/prog_guide/trace_lib.rst      |  32 ++
> >> >  lib/eal/arm/include/meson.build          |   1 +
> >> >  lib/eal/arm/include/rte_pmu_pmc.h        |  39 ++
> >> >  lib/eal/arm/meson.build                  |   4 +
> >> >  lib/eal/arm/rte_pmu.c                    | 104 +++++
> >> >  lib/eal/common/eal_common_trace_points.c |   3 +
> >> >  lib/eal/common/meson.build               |   3 +
> >> >  lib/eal/common/pmu_private.h             |  41 ++
> >> >  lib/eal/common/rte_pmu.c                 | 504
> >> +++++++++++++++++++++++
> >> >  lib/eal/include/meson.build              |   1 +
> >> >  lib/eal/include/rte_eal_trace.h          |  10 +
> >> >  lib/eal/include/rte_pmu.h                | 202 +++++++++
> >> >  lib/eal/linux/eal.c                      |   4 +
> >> >  lib/eal/version.map                      |   7 +
> >> >  lib/eal/x86/include/meson.build          |   1 +
> >> >  lib/eal/x86/include/rte_pmu_pmc.h        |  33 ++
> >> >  20 files changed, 1054 insertions(+)  create mode 100644
> >> > app/test/test_pmu.c  create mode 100644
> >> > lib/eal/arm/include/rte_pmu_pmc.h  create mode 100644
> >> > lib/eal/arm/rte_pmu.c  create mode 100644
> >> > lib/eal/common/pmu_private.h  create mode 100644
> >> > lib/eal/common/rte_pmu.c  create mode 100644
> >> > lib/eal/include/rte_pmu.h  create mode 100644
> >> > lib/eal/x86/include/rte_pmu_pmc.h
> >> >
> >> > --
> >> > 2.34.1
> >
> >[Moved Tyler's post down here.]
> >
> >>
> >> hi,
> >>
> >> don't interpret this as an objection to the functionality but this
> >> looks like a clear example of something that doesn't belong in the
> >> EAL. has there been a discussion as to whether or not this should be
> >> in a separate library?
> >
> >IIRC, there has been no such discussion.
> >
> >Although I agree that this doesn't belong in EAL, I would point to the trace library as a reference
> >for allowing it into the EAL.
> >
> >For the records, I also oppose to the trace library being part of the EAL.
> >
> >On the other hand, it would be interesting to determine if it is *impossible* adding this
> >functionality as any other normal DPDK library, i.e. outside of the EAL, or if there is an
> >unavoidable tie-in to the EAL.
> >
> >@Tomasz, if this is impossible, please describe the unavoidable tie-in to the EAL. No need for a
> >long report, just a few words. You (and this functionality) shouldn't suffer from our long term
> >ambition to move stuff out of the EAL.
> >
> 
> You can read about rationale here https://lore.kernel.org/dpdk-dev/DM4PR18MB436872EBC5922084C5DAFC1DD2FC9@DM4PR18MB4368.namprd18.prod.outlook.com/#t
> 
> As for the NO-NO there isn't any in fact. There are some tradeoffs though. 
> 
> For example, seems eal cannot depend on other libs so if someone needs to
> finetune some part of EAL for whatever reason, then relevant part needs to 
> modified each and every time. I.e specific includes and trcepoints need to be added each time.
>
Well, EAL can depend on other libs, but then those libs cannot in turn
directly depend upon DPDK. This is where breaking out first some of the
smaller widely used parts of DPDK  e.g. logging, would be good, as it would
then in turn allow other, potentially bigger parts of EAL to be taken out.

See [1] for a rough first attempt at this, which allows simlification of
telemetry as it no longer needs a "dependency injection" style to have
logging. Moving out logging would also allow logging from kvargs library
too - another lib which is used by EAL rather than depending on it.
Similarly for tracing functionality - if that were pulled out of EAL, it
could be used by telemetry, kvargs and any other parts removed from EAL.

/Bruce

[1] http://patches.dpdk.org/project/dpdk/list/?series=24453