From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DC142A0562; Fri, 3 Apr 2020 17:44:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2A3EF1C2DA; Fri, 3 Apr 2020 17:39:14 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 154C01C22B for ; Fri, 3 Apr 2020 17:39:11 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 033FaGHt029204; Fri, 3 Apr 2020 08:39:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=eDEr0ls55uAdbeg6+TQAC0HPzfAuCo1flhj1C0/Hwoo=; b=m5KJAUIKuxWzj8PSZYJ9ZBZbfq90n3URhYPJWSSshPGsynGv7arPOEIIqmnJdClxYZ1k lVm7+HMeh1IfpvlvXk4KrGY6YSJ3Jys3RKfN+cS2W8SCZ9AqzvR7mYP8n87mR2Nd+8Jo SrZt2t0NMfb4PI4qwq4R2TQbm1id799B+oJPEfiGUGNdXPzSmMshlhcYMMMIhVJl7FBI hhlZiOQqD7gHyzrYQ+7HZefZ3f1Px0I3Cj7JqEAL2fstCIVqQQBhCmibVMqBQP5qp0vg k7pjdbSL5ijlrjFGXm6yGXgSBzlfpyLAj9xPwR9DaqTZ/ORODAIVQOM1u7vG+BCA9Wu5 GA== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 304855y7mt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 03 Apr 2020 08:39:11 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 3 Apr 2020 08:39:09 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 3 Apr 2020 08:39:08 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 3 Apr 2020 08:39:08 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id A1DD33F705B; Fri, 3 Apr 2020 08:39:05 -0700 (PDT) From: To: Thomas Monjalon , John McNamara , Marko Kovacevic , "Jerin Jacob" , Sunil Kumar Kori CC: , , , Date: Fri, 3 Apr 2020 21:07:09 +0530 Message-ID: <20200403153709.3703448-34-jerinj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200403153709.3703448-1-jerinj@marvell.com> References: <20200329144342.1543749-1-jerinj@marvell.com> <20200403153709.3703448-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-03_11:2020-04-03, 2020-04-03 signatures=0 Subject: [dpdk-dev] [PATCH v4 33/33] doc: add trace library guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add programmar's guide for trace library support. Signed-off-by: Jerin Jacob Signed-off-by: Sunil Kumar Kori --- MAINTAINERS | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/trace_lib.rst | 339 +++++++++++++++++++++++++ doc/guides/rel_notes/release_20_05.rst | 9 + 4 files changed, 350 insertions(+) create mode 100644 doc/guides/prog_guide/trace_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 49a97e592..b15062270 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -200,6 +200,7 @@ M: Sunil Kumar Kori F: lib/librte_eal/include/rte_trace*.h F: lib/librte_eal/common/eal_common_trace*.c F: lib/librte_eal/common/eal_trace.h +F: doc/guides/prog_guide/trace_lib.rst Memory Allocation M: Anatoly Burakov diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf5..0daa08acc 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -35,6 +35,7 @@ Programmer's Guide lpm_lib lpm6_lib flow_classify_lib + trace_lib packet_distrib_lib reorder_lib ip_fragment_reassembly_lib diff --git a/doc/guides/prog_guide/trace_lib.rst b/doc/guides/prog_guide/trace_lib.rst new file mode 100644 index 000000000..186b986cd --- /dev/null +++ b/doc/guides/prog_guide/trace_lib.rst @@ -0,0 +1,339 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2020 Marvell International Ltd. + +Trace Library +============= + +Overview +-------- + +*Tracing* is a technique used to understand what goes on in a running software +system. The software used for tracing is called a *tracer*, which is conceptually +similar to a tape recorder. When recording, specific instrumentation points +placed in the software source code generate events that are saved on a giant +tape: a trace file. The trace file then later can be opened in *trace viewers* to +visualize and analyze the trace events with timestamps and multi-core views. +Such a mechanism will be useful for resolving a wide range of problems such as +multi-core synchronization issues, latency measurements, finding out the +post analyses information like CPU idle time, etc that would otherwise be +extremely challenging. + +Tracing is often compared to *logging*. However, tracers and loggers are two +different tools, serving two different purposes. Tracers are designed to record +much lower-level events that occur much more frequently than log messages, often +in the range of thousands per second, with very little execution overhead. +Logging is more appropriate for a very high-level analysis of less frequent +events: user accesses, exceptional conditions (errors and warnings, for +example), database transactions, instant messaging communications, and such. +Simply put, logging is one of the many use cases that can be satisfied with +tracing. + +DPDK tracing library features +----------------------------- + +- A framework to add tracepoints in control and fast APIs with minimum impact on + performance. Typical trace overhead is ~20 cycles and instrumentation + overhead is 1 cycle. +- Enable and disable the tracepoints at runtime. +- Save the trace buffer to the filesystem at any point in time. +- Supports ``overwrite`` and ``discard`` trace mode operations. +- String-based tracepoint object lookup. +- Enable and disable a set of tracepoints based on regular expression and/or + shell pattern. +- Generate trace in ``common trace format(CTF)``. ``CTF`` is an open-source trace + format and it is compatible with ``LTTng``. + For detailed information, refer `Common Trace Format `_ + +How to add a tracepoint? +------------------------ + +This section steps you through the details of adding a simple tracepoint. + +.. _create_provider_header_file: + +Create the tracepoint provider header file +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. code-block:: c + + #include + + RTE_TRACE_POINT(app_trace_string, + RTE_TRACE_POINT_ARGS(const char *str), + rte_trace_ctf_string(str); + ) + +The above macro creates ``app_trace_string`` tracepoint. The user can choose +any name for the tracepoint. However, when adding the tracepoint in the DPDK +library, the ``rte_trace_lib__[]_`` naming convention +must be followed. The examples are ``rte_trace_lib_eal_generic_str``, +``rte_trace_lib_mempool_create``. + +When ``RTE_TRACE_POINT`` expands for above the definition, it creates the +following function template. The consumer of this tracepoint can invoke +``app_trace_string(const char *str)`` to emit the trace event to the trace buffer. + +.. code-block:: c + + static __rte_always_inline void + app_trace_string(const char *str) + { + /* Trace subsystem hooks */ + ... + rte_trace_ctf_string(str); + } + +Register the tracepoint +~~~~~~~~~~~~~~~~~~~~~~~ +.. code-block:: c + + #define RTE_TRACE_POINT_REGISTER_SELECT /* Select trace point register macros */ + + #include + + RTE_TRACE_POINT_DEFINE(app_trace_string); + + RTE_INIT(eal_trace_init) + { + RTE_TRACE_POINT_REGISTER(app_trace_string, app.trace.string, INFO); + } + +The above code snippet registers the ``app_trace_string`` tracepoint to +trace library. Here, the ``tracepoint_provider_header_file.h`` is the header file +that user created in the first step :ref:`create_provider_header_file` + +The second argument for the ``RTE_TRACE_POINT_REGISTER`` is the name for the +tracepoint. This string will be used for tracepoint lookup or regular expression +and/or shell-pattern based tracepoint operations. There is no requirement for +the trace function and its name to be similar. However, it is recommended to +have a similar name for a better naming convention. + +The user must register the tracepoint before the ``rte_eal_init`` invocation. +The user can use the ``RTE_INIT`` construction scheme to achieve the same. + +.. note:: + + The ``RTE_TRACE_POINT_DEFINE`` defines the tracepoint of ``rte_trace_t`` + type. When the tracepoint defined in the shared library, the user must + update the ``.map`` file with ``__`` symbol. + For example, ``__app_trace_string`` will be the exported symbol in the + above example. + +Datapath tracepoint +------------------- + +In order to avoid performance impact for the datapath tracepoint, the library +introduced ``RTE_TRACE_POINT_DP``. When adding the tracepoint in datapath code, +The user must use ``RTE_TRACE_POINT_DP`` instead of ``RTE_TRACE_POINT``. + +``RTE_TRACE_POINT_DP`` is compiled out by default and it can be enabled using +``CONFIG_RTE_ENABLE_TRACE_DP`` configuration parameter. The ``enable_trace_dp`` +build option shall be used for the same for meson build .The application may use +``rte_trace_is_dp_enabled()`` to get status of ``CONFIG_RTE_ENABLE_TRACE_DP``. + +Event record mode +----------------- + +Event record mode is an attribute of trace buffers. Trace library exposes the +following modes: + +.. _table_event_record_modes: + +.. table:: Event record modes. + + +-----------+-----------------------------------------------------------------+ + | Mode | Description | + | | | + +===========+=================================================================+ + | Overwrite | When trace buffer is full, new trace events overwrites the | + | | existing captured events in the trace buffer. | + +-----------+-----------------------------------------------------------------+ + | Discard | When trace buffer is full, new trace events will be discarded. | + +-----------+-----------------------------------------------------------------+ + +The mode can be configured either using EAL command line parameters on +application boot up or use ``rte_trace_mode_set()`` or +``rte_trace_global_mode_set()`` API to configure at runtime. +Refer :doc:`../linux_gsg/linux_eal_parameters` for EAL command line options. + +Trace file location +------------------- +On ``rte_trace_save()`` or ``rte_eal_cleanup()`` invocation, the library saves +the trace buffers to the filesystem. By default, library saves trace files at +``$HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/``. It can be overridden by +the ``--trace-dir=`` EAL command line option. + +For more information, refer :doc:`../linux_gsg/linux_eal_parameters` for trace +EAL command line options. + + +View and analyze the recorded events +------------------------------------ +Once the trace directory is available, the user can view/inspect the recorded events. + +There are many tools you can use to read DPDK traces: + +1. ``babeltrace`` is a command-line utility that converts trace formats; it +supports the format that DPDK trace library produces, CTF, as well as a +basic text output that can be grep ed. The babeltrace command is part of the +opensource ``Babeltrace`` project. + +2. ``Trace Compass`` is a graphical user interface for viewing and analyzing any +type of logs or traces, including DPDK traces. + +Use the babeltrace command-line tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The simplest way to list all the recorded events of a trace is to pass its path +to babeltrace with no options:: + + babeltrace + +``babeltrace`` finds all traces recursively within the given path and prints all +their events, merging them in chronological order. + +You can pipe the output of the babeltrace into a tool like grep(1) for further +filtering. Below example grep the events for ``ethdev`` only:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev + +You can pipe the output of babeltrace into a tool like wc(1) to count the +recorded events. Below example count the number of ``ethdev`` events:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev | wc --lines + +Use the tracecompass GUI tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +``Tracecompass`` is another tool to view/analyze the DPDK traces which gives +a graphical view of events. Like ``babeltrace``, tracecompass also provides +an interface to search for a particular event. To use ``tracecompass``, +following are the minimum required steps: + +- Install ``tracecompass`` to the localhost. Variants are available for Linux, + Windows, and OS-X. +- Launch ``tracecompass`` which will open a graphical window with trace management + interfaces. +- Open a trace using ``File->Open Trace`` option and select metadata file which + is to be viewed/analyzed. + +For more details, refer `Trace Compass `_ + +Quick start +----------- +This section steps you through the details of generating trace and viewing it. + +- Delete the existing dpdk-traces directory if needed:: + + rm -rf $HOME/dpdk-traces/ + +- Start the dpdk-test:: + + echo "quit" | ./build/app/test/dpdk-test --no-huge --trace-level=8 + +- View the traces with babeltrace viewer:: + + babeltrace $HOME/dpdk-traces + +Implementation details +---------------------- +As DPDK trace library is designed to generate traces that uses ``Common Trace +Format (CTF)``. ``CTF`` specification consists of the following units to create +a trace. + +- ``Stream`` Sequence of packets. +- ``Packet`` Header and one or more events. +- ``Event`` Header and payload. + +For detailed information, refer `Common Trace Format `_ + +The implementation details broadly divided into the following areas: + +Trace metadata creation +~~~~~~~~~~~~~~~~~~~~~~~ + +Based on the ``CTF`` specification, One of a CTF trace's streams is mandatory: +the metadata stream. It contains exactly what you would expect: data about the +trace itself. The metadata stream contains a textual description of the binary +layouts of all the other streams. + +This description is written using the Trace Stream Description Language (TSDL), +a declarative language that exists only in the realm of CTF. The purpose of the +metadata stream is to make CTF readers know how to parse a trace's binary +streams of events without CTF specifying any fixed layout. The only stream +layout known in advance is, in fact, the metadata stream's one. + +The internal ``trace_metadata_create()`` function generates the metadata. + +Trace memory +~~~~~~~~~~~~ + +The trace memory will be allocated through an internal function +``__rte_trace_mem_per_thread_alloc()``. The trace memory will be allocated +per thread to enable lock less trace-emit function. The memory for the +trace memory for DPDK lcores will be allocated on ``rte_eal_init()`` if the trace +is enabled through a EAL option. For non DPDK threads, on the first trace +emission, the memory will be allocated. + +Trace memory layout +~~~~~~~~~~~~~~~~~~~ +.. _table_trace_mem_layout: + +.. table:: Trace memory layout. + + +-------------------+ + | packet.header | + +-------------------+ + | packet.context | + +-------------------+ + | trace 0 header | + +-------------------+ + | trace 0 payload | + +-------------------+ + | trace 1 header | + +-------------------+ + | trace 1 payload | + +-------------------+ + | trace N header | + +-------------------+ + | trace N payload | + +-------------------+ + +packet.header +^^^^^^^^^^^^^ +.. _table_packet_header: + +.. table:: Packet header layout. + + +-------------------+ + | uint32_t magic | + +-------------------+ + | rte_uuid_t uuid | + +-------------------+ + +packet.context +^^^^^^^^^^^^^^ +.. _table_packet_context: + +.. table:: Packet context layout. + + +----------------------+ + | uint32_t thread_id | + +----------------------+ + | char thread_name[32] | + +----------------------+ + +trace.header +^^^^^^^^^^^^ +.. _table_trace_header: + +.. table:: Packet context layout. + + +----------------------+ + | event_id [63:48] | + +----------------------+ + | timestamp [47:0] | + +----------------------+ + +The trace header is 64bit, it consists of 48bit of timestamp and 16bit event ID. + +The ``packet.header`` and ``packet.context`` will be written in the slow path +at the time of trace memory creation. The ``trace.header`` and trace payout +will be emitted when the trace function invoked. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 9059f9563..9af0eaaed 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -68,6 +68,15 @@ New Features meson.build in app, lib and drivers instead enable ALLOW_EXPERIMENTAL_API flag globally. +* **Added Trace Library and Tracepoints** + + A native implementation of ``common trace format(CTF)`` based trace library added + to provide the ability to add tracepoints in application/library to get runtime + trace/debug information for control and fast APIs with minimum impact on + fast path performance. Typical trace overhead is ~20 cycles and instrumentation + overhead is 1 cycle. Added tracepoints in ``EAL``, ``ethdev``, ``cryptodev``, + ``eventdev`` and ``mempool`` libraries for important functions. + Removed Items ------------- -- 2.25.1