From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id ACFC5A00C2; Wed, 22 Apr 2020 21:09:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 696AA1D67A; Wed, 22 Apr 2020 21:06:11 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 49FD81D6D3 for ; Wed, 22 Apr 2020 21:06:07 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03MJ5Y2S020002; Wed, 22 Apr 2020 12:06:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=vle10S9gusTzRVDA4QtuVwTP7McCQk9joB9dH+nGq1s=; b=azxNfEE0lQIgGwV9X5nAYB5DnRPrliX6WNMCCWdzimLQn0RrR1Nfx+TVqqvm8xKLQfuo kjsrrUe722PWSXWvKe1HesXJa5hDWPYpsRLZW6v0YPKJExcm60DUb/bIVFRCChbK4Bch cysPvJ3Gz9p6kAySUiknmucaV7LyfzYgX7OinutMC0YtyWYJTnDIWCor8fASs0YnjFZI vl60eKwvgV7rzJhFiu4ONUlUvp3f76RtMcq1qllUHoN39w3O1Bo7MllN7uxdgmiFeH8I 9lOYP8R0FWm7/8ngII97DYM/ve1EkaD/t4Ja9QoiwnYhmXK44EiiKDkpzner5yAXO9Ik Ag== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 30fxwpk06b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Apr 2020 12:06:06 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Apr 2020 12:06:04 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 22 Apr 2020 12:06:04 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id A320D3F703F; Wed, 22 Apr 2020 12:05:59 -0700 (PDT) From: To: Thomas Monjalon , John McNamara , Marko Kovacevic , "Jerin Jacob" , Sunil Kumar Kori CC: , , , Date: Thu, 23 Apr 2020 00:33:49 +0530 Message-ID: <20200422190349.3768487-33-jerinj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200422190349.3768487-1-jerinj@marvell.com> References: <20200419100133.3232316-1-jerinj@marvell.com> <20200422190349.3768487-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-22_06:2020-04-22, 2020-04-22 signatures=0 Subject: [dpdk-dev] [PATCH v7 32/32] doc: add trace library guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add programmar's guide for trace library support. Signed-off-by: Jerin Jacob Signed-off-by: Sunil Kumar Kori --- MAINTAINERS | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/trace_lib.rst | 348 +++++++++++++++++++++++++ doc/guides/rel_notes/release_20_05.rst | 9 + 4 files changed, 359 insertions(+) create mode 100644 doc/guides/prog_guide/trace_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 56217c9da..4dbf48e4d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -200,6 +200,7 @@ M: Sunil Kumar Kori F: lib/librte_eal/include/rte_trace*.h F: lib/librte_eal/common/eal_common_trace*.c F: lib/librte_eal/common/eal_trace.h +F: doc/guides/prog_guide/trace_lib.rst Memory Allocation M: Anatoly Burakov diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf5..0daa08acc 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -35,6 +35,7 @@ Programmer's Guide lpm_lib lpm6_lib flow_classify_lib + trace_lib packet_distrib_lib reorder_lib ip_fragment_reassembly_lib diff --git a/doc/guides/prog_guide/trace_lib.rst b/doc/guides/prog_guide/trace_lib.rst new file mode 100644 index 000000000..4ccfab49e --- /dev/null +++ b/doc/guides/prog_guide/trace_lib.rst @@ -0,0 +1,348 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2020 Marvell International Ltd. + +Trace Library +============= + +Overview +-------- + +*Tracing* is a technique used to understand what goes on in a running software +system. The software used for tracing is called a *tracer*, which is +conceptually similar to a tape recorder. +When recording, specific instrumentation points placed in the software source +code generate events that are saved on a giant tape: a trace file. +The trace file then later can be opened in *trace viewers* to visualize and +analyze the trace events with timestamps and multi-core views. +Such a mechanism will be useful for resolving a wide range of problems such as +multi-core synchronization issues, latency measurements, finding out the +post analyses information like CPU idle time, etc that would otherwise be +extremely challenging. + +Tracing is often compared to *logging*. However, tracers and loggers are two +different tools, serving two different purposes. +Tracers are designed to record much lower-level events that occur much more +frequently than log messages, often in the range of thousands per second, with +very little execution overhead. +Logging is more appropriate for a very high-level analysis of less frequent +events: user accesses, exceptional conditions (errors and warnings, for +example), database transactions, instant messaging communications, and such. +Simply put, logging is one of the many use cases that can be satisfied with +tracing. + +DPDK tracing library features +----------------------------- + +- A framework to add tracepoints in control and fast path APIs with minimum + impact on performance. + Typical trace overhead is ~20 cycles and instrumentation overhead is 1 cycle. +- Enable and disable the tracepoints at runtime. +- Save the trace buffer to the filesystem at any point in time. +- Supports ``overwrite`` and ``discard`` trace mode operations. +- String-based tracepoint object lookup. +- Enable and disable a set of tracepoints based on regular expression and/or + globbing. +- Generate trace in ``Common Trace Format (CTF)``. ``CTF`` is an open-source + trace format and is compatible with ``LTTng``. + For detailed information, refer to `Common Trace Format `_ + +How to add a tracepoint? +------------------------ + +This section steps you through the details of adding a simple tracepoint. + +.. _create_provider_header_file: + +Create the tracepoint provider header file +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + #include + + RTE_TRACE_POINT(app_trace_string, + RTE_TRACE_POINT_ARGS(const char *str), + rte_trace_point_emit_string(str); + ) + +The above macro creates ``app_trace_string`` tracepoint. The user can choose any +name for the tracepoint. However, when adding the tracepoint in the DPDK library, +the ``rte__trace_[_]`` naming convention must be +followed. +The examples are ``rte_eal_trace_generic_str``, ``rte_mempool_trace_create``. + +When ``RTE_TRACE_POINT`` expands from above definition, it creates the following +function template: + +.. code-block:: c + + static __rte_always_inline void + app_trace_string(const char *str) + { + /* Trace subsystem hooks */ + ... + rte_trace_point_emit_string(str); + } + +The consumer of this tracepoint can invoke ``app_trace_string(const char *str)`` +to emit the trace event to the trace buffer. + +Register the tracepoint +~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: c + + #define RTE_TRACE_POINT_REGISTER_SELECT /* Select trace point register macros */ + + #include + + RTE_TRACE_POINT_DEFINE(app_trace_string); + + RTE_INIT(app_trace_init) + { + RTE_TRACE_POINT_REGISTER(app_trace_string, app.trace.string); + } + +The above code snippet registers the ``app_trace_string`` tracepoint to +trace library. Here, the ``my_tracepoint_provider.h`` is the header file +that user created in the first step :ref:`create_provider_header_file` + +The second argument for the ``RTE_TRACE_POINT_REGISTER`` is the name for the +tracepoint. This string will be used for tracepoint lookup or regular expression +and/or glob based tracepoint operations. +There is no requirement for the trace function and its name to be similar. +However, it is recommended to have a similar name for a better naming convention. + +The user must register the tracepoint before the ``rte_eal_init`` invocation. +The user can use the ``RTE_INIT`` construction scheme to achieve the same. + +.. note:: + + The ``RTE_TRACE_POINT_REGISTER_SELECT`` must be defined before including the + header for the tracepoint registration to work properly. + +.. note:: + + The ``RTE_TRACE_POINT_DEFINE`` defines the tracepoint of ``rte_trace_point_t`` + type. When the tracepoint defined in a shared library, the user must + update the ``.map`` file with ``__`` symbol as all the + libraries can compiled in shared mode. + For example, ``__app_trace_string`` will be the exported symbol in the + above example. + +Datapath tracepoint +------------------- + +In order to avoid performance impact for the datapath tracepoint, the library +introduced ``RTE_TRACE_POINT_FP``. When adding the tracepoint in datapath code, +user must use ``RTE_TRACE_POINT_FP`` instead of ``RTE_TRACE_POINT``. + +``RTE_TRACE_POINT_FP`` is compiled out by default and it can be enabled using +``CONFIG_RTE_ENABLE_TRACE_FP`` configuration parameter. +The ``enable_trace_fp`` build option shall be used for the same for meson build. + +Event record mode +----------------- + +Event record mode is an attribute of trace buffers. Trace library exposes the +following modes: + +Overwrite + When trace buffer is full, new trace events overwrites the existing captured + events in the trace buffer. +Discard + When trace buffer is full, new trace events will be discarded. + +The mode can be configured either using EAL command line parameter +``--trace-mode`` on application boot up or use ``rte_trace_mode_set()`` API to +configure at runtime. + +Trace file location +------------------- + +On ``rte_trace_save()`` or ``rte_eal_cleanup()`` invocation, the library saves +the trace buffers to the filesystem. By default, library saves trace files at +``$HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/``. +It can be overridden by the ``--trace-dir=`` EAL command line +option. + +For more information, refer :doc:`../linux_gsg/linux_eal_parameters` for trace +EAL command line options. + + +View and analyze the recorded events +------------------------------------ + +Once the trace directory is available, the user can view/inspect the recorded events. + +There are many tools you can use to read DPDK traces: + +1. ``babeltrace`` is a command-line utility that converts trace formats; it +supports the format that DPDK trace library produces, CTF, as well as a +basic text output that can be grep'ed. +The babeltrace command is part of the Open Source Babeltrace project. + +2. ``Trace Compass`` is a graphical user interface for viewing and analyzing any +type of logs or traces, including DPDK traces. + +Use the babeltrace command-line tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The simplest way to list all the recorded events of a trace is to pass its path +to babeltrace with no options:: + + babeltrace + +``babeltrace`` finds all traces recursively within the given path and prints all +their events, merging them in chronological order. + +You can pipe the output of the babeltrace into a tool like grep(1) for further +filtering. Below example grep the events for ``ethdev`` only:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev + +You can pipe the output of babeltrace into a tool like wc(1) to count the +recorded events. Below example count the number of ``ethdev`` events:: + + babeltrace /tmp/my-dpdk-trace | grep ethdev | wc --lines + +Use the tracecompass GUI tool +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``Tracecompass`` is another tool to view/analyze the DPDK traces which gives +a graphical view of events. Like ``babeltrace``, tracecompass also provides +an interface to search for a particular event. +To use ``tracecompass``, following are the minimum required steps: + +- Install ``tracecompass`` to the localhost. Variants are available for Linux, + Windows, and OS-X. +- Launch ``tracecompass`` which will open a graphical window with trace management + interfaces. +- Open a trace using ``File->Open Trace`` option and select metadata file which + is to be viewed/analyzed. + +For more details, refer `Trace Compass `_ + +Quick start +----------- + +This section steps you through the details of generating trace and viewing it. + +- Start the dpdk-test:: + + echo "quit" | ./build/app/test/dpdk-test --no-huge --trace=.* + +- View the traces with babeltrace viewer:: + + babeltrace $HOME/dpdk-traces/rte-yyyy-mm-dd-[AP]M-hh-mm-ss/ + +Implementation details +---------------------- + +As DPDK trace library is designed to generate traces that uses ``Common Trace +Format (CTF)``. ``CTF`` specification consists of the following units to create +a trace. + +- ``Stream`` Sequence of packets. +- ``Packet`` Header and one or more events. +- ``Event`` Header and payload. + +For detailed information, refer `Common Trace Format `_ + +The implementation details broadly divided into the following areas: + +Trace metadata creation +~~~~~~~~~~~~~~~~~~~~~~~ + +Based on the ``CTF`` specification, One of a CTF trace's streams is mandatory: +the metadata stream. It contains exactly what you would expect: data about the +trace itself. The metadata stream contains a textual description of the binary +layouts of all the other streams. + +This description is written using the Trace Stream Description Language (TSDL), +a declarative language that exists only in the realm of CTF. +The purpose of the metadata stream is to make CTF readers know how to parse a +trace's binary streams of events without CTF specifying any fixed layout. +The only stream layout known in advance is, in fact, the metadata stream's one. + +The internal ``trace_metadata_create()`` function generates the metadata. + +Trace memory +~~~~~~~~~~~~ + +The trace memory will be allocated through an internal function +``__rte_trace_mem_per_thread_alloc()``. The trace memory will be allocated +per thread to enable lock less trace-emit function. +The memory for the trace memory for DPDK lcores will be allocated on +``rte_eal_init()`` if the trace is enabled through a EAL option. +For non DPDK threads, on the first trace emission, the memory will be allocated. + +Trace memory layout +~~~~~~~~~~~~~~~~~~~ + +.. _table_trace_mem_layout: + +.. table:: Trace memory layout. + + +-------------------+ + | packet.header | + +-------------------+ + | packet.context | + +-------------------+ + | trace 0 header | + +-------------------+ + | trace 0 payload | + +-------------------+ + | trace 1 header | + +-------------------+ + | trace 1 payload | + +-------------------+ + | trace N header | + +-------------------+ + | trace N payload | + +-------------------+ + +packet.header +^^^^^^^^^^^^^ + +.. _table_packet_header: + +.. table:: Packet header layout. + + +-------------------+ + | uint32_t magic | + +-------------------+ + | rte_uuid_t uuid | + +-------------------+ + +packet.context +^^^^^^^^^^^^^^ + +.. _table_packet_context: + +.. table:: Packet context layout. + + +----------------------+ + | uint32_t thread_id | + +----------------------+ + | char thread_name[32] | + +----------------------+ + +trace.header +^^^^^^^^^^^^ + +.. _table_trace_header: + +.. table:: Packet context layout. + + +----------------------+ + | event_id [63:48] | + +----------------------+ + | timestamp [47:0] | + +----------------------+ + +The trace header is 64bit, it consists of 48bit of timestamp and 16bit event ID. + +The ``packet.header`` and ``packet.context`` will be written in the slow path +at the time of trace memory creation. The ``trace.header`` and trace payout +will be emitted when the trace function invoked. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 7f2049a0f..ecc746882 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -135,6 +135,15 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Added Trace Library and Tracepoints** + + A native implementation of ``common trace format(CTF)`` based trace library added + to provide the ability to add tracepoints in application/library to get runtime + trace/debug information for control and fast APIs with minimum impact on + fast path performance. Typical trace overhead is ~20 cycles and instrumentation + overhead is 1 cycle. Added tracepoints in ``EAL``, ``ethdev``, ``cryptodev``, + ``eventdev`` and ``mempool`` libraries for important functions. + Removed Items ------------- -- 2.25.1