From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 242F4A0C43; Fri, 15 Oct 2021 20:30:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5430D41187; Fri, 15 Oct 2021 20:29:27 +0200 (CEST) Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by mails.dpdk.org (Postfix) with ESMTP id 6B5D34117A for ; Fri, 15 Oct 2021 20:29:24 +0200 (CEST) Received: by mail-pg1-f178.google.com with SMTP id f5so9291299pgc.12 for ; Fri, 15 Oct 2021 11:29:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t2TOlTvAEsRGa72VW4eNl+IWogNKZqjHPiyG7bNkVmw=; b=Tc11PKDC7K6BR0qXLNBlaHENRWlE/8+mgl3rwCcayjjU8yg2QiWGNvIyvK48aTE6ZF PVrVM04Gz/KnMovD8+7zXghhu0/XwWZ7r3S4GQXY8wXM64SYYx89XIDGfYR+bwm2xxD1 51yoUyMVtmu8Nl7FHZ1AjZIP5tli7gOZ0kbe8Js3UBHc5s+LiWw+ruTzZF9K5oY7Ucsd TBW5U+fPGlceGCDNH4AK/NmooOweI6NTO4AFeACHmWWC6/u/ht7KffnV50sOHLYWlUrg pzAEvZYgXJN4fNSmXtBNa1NiEnyM+rRdctdnwyB/4tOLF/BkZLJLOCO6KEG+bp1KGXaN GdjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t2TOlTvAEsRGa72VW4eNl+IWogNKZqjHPiyG7bNkVmw=; b=koRFjN1uTL7hh5rWarZM6M3CtVaQQXGAJQ83iUauGdqjoBMm4KfeRzceU2pOD3XDTV N83satK5myyU3BL+0w3r9PU8qITzMhi6SiS+JaoZpwZ6E0j8R+bL7/DGWYqv/EKWhmLL zAHeN53l2jNKFZezmazYJKQAiS2/+fpjq8/SoyVkKp2vtu6eEckfOge7ZhAkfTQuhWcI qeLIr7asVAN+g0rCrkw8D/wHokt9pU6D5gqbquid4wdjwkIq5Edl9QQWucxMNezNfMEo tvcYyuLiKFCORVfgTEfcBStEuN3uHNcsLxYXkK4jRl1EqnolsTGjbRcH7P1OPoKaQM62 A6Xg== X-Gm-Message-State: AOAM533wLOKHlPBXct5V+3uvpaQEtGzcAj+pT5GWK1UzDT6UlBSlarzp LxIrcLW8E7c+NfoM36nxkiqeA93NEY8VTg== X-Google-Smtp-Source: ABdhPJwFfGfqPBVh4Tv33QRSMJruAiUTCfz1FME2ivWV6Ng83SHf5dbFEU4UK4vznimd4h+mcbYjnA== X-Received: by 2002:a63:84c8:: with SMTP id k191mr10245548pgd.158.1634322562647; Fri, 15 Oct 2021 11:29:22 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id p189sm5526545pfp.167.2021.10.15.11.29.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Oct 2021 11:29:22 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger , Reshma Pattan Date: Fri, 15 Oct 2021 11:29:02 -0700 Message-Id: <20211015182903.52615-12-stephen@networkplumber.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211015182903.52615-1-stephen@networkplumber.org> References: <20210903004732.109023-1-stephen@networkplumber.org> <20211015182903.52615-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v13 11/12] doc: changes for new pcapng and dumpcap utility X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Describe the new packet capture library and utility. Signed-off-by: Stephen Hemminger --- doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf.in | 1 + .../howto/img/packet_capture_framework.svg | 96 +++++++++---------- doc/guides/howto/packet_capture_framework.rst | 69 ++++++------- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pcapng_lib.rst | 25 +++++ doc/guides/prog_guide/pdump_lib.rst | 28 ++++-- doc/guides/rel_notes/release_21_11.rst | 10 ++ doc/guides/tools/dumpcap.rst | 86 +++++++++++++++++ doc/guides/tools/index.rst | 1 + 10 files changed, 230 insertions(+), 88 deletions(-) create mode 100644 doc/guides/prog_guide/pcapng_lib.rst create mode 100644 doc/guides/tools/dumpcap.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 1992107a0356..ee07394d1c78 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -223,3 +223,4 @@ The public API headers are grouped by topics: [experimental APIs] (@ref rte_compat.h), [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h) + [pcapng] (@ref rte_pcapng.h) diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 325a0195c6ab..aba17799a9a1 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -58,6 +58,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/metrics \ @TOPDIR@/lib/node \ @TOPDIR@/lib/net \ + @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ diff --git a/doc/guides/howto/img/packet_capture_framework.svg b/doc/guides/howto/img/packet_capture_framework.svg index a76baf71fdee..1c2646a81096 100644 --- a/doc/guides/howto/img/packet_capture_framework.svg +++ b/doc/guides/howto/img/packet_capture_framework.svg @@ -1,6 +1,4 @@ - - + inkscape:version="1.0.2 (e86c870879, 2021-01-15)" + sodipodi:docname="packet_capture_framework.svg"> + gradientTransform="matrix(1.1457977,0,0,0.99944907,-151.97019,745.05014)" /> + inkscape:window-width="2560" + inkscape:window-height="1414" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" + inkscape:document-rotation="0" /> @@ -296,7 +295,7 @@ image/svg+xml - + @@ -321,15 +320,15 @@ y="790.82452" /> DPDK Primary Application + y="807.3205" + style="font-size:12.5px;line-height:1.25">DPDK Primary Application dpdk-pdumpdpdk-dumpcaptool + id="tspan4193" + style="font-size:12.5px;line-height:1.25">tool PCAP PMD + id="tspan4193-3" + style="font-size:12.5px;line-height:1.25">librte_pcapng dpdk_port0 + id="tspan4193-6" + style="font-size:12.5px;line-height:1.25">dpdk_port0 librte_pdump + id="tspan4193-2" + style="font-size:12.5px;line-height:1.25">librte_pdump + width="108.21974" + height="35.335861" + x="297.9809" + y="985.62219" /> capture.pcap + id="tspan4193-3-2" + style="font-size:12.5px;line-height:1.25">capture.pcapng Traffic Generator + id="tspan4193-3-2-7" + style="font-size:12.5px;line-height:1.25">Traffic Generator ` tool is developed based on the -``librte_pdump`` library. It runs as a DPDK secondary process and is capable -of enabling or disabling packet capture on DPDK ports. The ``dpdk-pdump`` tool -provides command-line options with which users can request enabling or -disabling of the packet capture on DPDK ports. +The :ref:`librte_pcapng ` library provides the APIs to format +packets and write them to a file in Pcapng format. + + +The :ref:`dpdk-dumpcap ` is a tool that captures packets in +like Wireshark dumpcap does for Linux. It runs as a DPDK secondary process and +captures packets from one or more interfaces and writes them to a file +in Pcapng format. The ``dpdk-dumpcap`` tool is designed to take +most of the same options as the Wireshark ``dumpcap`` command. -The application which initializes the packet capture framework will be a primary process -and the application that enables or disables the packet capture will -be a secondary process. The primary process sends the Rx and Tx packets from the DPDK ports -to the secondary process. +Without any options it will use the packet capture framework to +capture traffic from the first available DPDK port. In DPDK the ``testpmd`` application can be used to initialize the packet -capture framework and acts as a server, and the ``dpdk-pdump`` tool acts as a +capture framework and acts as a server, and the ``dpdk-dumpcap`` tool acts as a client. To view Rx or Tx packets of ``testpmd``, the application should be -launched first, and then the ``dpdk-pdump`` tool. Packets from ``testpmd`` -will be sent to the tool, which then sends them on to the Pcap PMD device and -that device writes them to the Pcap file or to an external interface depending -on the command-line option used. +launched first, and then the ``dpdk-dumpcap`` tool. Packets from ``testpmd`` +will be sent to the tool, and then to the Pcapng file. Some things to note: -* The ``dpdk-pdump`` tool can only be used in conjunction with a primary +* All tools using ``librte_pdump`` can only be used in conjunction with a primary application which has the packet capture framework initialized already. In dpdk, only ``testpmd`` is modified to initialize packet capture framework, - other applications remain untouched. So, if the ``dpdk-pdump`` tool has to + other applications remain untouched. So, if the ``dpdk-dumpcap`` tool has to be used with any application other than the testpmd, the user needs to explicitly modify that application to call the packet capture framework initialization code. Refer to the ``app/test-pmd/testpmd.c`` code and look for ``pdump`` keyword to see how this is done. -* The ``dpdk-pdump`` tool depends on the libpcap based PMD. +* The ``dpdk-pdump`` tool is an older tool created as demonstration of ``librte_pdump`` + library. The ``dpdk-pdump`` tool provides more limited functionality and + and depends on the Pcap PMD. It is retained only for compatibility reasons; + users should use ``dpdk-dumpcap`` instead. Test Environment ---------------- -The overview of using the Packet Capture Framework and the ``dpdk-pdump`` tool +The overview of using the Packet Capture Framework and the ``dpdk-dumpcap`` utility for packet capturing on the DPDK port in :numref:`figure_packet_capture_framework`. @@ -66,13 +70,13 @@ for packet capturing on the DPDK port in .. figure:: img/packet_capture_framework.* - Packet capturing on a DPDK port using the dpdk-pdump tool. + Packet capturing on a DPDK port using the dpdk-dumpcap utility. Running the Application ----------------------- -The following steps demonstrate how to run the ``dpdk-pdump`` tool to capture +The following steps demonstrate how to run the ``dpdk-dumpcap`` tool to capture Rx side packets on dpdk_port0 in :numref:`figure_packet_capture_framework` and inspect them using ``tcpdump``. @@ -80,16 +84,15 @@ inspect them using ``tcpdump``. sudo /app/dpdk-testpmd -c 0xf0 -n 4 -- -i --port-topology=chained -#. Launch the pdump tool as follows:: +#. Launch the dpdk-dumpcap as follows:: - sudo /app/dpdk-pdump -- \ - --pdump 'port=0,queue=*,rx-dev=/tmp/capture.pcap' + sudo /app/dpdk-dumpcap -w /tmp/capture.pcapng #. Send traffic to dpdk_port0 from traffic generator. - Inspect packets captured in the file capture.pcap using a tool - that can interpret Pcap files, for example tcpdump:: + Inspect packets captured in the file capture.pcapng using a tool such as + tcpdump or tshark that can interpret Pcapng files:: - $tcpdump -nr /tmp/capture.pcap + $ tcpdump -nr /tmp/capture.pcapng reading from file /tmp/capture.pcap, link-type EN10MB (Ethernet) 11:11:36.891404 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 11:11:36.891442 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 2dce507f46a3..b440c77c2ba1 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -43,6 +43,7 @@ Programmer's Guide ip_fragment_reassembly_lib generic_receive_offload_lib generic_segmentation_offload_lib + pcapng_lib pdump_lib multi_proc_support kernel_nic_interface diff --git a/doc/guides/prog_guide/pcapng_lib.rst b/doc/guides/prog_guide/pcapng_lib.rst new file mode 100644 index 000000000000..7b2d82d7bd3b --- /dev/null +++ b/doc/guides/prog_guide/pcapng_lib.rst @@ -0,0 +1,25 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016 Intel Corporation. + +.. _pcapng_library: + +Packet Capture File Writer +========================== + +Pcapng is a library for creating files in Pcapng file format. +The Pcapng file format is the default capture file format for modern +network capture processing tools. It can be read by wireshark and tcpdump. + +Usage +----- + +Before the library can be used the function ``rte_pcapng_init`` +should be called once to initialize timestamp computation. + + +References +---------- +* Project repository https://github.com/pcapng/pcapng/ + +* PCAP Next Generation (pcapng) Capture File Format +https://pcapng.github.io/pcapng/draft-tuexen-opsawg-pcapng.html diff --git a/doc/guides/prog_guide/pdump_lib.rst b/doc/guides/prog_guide/pdump_lib.rst index 62c0b015b2fe..d04d9709e364 100644 --- a/doc/guides/prog_guide/pdump_lib.rst +++ b/doc/guides/prog_guide/pdump_lib.rst @@ -3,10 +3,10 @@ .. _pdump_library: -The librte_pdump Library -======================== +The Packet Capture Library +========================== -The ``librte_pdump`` library provides a framework for packet capturing in DPDK. +The DPDK ``pdump`` library provides a framework for packet capturing in DPDK. The library does the complete copy of the Rx and Tx mbufs to a new mempool and hence it slows down the performance of the applications, so it is recommended to use this library for debugging purposes. @@ -23,11 +23,19 @@ or disable the packet capture, and to uninitialize it. * ``rte_pdump_enable()``: This API enables the packet capture on a given port and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf()`` + This API enables the packet capture on a given port and queue. + It also allows setting an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_enable_by_deviceid()``: This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf_by_deviceid()`` + This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. + It also allows setting an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_disable()``: This API disables the packet capture on a given port and queue. @@ -61,6 +69,12 @@ and enables the packet capture by registering the Ethernet RX and TX callbacks f and queue combinations. Then the primary process will mirror the packets to the new mempool and enqueue them to the rte_ring that secondary process have passed to these APIs. +The packet ring supports one of two formats. The default format enqueues copies of the original packets +into the rte_ring. If the ``RTE_PDUMP_FLAG_PCAPNG`` is set the mbuf data is extended with header and trailer +to match the format of Pcapng enhanced packet block. The enhanced packet block has meta-data such as the +timestamp, port and queue the packet was captured on. It is up to the application consuming the +packets from the ring to select the format desired. + The library APIs ``rte_pdump_disable()`` and ``rte_pdump_disable_by_deviceid()`` disables the packet capture. For the calls to these APIs from secondary process, the library creates the "pdump disable" request and sends the request to the primary process over the multi process channel. The primary process takes this request and @@ -74,5 +88,5 @@ function. Use Case: Packet Capturing -------------------------- -The DPDK ``app/pdump`` tool is developed based on this library to capture packets in DPDK. -Users can use this as an example to develop their own packet capturing tools. +The DPDK ``app/dpdk-dumpcap`` utility uses this library +to capture packets in DPDK. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 4c56cdfeaaa2..0909f4258cf8 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -159,6 +159,16 @@ New Features * Added tests to verify tunnel header verification in IPsec inbound. * Added tests to verify inner checksum. +* **Revised packet capture framework.** + + * New dpdk-dumpcap program that has most of the features of the + wireshark dumpcap utility including: capture of multiple interfaces, + filtering, and stopping after number of bytes, packets. + * New library for writing pcapng packet capture files. + * Enhancements to the pdump library to support: + * Packet filter with BPF. + * Pcapng format with timestamps and meta-data. + * Fixes packet capture with stripped VLAN tags. Removed Items ------------- diff --git a/doc/guides/tools/dumpcap.rst b/doc/guides/tools/dumpcap.rst new file mode 100644 index 000000000000..664ea0c79802 --- /dev/null +++ b/doc/guides/tools/dumpcap.rst @@ -0,0 +1,86 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Microsoft Corporation. + +.. _dumpcap_tool: + +dpdk-dumpcap Application +======================== + +The ``dpdk-dumpcap`` tool is a Data Plane Development Kit (DPDK) +network traffic dump tool. The interface is similar to the dumpcap tool in Wireshark. +It runs as a secondary DPDK process and lets you capture packets that are +coming into and out of a DPDK primary process. +The ``dpdk-dumpcap`` writes files in Pcapng packet format using +capture file format is pcapng. + +Without any options set it will use DPDK to capture traffic from the first +available DPDK interface and write the received raw packet data, along +with timestamps into a pcapng file. + +If the ``-w`` option is not specified, ``dpdk-dumpcap`` writes to a newly +create file with a name chosen based on interface name and timestamp. +If ``-w`` option is specified, then that file is used. + + .. Note:: + * The ``dpdk-dumpcap`` tool can only be used in conjunction with a primary + application which has the packet capture framework initialized already. + In dpdk, only the ``testpmd`` is modified to initialize packet capture + framework, other applications remain untouched. So, if the ``dpdk-dumpcap`` + tool has to be used with any application other than the testpmd, user + needs to explicitly modify that application to call packet capture + framework initialization code. Refer ``app/test-pmd/testpmd.c`` + code to see how this is done. + + * The ``dpdk-dumpcap`` tool runs as a DPDK secondary process. It exits when + the primary application exits. + + +Running the Application +----------------------- + +To list interfaces available for capture use ``--list-interfaces``. + +To filter packets in style of *tshark* use the ``-f`` flag. + +To capture on multiple interfaces at once, use multiple ``-I`` flags. + +Example +------- + +.. code-block:: console + + # .//app/dpdk-dumpcap --list-interfaces + 0. 000:00:03.0 + 1. 000:00:03.1 + + # .//app/dpdk-dumpcap -I 0000:00:03.0 -c 6 -w /tmp/sample.pcapng + Packets captured: 6 + Packets received/dropped on interface '0000:00:03.0' 6/0 + + # .//app/dpdk-dumpcap -f 'tcp port 80' + Packets captured: 6 + Packets received/dropped on interface '0000:00:03.0' 10/8 + + +Limitations +----------- +The following option of Wireshark ``dumpcap`` is not yet implemented: + + * ``-b|--ring-buffer`` -- more complex file management. + +The following options do not make sense in the context of DPDK. + + * ``-C `` -- its a kernel thing + + * ``-t`` -- use a thread per interface + + * Timestamp type. + + * Link data types. Only EN10MB (Ethernet) is supported. + + * Wireless related options: ``-I|--monitor-mode`` and ``-k `` + + +.. Note:: + * The options to ``dpdk-dumpcap`` are like the Wireshark dumpcap program and + are not the same as ``dpdk-pdump`` and other DPDK applications. diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst index 93dde4148e90..b71c12b8f2dd 100644 --- a/doc/guides/tools/index.rst +++ b/doc/guides/tools/index.rst @@ -8,6 +8,7 @@ DPDK Tools User Guides :maxdepth: 2 :numbered: + dumpcap proc_info pdump pmdinfo -- 2.30.2