From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D10C2A0C4D; Fri, 3 Sep 2021 02:48:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 357FE41100; Fri, 3 Sep 2021 02:47:44 +0200 (CEST) Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by mails.dpdk.org (Postfix) with ESMTP id 234F240041 for ; Fri, 3 Sep 2021 02:47:41 +0200 (CEST) Received: by mail-pg1-f173.google.com with SMTP id r2so3825617pgl.10 for ; Thu, 02 Sep 2021 17:47:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BD/za4PZGzBtcQ40Y5JqOc5jLeyflDSGbqRv4jFc1Ug=; b=sGpl/i/6XizuSfeHVz3yKpG4gHAQr8jmxLBls83n0CS4bbKmeXfNDPnJy7itA6poQz 2AYbhSP12/Fm3sIsc8EUbf5bZa4hVnIU3dIDETezI/2PeeQ/1P1p8m7263qwPS4iUF8+ Hdi2lX+l4OXGktXEMaFdcKG7N6c2tbZEL5Ao2CHZphrQvHYjQYb83OJH/rAeGrFLh86U uc5WtUSVSkW1qkBrVDnsE+g/wGH1XN0rZ3KP4ZGqOQes3caOlxrTvV5LSnt5pCRgjs5h ja0UoB4PoEM+UAKR0vRpADYvsgvRonAvegpB8tvnpvV03RqBo+vhmCHGSzveY4e1hv6q mklg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BD/za4PZGzBtcQ40Y5JqOc5jLeyflDSGbqRv4jFc1Ug=; b=S/3XkpgQ7Oi1ZwQCwRYjbC23BnQQu4thQW+EJezn7ykKyDD4zVb+zcwN+6Z1cDlYjm 40q8GcMlI+r5SFDr43qr96f378kz+hvB1yGdku6dXVEmt9eXUQclxDYL7xh65V4eDQCi mivoflLipB0rdJLPZiVnS1KB6GbOMtAig+FTo4LEFiScRnAw8Bl5hD23JQO7z9569ttz y950G+bZxPkLeZlL0AU+C41V8ikheAaiocZsaqm0GPgZNtQDJjwOG0zzVWgGH1CnbLrA COODm0RRWUVFBe87OVuRt3O0Uc+6TOT9dYFwnjZp5Tp39KkxkaFQ2zlNRhBMDfDnK/4J vKLg== X-Gm-Message-State: AOAM5320N7H1ENomFdmSDGKM2C8xb2umVrt98PPfqY3pOEVDjUOaDYqp 6lZ3ZII4PePtBSC4DQ7D2K3FwKFxDJyRLg== X-Google-Smtp-Source: ABdhPJwx6jLx1BvgriuzCk1A37BTn+d1eh4yMivPZXgDeWZ5Db/csGYGeYe8YEcw67V9WXn5E2apXg== X-Received: by 2002:a05:6a00:1255:b0:3f5:26ee:ca2c with SMTP id u21-20020a056a00125500b003f526eeca2cmr1031758pfi.62.1630630059412; Thu, 02 Sep 2021 17:47:39 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id cq8sm3179033pjb.31.2021.09.02.17.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Sep 2021 17:47:38 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Date: Thu, 2 Sep 2021 17:47:31 -0700 Message-Id: <20210903004732.109023-5-stephen@networkplumber.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210903004732.109023-1-stephen@networkplumber.org> References: <20210903004732.109023-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH 4/5] doc: changes for new pcapng and dumpcap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Describe the new packet capture library and utilities Signed-off-by: Stephen Hemminger --- doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf.in | 1 + doc/guides/howto/packet_capture_framework.rst | 67 ++++++++-------- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pcapng_lib.rst | 24 ++++++ doc/guides/prog_guide/pdump_lib.rst | 28 +++++-- doc/guides/rel_notes/release_21_11.rst | 10 +++ doc/guides/tools/dumpcap.rst | 80 +++++++++++++++++++ doc/guides/tools/index.rst | 1 + 9 files changed, 174 insertions(+), 39 deletions(-) create mode 100644 doc/guides/prog_guide/pcapng_lib.rst create mode 100644 doc/guides/tools/dumpcap.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 1992107a0356..ee07394d1c78 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -223,3 +223,4 @@ The public API headers are grouped by topics: [experimental APIs] (@ref rte_compat.h), [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h) + [pcapng] (@ref rte_pcapng.h) diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 325a0195c6ab..aba17799a9a1 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -58,6 +58,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/metrics \ @TOPDIR@/lib/node \ @TOPDIR@/lib/net \ + @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ diff --git a/doc/guides/howto/packet_capture_framework.rst b/doc/guides/howto/packet_capture_framework.rst index c31bac52340e..78baa609a021 100644 --- a/doc/guides/howto/packet_capture_framework.rst +++ b/doc/guides/howto/packet_capture_framework.rst @@ -1,18 +1,19 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright(c) 2017 Intel Corporation. -DPDK pdump Library and pdump Tool -================================= +DPDK packet capture libraries and tools +======================================= This document describes how the Data Plane Development Kit (DPDK) Packet Capture Framework is used for capturing packets on DPDK ports. It is intended for users of DPDK who want to know more about the Packet Capture feature and for those who want to monitor traffic on DPDK-controlled devices. -The DPDK packet capture framework was introduced in DPDK v16.07. The DPDK -packet capture framework consists of the DPDK pdump library and DPDK pdump -tool. - +The DPDK packet capture framework was introduced in DPDK v16.07 and +enhanced in 21.1. The DPDK packet capture framework consists of the +libraries for collecting packets ``librte_pdump`` and writing packets +to a file ``librte_pcapng``. There are two sample applications: +``dpdk-dumpcap`` and older ``dpdk-pdump``. Introduction ------------ @@ -22,43 +23,46 @@ allow users to initialize the packet capture framework and to enable or disable packet capture. The library works on a multi process communication model and its usage is recommended for debugging purposes. -The :ref:`dpdk-pdump ` tool is developed based on the -``librte_pdump`` library. It runs as a DPDK secondary process and is capable -of enabling or disabling packet capture on DPDK ports. The ``dpdk-pdump`` tool -provides command-line options with which users can request enabling or -disabling of the packet capture on DPDK ports. +The :ref:`librte_pcapng ` library provides the APIs to format +packets and write them to a file in Pcapng format. + + +The :ref:`dpdk-dumpcap ` is a tool that captures packets in +like Wireshark dumpcap does for Linux. It runs as a DPDK secondary process and +captures packets from one or more interfaces and writes them to a file +in Pcapng format. The ``dpdk-dumpcap`` tool is designed to take +most of the same options as the Wireshark ``dumpcap`` command. -The application which initializes the packet capture framework will be a primary process -and the application that enables or disables the packet capture will -be a secondary process. The primary process sends the Rx and Tx packets from the DPDK ports -to the secondary process. +Without any options it will use the packet capture framework to +capture traffic from the first available DPDK port. In DPDK the ``testpmd`` application can be used to initialize the packet -capture framework and acts as a server, and the ``dpdk-pdump`` tool acts as a +capture framework and acts as a server, and the ``dpdk-dumpcap`` tool acts as a client. To view Rx or Tx packets of ``testpmd``, the application should be -launched first, and then the ``dpdk-pdump`` tool. Packets from ``testpmd`` -will be sent to the tool, which then sends them on to the Pcap PMD device and -that device writes them to the Pcap file or to an external interface depending -on the command-line option used. +launched first, and then the ``dpdk-dumpcap`` tool. Packets from ``testpmd`` +will be sent to the tool, and then to the Pcapng file. Some things to note: -* The ``dpdk-pdump`` tool can only be used in conjunction with a primary +* All tools using ``librte_pdump`` can only be used in conjunction with a primary application which has the packet capture framework initialized already. In dpdk, only ``testpmd`` is modified to initialize packet capture framework, - other applications remain untouched. So, if the ``dpdk-pdump`` tool has to + other applications remain untouched. So, if the ``dpdk-dumpcap`` tool has to be used with any application other than the testpmd, the user needs to explicitly modify that application to call the packet capture framework initialization code. Refer to the ``app/test-pmd/testpmd.c`` code and look for ``pdump`` keyword to see how this is done. -* The ``dpdk-pdump`` tool depends on the libpcap based PMD. +* The ``dpdk-pdump`` tool is an older tool created as demonstration of ``librte_pdump`` + library. The ``dpdk-pdump`` tool provides more limited functionality and + and depends on the Pcap PMD. It is retained only for compatibility reasons; + users should use ``dpdk-dumpcap`` instead. Test Environment ---------------- -The overview of using the Packet Capture Framework and the ``dpdk-pdump`` tool +The overview of using the Packet Capture Framework and the ``dpdk-dumpcap`` utility for packet capturing on the DPDK port in :numref:`figure_packet_capture_framework`. @@ -66,13 +70,13 @@ for packet capturing on the DPDK port in .. figure:: img/packet_capture_framework.* - Packet capturing on a DPDK port using the dpdk-pdump tool. + Packet capturing on a DPDK port using the dpdk-dumpcap utility. Running the Application ----------------------- -The following steps demonstrate how to run the ``dpdk-pdump`` tool to capture +The following steps demonstrate how to run the ``dpdk-dumpcap`` tool to capture Rx side packets on dpdk_port0 in :numref:`figure_packet_capture_framework` and inspect them using ``tcpdump``. @@ -80,16 +84,15 @@ inspect them using ``tcpdump``. sudo /app/dpdk-testpmd -c 0xf0 -n 4 -- -i --port-topology=chained -#. Launch the pdump tool as follows:: +#. Launch the dpdk-dump as follows:: - sudo /app/dpdk-pdump -- \ - --pdump 'port=0,queue=*,rx-dev=/tmp/capture.pcap' + sudo /app/dpdk-dumpcap -w /tmp/capture.pcapng #. Send traffic to dpdk_port0 from traffic generator. - Inspect packets captured in the file capture.pcap using a tool - that can interpret Pcap files, for example tcpdump:: + Inspect packets captured in the file capture.pcap using a tool such as + tcpdump or tshark that can interpret Pcapng files:: - $tcpdump -nr /tmp/capture.pcap + $ tcpdump -nr /tmp/capture.pcapng reading from file /tmp/capture.pcap, link-type EN10MB (Ethernet) 11:11:36.891404 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 11:11:36.891442 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 2dce507f46a3..b440c77c2ba1 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -43,6 +43,7 @@ Programmer's Guide ip_fragment_reassembly_lib generic_receive_offload_lib generic_segmentation_offload_lib + pcapng_lib pdump_lib multi_proc_support kernel_nic_interface diff --git a/doc/guides/prog_guide/pcapng_lib.rst b/doc/guides/prog_guide/pcapng_lib.rst new file mode 100644 index 000000000000..36379b530a57 --- /dev/null +++ b/doc/guides/prog_guide/pcapng_lib.rst @@ -0,0 +1,24 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016 Intel Corporation. + +.. _pcapng_library: + +Packet Capture File Writer +========================== + +Pcapng is a library for creating files in Pcapng file format. +The Pcapng file format is the default capture file format for modern +network capture processing tools. It can be read by wireshark and tcpdump. + +Usage +----- + +Before the library can be used the function ``rte_pcapng_init`` +should be called once to initialize timestamp computation. + + +References +---------- +* Draft RFC https://www.ietf.org/id/draft-tuexen-opsawg-pcapng-03.html + +* Project repository https://github.com/pcapng/pcapng/ diff --git a/doc/guides/prog_guide/pdump_lib.rst b/doc/guides/prog_guide/pdump_lib.rst index 62c0b015b2fe..9af91415e5ea 100644 --- a/doc/guides/prog_guide/pdump_lib.rst +++ b/doc/guides/prog_guide/pdump_lib.rst @@ -3,10 +3,10 @@ .. _pdump_library: -The librte_pdump Library -======================== +The Packet Capture Library +========================== -The ``librte_pdump`` library provides a framework for packet capturing in DPDK. +The DPDK ``pdump`` library provides a framework for packet capturing in DPDK. The library does the complete copy of the Rx and Tx mbufs to a new mempool and hence it slows down the performance of the applications, so it is recommended to use this library for debugging purposes. @@ -23,11 +23,19 @@ or disable the packet capture, and to uninitialize it. * ``rte_pdump_enable()``: This API enables the packet capture on a given port and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf()`` + This API enables the packet capture on a given port and queue. + It also allows setting an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_enable_by_deviceid()``: This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf_by_deviceid()`` + This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. + It also allows seating an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_disable()``: This API disables the packet capture on a given port and queue. @@ -61,6 +69,12 @@ and enables the packet capture by registering the Ethernet RX and TX callbacks f and queue combinations. Then the primary process will mirror the packets to the new mempool and enqueue them to the rte_ring that secondary process have passed to these APIs. +The packet ring supports one of two formats. The default format enqueues copies of the original packets +into the rte_ring. If the ``RTE_PDUMP_FLAG_PCAPNG`` is set the mbuf data is extended with header and trailer +to match the format of Pcapng enhanced packet block. The enhanced packet block has meta-data such as the +timestamp, port and queue the packet was captured on. It is up to the application consuming the +packets from the ring to select the format desired. + The library APIs ``rte_pdump_disable()`` and ``rte_pdump_disable_by_deviceid()`` disables the packet capture. For the calls to these APIs from secondary process, the library creates the "pdump disable" request and sends the request to the primary process over the multi process channel. The primary process takes this request and @@ -74,5 +88,5 @@ function. Use Case: Packet Capturing -------------------------- -The DPDK ``app/pdump`` tool is developed based on this library to capture packets in DPDK. -Users can use this as an example to develop their own packet capturing tools. +The DPDK ``app/dpdk-dumpcap`` utility uses this library +to capture packets in DPDK. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 675b5738348b..ee24cbfdb99d 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -62,6 +62,16 @@ New Features * Added bus-level parsing of the devargs syntax. * Kept compatibility with the legacy syntax as parsing fallback. +* **Enhance Packet capture.** + + * New dpdk-dumpcap program that has most of the features of the + wireshark dumpcap utility including capture of multiple interfaces, + stopping after number of bytes, packets. + * New library for writing pcapng packet capture files. + * Enhancement to the pdump library to support: + * Packet filter with BPF. + * Pcapng format with timestamps and meta-data. + * Fixes packet capture with stripped VLAN tags. Removed Items ------------- diff --git a/doc/guides/tools/dumpcap.rst b/doc/guides/tools/dumpcap.rst new file mode 100644 index 000000000000..90993bd5be5e --- /dev/null +++ b/doc/guides/tools/dumpcap.rst @@ -0,0 +1,80 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Microsoft Corporation. + +.. _dumpcap_tool: + +dpdk-dumpcap Application +======================== + +The ``dpdk-dumpcap`` tool is a Data Plane Development Kit (DPDK) +network traffic dump tool. The interface is similar to the dumpcap tool in Wireshark. +It runs as a secondary DPDK process and lets you capture packets that are +coming into and out of a DPDK primary process. +The ``dpdk-dumpcap`` writes files in Pcapng packet format using +capture file format is pcapng. + +Without any options set it will use DPDK to capture traffic from the first +available DPDK interface and write the received raw packet data, along +with timestamps into a pcapng file. + +If the ``-w`` option is not specified, ``dpdk-dumpcap`` writes to a newly +create file with a name chosen based on interface name and timestamp. +If ``-w`` option is specified, then that file is used. + + .. Note:: + * The ``dpdk-dumpcap`` tool can only be used in conjunction with a primary + application which has the packet capture framework initialized already. + In dpdk, only the ``testpmd`` is modified to initialize packet capture + framework, other applications remain untouched. So, if the ``dpdk-dumpcap`` + tool has to be used with any application other than the testpmd, user + needs to explicitly modify that application to call packet capture + framework initialization code. Refer ``app/test-pmd/testpmd.c`` + code to see how this is done. + + * The ``dpdk-dumpcap`` tool runs as a DPDK secondary process. It exits when + the primary application exits. + + +Running the Application +----------------------- + +To list interfaces available for capture use ``--list-interfaces``. + +To capture on multiple interfaces at once, use multiple ``-I`` flags. + +Example +------- + +.. code-block:: console + + # .//app/dpdk-dumpcap --list-interfaces + 0. 000:00:03.0 + 1. 000:00:03.1 + + # .//app/dpdk-dumpcap -I 0000:00:03.0 -c 6 -w /tmp/sample.pcapng + Packets captured: 6 + Packets received/dropped on interface '0000:00:03.0' 6/0 + + +Limitations +----------- +The following options of Wireshark ``dumpcap`` are not yet implemented: + + * ``-f `` -- needs translation from classic BPF to eBPF. + + * ``-b|--ring-buffer`` -- more complex file management. + + * ``-C `` -- doesn't make sense in DPDK model. + + * ``-t`` -- use a thread per interface + + * Timestamp type. + + * Link data types. Only EN10MB (Ethernet) is supported. + + * Wireless related options: ``-I|--monitor-mode`` and ``-k `` + + +.. Note:: + * The options to ``dpdk-dumpcap`` are like the Wireshark dumpcap program and + are not the same as ``dpdk-pdump`` and other DPDK applications. diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst index 93dde4148e90..b71c12b8f2dd 100644 --- a/doc/guides/tools/index.rst +++ b/doc/guides/tools/index.rst @@ -8,6 +8,7 @@ DPDK Tools User Guides :maxdepth: 2 :numbered: + dumpcap proc_info pdump pmdinfo -- 2.30.2