From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0FE97A0547; Fri, 10 Sep 2021 01:34:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 248F941162; Fri, 10 Sep 2021 01:33:51 +0200 (CEST) Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by mails.dpdk.org (Postfix) with ESMTP id 97EE74114F for ; Fri, 10 Sep 2021 01:33:44 +0200 (CEST) Received: by mail-pl1-f176.google.com with SMTP id bg1so2133035plb.13 for ; Thu, 09 Sep 2021 16:33:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+vkeq9G4kEH23sOuf26gc9ZgFod9x5H9VZ1fsvuCTQ0=; b=Qnvj+j5ao+GMcQ9SdN+g+UbUKfADkJo3NGlta+ynjFIQcIVnKfMb/v1YBdmpG9rGbN XpbFt5KSAa0WJ44tPa3qorLJN/PiUARebhczEVfPnn/n8lu+VIpuPmBSZd00TCdTTn+S D1uPenT5DOHTfuFZHOa4E4zguQdcAamUbC5GYuVOS2rKSwr62NJ1KQGQOwT3QqYu7OVT 2+eM/Y3ZYUyvxVtTsqO5mpLINtgwwrDP8v5EDB+ovlyt6MiD1LIgN5/UYTIRHF+HrQCv lRo3y6yRtjXUIUJme5zKWWjGnXW0NtayK4VGcCotZ8VIqsC7vXycC7qYAmWg+BUI/7Yd fFtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+vkeq9G4kEH23sOuf26gc9ZgFod9x5H9VZ1fsvuCTQ0=; b=is+zePSlirFoVIGRYIm2I/JAjX9BVAl4T08OzW+0FtVYJNKPVt/M8kk3b8ufio8Tog Syday7BJOS1x5DG4fDGFan5Q9FtptwAww8c8WnHrvKuEZhl1z7cbAbVO0/kjX4aiCvcZ 8Pibdj5pBNkz/ylFccveqbFu3wUv03sGg5cDp5viOUAlwIShnyXLeAxu1EfKjGMoQ0Zt udVB4uNS/xtrwxRbLBobiLKf2Uvd372ulQcMkgl0B1uct9PhL1THjDuZ29hPKP7EjEHo ePAY+S4v/kbU9i09508RQ/gyVhH7y9HmFMA1aFhkWfIcNVLgP/cwN7LV9skYIcF9Vrrk QXug== X-Gm-Message-State: AOAM533IV31o/2cbZ/aTsXwlVTbEllHXAEylouj4Rd/ifFeAsDmDaq1x WxoxpyOB0dhPkUkrzbrF5HKR0iNykd3D9g== X-Google-Smtp-Source: ABdhPJxtcAlOsS1L0M0rM3Sus0lz2Ao5VwI1Mp5D+mItem1OvPnySYWDq70d//qcqdgjtPTp7N9F+Q== X-Received: by 2002:a17:902:784c:b0:138:f4e5:9df8 with SMTP id e12-20020a170902784c00b00138f4e59df8mr4897832pln.14.1631230422932; Thu, 09 Sep 2021 16:33:42 -0700 (PDT) Received: from hermes.local (204-195-33-123.wavecable.com. [204.195.33.123]) by smtp.gmail.com with ESMTPSA id z6sm3172255pjn.27.2021.09.09.16.33.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Sep 2021 16:33:41 -0700 (PDT) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Date: Thu, 9 Sep 2021 16:33:28 -0700 Message-Id: <20210909233329.190021-10-stephen@networkplumber.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210909233329.190021-1-stephen@networkplumber.org> References: <20210903004732.109023-1-stephen@networkplumber.org> <20210909233329.190021-1-stephen@networkplumber.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v6 09/10] doc: changes for new pcapng and dumpcap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Describe the new packet capture library and utilities Signed-off-by: Stephen Hemminger --- doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf.in | 1 + .../howto/img/packet_capture_framework.svg | 96 +++++++++---------- doc/guides/howto/packet_capture_framework.rst | 67 ++++++------- doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pcapng_lib.rst | 24 +++++ doc/guides/prog_guide/pdump_lib.rst | 28 ++++-- doc/guides/rel_notes/release_21_11.rst | 10 ++ doc/guides/tools/dumpcap.rst | 86 +++++++++++++++++ doc/guides/tools/index.rst | 1 + 10 files changed, 228 insertions(+), 87 deletions(-) create mode 100644 doc/guides/prog_guide/pcapng_lib.rst create mode 100644 doc/guides/tools/dumpcap.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index 1992107a0356..ee07394d1c78 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -223,3 +223,4 @@ The public API headers are grouped by topics: [experimental APIs] (@ref rte_compat.h), [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h) + [pcapng] (@ref rte_pcapng.h) diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 325a0195c6ab..aba17799a9a1 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -58,6 +58,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/metrics \ @TOPDIR@/lib/node \ @TOPDIR@/lib/net \ + @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ diff --git a/doc/guides/howto/img/packet_capture_framework.svg b/doc/guides/howto/img/packet_capture_framework.svg index a76baf71fdee..1c2646a81096 100644 --- a/doc/guides/howto/img/packet_capture_framework.svg +++ b/doc/guides/howto/img/packet_capture_framework.svg @@ -1,6 +1,4 @@ - - + inkscape:version="1.0.2 (e86c870879, 2021-01-15)" + sodipodi:docname="packet_capture_framework.svg"> + gradientTransform="matrix(1.1457977,0,0,0.99944907,-151.97019,745.05014)" /> + inkscape:window-width="2560" + inkscape:window-height="1414" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" + inkscape:document-rotation="0" /> @@ -296,7 +295,7 @@ image/svg+xml - + @@ -321,15 +320,15 @@ y="790.82452" /> DPDK Primary Application + y="807.3205" + style="font-size:12.5px;line-height:1.25">DPDK Primary Application dpdk-pdumpdpdk-dumpcaptool + id="tspan4193" + style="font-size:12.5px;line-height:1.25">tool PCAP PMD + id="tspan4193-3" + style="font-size:12.5px;line-height:1.25">librte_pcapng dpdk_port0 + id="tspan4193-6" + style="font-size:12.5px;line-height:1.25">dpdk_port0 librte_pdump + id="tspan4193-2" + style="font-size:12.5px;line-height:1.25">librte_pdump + width="108.21974" + height="35.335861" + x="297.9809" + y="985.62219" /> capture.pcap + id="tspan4193-3-2" + style="font-size:12.5px;line-height:1.25">capture.pcapng Traffic Generator + id="tspan4193-3-2-7" + style="font-size:12.5px;line-height:1.25">Traffic Generator ` tool is developed based on the -``librte_pdump`` library. It runs as a DPDK secondary process and is capable -of enabling or disabling packet capture on DPDK ports. The ``dpdk-pdump`` tool -provides command-line options with which users can request enabling or -disabling of the packet capture on DPDK ports. +The :ref:`librte_pcapng ` library provides the APIs to format +packets and write them to a file in Pcapng format. + + +The :ref:`dpdk-dumpcap ` is a tool that captures packets in +like Wireshark dumpcap does for Linux. It runs as a DPDK secondary process and +captures packets from one or more interfaces and writes them to a file +in Pcapng format. The ``dpdk-dumpcap`` tool is designed to take +most of the same options as the Wireshark ``dumpcap`` command. -The application which initializes the packet capture framework will be a primary process -and the application that enables or disables the packet capture will -be a secondary process. The primary process sends the Rx and Tx packets from the DPDK ports -to the secondary process. +Without any options it will use the packet capture framework to +capture traffic from the first available DPDK port. In DPDK the ``testpmd`` application can be used to initialize the packet -capture framework and acts as a server, and the ``dpdk-pdump`` tool acts as a +capture framework and acts as a server, and the ``dpdk-dumpcap`` tool acts as a client. To view Rx or Tx packets of ``testpmd``, the application should be -launched first, and then the ``dpdk-pdump`` tool. Packets from ``testpmd`` -will be sent to the tool, which then sends them on to the Pcap PMD device and -that device writes them to the Pcap file or to an external interface depending -on the command-line option used. +launched first, and then the ``dpdk-dumpcap`` tool. Packets from ``testpmd`` +will be sent to the tool, and then to the Pcapng file. Some things to note: -* The ``dpdk-pdump`` tool can only be used in conjunction with a primary +* All tools using ``librte_pdump`` can only be used in conjunction with a primary application which has the packet capture framework initialized already. In dpdk, only ``testpmd`` is modified to initialize packet capture framework, - other applications remain untouched. So, if the ``dpdk-pdump`` tool has to + other applications remain untouched. So, if the ``dpdk-dumpcap`` tool has to be used with any application other than the testpmd, the user needs to explicitly modify that application to call the packet capture framework initialization code. Refer to the ``app/test-pmd/testpmd.c`` code and look for ``pdump`` keyword to see how this is done. -* The ``dpdk-pdump`` tool depends on the libpcap based PMD. +* The ``dpdk-pdump`` tool is an older tool created as demonstration of ``librte_pdump`` + library. The ``dpdk-pdump`` tool provides more limited functionality and + and depends on the Pcap PMD. It is retained only for compatibility reasons; + users should use ``dpdk-dumpcap`` instead. Test Environment ---------------- -The overview of using the Packet Capture Framework and the ``dpdk-pdump`` tool +The overview of using the Packet Capture Framework and the ``dpdk-dumpcap`` utility for packet capturing on the DPDK port in :numref:`figure_packet_capture_framework`. @@ -66,13 +70,13 @@ for packet capturing on the DPDK port in .. figure:: img/packet_capture_framework.* - Packet capturing on a DPDK port using the dpdk-pdump tool. + Packet capturing on a DPDK port using the dpdk-dumpcap utility. Running the Application ----------------------- -The following steps demonstrate how to run the ``dpdk-pdump`` tool to capture +The following steps demonstrate how to run the ``dpdk-dumpcap`` tool to capture Rx side packets on dpdk_port0 in :numref:`figure_packet_capture_framework` and inspect them using ``tcpdump``. @@ -80,16 +84,15 @@ inspect them using ``tcpdump``. sudo /app/dpdk-testpmd -c 0xf0 -n 4 -- -i --port-topology=chained -#. Launch the pdump tool as follows:: +#. Launch the dpdk-dump as follows:: - sudo /app/dpdk-pdump -- \ - --pdump 'port=0,queue=*,rx-dev=/tmp/capture.pcap' + sudo /app/dpdk-dumpcap -w /tmp/capture.pcapng #. Send traffic to dpdk_port0 from traffic generator. - Inspect packets captured in the file capture.pcap using a tool - that can interpret Pcap files, for example tcpdump:: + Inspect packets captured in the file capture.pcap using a tool such as + tcpdump or tshark that can interpret Pcapng files:: - $tcpdump -nr /tmp/capture.pcap + $ tcpdump -nr /tmp/capture.pcapng reading from file /tmp/capture.pcap, link-type EN10MB (Ethernet) 11:11:36.891404 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 11:11:36.891442 IP 4.4.4.4.whois++ > 3.3.3.3.whois++: UDP, length 18 diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 2dce507f46a3..b440c77c2ba1 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -43,6 +43,7 @@ Programmer's Guide ip_fragment_reassembly_lib generic_receive_offload_lib generic_segmentation_offload_lib + pcapng_lib pdump_lib multi_proc_support kernel_nic_interface diff --git a/doc/guides/prog_guide/pcapng_lib.rst b/doc/guides/prog_guide/pcapng_lib.rst new file mode 100644 index 000000000000..36379b530a57 --- /dev/null +++ b/doc/guides/prog_guide/pcapng_lib.rst @@ -0,0 +1,24 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2016 Intel Corporation. + +.. _pcapng_library: + +Packet Capture File Writer +========================== + +Pcapng is a library for creating files in Pcapng file format. +The Pcapng file format is the default capture file format for modern +network capture processing tools. It can be read by wireshark and tcpdump. + +Usage +----- + +Before the library can be used the function ``rte_pcapng_init`` +should be called once to initialize timestamp computation. + + +References +---------- +* Draft RFC https://www.ietf.org/id/draft-tuexen-opsawg-pcapng-03.html + +* Project repository https://github.com/pcapng/pcapng/ diff --git a/doc/guides/prog_guide/pdump_lib.rst b/doc/guides/prog_guide/pdump_lib.rst index 62c0b015b2fe..9af91415e5ea 100644 --- a/doc/guides/prog_guide/pdump_lib.rst +++ b/doc/guides/prog_guide/pdump_lib.rst @@ -3,10 +3,10 @@ .. _pdump_library: -The librte_pdump Library -======================== +The Packet Capture Library +========================== -The ``librte_pdump`` library provides a framework for packet capturing in DPDK. +The DPDK ``pdump`` library provides a framework for packet capturing in DPDK. The library does the complete copy of the Rx and Tx mbufs to a new mempool and hence it slows down the performance of the applications, so it is recommended to use this library for debugging purposes. @@ -23,11 +23,19 @@ or disable the packet capture, and to uninitialize it. * ``rte_pdump_enable()``: This API enables the packet capture on a given port and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf()`` + This API enables the packet capture on a given port and queue. + It also allows setting an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_enable_by_deviceid()``: This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. - Note: The filter option in the API is a place holder for future enhancements. + +* ``rte_pdump_enable_bpf_by_deviceid()`` + This API enables the packet capture on a given device id (``vdev name or pci address``) and queue. + It also allows seating an optional filter using DPDK BPF interpreter and + setting the captured packet length. * ``rte_pdump_disable()``: This API disables the packet capture on a given port and queue. @@ -61,6 +69,12 @@ and enables the packet capture by registering the Ethernet RX and TX callbacks f and queue combinations. Then the primary process will mirror the packets to the new mempool and enqueue them to the rte_ring that secondary process have passed to these APIs. +The packet ring supports one of two formats. The default format enqueues copies of the original packets +into the rte_ring. If the ``RTE_PDUMP_FLAG_PCAPNG`` is set the mbuf data is extended with header and trailer +to match the format of Pcapng enhanced packet block. The enhanced packet block has meta-data such as the +timestamp, port and queue the packet was captured on. It is up to the application consuming the +packets from the ring to select the format desired. + The library APIs ``rte_pdump_disable()`` and ``rte_pdump_disable_by_deviceid()`` disables the packet capture. For the calls to these APIs from secondary process, the library creates the "pdump disable" request and sends the request to the primary process over the multi process channel. The primary process takes this request and @@ -74,5 +88,5 @@ function. Use Case: Packet Capturing -------------------------- -The DPDK ``app/pdump`` tool is developed based on this library to capture packets in DPDK. -Users can use this as an example to develop their own packet capturing tools. +The DPDK ``app/dpdk-dumpcap`` utility uses this library +to capture packets in DPDK. diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 675b5738348b..ee24cbfdb99d 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -62,6 +62,16 @@ New Features * Added bus-level parsing of the devargs syntax. * Kept compatibility with the legacy syntax as parsing fallback. +* **Enhance Packet capture.** + + * New dpdk-dumpcap program that has most of the features of the + wireshark dumpcap utility including capture of multiple interfaces, + stopping after number of bytes, packets. + * New library for writing pcapng packet capture files. + * Enhancement to the pdump library to support: + * Packet filter with BPF. + * Pcapng format with timestamps and meta-data. + * Fixes packet capture with stripped VLAN tags. Removed Items ------------- diff --git a/doc/guides/tools/dumpcap.rst b/doc/guides/tools/dumpcap.rst new file mode 100644 index 000000000000..664ea0c79802 --- /dev/null +++ b/doc/guides/tools/dumpcap.rst @@ -0,0 +1,86 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Microsoft Corporation. + +.. _dumpcap_tool: + +dpdk-dumpcap Application +======================== + +The ``dpdk-dumpcap`` tool is a Data Plane Development Kit (DPDK) +network traffic dump tool. The interface is similar to the dumpcap tool in Wireshark. +It runs as a secondary DPDK process and lets you capture packets that are +coming into and out of a DPDK primary process. +The ``dpdk-dumpcap`` writes files in Pcapng packet format using +capture file format is pcapng. + +Without any options set it will use DPDK to capture traffic from the first +available DPDK interface and write the received raw packet data, along +with timestamps into a pcapng file. + +If the ``-w`` option is not specified, ``dpdk-dumpcap`` writes to a newly +create file with a name chosen based on interface name and timestamp. +If ``-w`` option is specified, then that file is used. + + .. Note:: + * The ``dpdk-dumpcap`` tool can only be used in conjunction with a primary + application which has the packet capture framework initialized already. + In dpdk, only the ``testpmd`` is modified to initialize packet capture + framework, other applications remain untouched. So, if the ``dpdk-dumpcap`` + tool has to be used with any application other than the testpmd, user + needs to explicitly modify that application to call packet capture + framework initialization code. Refer ``app/test-pmd/testpmd.c`` + code to see how this is done. + + * The ``dpdk-dumpcap`` tool runs as a DPDK secondary process. It exits when + the primary application exits. + + +Running the Application +----------------------- + +To list interfaces available for capture use ``--list-interfaces``. + +To filter packets in style of *tshark* use the ``-f`` flag. + +To capture on multiple interfaces at once, use multiple ``-I`` flags. + +Example +------- + +.. code-block:: console + + # .//app/dpdk-dumpcap --list-interfaces + 0. 000:00:03.0 + 1. 000:00:03.1 + + # .//app/dpdk-dumpcap -I 0000:00:03.0 -c 6 -w /tmp/sample.pcapng + Packets captured: 6 + Packets received/dropped on interface '0000:00:03.0' 6/0 + + # .//app/dpdk-dumpcap -f 'tcp port 80' + Packets captured: 6 + Packets received/dropped on interface '0000:00:03.0' 10/8 + + +Limitations +----------- +The following option of Wireshark ``dumpcap`` is not yet implemented: + + * ``-b|--ring-buffer`` -- more complex file management. + +The following options do not make sense in the context of DPDK. + + * ``-C `` -- its a kernel thing + + * ``-t`` -- use a thread per interface + + * Timestamp type. + + * Link data types. Only EN10MB (Ethernet) is supported. + + * Wireless related options: ``-I|--monitor-mode`` and ``-k `` + + +.. Note:: + * The options to ``dpdk-dumpcap`` are like the Wireshark dumpcap program and + are not the same as ``dpdk-pdump`` and other DPDK applications. diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst index 93dde4148e90..b71c12b8f2dd 100644 --- a/doc/guides/tools/index.rst +++ b/doc/guides/tools/index.rst @@ -8,6 +8,7 @@ DPDK Tools User Guides :maxdepth: 2 :numbered: + dumpcap proc_info pdump pmdinfo -- 2.30.2