From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from tama50.ecl.ntt.co.jp (tama50.ecl.ntt.co.jp [129.60.39.147]) by dpdk.org (Postfix) with ESMTP id 46F495F2F for ; Mon, 18 Feb 2019 12:50:49 +0100 (CET) Received: from vc1.ecl.ntt.co.jp (vc1.ecl.ntt.co.jp [129.60.86.153]) by tama50.ecl.ntt.co.jp (8.13.8/8.13.8) with ESMTP id x1IBomtd029194; Mon, 18 Feb 2019 20:50:48 +0900 Received: from vc1.ecl.ntt.co.jp (localhost [127.0.0.1]) by vc1.ecl.ntt.co.jp (Postfix) with ESMTP id 636D8EA8180; Mon, 18 Feb 2019 20:50:48 +0900 (JST) Received: from localhost.localdomain (lobster.nslab.ecl.ntt.co.jp [129.60.13.95]) by vc1.ecl.ntt.co.jp (Postfix) with ESMTP id 410CBEA817A; Mon, 18 Feb 2019 20:50:48 +0900 (JST) From: ogawa.yasufumi@lab.ntt.co.jp To: spp@dpdk.org, ferruh.yigit@intel.com, ogawa.yasufumi@lab.ntt.co.jp Date: Mon, 18 Feb 2019 20:48:12 +0900 Message-Id: <1550490511-31683-2-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550490511-31683-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> References: <1550490511-31683-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-TM-AS-MML: disable Subject: [spp] [PATCH 01/20] docs: move design of SPP VF X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Feb 2019 11:50:51 -0000 From: Yasufumi Ogawa To revise contents of docs, move design of chapter `SPP VF` to `Design`. Signed-off-by: Yasufumi Ogawa --- docs/guides/commands/secondary/spp_mirror.rst | 6 +- docs/guides/commands/secondary/spp_vf.rst | 4 +- docs/guides/design/spp_secondary.rst | 159 +++ docs/guides/images/spp_vf/spp_mirror_overview.svg | 1433 --------------------- docs/guides/spp_vf/design.rst | 140 -- docs/guides/spp_vf/index.rst | 2 - docs/guides/spp_vf/overview.rst | 107 -- 7 files changed, 164 insertions(+), 1687 deletions(-) delete mode 100644 docs/guides/images/spp_vf/spp_mirror_overview.svg delete mode 100644 docs/guides/spp_vf/design.rst delete mode 100644 docs/guides/spp_vf/overview.rst diff --git a/docs/guides/commands/secondary/spp_mirror.rst b/docs/guides/commands/secondary/spp_mirror.rst index bb60f05..e92967a 100644 --- a/docs/guides/commands/secondary/spp_mirror.rst +++ b/docs/guides/commands/secondary/spp_mirror.rst @@ -199,8 +199,8 @@ by duplicating the packets. spp > mirror 2; port add vhost:1 tx mr1 Adding port may cause component to start packet forwarding. Please see -detail in -:ref:`design spp_mirror`. +details in +:ref:`design spp_mirror`. Until one rx and two tx ports are registered, ``spp_mirror`` does not start forwarding. If it is requested to add more than one rx and two tx ports, it @@ -230,7 +230,7 @@ Here is some examples. .. note:: Deleting port may cause component to stop packet forwarding. - Please see detail in :ref:`design spp_mirror`. + Please see detail in :ref:`design spp_mirror`. exit ---- diff --git a/docs/guides/commands/secondary/spp_vf.rst b/docs/guides/commands/secondary/spp_vf.rst index d31688e..7d63111 100644 --- a/docs/guides/commands/secondary/spp_vf.rst +++ b/docs/guides/commands/secondary/spp_vf.rst @@ -278,7 +278,7 @@ adds VLAN tag before sending packet outside. Adding port may cause component to start packet forwarding. Please see detail in -:ref:`design spp_vf`. +:ref:`design spp_vf`. Until one rx port and one tx port are added, forwarder does not start packet forwarding. If it is requested to add more than one rx and one tx port, it @@ -315,7 +315,7 @@ Here is an example. .. note:: Deleting port may cause component to stop packet forwarding. - Please see detail in :ref:`design spp_vf`. + Please see detail in :ref:`design spp_vf`. .. _commands_spp_vf_classifier_table: diff --git a/docs/guides/design/spp_secondary.rst b/docs/guides/design/spp_secondary.rst index a784e10..38c3d01 100644 --- a/docs/guides/design/spp_secondary.rst +++ b/docs/guides/design/spp_secondary.rst @@ -84,3 +84,162 @@ main thread accepts it. :width: 70% Main thread and worker thread in spp_nfv + + +.. _spp_design_spp_sec_vf: + +spp_vf +------ + +``spp_vf`` provides a SR-IOV like network feature. + +``spp_vf`` forwards incoming packets to several destination VMs by referring +MAC address like as a Virtual Function (VF) of SR-IOV. + +``spp_vf`` is a multi-process and multi-thread application. +Each of ``spp_vf`` has one manager thread and worker threads called as +components. +The manager thread provides a function for parsing a command and creating the +components. +The component threads have its own multiple components, ports and classifier +tables including Virtual MAC address. +There are three types of components, ``forwarder``, +``merger`` and ``classifier``. + +This is an example of network configuration, in which one +``classifier_mac``, +one merger and four forwarders are running in ``spp_vf`` process +for two destinations of vhost interface. +Incoming packets from rx on host1 are sent to each of vhosts of VM +by looking up destination MAC address in the packet. + +.. figure:: ../images/spp_vf/spp_vf_overview.* + :width: 72% + + Classification of spp_vf for two VMs + + +Forwarder +^^^^^^^^^ + +Simply forwards packets from rx to tx port. +Forwarder does not start forwarding until when at least one rx and one tx are +added. + +Merger +^^^^^^ + +Receives packets from multiple rx ports to aggregate +packets and sends to a desctination port. +Merger does not start forwarding until when at least two rx and one tx are +added. + +Classifier +^^^^^^^^^^ + +Sends packets to multiple tx ports based on entries of +MAC address and destination port in a classifier table. +This component also supports VLAN tag. + +For VLAN addressing, classifier has other tables than defalut. +Classifier prepares tables for each of VLAN ID and decides +which of table is referred +if TPID (Tag Protocol Indetifier) is included in a packet and +equals to 0x8100 as defined in IEEE 802.1Q standard. +Classifier does not start forwarding until when at least one rx and two tx +are added. + + +.. _spp_design_spp_sec_mirror: + +spp_mirror +---------- + +``spp_mirror`` is an implementation of +`TaaS +`_ +as a SPP secondary process for port mirroring. +TaaS stands for TAP as a Service. +The keyword ``mirror`` means that it duplicates incoming packets and forwards +to additional destination. + +Mirror +^^^^^^ + +``mirror`` component has one ``rx`` port and two ``tx`` ports. Incoming packets +from ``rx`` port are duplicated and sent to each of ``tx`` ports. + +.. _figure_spp_mirror_design: + +.. figure:: ../images/spp_vf/spp_mirror_design.* + :width: 45% + + Spp_mirror component + +In general, copying packet is time-consuming because it requires to make a new +region on memory space. Considering to minimize impact for performance, +``spp_mirror`` provides a choice of copying methods, ``shallowocopy`` or +``deepcopy``. +The difference between those methods is ``shallowocopy`` does not copy whole of +packet data but share without header actually. +``shallowcopy`` is to share mbuf between packets to get better performance +than ``deepcopy``, but it should be used for read only for the packet. + +.. note:: + + ``shallowcopy`` calls ``rte_pktmbuf_clone()`` internally and + ``deepcopy`` create a new mbuf region. + +You should choose ``deepcopy`` if you use VLAN feature to make no change for +original packet while copied packet is modified. + + +.. _spp_design_spp_sec_pcap: + +spp_pcap +-------- + +SPP provides a connectivity between VM and NIC as a virtual patch panel. +However, for more practical use, operator and/or developer needs to capture +packets. For such use, spp_pcap provides packet capturing feature from +specific port. It is aimed to capture up to 10Gbps packets. + +``spp_pcap`` is a SPP secondary process for capturing packets from specific +``port``. :numref:`figure_spp_pcap_overview` shows an overview of use of +``spp_pcap`` in which ``spp_pcap`` process receives packets from ``phy:0`` +for capturing. + +``spp_pcap`` provides packet capturing capability as a SPP secondary process. +``spp_pcap`` has one manager thread like spp_vf, and has two types of worker +threads unlike spp_vf. + + +.. _figure_spp_pcap_overview: + +.. figure:: ../images/spp_vf/spp_pcap_overview.* + :width: 50% + + Overview of spp_pcap + +``spp_pcap`` cosisits of main thread, ``receiver`` thread and one or more +``wirter`` threads. As design policy, the number of ``receiver`` is fixed +to 1 because to make it simple and it is enough for task of receiving. +``spp_pcap`` requires at least three lcores, and assign to from master, +``receiver`` and then the rest of ``writer`` threads respectively. + +Incoming packets are received by ``receiver`` thread and transferred to +``writer`` threads via ring buffers between threads. + +Several ``writer`` work in parallel to store packets as files in LZ4 +format. You can capture a certain amount of heavy traffic by using much +``writer`` threads. + +:numref:`figure_spp_pcap_design` shows an usecase of ``spp_pcap`` in which +packets from ``phy:0`` are captured by using three ``writer`` threads. + +.. _figure_spp_pcap_design: + +.. figure:: ../images/spp_vf/spp_pcap_design.* + :width: 55% + + spp_pcap internal structure diff --git a/docs/guides/images/spp_vf/spp_mirror_overview.svg b/docs/guides/images/spp_vf/spp_mirror_overview.svg deleted file mode 100644 index 9414fdf..0000000 --- a/docs/guides/images/spp_vf/spp_mirror_overview.svg +++ /dev/null @@ -1,1433 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - host - NIC - - - NIC - NIC - VM2 - VM1 - - - NIC - VM3 - - - spp_vf - App1 - App2 - Monitor - spp_mirror - - - forwarder0 - forwarder1 - forwarder2 - forwarde3 - merger0 - classifier - mirror - merger1 - - - ring:2 - ring:1 - ring:3 - ring:0 - vhost:0 - vhost:1 - vhost:4 - ring:5 - ring:4 - vhost:2 - vhost:3 - - diff --git a/docs/guides/spp_vf/design.rst b/docs/guides/spp_vf/design.rst deleted file mode 100644 index b1e864a..0000000 --- a/docs/guides/spp_vf/design.rst +++ /dev/null @@ -1,140 +0,0 @@ -.. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2010-2014 Intel Corporation - -.. _spp_vf_design: - -Design -====== - -.. _spp_vf_design_port: - -Ports ------ - -Both of ``spp_vf`` and ``spp_mirror`` support three types of port, -``phy``, ``ring`` and ``vhost``. -``phy`` port is used to retrieve packets from specific physical NIC or sent to. -``ring`` is basically used to connect a process or a thread to make a network -path between them. -``vhost`` is used to forward packets from a VM or sent to. - - -.. _spp_vf_design_spp_vf: - -spp_vf ------- - -``spp_vf`` is a kind of secondary process and consists of several -threads called component. -There are three types of components, ``forwarder``, -``merger`` and ``classifier``. - -.. figure:: ../images/spp_vf/spp_vf_overview.* - :width: 75% - - SPP VF components - -Forwarder -^^^^^^^^^ - -Simply forwards packets from rx to tx port. -Forwarder does not start forwarding until when at least one rx and one tx are -added. - -Merger -^^^^^^ - -Receives packets from multiple rx ports to aggregate -packets and sends to a desctination port. -Merger does not start forwarding until when at least two rx and one tx are -added. - -Classifier -^^^^^^^^^^ - -Sends packets to multiple tx ports based on entries of -MAC address and destination port in a classifier table. -This component also supports VLAN tag. - -For VLAN addressing, classifier has other tables than defalut. -Classifier prepares tables for each of VLAN ID and decides -which of table is referred -if TPID (Tag Protocol Indetifier) is included in a packet and -equals to 0x8100 as defined in IEEE 802.1Q standard. -Classifier does not start forwarding until when at least one rx and two tx are -added. - - -.. _spp_vf_design_spp_mirror: - -spp_mirror ----------- - -``spp_mirror`` is another kind of secondary process. The keyword ``mirror`` -means that it duplicates incoming packets and forwards to additional -destination. -It supports only one type of component called ``mirror`` for duplicating. -In :numref:`figure_spp_mirror_design`, incoming packets are duplicated with -``mirror`` component and sent to original and additional destinations. - -.. _figure_spp_mirror_design: - -.. figure:: ../images/spp_vf/spp_mirror_design.* - :width: 45% - - Spp_mirror component - -Mirror -^^^^^^ - -``mirror`` component has one ``rx`` port and two ``tx`` ports. Incoming packets -from ``rx`` port are duplicated and sent to each of ``tx`` ports. - -In general, copying packet is time-consuming because it requires to make a new -region on memory space. Considering to minimize impact for performance, -``spp_mirror`` provides a choice of copying methods, ``shallowocopy`` or -``deepcopy``. -The difference between those methods is ``shallowocopy`` does not copy whole of -packet data but share without header actually. -``shallowcopy`` is to share mbuf between packets to get better performance -than ``deepcopy``, but it should be used for read only for the packet. - -.. note:: - - ``shallowcopy`` calls ``rte_pktmbuf_clone()`` internally and - ``deepcopy`` create a new mbuf region. - -You should choose ``deepcopy`` if you use VLAN feature to make no change for -original packet while copied packet is modified. - - -.. _spp_vf_design_spp_pcap: - -spp_pcap --------- - -``spp_pcap`` cosisits of main thread, ``receiver`` thread and one or more -``wirter`` threads. As design policy, the number of ``receiver`` is fixed -to 1 because to make it simple and it is enough for task of receiving. -``spp_pcap`` requires at least three lcores, and assign to from master, -``receiver`` and then the rest of ``writer`` threads respectively. - -Incoming packets are received by ``receiver`` thread and transferred to -``writer`` threads via ring buffers between threads. - -Several ``writer`` work in parallel to store packets as files in LZ4 -format. You can capture a certain amount of heavy traffic by using much -``writer`` threads. - -:numref:`figure_spp_pcap_design` shows an usecase of ``spp_pcap`` in which -packets from ``phy:0`` are captured by using three ``writer`` threads. - -.. _figure_spp_pcap_design: - -.. figure:: ../images/spp_vf/spp_pcap_design.* - :width: 55% - - spp_pcap internal structure - -.. _spp_pcap_design_output_file_format: - diff --git a/docs/guides/spp_vf/index.rst b/docs/guides/spp_vf/index.rst index c0ce125..e5b79ba 100644 --- a/docs/guides/spp_vf/index.rst +++ b/docs/guides/spp_vf/index.rst @@ -8,8 +8,6 @@ SPP VF :maxdepth: 2 :numbered: - overview - design gsg/index use_cases/index explain/index diff --git a/docs/guides/spp_vf/overview.rst b/docs/guides/spp_vf/overview.rst deleted file mode 100644 index adaced6..0000000 --- a/docs/guides/spp_vf/overview.rst +++ /dev/null @@ -1,107 +0,0 @@ -.. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2019 Nippon Telegraph and Telephone Corporation - -.. _spp_vf_overview: - -Overview -======== - -This section describes an overview of SPP's extensions, ``spp_vf`` , ``spp_mirror`` and ``spp_pcap``. -SPP provides a connectivity between DPDK processes as a virtual patch panel. -However, It is not sufficient considering more practical usecases. -For instance, you might need a classification for incoming packets if you have -multiple processes as destinations. You might also need to duplicate packets -for monitoring or inspection. - -``spp_vf`` is a SPP secondary process for forwarding packets to multiple -destination ports with a MAC address classification. It is also supported to -merge from several source ports to aggregate packets to a destination port. - -``spp_mirror`` is an another kind of secondary process to duplicate incoming -packets. This feature is intended to use as a -`TaaS -`_ -feature for port mirroring introduced in OpenStack. - -``spp_pcap`` is an another kind of secondary process to capture incoming packets. This feature is intended to capture up to 10Gbps packets. - - -spp_vf ------- - -``spp_vf`` provides a SR-IOV like network feature. - -``spp_vf`` forwards incoming packets to several destination VMs by referring -MAC address like as a Virtual Function (VF) of SR-IOV. - -``spp_vf`` is a multi-process and multi-thread application. -Each of ``spp_vf`` has one manager thread and worker threads called as -components. -The manager thread provides a function for parsing a command and creating the -components. -The component threads have its own multiple components, ports and classifier -tables including Virtual MAC address. - -This is an example of network configuration, in which one -``classifier_mac``, -one merger and four forwarders are running in ``spp_vf`` process -for two destinations of vhost interface. -Incoming packets from rx on host1 are sent to each of vhosts of VM -by looking up destination MAC address in the packet. - -.. figure:: ../images/spp_vf/spp_vf_overview.* - :width: 70% - - Overview of spp_vf - - -spp_mirror ----------- - -``spp_mirror`` is an implementation of -`TaaS -`_ -as a SPP secondary process for port mirroring. -TaaS stands for TAP as a Service. - -``spp_mirror`` consists of one manager thread and worker threads as similar to -``spp_vf``, but just one worker role for mirroring. -:numref:`figure_spp_mirror_overview` shows an overview of use of ``spp_mirror`` -in which ``spp_mirror`` process receives packets from ``ring:0`` in ``spp_vf`` -for duplicating the packets for monitoring. -Worker thread ``mirror`` has two of tx rings for forwaring the packets to the -destination and a monitor application running on VM3. - -.. _figure_spp_mirror_overview: - -.. figure:: ../images/spp_vf/spp_mirror_overview.* - :width: 70% - - Overview of spp_mirror - - -spp_pcap --------- - -This section describes an overview of SPP's extensions, ``spp_pcap``. -SPP provides a connectivity between VM and NIC as a virtual patch panel. -However, for more practical use, operator and/or developer needs to capture -packets. For such use, spp_pcap provides packet capturing feature from -specific port. It is aimed to capture up to 10Gbps packets. - -``spp_pcap`` is a SPP secondary process for capturing packets from specific -``port``. :numref:`figure_spp_pcap_overview` shows an overview of use of -``spp_pcap`` in which ``spp_pcap`` process receives packets from ``phy:0`` -for capturing. - -``spp_pcap`` provides packet capturing capability as a SPP secondary process. -``spp_pcap`` has one manager thread like spp_vf, and has two types of worker -threads unlike spp_vf. - - -.. _figure_spp_pcap_overview: - -.. figure:: ../images/spp_vf/spp_pcap_overview.* - :width: 50% - - Overview of spp_pcap -- 2.7.4