* [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references
@ 2015-04-24 13:11 John McNamara
2015-04-24 13:11 ` [dpdk-dev] [PATCH 1/2] doc: refactored figure numbers " John McNamara
` (4 more replies)
0 siblings, 5 replies; 17+ messages in thread
From: John McNamara @ 2015-04-24 13:11 UTC (permalink / raw)
To: dev
This patch adds automatic figure and table references to the docs. The
figure and table numbers in the generated Html and PDF docs can now be
automatically numbered based on section.
Requires Sphinx >= 1.3.
Advantages:
* Figues/tables are numbered automatically and adding/removing items
no longer requires manual fixing.
Disadvantages:
* Requires Sphinx >= 1.3.
* Table captions are above tables while figure captions are below. This is
a current Sphinx/DocUtils limitation.
* Both the :numref: and :ref: directives are required to link to a figure
or table number and caption.
John McNamara (2):
doc: refactored figure numbers into references
doc: refactored table numbers into references
doc/guides/conf.py | 2 +
doc/guides/nics/index.rst | 18 +-
doc/guides/nics/intel_vf.rst | 37 +-
doc/guides/nics/virtio.rst | 18 +-
doc/guides/nics/vmxnet3.rst | 18 +-
doc/guides/prog_guide/env_abstraction_layer.rst | 8 +-
doc/guides/prog_guide/index.rst | 162 ++-
doc/guides/prog_guide/ivshmem_lib.rst | 8 +-
doc/guides/prog_guide/kernel_nic_interface.rst | 40 +-
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 +-
doc/guides/prog_guide/lpm6_lib.rst | 8 +-
doc/guides/prog_guide/lpm_lib.rst | 8 +-
doc/guides/prog_guide/malloc_lib.rst | 9 +-
doc/guides/prog_guide/mbuf_lib.rst | 20 +-
doc/guides/prog_guide/mempool_lib.rst | 32 +-
doc/guides/prog_guide/multi_proc_support.rst | 9 +-
doc/guides/prog_guide/overview.rst | 9 +-
doc/guides/prog_guide/packet_distrib_lib.rst | 15 +-
doc/guides/prog_guide/packet_framework.rst | 1245 ++++++++--------
doc/guides/prog_guide/qos_framework.rst | 1511 ++++++++++----------
doc/guides/prog_guide/ring_lib.rst | 159 +-
doc/guides/sample_app_ug/dist_app.rst | 20 +-
doc/guides/sample_app_ug/exception_path.rst | 8 +-
doc/guides/sample_app_ug/index.rst | 64 +-
doc/guides/sample_app_ug/intel_quickassist.rst | 11 +-
doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +-
doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +-
.../sample_app_ug/l2_forward_real_virtual.rst | 22 +-
.../sample_app_ug/l3_forward_access_ctrl.rst | 21 +-
doc/guides/sample_app_ug/load_balancer.rst | 9 +-
doc/guides/sample_app_ug/multi_process.rst | 36 +-
doc/guides/sample_app_ug/qos_metering.rst | 46 +-
doc/guides/sample_app_ug/qos_scheduler.rst | 55 +-
doc/guides/sample_app_ug/quota_watermark.rst | 36 +-
doc/guides/sample_app_ug/test_pipeline.rst | 311 ++--
doc/guides/sample_app_ug/vhost.rst | 45 +-
doc/guides/sample_app_ug/vm_power_management.rst | 18 +-
doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +-
doc/guides/xen/pkt_switch.rst | 30 +-
39 files changed, 2029 insertions(+), 2125 deletions(-)
--
1.8.1.4
^ permalink raw reply [flat|nested] 17+ messages in thread* [dpdk-dev] [PATCH 1/2] doc: refactored figure numbers into references 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara @ 2015-04-24 13:11 ` John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 2/2] doc: refactored table " John McNamara ` (3 subsequent siblings) 4 siblings, 0 replies; 17+ messages in thread From: John McNamara @ 2015-04-24 13:11 UTC (permalink / raw) To: dev This change adds automatic figure references to the docs. The figure numbers in the generated Html and PDF docs are now automatically numbered based on section. Requires Sphinx >= 1.3. The patch makes the following changes. * Changes image:: tag to figure:: and moves image caption to the figure. * Adds captions to figures that didn't previously have any. * Un-templates the |image-name| substitution definitions into explicit figure:: tags. They weren't used more than once anyway and Sphinx doesn't support them for figure. * Adds a target to each image that didn't previously have one so that they can be cross-referenced. * Renamed existing image target to match the image name for consistency. * Replaces the Figures lists with automatic :numref: :ref: entries to generate automatic numbering and captions. * Replaces "Figure" references with automatic :numref: references. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/conf.py | 2 + doc/guides/nics/index.rst | 18 ++- doc/guides/nics/intel_vf.rst | 37 ++--- doc/guides/nics/virtio.rst | 18 ++- doc/guides/nics/vmxnet3.rst | 18 ++- doc/guides/prog_guide/env_abstraction_layer.rst | 8 +- doc/guides/prog_guide/index.rst | 92 +++++++----- doc/guides/prog_guide/ivshmem_lib.rst | 8 +- doc/guides/prog_guide/kernel_nic_interface.rst | 40 ++--- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 ++++-- doc/guides/prog_guide/lpm6_lib.rst | 8 +- doc/guides/prog_guide/lpm_lib.rst | 8 +- doc/guides/prog_guide/malloc_lib.rst | 9 +- doc/guides/prog_guide/mbuf_lib.rst | 20 +-- doc/guides/prog_guide/mempool_lib.rst | 32 ++-- doc/guides/prog_guide/multi_proc_support.rst | 9 +- doc/guides/prog_guide/overview.rst | 9 +- doc/guides/prog_guide/packet_distrib_lib.rst | 15 +- doc/guides/prog_guide/packet_framework.rst | 81 +++++----- doc/guides/prog_guide/qos_framework.rst | 163 +++++++-------------- doc/guides/prog_guide/ring_lib.rst | 159 +++++++++++--------- doc/guides/sample_app_ug/dist_app.rst | 20 ++- doc/guides/sample_app_ug/exception_path.rst | 8 +- doc/guides/sample_app_ug/index.rst | 58 ++++---- doc/guides/sample_app_ug/intel_quickassist.rst | 11 +- doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +- doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +-- .../sample_app_ug/l2_forward_real_virtual.rst | 22 +-- .../sample_app_ug/l3_forward_access_ctrl.rst | 21 ++- doc/guides/sample_app_ug/load_balancer.rst | 9 +- doc/guides/sample_app_ug/multi_process.rst | 36 ++--- doc/guides/sample_app_ug/qos_scheduler.rst | 9 +- doc/guides/sample_app_ug/quota_watermark.rst | 36 ++--- doc/guides/sample_app_ug/test_pipeline.rst | 9 +- doc/guides/sample_app_ug/vhost.rst | 45 ++---- doc/guides/sample_app_ug/vm_power_management.rst | 18 +-- doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +- doc/guides/xen/pkt_switch.rst | 30 ++-- 38 files changed, 539 insertions(+), 633 deletions(-) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index b1ef323..1bc031f 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -41,6 +41,8 @@ release = version master_doc = 'index' +numfig = True + latex_documents = [ ('index', 'doc.tex', diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index aadbae3..1ee67fa 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -50,14 +50,20 @@ Network Interface Controller Drivers **Figures** -:ref:`Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode <nic_figure_1>` +:numref:`figure_single_port_nic` :ref:`figure_single_port_nic` -:ref:`Figure 2. SR-IOV Performance Benchmark Setup <nic_figure_2>` +:numref:`figure_perf_benchmark` :ref:`figure_perf_benchmark` -:ref:`Figure 3. Fast Host-based Packet Processing <nic_figure_3>` +:numref:`figure_fast_pkt_proc` :ref:`figure_fast_pkt_proc` -:ref:`Figure 4. SR-IOV Inter-VM Communication <nic_figure_4>` +:numref:`figure_inter_vm_comms` :ref:`figure_inter_vm_comms` -:ref:`Figure 5. Virtio Host2VM Communication Example Using KNI vhost Back End <nic_figure_5>` +:numref:`figure_host_vm_comms` :ref:`figure_host_vm_comms` -:ref:`Figure 6. Virtio Host2VM Communication Example Using Qemu vhost Back End <nic_figure_6>` +:numref:`figure_host_vm_comms_qemu` :ref:`figure_host_vm_comms_qemu` + +:numref:`figure_vmxnet3_int` :ref:`figure_vmxnet3_int` + +:numref:`figure_vswitch_vm` :ref:`figure_vswitch_vm` + +:numref:`figure_vm_vm_comms` :ref:`figure_vm_vm_comms` diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst index e773627..17e83a2 100644 --- a/doc/guides/nics/intel_vf.rst +++ b/doc/guides/nics/intel_vf.rst @@ -49,9 +49,9 @@ SR-IOV Mode Utilization in a DPDK Environment The DPDK uses the SR-IOV feature for hardware-based I/O sharing in IOV mode. Therefore, it is possible to partition SR-IOV capability on Ethernet controller NIC resources logically and expose them to a virtual machine as a separate PCI function called a "Virtual Function". -Refer to Figure 10. +Refer to :numref:`figure_single_port_nic`. -Therefore, a NIC is logically distributed among multiple virtual machines (as shown in Figure 10), +Therefore, a NIC is logically distributed among multiple virtual machines (as shown in :numref:`figure_single_port_nic`), while still having global data in common to share with the Physical Function and other Virtual Functions. The DPDK fm10kvf, i40evf, igbvf or ixgbevf as a Poll Mode Driver (PMD) serves for the Intel® 82576 Gigabit Ethernet Controller, Intel® Ethernet Controller I350 family, Intel® 82599 10 Gigabit Ethernet Controller NIC, @@ -72,11 +72,12 @@ For more detail on SR-IOV, please refer to the following documents: * `Scalable I/O Virtualized Servers <http://www.intel.com/content/www/us/en/virtualization/server-virtualization/scalable-i-o-virtualized-servers-paper.html>`_ -.. _nic_figure_1: +.. _figure_single_port_nic: -**Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode** +.. figure:: img/single_port_nic.* + + Virtualization for a Single Port NIC in SR-IOV Mode -.. image:: img/single_port_nic.* Physical and Virtual Function Infrastructure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -548,13 +549,14 @@ The setup procedure is as follows: can also be used to bind and unbind devices to a virtual machine in Ubuntu. If this option is used, step 6 in the instructions provided will be different. - * The Virtual Machine Monitor (see Figure 11) is equivalent to a Host OS with KVM installed as described in the instructions. + * The Virtual Machine Monitor (see :numref:`figure_perf_benchmark`) is equivalent to a Host OS with KVM installed as described in the instructions. + +.. _figure_perf_benchmark: -.. _nic_figure_2: +.. figure:: img/perf_benchmark.* -**Figure 2. Performance Benchmark Setup** + Performance Benchmark Setup -.. image:: img/perf_benchmark.* DPDK SR-IOV PMD PF/VF Driver Usage Model ---------------------------------------- @@ -569,14 +571,15 @@ the DPDK VF PMD driver performs the same throughput result as a non-VT native en With such host instance fast packet processing, lots of services such as filtering, QoS, DPI can be offloaded on the host fast path. -Figure 12 shows the scenario where some VMs directly communicate externally via a VFs, +:numref:`figure_fast_pkt_proc` shows the scenario where some VMs directly communicate externally via a VFs, while others connect to a virtual switch and share the same uplink bandwidth. -.. _nic_figure_3: +.. _figure_fast_pkt_proc: + +.. figure:: img/fast_pkt_proc.* -**Figure 3. Fast Host-based Packet Processing** + Fast Host-based Packet Processing -.. image:: img/fast_pkt_proc.* SR-IOV (PF/VF) Approach for Inter-VM Communication -------------------------------------------------- @@ -587,7 +590,7 @@ So VF-to-VF traffic within the same physical port (VM0<->VM1) have hardware acce However, when VF crosses physical ports (VM0<->VM2), there is no such hardware bridge. In this case, the DPDK PMD PF driver provides host forwarding between such VMs. -Figure 13 shows an example. +:numref:`figure_inter_vm_comms` shows an example. In this case an update of the MAC address lookup tables in both the NIC and host DPDK application is required. In the NIC, writing the destination of a MAC address belongs to another cross device VM to the PF specific pool. @@ -598,8 +601,8 @@ that is, the packet is forwarded to the correct PF pool. The SR-IOV NIC switch forwards the packet to a specific VM according to the MAC destination address which belongs to the destination VF on the VM. -.. _nic_figure_4: +.. _figure_inter_vm_comms: -**Figure 4. Inter-VM Communication** +.. figure:: img/inter_vm_comms.* -.. image:: img/inter_vm_comms.* + Inter-VM Communication diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst index 073d980..9f18b3a 100644 --- a/doc/guides/nics/virtio.rst +++ b/doc/guides/nics/virtio.rst @@ -106,11 +106,12 @@ Virtio with kni vhost Back End This section demonstrates kni vhost back end example setup for Phy-VM Communication. -.. _nic_figure_5: +.. _figure_host_vm_comms: -**Figure 5. Host2VM Communication Example Using kni vhost Back End** +.. figure:: img/host_vm_comms.* + + Host2VM Communication Example Using kni vhost Back End -.. image:: img/host_vm_comms.* Host2VM communication example @@ -174,7 +175,9 @@ Host2VM communication example We use testpmd as the forwarding application in this example. - .. image:: img/console.* + .. figure:: img/console.* + + Running testpmd #. Use IXIA packet generator to inject a packet stream into the KNI physical port. @@ -185,11 +188,12 @@ Host2VM communication example Virtio with qemu virtio Back End -------------------------------- -.. _nic_figure_6: +.. _figure_host_vm_comms_qemu: + +.. figure:: img/host_vm_comms_qemu.* -**Figure 6. Host2VM Communication Example Using qemu vhost Back End** + Host2VM Communication Example Using qemu vhost Back End -.. image:: img/host_vm_comms_qemu.* .. code-block:: console diff --git a/doc/guides/nics/vmxnet3.rst b/doc/guides/nics/vmxnet3.rst index 830530d..e919088 100644 --- a/doc/guides/nics/vmxnet3.rst +++ b/doc/guides/nics/vmxnet3.rst @@ -121,7 +121,11 @@ The following prerequisites apply: * Before starting a VM, a VMXNET3 interface to a VM through VMware vSphere Client must be assigned. This is shown in the figure below. -.. image:: img/vmxnet3_int.* +.. _figure_vmxnet3_int: + +.. figure:: img/vmxnet3_int.* + + Assigning a VMXNET3 interface to a VM using VMware vSphere Client .. note:: @@ -142,7 +146,11 @@ VMXNET3 with a Native NIC Connected to a vSwitch This section describes an example setup for Phy-vSwitch-VM-Phy communication. -.. image:: img/vswitch_vm.* +.. _figure_vswitch_vm: + +.. figure:: img/vswitch_vm.* + + VMXNET3 with a Native NIC Connected to a vSwitch .. note:: @@ -164,7 +172,11 @@ VMXNET3 Chaining VMs Connected to a vSwitch The following figure shows an example VM-to-VM communication over a Phy-VM-vSwitch-VM-Phy communication channel. -.. image:: img/vm_vm_comms.* +.. _figure_vm_vm_comms: + +.. figure:: img/vm_vm_comms.* + + VMXNET3 Chaining VMs Connected to a vSwitch .. note:: diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 06289ed..05a7746 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -85,13 +85,12 @@ A check is also performed at initialization time to ensure that the micro archit Then, the main() function is called. The core initialization and launch is done in rte_eal_init() (see the API documentation). It consist of calls to the pthread library (more specifically, pthread_self(), pthread_create(), and pthread_setaffinity_np()). -.. _pg_figure_2: +.. _figure_linuxapp_launch: -**Figure 2. EAL Initialization in a Linux Application Environment** +.. figure:: img/linuxapp_launch.* -.. image3_png has been replaced + EAL Initialization in a Linux Application Environment -|linuxapp_launch| .. note:: @@ -367,4 +366,3 @@ We expect only 50% of CPU spend on packet IO. echo 50000 > pkt_io/cpu.cfs_quota_us -.. |linuxapp_launch| image:: img/linuxapp_launch.* diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index a9966a0..84a657e 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -80,71 +80,97 @@ Programmer's Guide **Figures** -:ref:`Figure 1. Core Components Architecture <pg_figure_1>` +:numref:`figure_architecture-overview` :ref:`figure_architecture-overview` -:ref:`Figure 2. EAL Initialization in a Linux Application Environment <pg_figure_2>` +:numref:`figure_linuxapp_launch` :ref:`figure_linuxapp_launch` -:ref:`Figure 3. Example of a malloc heap and malloc elements within the malloc library <pg_figure_3>` +:numref:`figure_malloc_heap` :ref:`figure_malloc_heap` -:ref:`Figure 4. Ring Structure <pg_figure_4>` +:numref:`figure_ring1` :ref:`figure_ring1` -:ref:`Figure 5. Two Channels and Quad-ranked DIMM Example <pg_figure_5>` +:numref:`figure_ring-enqueue1` :ref:`figure_ring-enqueue1` -:ref:`Figure 6. Three Channels and Two Dual-ranked DIMM Example <pg_figure_6>` +:numref:`figure_ring-enqueue2` :ref:`figure_ring-enqueue2` -:ref:`Figure 7. A mempool in Memory with its Associated Ring <pg_figure_7>` +:numref:`figure_ring-enqueue3` :ref:`figure_ring-enqueue3` -:ref:`Figure 8. An mbuf with One Segment <pg_figure_8>` +:numref:`figure_ring-dequeue1` :ref:`figure_ring-dequeue1` -:ref:`Figure 9. An mbuf with Three Segments <pg_figure_9>` +:numref:`figure_ring-dequeue2` :ref:`figure_ring-dequeue2` -:ref:`Figure 16. Memory Sharing inthe Intel® DPDK Multi-process Sample Application <pg_figure_16>` +:numref:`figure_ring-dequeue3` :ref:`figure_ring-dequeue3` -:ref:`Figure 17. Components of an Intel® DPDK KNI Application <pg_figure_17>` +:numref:`figure_ring-mp-enqueue1` :ref:`figure_ring-mp-enqueue1` -:ref:`Figure 18. Packet Flow via mbufs in the Intel DPDK® KNI <pg_figure_18>` +:numref:`figure_ring-mp-enqueue2` :ref:`figure_ring-mp-enqueue2` -:ref:`Figure 19. vHost-net Architecture Overview <pg_figure_19>` +:numref:`figure_ring-mp-enqueue3` :ref:`figure_ring-mp-enqueue3` -:ref:`Figure 20. KNI Traffic Flow <pg_figure_20>` +:numref:`figure_ring-mp-enqueue4` :ref:`figure_ring-mp-enqueue4` -:ref:`Figure 21. Complex Packet Processing Pipeline with QoS Support <pg_figure_21>` +:numref:`figure_ring-mp-enqueue5` :ref:`figure_ring-mp-enqueue5` -:ref:`Figure 22. Hierarchical Scheduler Block Internal Diagram <pg_figure_22>` +:numref:`figure_ring-modulo1` :ref:`figure_ring-modulo1` -:ref:`Figure 23. Scheduling Hierarchy per Port <pg_figure_23>` +:numref:`figure_ring-modulo2` :ref:`figure_ring-modulo2` -:ref:`Figure 24. Internal Data Structures per Port <pg_figure_24>` +:numref:`figure_memory-management` :ref:`figure_memory-management` -:ref:`Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation <pg_figure_25>` +:numref:`figure_memory-management2` :ref:`figure_memory-management2` -:ref:`Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation <pg_figure_26>` +:numref:`figure_mempool` :ref:`figure_mempool` -:ref:`Figure 27. High-level Block Diagram of the Intel® DPDK Dropper <pg_figure_27>` +:numref:`figure_mbuf1` :ref:`figure_mbuf1` -:ref:`Figure 28. Flow Through the Dropper <pg_figure_28>` +:numref:`figure_mbuf2` :ref:`figure_mbuf2` -:ref:`Figure 29. Example Data Flow Through Dropper <pg_figure_29>` +:numref:`figure_multi_process_memory` :ref:`figure_multi_process_memory` -:ref:`Figure 30. Packet Drop Probability for a Given RED Configuration <pg_figure_30>` +:numref:`figure_kernel_nic_intf` :ref:`figure_kernel_nic_intf` -:ref:`Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) <pg_figure_31>` +:numref:`figure_pkt_flow_kni` :ref:`figure_pkt_flow_kni` -:ref:`Figure 32. Example of packet processing pipeline. The input ports 0 and 1 are connected with the output ports 0, 1 and 2 through tables 0 and 1. <pg_figure_32>` +:numref:`figure_vhost_net_arch2` :ref:`figure_vhost_net_arch2` -:ref:`Figure 33. Sequence of steps for hash table operations in packet processing context <pg_figure_33>` +:numref:`figure_kni_traffic_flow` :ref:`figure_kni_traffic_flow` -:ref:`Figure 34. Data structures for configurable key size hash tables <pg_figure_34>` -:ref:`Figure 35. Bucket search pipeline for key lookup operation (configurable key size hash tables) <pg_figure_35>` +:numref:`figure_pkt_proc_pipeline_qos` :ref:`figure_pkt_proc_pipeline_qos` -:ref:`Figure 36. Pseudo-code for match, match_many and match_pos <pg_figure_36>` +:numref:`figure_hier_sched_blk` :ref:`figure_hier_sched_blk` -:ref:`Figure 37. Data structures for 8-byte key hash tables <pg_figure_37>` +:numref:`figure_sched_hier_per_port` :ref:`figure_sched_hier_per_port` -:ref:`Figure 38. Data structures for 16-byte key hash tables <pg_figure_38>` +:numref:`figure_data_struct_per_port` :ref:`figure_data_struct_per_port` + +:numref:`figure_prefetch_pipeline` :ref:`figure_prefetch_pipeline` + +:numref:`figure_pipe_prefetch_sm` :ref:`figure_pipe_prefetch_sm` + +:numref:`figure_blk_diag_dropper` :ref:`figure_blk_diag_dropper` + +:numref:`figure_flow_tru_droppper` :ref:`figure_flow_tru_droppper` + +:numref:`figure_ex_data_flow_tru_dropper` :ref:`figure_ex_data_flow_tru_dropper` + +:numref:`figure_pkt_drop_probability` :ref:`figure_pkt_drop_probability` + +:numref:`figure_drop_probability_graph` :ref:`figure_drop_probability_graph` + +:numref:`figure_figure32` :ref:`figure_figure32` + +:numref:`figure_figure33` :ref:`figure_figure33` + +:numref:`figure_figure34` :ref:`figure_figure34` + +:numref:`figure_figure35` :ref:`figure_figure35` + +:numref:`figure_figure37` :ref:`figure_figure37` + +:numref:`figure_figure38` :ref:`figure_figure38` + +:numref:`figure_figure39` :ref:`figure_figure39` -:ref:`Figure 39. Bucket search pipeline for key lookup operation (single key size hash tables) <pg_figure_39>` **Tables** diff --git a/doc/guides/prog_guide/ivshmem_lib.rst b/doc/guides/prog_guide/ivshmem_lib.rst index c76d2b3..af4c7a9 100644 --- a/doc/guides/prog_guide/ivshmem_lib.rst +++ b/doc/guides/prog_guide/ivshmem_lib.rst @@ -43,9 +43,11 @@ they are automatically recognized by the DPDK Environment Abstraction Layer (EAL A typical DPDK IVSHMEM use case looks like the following. -.. image28_png has been renamed -|ivshmem| +.. figure:: img/ivshmem.* + + Typical Ivshmem use case + The same could work with several virtual machines, providing host-to-VM or VM-to-VM communication. The maximum number of metadata files is 32 (by default) and each metadata file can contain different (or even the same) hugepages. @@ -154,5 +156,3 @@ It is important to note that once QEMU is started, it holds on to the hugepages As a result, if the user wishes to shut down or restart the IVSHMEM host application, it is not enough to simply shut the application down. The virtual machine must also be shut down (if not, it will hold onto outdated host data). - -.. |ivshmem| image:: img/ivshmem.* diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index bac2215..3402fd2 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -42,15 +42,14 @@ The benefits of using the DPDK KNI are: * Allows an interface with the kernel network stack. -The components of an application using the DPDK Kernel NIC Interface are shown in Figure 17. +The components of an application using the DPDK Kernel NIC Interface are shown in :numref:`figure_kernel_nic_intf`. -.. _pg_figure_17: +.. _figure_kernel_nic_intf: -**Figure 17. Components of a DPDK KNI Application** +.. figure:: img/kernel_nic_intf.* -.. image43_png has been renamed + Components of a DPDK KNI Application -|kernel_nic_intf| The DPDK KNI Kernel Module -------------------------- @@ -114,15 +113,14 @@ To minimize the amount of DPDK code running in kernel space, the mbuf mempool is The kernel module will be aware of mbufs, but all mbuf allocation and free operations will be handled by the DPDK application only. -Figure 18 shows a typical scenario with packets sent in both directions. +:numref:`figure_pkt_flow_kni` shows a typical scenario with packets sent in both directions. -.. _pg_figure_18: +.. _figure_pkt_flow_kni: -**Figure 18. Packet Flow via mbufs in the DPDK KNI** +.. figure:: img/pkt_flow_kni.* -.. image44_png has been renamed + Packet Flow via mbufs in the DPDK KNI -|pkt_flow_kni| Use Case: Ingress ----------------- @@ -189,13 +187,12 @@ it naturally supports both legacy virtio -net and the DPDK PMD virtio. There is a little penalty that comes from the non-polling mode of vhost. However, it scales throughput well when using KNI in multi-thread mode. -.. _pg_figure_19: +.. _figure_vhost_net_arch2: -**Figure 19. vHost-net Architecture Overview** +.. figure:: img/vhost_net_arch.* -.. image45_png has been renamed + vHost-net Architecture Overview -|vhost_net_arch| Packet Flow ~~~~~~~~~~~ @@ -208,13 +205,12 @@ All the packet copying, irrespective of whether it is on the transmit or receive happens in the context of vhost kthread. Every vhost-net device is exposed to a front end virtio device in the guest. -.. _pg_figure_20: +.. _figure_kni_traffic_flow: -**Figure 20. KNI Traffic Flow** +.. figure:: img/kni_traffic_flow.* -.. image46_png has been renamed + KNI Traffic Flow -|kni_traffic_flow| Sample Usage ~~~~~~~~~~~~ @@ -280,11 +276,3 @@ since the kni-vhost does not yet support those features. Even if the option is turned on, kni-vhost will ignore the information that the header contains. When working with legacy virtio on the guest, it is better to turn off unsupported offload features using ethtool -K. Otherwise, there may be problems such as an incorrect L4 checksum error. - -.. |kni_traffic_flow| image:: img/kni_traffic_flow.* - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |pkt_flow_kni| image:: img/pkt_flow_kni.* - -.. |kernel_nic_intf| image:: img/kernel_nic_intf.* diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index 24a1a36..fd3ac5e 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -35,7 +35,10 @@ In addition to Poll Mode Drivers (PMDs) for physical and virtual hardware, DPDK also includes a pure-software library that allows physical PMD's to be bonded together to create a single logical PMD. -|bond-overview| +.. figure:: img/bond-overview.* + + Bonded PMDs + The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups of ``rte_eth_dev`` ports of the same speed and duplex to provide @@ -62,7 +65,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Round-Robin (Mode 0):** -|bond-mode-0| +.. figure:: img/bond-mode-0.* + + Round-Robin (Mode 0) + This mode provides load balancing and fault tolerance by transmission of packets in sequential order from the first available slave device through @@ -72,7 +78,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Active Backup (Mode 1):** -|bond-mode-1| +.. figure:: img/bond-mode-1.* + + Active Backup (Mode 1) + In this mode only one slave in the bond is active at any time, a different slave becomes active if, and only if, the primary active slave fails, @@ -82,7 +91,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Balance XOR (Mode 2):** -|bond-mode-2| +.. figure:: img/bond-mode-2.* + + Balance XOR (Mode 2) + This mode provides transmit load balancing (based on the selected transmission policy) and fault tolerance. The default policy (layer2) uses @@ -101,14 +113,20 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Broadcast (Mode 3):** -|bond-mode-3| +.. figure:: img/bond-mode-3.* + + Broadcast (Mode 3) + This mode provides fault tolerance by transmission of packets on all slave ports. * **Link Aggregation 802.3AD (Mode 4):** -|bond-mode-4| +.. figure:: img/bond-mode-4.* + + Link Aggregation 802.3AD (Mode 4) + This mode provides dynamic link aggregation according to the 802.3ad specification. It negotiates and monitors aggregation groups that share the @@ -128,7 +146,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Transmit Load Balancing (Mode 5):** -|bond-mode-5| +.. figure:: img/bond-mode-5.* + + Transmit Load Balancing (Mode 5) + This mode provides an adaptive transmit load balancing. It dynamically changes the transmitting slave, according to the computed load. Statistics @@ -433,11 +454,3 @@ Create a bonded device in balance mode with two slaves specified by their PCI ad .. code-block:: console $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=2, slave=0000:00a:00.01,slave=0000:004:00.00,xmit_policy=l34' -- --port-topology=chained - -.. |bond-overview| image:: img/bond-overview.* -.. |bond-mode-0| image:: img/bond-mode-0.* -.. |bond-mode-1| image:: img/bond-mode-1.* -.. |bond-mode-2| image:: img/bond-mode-2.* -.. |bond-mode-3| image:: img/bond-mode-3.* -.. |bond-mode-4| image:: img/bond-mode-4.* -.. |bond-mode-5| image:: img/bond-mode-5.* diff --git a/doc/guides/prog_guide/lpm6_lib.rst b/doc/guides/prog_guide/lpm6_lib.rst index abc5adb..87f5066 100644 --- a/doc/guides/prog_guide/lpm6_lib.rst +++ b/doc/guides/prog_guide/lpm6_lib.rst @@ -108,9 +108,11 @@ This is not feasible due to resource restrictions. By splitting the process in different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access per level). -.. image40_png has been renamed -|tbl24_tbl8_tbl8| +.. figure:: img/tbl24_tbl8_tbl8.* + + Table split into different levels + An entry in a table contains the following fields: @@ -231,5 +233,3 @@ Use Case: IPv6 Forwarding ------------------------- The LPM algorithm is used to implement the Classless Inter-Domain Routing (CIDR) strategy used by routers implementing IP forwarding. - -.. |tbl24_tbl8_tbl8| image:: img/tbl24_tbl8_tbl8.* diff --git a/doc/guides/prog_guide/lpm_lib.rst b/doc/guides/prog_guide/lpm_lib.rst index 692e37f..c33e469 100644 --- a/doc/guides/prog_guide/lpm_lib.rst +++ b/doc/guides/prog_guide/lpm_lib.rst @@ -90,9 +90,11 @@ Instead, this approach takes advantage of the fact that rules longer than 24 bit By splitting the process in two different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access, most of the times). -.. image39 has been renamed -|tbl24_tbl8| +.. figure:: img/tbl24_tbl8.* + + Table split into different levels + An entry in tbl24 contains the following fields: @@ -219,5 +221,3 @@ References * Pankaj Gupta, Algorithms for Routing Lookups and Packet Classification, PhD Thesis, Stanford University, 2000 (`http://klamath.stanford.edu/~pankaj/thesis/ thesis_1sided.pdf <http://klamath.stanford.edu/~pankaj/thesis/%20thesis_1sided.pdf>`_ ) - -.. |tbl24_tbl8| image:: img/tbl24_tbl8.* diff --git a/doc/guides/prog_guide/malloc_lib.rst b/doc/guides/prog_guide/malloc_lib.rst index b9298f8..6418fab 100644 --- a/doc/guides/prog_guide/malloc_lib.rst +++ b/doc/guides/prog_guide/malloc_lib.rst @@ -117,13 +117,12 @@ The key fields of the heap structure and their function are described below (see since these are never touched except when they are to be freed again - at which point the pointer to the block is an input to the free() function. -.. _pg_figure_3: +.. _figure_malloc_heap: -**Figure 3. Example of a malloc heap and malloc elements within the malloc library** +.. figure:: img/malloc_heap.* -.. image4_png has been renamed + Example of a malloc heap and malloc elements within the malloc library -|malloc_heap| Structure: malloc_elem ^^^^^^^^^^^^^^^^^^^^^^ @@ -232,5 +231,3 @@ These next and previous elements are then checked to see if they too are free, and if so, they are merged with the current elements. This means that we can never have two free memory blocks adjacent to one another, they are always merged into a single block. - -.. |malloc_heap| image:: img/malloc_heap.* diff --git a/doc/guides/prog_guide/mbuf_lib.rst b/doc/guides/prog_guide/mbuf_lib.rst index 8f546e0..8845039 100644 --- a/doc/guides/prog_guide/mbuf_lib.rst +++ b/doc/guides/prog_guide/mbuf_lib.rst @@ -71,23 +71,21 @@ Message buffers may be used to carry control information, packets, events, and so on between different entities in the system. Message buffers may also use their buffer pointers to point to other message buffer data sections or other structures. -Figure 8 and Figure 9 show some of these scenarios. +:numref:`figure_mbuf1` and :numref:`figure_mbuf2` show some of these scenarios. -.. _pg_figure_8: +.. _figure_mbuf1: -**Figure 8. An mbuf with One Segment** +.. figure:: img/mbuf1.* -.. image22_png has been replaced + An mbuf with One Segment -|mbuf1| -.. _pg_figure_9: +.. _figure_mbuf2: -**Figure 9. An mbuf with Three Segments** +.. figure:: img/mbuf2.* -.. image23_png has been replaced + An mbuf with Three Segments -|mbuf2| The Buffer Manager implements a fairly standard set of buffer access functions to manipulate network packets. @@ -277,7 +275,3 @@ Use Cases --------- All networking application should use mbufs to transport network packets. - -.. |mbuf1| image:: img/mbuf1.* - -.. |mbuf2| image:: img/mbuf2.* diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f9b7cfe..f0ca06f 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -74,28 +74,27 @@ When running an application, the EAL command line options provide the ability to The command line must always have the number of memory channels specified for the processor. -Examples of alignment for different DIMM architectures are shown in Figure 5 and Figure 6. +Examples of alignment for different DIMM architectures are shown in +:numref:`figure_memory-management` and :numref:`figure_memory-management2`. -.. _pg_figure_5: +.. _figure_memory-management: -**Figure 5. Two Channels and Quad-ranked DIMM Example** +.. figure:: img/memory-management.* -.. image19_png has been replaced + Two Channels and Quad-ranked DIMM Example -|memory-management| In this case, the assumption is that a packet is 16 blocks of 64 bytes, which is not true. The Intel® 5520 chipset has three channels, so in most cases, no padding is required between objects (except for objects whose size are n x 3 x 64 bytes blocks). -.. _pg_figure_6: +.. _figure_memory-management2: -**Figure 6. Three Channels and Two Dual-ranked DIMM Example** +.. figure:: img/memory-management2.* -.. image20_png has been replaced + Three Channels and Two Dual-ranked DIMM Example -|memory-management2| When creating a new pool, the user can specify to use this feature or not. @@ -119,15 +118,14 @@ This cache can be enabled or disabled at creation of the pool. The maximum size of the cache is static and is defined at compilation time (CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE). -Figure 7 shows a cache in operation. +:numref:`figure_mempool` shows a cache in operation. -.. _pg_figure_7: +.. _figure_mempool: -**Figure 7. A mempool in Memory with its Associated Ring** +.. figure:: img/mempool.* -.. image21_png has been replaced + A mempool in Memory with its Associated Ring -|mempool| Use Cases --------- @@ -140,9 +138,3 @@ Below are some examples: * :ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` , for logging service * Any application that needs to allocate fixed-sized objects in the data plane and that will be continuously utilized by the system. - -.. |memory-management| image:: img/memory-management.* - -.. |memory-management2| image:: img/memory-management2.* - -.. |mempool| image:: img/mempool.* diff --git a/doc/guides/prog_guide/multi_proc_support.rst b/doc/guides/prog_guide/multi_proc_support.rst index 25a6056..6562f0d 100644 --- a/doc/guides/prog_guide/multi_proc_support.rst +++ b/doc/guides/prog_guide/multi_proc_support.rst @@ -83,13 +83,12 @@ and point to the same objects, in both processes. Refer to Section 23.3 "Multi-process Limitations" for details of how Linux kernel Address-Space Layout Randomization (ASLR) can affect memory sharing. -.. _pg_figure_16: +.. _figure_multi_process_memory: -**Figure 16. Memory Sharing in the DPDK Multi-process Sample Application** +.. figure:: img/multi_process_memory.* -.. image42_png has been replaced + Memory Sharing in the DPDK Multi-process Sample Application -|multi_process_memory| The EAL also supports an auto-detection mode (set by EAL --proc-type=auto flag ), whereby an DPDK process is started as a secondary instance if a primary instance is already running. @@ -199,5 +198,3 @@ instead of the functions which do the hashing internally, such as rte_hash_add() which means that only the first, primary DPDK process instance can open and mmap /dev/hpet. If the number of required DPDK processes exceeds that of the number of available HPET comparators, the TSC (which is the default timer in this release) must be used as a time source across all processes instead of the HPET. - -.. |multi_process_memory| image:: img/multi_process_memory.* diff --git a/doc/guides/prog_guide/overview.rst b/doc/guides/prog_guide/overview.rst index 062d923..cef6ca7 100644 --- a/doc/guides/prog_guide/overview.rst +++ b/doc/guides/prog_guide/overview.rst @@ -120,13 +120,12 @@ Core Components The *core components* are a set of libraries that provide all the elements needed for high-performance packet processing applications. -.. _pg_figure_1: +.. _figure_architecture-overview: -**Figure 1. Core Components Architecture** +.. figure:: img/architecture-overview.* -.. image2_png has been replaced + Core Components Architecture -|architecture-overview| Memory Manager (librte_malloc) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -203,5 +202,3 @@ librte_net The librte_net library is a collection of IP protocol definitions and convenience macros. It is based on code from the FreeBSD* IP stack and contains protocol numbers (for use in IP headers), IP-related macros, IPv4/IPv6 header structures and TCP, UDP and SCTP header structures. - -.. |architecture-overview| image:: img/architecture-overview.* diff --git a/doc/guides/prog_guide/packet_distrib_lib.rst b/doc/guides/prog_guide/packet_distrib_lib.rst index 767accc..b5bdabb 100644 --- a/doc/guides/prog_guide/packet_distrib_lib.rst +++ b/doc/guides/prog_guide/packet_distrib_lib.rst @@ -38,7 +38,10 @@ which is responsible for load balancing or distributing packets, and a set of worker lcores which are responsible for receiving the packets from the distributor and operating on them. The model of operation is shown in the diagram below. -|packet_distributor1| +.. figure:: img/packet_distributor1.* + + Packet Distributor mode of operation + Distributor Core Operation -------------------------- @@ -91,9 +94,11 @@ No packet ordering guarantees are made about packets which do not share a common Using the process and returned_pkts API, the following application workflow can be used, while allowing packet order within a packet flow -- identified by a tag -- to be maintained. -.. image41_png has been renamed -|packet_distributor2| +.. figure:: img/packet_distributor2.* + + Application workflow + The flush and clear_returns API calls, mentioned previously, are likely of less use that the process and returned_pkts APIS, and are principally provided to aid in unit testing of the library. @@ -110,7 +115,3 @@ Since it may be desirable to vary the number of worker cores, depending on the t i.e. to save power at times of lighter load, it is possible to have a worker stop processing packets by calling "rte_distributor_return_pkt()" to indicate that it has finished the current packet and does not want a new one. - -.. |packet_distributor1| image:: img/packet_distributor1.* - -.. |packet_distributor2| image:: img/packet_distributor2.* diff --git a/doc/guides/prog_guide/packet_framework.rst b/doc/guides/prog_guide/packet_framework.rst index 8e8e32f..42bbbaa 100644 --- a/doc/guides/prog_guide/packet_framework.rst +++ b/doc/guides/prog_guide/packet_framework.rst @@ -66,15 +66,15 @@ one of the table entries (on lookup hit) or the default table entry (on lookup m provides the set of actions to be applied on the current packet, as well as the next hop for the packet, which can be either another table, an output port or packet drop. -An example of packet processing pipeline is presented in Figure 32: +An example of packet processing pipeline is presented in :numref:`figure_figure32`: -.. _pg_figure_32: +.. _figure_figure32: -**Figure 32 Example of Packet Processing Pipeline where Input Ports 0 and 1 are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1** +.. figure:: img/figure32.* -.. Object_1_png has been renamed + Example of Packet Processing Pipeline where Input Ports 0 and 1 + are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1 -|figure32| Port Library Design ------------------- @@ -344,13 +344,14 @@ considering *n_bits* as the number of bits set in *bucket_mask = n_buckets - 1*, this means that all the keys that end up in the same hash table bucket have the lower *n_bits* of their signature identical. In order to reduce the number of keys in the same bucket (collisions), the number of hash table buckets needs to be increased. -In packet processing context, the sequence of operations involved in hash table operations is described in Figure 33: +In packet processing context, the sequence of operations involved in hash table operations is described in :numref:`figure_figure33`: -.. _pg_figure_33: +.. _figure_figure33: -**Figure 33 Sequence of Steps for Hash Table Operations in a Packet Processing Context** +.. figure:: img/figure33.* + + Sequence of Steps for Hash Table Operations in a Packet Processing Context -|figure33| Hash Table Use Cases @@ -553,16 +554,15 @@ This avoids the important cost associated with flushing the CPU core execution p Configurable Key Size Hash Table """""""""""""""""""""""""""""""" -Figure 34, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, +:numref:`figure_figure34`, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_34: +.. _figure_figure34: -**Figure 34 Data Structures for Configurable Key Size Hash Tables** +.. figure:: img/figure34.* -.. image65_png has been renamed + Data Structures for Configurable Key Size Hash Tables -|figure34| .. _pg_table_25: @@ -627,15 +627,17 @@ either with pre-computed signature or "do-sig"). +---+------------------+--------------------+------------------------------------------------------------------+ -Figure 35 and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, +:numref:`figure_figure35` and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_35: +.. _figure_figure35: + +.. figure:: img/figure35.* -**Figure 35 Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash Tables)** + Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash + Tables) -|figure35| .. _pg_table_27: @@ -814,11 +816,8 @@ Given the input *mask*, the values for *match*, *match_many* and *match_pos* can | | | | +------------+------------------------------------------+-------------------+ -The pseudo-code is displayed in Figure 36. - -.. _pg_figure_36: -**Figure 36 Pseudo-code for match, match_many and match_pos** +The pseudo-code for match, match_many and match_pos is:: match = (0xFFFELLU >> mask) & 1; @@ -829,24 +828,22 @@ The pseudo-code is displayed in Figure 36. Single Key Size Hash Tables """"""""""""""""""""""""""" -Figure 37, Figure 38, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables +:numref:`figure_figure37`, :numref:`figure_figure38`, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_37: +.. _figure_figure37: -**Figure 37 Data Structures for 8-byte Key Hash Tables** +.. figure:: img/figure37.* -.. image66_png has been renamed + Data Structures for 8-byte Key Hash Tables -|figure37| -.. _pg_figure_38: +.. _figure_figure38: -**Figure 38 Data Structures for 16-byte Key Hash Tables** +.. figure:: img/figure38.* -.. image67_png has been renamed + Data Structures for 16-byte Key Hash Tables -|figure38| .. _pg_table_30: @@ -914,11 +911,13 @@ and detail the bucket search pipeline used to implement 8-byte and 16-byte key h either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_39: +.. _figure_figure39: -**Figure 39 Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash Tables)** +.. figure:: img/figure39.* + + Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash + Tables) -|figure39| .. _pg_table_32: @@ -1167,17 +1166,3 @@ Usually, to support a specific functional block, specific implementation of Pack with all the implementations sharing the same API: pure SW implementation (no acceleration), implementation using accelerator A, implementation using accelerator B, etc. The selection between these implementations could be done at build time or at run-time (recommended), based on which accelerators are present in the system, with no application changes required. - -.. |figure33| image:: img/figure33.* - -.. |figure35| image:: img/figure35.* - -.. |figure39| image:: img/figure39.* - -.. |figure34| image:: img/figure34.* - -.. |figure32| image:: img/figure32.* - -.. |figure37| image:: img/figure37.* - -.. |figure38| image:: img/figure38.* diff --git a/doc/guides/prog_guide/qos_framework.rst b/doc/guides/prog_guide/qos_framework.rst index b609841..59f7fb3 100644 --- a/doc/guides/prog_guide/qos_framework.rst +++ b/doc/guides/prog_guide/qos_framework.rst @@ -38,13 +38,12 @@ Packet Pipeline with QoS Support An example of a complex packet processing pipeline with QoS support is shown in the following figure. -.. _pg_figure_21: +.. _figure_pkt_proc_pipeline_qos: -**Figure 21. Complex Packet Processing Pipeline with QoS Support** +.. figure:: img/pkt_proc_pipeline_qos.* -.. image47_png has been renamed + Complex Packet Processing Pipeline with QoS Support -|pkt_proc_pipeline_qos| This pipeline can be built using reusable DPDK software libraries. The main blocks implementing QoS in this pipeline are: the policer, the dropper and the scheduler. @@ -139,13 +138,12 @@ It typically acts like a buffer that is able to temporarily store a large number as the NIC TX is requesting more packets for transmission, these packets are later on removed and handed over to the NIC TX with the packet selection logic observing the predefined SLAs (dequeue operation). -.. _pg_figure_22: +.. _figure_hier_sched_blk: -**Figure 22. Hierarchical Scheduler Block Internal Diagram** +.. figure:: img/hier_sched_blk.* -.. image48_png has been renamed + Hierarchical Scheduler Block Internal Diagram -|hier_sched_blk| The hierarchical scheduler is optimized for a large number of packet queues. When only a small number of queues are needed, message passing queues should be used instead of this block. @@ -154,7 +152,7 @@ See Section 26.2.5 "Worst Case Scenarios for Performance" for a more detailed di Scheduling Hierarchy ~~~~~~~~~~~~~~~~~~~~ -The scheduling hierarchy is shown in Figure 23. +The scheduling hierarchy is shown in :numref:`figure_sched_hier_per_port`. The first level of the hierarchy is the Ethernet TX port 1/10/40 GbE, with subsequent hierarchy levels defined as subport, pipe, traffic class and queue. @@ -163,13 +161,12 @@ Each traffic class is the representation of a different traffic type with specif delay and jitter requirements, such as voice, video or data transfers. Each queue hosts packets from one or multiple connections of the same type belonging to the same user. -.. _pg_figure_23: +.. _figure_sched_hier_per_port: -**Figure 23. Scheduling Hierarchy per Port** +.. figure:: img/sched_hier_per_port.* -.. image49_png has been renamed + Scheduling Hierarchy per Port -|sched_hier_per_port| The functionality of each hierarchical level is detailed in the following table. @@ -293,13 +290,12 @@ Internal Data Structures per Port A schematic of the internal data structures in shown in with details in. -.. _pg_figure_24: +.. _figure_data_struct_per_port: -**Figure 24. Internal Data Structures per Port** +.. figure:: img/data_struct_per_port.* -.. image50_png has been renamed + Internal Data Structures per Port -|data_struct_per_port| .. _pg_table_4: @@ -434,16 +430,15 @@ the processor should not attempt to access the data structure currently under pr The only other work available is to execute different stages of the enqueue sequence of operations on other input packets, thus resulting in a pipelined implementation for the enqueue operation. -Figure 25 illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. +:numref:`figure_prefetch_pipeline` illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. No input packet can be part of more than one pipeline stage at a given time. -.. _pg_figure_25: +.. _figure_prefetch_pipeline: -**Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation** +.. figure:: img/prefetch_pipeline.* -.. image51 has been renamed + Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation -|prefetch_pipeline| The congestion management scheme implemented by the enqueue pipeline described above is very basic: packets are enqueued until a specific queue becomes full, @@ -478,13 +473,13 @@ The dequeue pipe state machine exploits the data presence into the processor cac therefore it tries to send as many packets from the same pipe TC and pipe as possible (up to the available packets and credits) before moving to the next active TC from the same pipe (if any) or to another active pipe. -.. _pg_figure_26: +.. _figure_pipe_prefetch_sm: -**Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation** +.. figure:: img/pipe_prefetch_sm.* -.. image52 has been renamed + Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue + Operation -|pipe_prefetch_sm| Timing and Synchronization ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1173,17 +1168,16 @@ Dropper The purpose of the DPDK dropper is to drop packets arriving at a packet scheduler to avoid congestion. The dropper supports the Random Early Detection (RED), Weighted Random Early Detection (WRED) and tail drop algorithms. -Figure 1 illustrates how the dropper integrates with the scheduler. +:numref:`figure_blk_diag_dropper` illustrates how the dropper integrates with the scheduler. The DPDK currently does not support congestion management so the dropper provides the only method for congestion avoidance. -.. _pg_figure_27: +.. _figure_blk_diag_dropper: -**Figure 27. High-level Block Diagram of the DPDK Dropper** +.. figure:: img/blk_diag_dropper.* -.. image53_png has been renamed + High-level Block Diagram of the DPDK Dropper -|blk_diag_dropper| The dropper uses the Random Early Detection (RED) congestion avoidance algorithm as documented in the reference publication. The purpose of the RED algorithm is to monitor a packet queue, @@ -1202,16 +1196,15 @@ In the case of severe congestion, the dropper resorts to tail drop. This occurs when a packet queue has reached maximum capacity and cannot store any more packets. In this situation, all arriving packets are dropped. -The flow through the dropper is illustrated in Figure 28. +The flow through the dropper is illustrated in :numref:`figure_flow_tru_droppper`. The RED/WRED algorithm is exercised first and tail drop second. -.. _pg_figure_28: +.. _figure_flow_tru_droppper: -**Figure 28. Flow Through the Dropper** +.. figure:: img/flow_tru_droppper.* -.. image54_png has been renamed + Flow Through the Dropper -|flow_tru_droppper| The use cases supported by the dropper are: @@ -1270,17 +1263,16 @@ for example, a filter weight parameter value of 9 corresponds to a filter weight Enqueue Operation ~~~~~~~~~~~~~~~~~ -In the example shown in Figure 29, q (actual queue size) is the input value, +In the example shown in :numref:`figure_ex_data_flow_tru_dropper`, q (actual queue size) is the input value, avg (average queue size) and count (number of packets since the last drop) are run-time values, decision is the output value and the remaining values are configuration parameters. -.. _pg_figure_29: +.. _figure_ex_data_flow_tru_dropper: -**Figure 29. Example Data Flow Through Dropper** +.. figure:: img/ex_data_flow_tru_dropper.* -.. image55_png has been renamed + Example Data Flow Through Dropper -|ex_data_flow_tru_dropper| EWMA Filter Microblock ^^^^^^^^^^^^^^^^^^^^^^ @@ -1298,11 +1290,7 @@ Average Queue Size Calculation when the Queue is not Empty The definition of the EWMA filter is given in the following equation. -**Equation 1.** - -.. image56_png has been renamed - -|ewma_filter_eq_1| +.. image:: img/ewma_filter_eq_1.* Where: @@ -1326,11 +1314,7 @@ When the queue becomes empty, average queue size should decay gradually to zero or remaining stagnant at the last computed value. When a packet is enqueued on an empty queue, the average queue size is computed using the following formula: -**Equation 2.** - -.. image57_png has been renamed - -|ewma_filter_eq_2| +.. image:: img/ewma_filter_eq_2.* Where: @@ -1338,9 +1322,7 @@ Where: In the dropper module, *m* is defined as: -.. image58_png has been renamed - -|m_definition| +.. image:: img/m_definition.* Where: @@ -1374,15 +1356,13 @@ A numerical method is used to compute the factor (1-wq)^m that appears in Equati This method is based on the following identity: -.. image59_png has been renamed +.. image:: img/eq2_factor.* -|eq2_factor| This allows us to express the following: -.. image60_png has been renamed +.. image:: img/eq2_expression.* -|eq2_expression| In the dropper module, a look-up table is used to compute log2(1-wq) for each value of wq supported by the dropper module. The factor (1-wq)^m can then be obtained by multiplying the table value by *m* and applying shift operations. @@ -1465,11 +1445,7 @@ Initial Packet Drop Probability The initial drop probability is calculated using the following equation. -**Equation 3.** - -.. image61_png has been renamed - -|drop_probability_eq3| +.. image:: img/drop_probability_eq3.* Where: @@ -1481,19 +1457,18 @@ Where: * *maxth* = maximum threshold -The calculation of the packet drop probability using Equation 3 is illustrated in Figure 30. +The calculation of the packet drop probability using Equation 3 is illustrated in :numref:`figure_pkt_drop_probability`. If the average queue size is below the minimum threshold, an arriving packet is enqueued. If the average queue size is at or above the maximum threshold, an arriving packet is dropped. If the average queue size is between the minimum and maximum thresholds, a drop probability is calculated to determine if the packet should be enqueued or dropped. -.. _pg_figure_30: +.. _figure_pkt_drop_probability: -**Figure 30. Packet Drop Probability for a Given RED Configuration** +.. figure:: img/pkt_drop_probability.* -.. image62_png has been renamed + Packet Drop Probability for a Given RED Configuration -|pkt_drop_probability| Actual Drop Probability """"""""""""""""""""""" @@ -1501,11 +1476,7 @@ Actual Drop Probability If the average queue size is between the minimum and maximum thresholds, then the actual drop probability is calculated from the following equation. -**Equation 4.** - -.. image63_png has been renamed - -|drop_probability_eq4| +.. image:: img/drop_probability_eq4.* Where: @@ -1518,7 +1489,7 @@ given in the reference document where a value of 1 is used instead. It should be noted that the value pa computed from can be negative or greater than 1. If this is the case, then a value of 1 should be used instead. -The initial and actual drop probabilities are shown in Figure 31. +The initial and actual drop probabilities are shown in :numref:`figure_drop_probability_graph`. The actual drop probability is shown for the case where the formula given in the reference document1 is used (blue curve) and also for the case where the formula implemented in the dropper module, @@ -1528,13 +1499,13 @@ compared to the mark probability configuration parameter specified by the user. The choice to deviate from the reference document is simply a design decision and one that has been taken by other RED implementations, for example, FreeBSD* ALTQ RED. -.. _pg_figure_31: +.. _figure_drop_probability_graph: -**Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve)** +.. figure:: img/drop_probability_graph.* -.. image64_png has been renamed + Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using + a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) -|drop_probability_graph| .. _Queue_Empty_Operation: @@ -1727,39 +1698,3 @@ For each input packet, the steps for the srTCM / trTCM algorithms are: the input color of the packet is also considered. When the output color is not red, a number of tokens equal to the length of the IP packet are subtracted from the C or E /P or both buckets, depending on the algorithm and the output color of the packet. - -.. |flow_tru_droppper| image:: img/flow_tru_droppper.* - -.. |drop_probability_graph| image:: img/drop_probability_graph.* - -.. |drop_probability_eq3| image:: img/drop_probability_eq3.* - -.. |eq2_expression| image:: img/eq2_expression.* - -.. |drop_probability_eq4| image:: img/drop_probability_eq4.* - -.. |pkt_drop_probability| image:: img/pkt_drop_probability.* - -.. |pkt_proc_pipeline_qos| image:: img/pkt_proc_pipeline_qos.* - -.. |ex_data_flow_tru_dropper| image:: img/ex_data_flow_tru_dropper.* - -.. |ewma_filter_eq_1| image:: img/ewma_filter_eq_1.* - -.. |ewma_filter_eq_2| image:: img/ewma_filter_eq_2.* - -.. |data_struct_per_port| image:: img/data_struct_per_port.* - -.. |prefetch_pipeline| image:: img/prefetch_pipeline.* - -.. |pipe_prefetch_sm| image:: img/pipe_prefetch_sm.* - -.. |blk_diag_dropper| image:: img/blk_diag_dropper.* - -.. |m_definition| image:: img/m_definition.* - -.. |eq2_factor| image:: img/eq2_factor.* - -.. |sched_hier_per_port| image:: img/sched_hier_per_port.* - -.. |hier_sched_blk| image:: img/hier_sched_blk.* diff --git a/doc/guides/prog_guide/ring_lib.rst b/doc/guides/prog_guide/ring_lib.rst index 8547b38..3b92a8f 100644 --- a/doc/guides/prog_guide/ring_lib.rst +++ b/doc/guides/prog_guide/ring_lib.rst @@ -72,13 +72,12 @@ The disadvantages: A simplified representation of a Ring is shown in with consumer and producer head and tail pointers to objects stored in the data structure. -.. _pg_figure_4: +.. _figure_ring1: -**Figure 4. Ring Structure** +.. figure:: img/ring1.* -.. image5_png has been replaced + Ring Structure -|ring1| References for Ring Implementation in FreeBSD* ---------------------------------------------- @@ -155,9 +154,13 @@ The prod_next local variable points to the next element of the table, or several If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image6_png has been replaced -|ring-enqueue1| +.. _figure_ring-enqueue1: + +.. figure:: img/ring-enqueue1.* + + Enqueue first step + Enqueue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -166,9 +169,13 @@ The second step is to modify *ring->prod_head* in ring structure to point to the A pointer to the added object is copied in the ring (obj4). -.. image7_png has been replaced -|ring-enqueue2| +.. _figure_ring-enqueue2: + +.. figure:: img/ring-enqueue2.* + + Enqueue second step + Enqueue Last Step ^^^^^^^^^^^^^^^^^ @@ -176,9 +183,13 @@ Enqueue Last Step Once the object is added in the ring, ring->prod_tail in the ring structure is modified to point to the same location as *ring->prod_head*. The enqueue operation is finished. -.. image8_png has been replaced -|ring-enqueue3| +.. _figure_ring-enqueue3: + +.. figure:: img/ring-enqueue3.* + + Enqueue last step + Single Consumer Dequeue ~~~~~~~~~~~~~~~~~~~~~~~ @@ -196,9 +207,13 @@ The cons_next local variable points to the next element of the table, or several If there are not enough objects in the ring (this is detected by checking prod_tail), it returns an error. -.. image9_png has been replaced -|ring-dequeue1| +.. _figure_ring-dequeue1: + +.. figure:: img/ring-dequeue1.* + + Dequeue last step + Dequeue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -207,9 +222,13 @@ The second step is to modify ring->cons_head in the ring structure to point to t The pointer to the dequeued object (obj1) is copied in the pointer given by the user. -.. image10_png has been replaced -|ring-dequeue2| +.. _figure_ring-dequeue2: + +.. figure:: img/ring-dequeue2.* + + Dequeue second step + Dequeue Last Step ^^^^^^^^^^^^^^^^^ @@ -217,9 +236,13 @@ Dequeue Last Step Finally, ring->cons_tail in the ring structure is modified to point to the same location as ring->cons_head. The dequeue operation is finished. -.. image11_png has been replaced -|ring-dequeue3| +.. _figure_ring-dequeue3: + +.. figure:: img/ring-dequeue3.* + + Dequeue last step + Multiple Producers Enqueue ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -229,8 +252,8 @@ In this example, only the producer head and tail (prod_head and prod_tail) are m The initial state is to have a prod_head and prod_tail pointing at the same location. -MC Enqueue First Step -^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue First Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On both cores, *ring->prod_head* and ring->cons_tail are copied in local variables. The prod_next local variable points to the next element of the table, @@ -238,12 +261,16 @@ or several elements after in the case of bulk enqueue. If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image12_png has been replaced -|ring-mp-enqueue1| +.. _figure_ring-mp-enqueue1: + +.. figure:: img/ring-mp-enqueue1.* + + Multiple consumer enqueue first step + -MC Enqueue Second Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Second Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The second step is to modify ring->prod_head in the ring structure to point to the same location as prod_next. This operation is done using a Compare And Swap (CAS) instruction, which does the following operations atomically: @@ -256,41 +283,57 @@ This operation is done using a Compare And Swap (CAS) instruction, which does th In the figure, the operation succeeded on core 1, and step one restarted on core 2. -.. image13_png has been replaced -|ring-mp-enqueue2| +.. _figure_ring-mp-enqueue2: -MC Enqueue Third Step -^^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue2.* + + Multiple consumer enqueue second step + + +Multiple Consumer Enqueue Third Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The CAS operation is retried on core 2 with success. The core 1 updates one element of the ring(obj4), and the core 2 updates another one (obj5). -.. image14_png has been replaced -|ring-mp-enqueue3| +.. _figure_ring-mp-enqueue3: + +.. figure:: img/ring-mp-enqueue3.* + + Multiple consumer enqueue third step + -MC Enqueue Fourth Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Fourth Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Each core now wants to update ring->prod_tail. A core can only update it if ring->prod_tail is equal to the prod_head local variable. This is only true on core 1. The operation is finished on core 1. -.. image15_png has been replaced -|ring-mp-enqueue4| +.. _figure_ring-mp-enqueue4: -MC Enqueue Last Step -^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue4.* + + Multiple consumer enqueue fourth step + + +Multiple Consumer Enqueue Last Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Once ring->prod_tail is updated by core 1, core 2 is allowed to update it too. The operation is also finished on core 2. -.. image16_png has been replaced -|ring-mp-enqueue5| +.. _figure_ring-mp-enqueue5: + +.. figure:: img/ring-mp-enqueue5.* + + Multiple consumer enqueue last step + Modulo 32-bit Indexes ~~~~~~~~~~~~~~~~~~~~~ @@ -309,15 +352,23 @@ The following are two examples that help to explain how indexes are used in a ri In addition, the four indexes are defined as unsigned 16-bit integers, as opposed to unsigned 32-bit integers in the more realistic case. -.. image17_png has been replaced -|ring-modulo1| +.. _figure_ring-modulo1: + +.. figure:: img/ring-modulo1.* + + Modulo 32-bit indexes - Example 1 + This ring contains 11000 entries. -.. image18_png has been replaced -|ring-modulo2| +.. _figure_ring-modulo2: + +.. figure:: img/ring-modulo2.* + + Modulo 32-bit indexes - Example 2 + This ring contains 12536 entries. @@ -346,31 +397,3 @@ References * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_ (version 8) * `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_ - -.. |ring1| image:: img/ring1.* - -.. |ring-enqueue1| image:: img/ring-enqueue1.* - -.. |ring-enqueue2| image:: img/ring-enqueue2.* - -.. |ring-enqueue3| image:: img/ring-enqueue3.* - -.. |ring-dequeue1| image:: img/ring-dequeue1.* - -.. |ring-dequeue2| image:: img/ring-dequeue2.* - -.. |ring-dequeue3| image:: img/ring-dequeue3.* - -.. |ring-mp-enqueue1| image:: img/ring-mp-enqueue1.* - -.. |ring-mp-enqueue2| image:: img/ring-mp-enqueue2.* - -.. |ring-mp-enqueue3| image:: img/ring-mp-enqueue3.* - -.. |ring-mp-enqueue4| image:: img/ring-mp-enqueue4.* - -.. |ring-mp-enqueue5| image:: img/ring-mp-enqueue5.* - -.. |ring-modulo1| image:: img/ring-modulo1.* - -.. |ring-modulo2| image:: img/ring-modulo2.* diff --git a/doc/guides/sample_app_ug/dist_app.rst b/doc/guides/sample_app_ug/dist_app.rst index bcff0dd..25844e0 100644 --- a/doc/guides/sample_app_ug/dist_app.rst +++ b/doc/guides/sample_app_ug/dist_app.rst @@ -47,11 +47,12 @@ into each other. This application can be used to benchmark performance using the traffic generator as shown in the figure below. -.. _figure_22: +.. _figure_dist_perf: -**Figure 22. Performance Benchmarking Setup (Basic Environment)** +.. figure:: img/dist_perf.* + + Performance Benchmarking Setup (Basic Environment) -|dist_perf| Compiling the Application ------------------------- @@ -106,7 +107,7 @@ Explanation The distributor application consists of three types of threads: a receive thread (lcore_rx()), a set of worker threads(locre_worker()) and a transmit thread(lcore_tx()). How these threads work together is shown -in Fig2 below. The main() function launches threads of these three types. +in :numref:`figure_dist_app` below. The main() function launches threads of these three types. Each thread has a while loop which will be doing processing and which is terminated only upon SIGINT or ctrl+C. The receive and transmit threads communicate using a software ring (rte_ring structure). @@ -136,11 +137,12 @@ Users who wish to terminate the running of the application have to press ctrl+C in the application will terminate all running threads gracefully and print final statistics to the user. -.. _figure_23: +.. _figure_dist_app: + +.. figure:: img/dist_app.* -**Figure 23. Distributor Sample Application Layout** + Distributor Sample Application Layout -|dist_app| Debug Logging Support --------------------- @@ -171,7 +173,3 @@ Sample Application. See Section 9.4.4, "RX Queue Initialization". TX queue initialization is done in the same way as it is done in the L2 Forwarding Sample Application. See Section 9.4.5, "TX Queue Initialization". - -.. |dist_perf| image:: img/dist_perf.* - -.. |dist_app| image:: img/dist_app.* diff --git a/doc/guides/sample_app_ug/exception_path.rst b/doc/guides/sample_app_ug/exception_path.rst index 6c06959..3cc7cbe 100644 --- a/doc/guides/sample_app_ug/exception_path.rst +++ b/doc/guides/sample_app_ug/exception_path.rst @@ -46,13 +46,12 @@ The second thread reads from a TAP interface and writes the data unmodified to t The packet flow through the exception path application is as shown in the following figure. -.. _figure_1: +.. _figure_exception_path_example: -**Figure 1. Packet Flow** +.. figure:: img/exception_path_example.* -.. image2_png has been replaced + Packet Flow -|exception_path_example| To make throughput measurements, kernel bridges must be setup to forward data between the bridges appropriately. @@ -327,4 +326,3 @@ To remove bridges and persistent TAP interfaces, the following commands are used brctl delbr br0 openvpn --rmtun --dev tap_dpdk_00 -.. |exception_path_example| image:: img/exception_path_example.* diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index aaa95ef..745a7ac 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -74,57 +74,63 @@ Sample Applications User Guide **Figures** -:ref:`Figure 1.Packet Flow <figure_1>` +:numref:`figure_exception_path_example` :ref:`figure_exception_path_example` -:ref:`Figure 2.Kernel NIC Application Packet Flow <figure_2>` +:numref:`figure_kernel_nic` :ref:`figure_kernel_nic` -:ref:`Figure 3.Performance Benchmark Setup (Basic Environment) <figure_3>` +:numref:`figure_l2_fwd_benchmark_setup_jobstats` :ref:`figure_l2_fwd_benchmark_setup_jobstats` -:ref:`Figure 4.Performance Benchmark Setup (Virtualized Environment) <figure_4>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` :ref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` -:ref:`Figure 5.Load Balancer Application Architecture <figure_5>` +:numref:`figure_l2_fwd_benchmark_setup` :ref:`figure_l2_fwd_benchmark_setup` -:ref:`Figure 5.Example Rules File <figure_5_1>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup` :ref:`figure_l2_fwd_virtenv_benchmark_setup` -:ref:`Figure 6.Example Data Flow in a Symmetric Multi-process Application <figure_6>` +:numref:`figure_ipv4_acl_rule` :ref:`figure_ipv4_acl_rule` -:ref:`Figure 7.Example Data Flow in a Client-Server Symmetric Multi-process Application <figure_7>` +:numref:`figure_example_rules` :ref:`figure_example_rules` -:ref:`Figure 8.Master-slave Process Workflow <figure_8>` +:numref:`figure_load_bal_app_arch` :ref:`figure_load_bal_app_arch` -:ref:`Figure 9.Slave Process Recovery Process Flow <figure_9>` +:numref:`figure_sym_multi_proc_app` :ref:`figure_sym_multi_proc_app` -:ref:`Figure 10.QoS Scheduler Application Architecture <figure_10>` +:numref:`figure_client_svr_sym_multi_proc_app` :ref:`figure_client_svr_sym_multi_proc_app` -:ref:`Figure 11.Intel®QuickAssist Technology Application Block Diagram <figure_11>` +:numref:`figure_master_slave_proc` :ref:`figure_master_slave_proc` -:ref:`Figure 12.Pipeline Overview <figure_12>` +:numref:`figure_slave_proc_recov` :ref:`figure_slave_proc_recov` -:ref:`Figure 13.Ring-based Processing Pipeline Performance Setup <figure_13>` +:numref:`figure_qos_sched_app_arch` :ref:`figure_qos_sched_app_arch` -:ref:`Figure 14.Threads and Pipelines <figure_14>` +:numref:`figure_quickassist_block_diagram` :ref:`figure_quickassist_block_diagram` -:ref:`Figure 15.Packet Flow Through the VMDQ and DCB Sample Application <figure_15>` +:numref:`figure_pipeline_overview` :ref:`figure_pipeline_overview` -:ref:`Figure 16.QEMU Virtio-net (prior to vhost-net) <figure_16>` +:numref:`figure_ring_pipeline_perf_setup` :ref:`figure_ring_pipeline_perf_setup` -:ref:`Figure 17.Virtio with Linux* Kernel Vhost <figure_17>` +:numref:`figure_threads_pipelines` :ref:`figure_threads_pipelines` -:ref:`Figure 18.Vhost-net Architectural Overview <figure_18>` +:numref:`figure_vmdq_dcb_example` :ref:`figure_vmdq_dcb_example` -:ref:`Figure 19.Packet Flow Through the vhost-net Sample Application <figure_19>` +:numref:`figure_qemu_virtio_net` :ref:`figure_qemu_virtio_net` -:ref:`Figure 20.Packet Flow on TX in DPDK-testpmd <figure_20>` +:numref:`figure_virtio_linux_vhost` :ref:`figure_virtio_linux_vhost` -:ref:`Figure 21.Test Pipeline Application <figure_21>` +:numref:`figure_vhost_net_arch` :ref:`figure_vhost_net_arch` -:ref:`Figure 22.Performance Benchmarking Setup (Basic Environment) <figure_22>` +:numref:`figure_vhost_net_sample_app` :ref:`figure_vhost_net_sample_app` -:ref:`Figure 23.Distributor Sample Application Layout <figure_23>` +:numref:`figure_tx_dpdk_testpmd` :ref:`figure_tx_dpdk_testpmd` -:ref:`Figure 24.High level Solution <figure_24>` +:numref:`figure_test_pipeline_app` :ref:`figure_test_pipeline_app` -:ref:`Figure 25.VM request to scale frequency <figure_25>` +:numref:`figure_dist_perf` :ref:`figure_dist_perf` + +:numref:`figure_dist_app` :ref:`figure_dist_app` + +:numref:`figure_vm_power_mgr_highlevel` :ref:`figure_vm_power_mgr_highlevel` + +:numref:`figure_vm_power_mgr_vm_request_seq` :ref:`figure_vm_power_mgr_vm_request_seq` **Tables** diff --git a/doc/guides/sample_app_ug/intel_quickassist.rst b/doc/guides/sample_app_ug/intel_quickassist.rst index 7f55282..a80d4ca 100644 --- a/doc/guides/sample_app_ug/intel_quickassist.rst +++ b/doc/guides/sample_app_ug/intel_quickassist.rst @@ -46,17 +46,16 @@ For this sample application, there is a dependency on either of: Overview -------- -An overview of the application is provided in Figure 11. +An overview of the application is provided in :numref:`figure_quickassist_block_diagram`. For simplicity, only two NIC ports and one Intel® QuickAssist Technology device are shown in this diagram, although the number of NIC ports and Intel® QuickAssist Technology devices can be different. -.. _figure_11: +.. _figure_quickassist_block_diagram: -**Figure 11. Intel® QuickAssist Technology Application Block Diagram** +.. figure:: img/quickassist_block_diagram.* -.. image14_png has been renamed + Intel® QuickAssist Technology Application Block Diagram -|quickassist_block_diagram| The application allows the configuration of the following items: @@ -220,5 +219,3 @@ performing AES-CBC-128 encryption with AES-XCBC-MAC-96 hash, the following setti Refer to the *DPDK Test Report* for more examples of traffic generator setup and the application startup command lines. If no errors are generated in response to the startup commands, the application is running correctly. - -.. |quickassist_block_diagram| image:: img/quickassist_block_diagram.* diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index d6876e2..02dde59 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -71,13 +71,12 @@ it is just for performance testing, or it can work together with VMDq support in The packet flow through the Kernel NIC Interface application is as shown in the following figure. -.. _figure_2: +.. _figure_kernel_nic: -**Figure 2. Kernel NIC Application Packet Flow** +.. figure:: img/kernel_nic.* -.. image3_png has been renamed to kernel_nic.* + Kernel NIC Application Packet Flow -|kernel_nic| Compiling the Application ------------------------- @@ -616,5 +615,3 @@ Currently, setting a new MTU and configuring the network interface (up/ down) ar RTE_LOG(ERR, APP, "Failed to start port %d\n", port_id); return ret; } - -.. |kernel_nic| image:: img/kernel_nic.* diff --git a/doc/guides/sample_app_ug/l2_forward_job_stats.rst b/doc/guides/sample_app_ug/l2_forward_job_stats.rst index 54d25cb..f61af52 100644 --- a/doc/guides/sample_app_ug/l2_forward_job_stats.rst +++ b/doc/guides/sample_app_ug/l2_forward_job_stats.rst @@ -55,27 +55,24 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup_jobstats`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup_jobstats: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| +.. _figure_l2_fwd_virtenv_benchmark_setup_jobstats: -.. _figure_4: +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** + Performance Benchmark Setup (Virtualized Environment) -.. image5_png has been renamed - -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -631,7 +628,3 @@ however it improves performance: * in which it was called. */ rte_jobstats_finish(&qconf->flush_job, qconf->flush_job.target); } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst index 234d71d..9334e75 100644 --- a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst +++ b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst @@ -54,27 +54,25 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| -.. _figure_4: +.. _figure_l2_fwd_virtenv_benchmark_setup: -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -.. image5_png has been renamed + Performance Benchmark Setup (Virtualized Environment) -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -526,7 +524,3 @@ however it improves performance: prev_tsc = cur_tsc; } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst index 73fa4df..dbf47c7 100644 --- a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst +++ b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst @@ -142,9 +142,13 @@ Other lines types are considered invalid. * A typical IPv4 ACL rule line should have a format as shown below: -.. image6_png has been renamed -|ipv4_acl_rule| +.. _figure_ipv4_acl_rule: + +.. figure:: img/ipv4_acl_rule.* + + A typical IPv4 ACL rule + IPv4 addresses are specified in CIDR format as specified in RFC 4632. They consist of the dot notation for the address and a prefix length separated by '/'. @@ -164,15 +168,12 @@ For example: 6/0xfe matches protocol values 6 and 7. Rules File Example ~~~~~~~~~~~~~~~~~~ -.. _figure_5_1: +.. _figure_example_rules: -Figure 5 is an example of a rules file. This file has three rules, one for ACL and two for route information. +.. figure:: img/example_rules.* -**Figure 5.Example Rules File** + Rules example -.. image7_png has been renamed - -|example_rules| Each rule is explained as follows: @@ -397,7 +398,3 @@ Finally, the application creates contexts handler from the ACL library, adds rules parsed from the file into the database and build an ACL trie. It is important to note that the application creates an independent copy of each database for each socket CPU involved in the task to reduce the time for remote memory access. - -.. |ipv4_acl_rule| image:: img/ipv4_acl_rule.* - -.. |example_rules| image:: img/example_rules.* diff --git a/doc/guides/sample_app_ug/load_balancer.rst b/doc/guides/sample_app_ug/load_balancer.rst index 6237633..857eb8a 100644 --- a/doc/guides/sample_app_ug/load_balancer.rst +++ b/doc/guides/sample_app_ug/load_balancer.rst @@ -44,13 +44,12 @@ Overview The architecture of the Load Balance application is presented in the following figure. -.. _figure_5: +.. _figure_load_bal_app_arch: -**Figure 5. Load Balancer Application Architecture** +.. figure:: img/load_bal_app_arch.* -.. image8_png has been renamed + Load Balancer Application Architecture -|load_bal_app_arch| For the sake of simplicity, the diagram illustrates a specific case of two I/O RX and two I/O TX lcores off loading the packet I/O overhead incurred by four NIC ports from four worker cores, with each I/O lcore handling RX/TX for two NIC ports. @@ -241,5 +240,3 @@ are on the same or different CPU sockets, the following run-time scenarios are p #. ABC: The packet is received on socket A, it is processed by an lcore on socket B, then it has to be transmitted out by a NIC connected to socket C. The performance price for crossing the CPU socket boundary is paid twice for this packet. - -.. |load_bal_app_arch| image:: img/load_bal_app_arch.* diff --git a/doc/guides/sample_app_ug/multi_process.rst b/doc/guides/sample_app_ug/multi_process.rst index 7ca71ca..f42cb9a 100644 --- a/doc/guides/sample_app_ug/multi_process.rst +++ b/doc/guides/sample_app_ug/multi_process.rst @@ -190,13 +190,12 @@ such as a client-server mode of operation seen in the next example, where different processes perform different tasks, yet co-operate to form a packet-processing system.) The following diagram shows the data-flow through the application, using two processes. -.. _figure_6: +.. _figure_sym_multi_proc_app: -**Figure 6. Example Data Flow in a Symmetric Multi-process Application** +.. figure:: img/sym_multi_proc_app.* -.. image9_png has been renamed + Example Data Flow in a Symmetric Multi-process Application -|sym_multi_proc_app| As the diagram shows, each process reads packets from each of the network ports in use. RSS is used to distribute incoming packets on each port to different hardware RX queues. @@ -296,13 +295,12 @@ In this case, the client applications just perform level-2 forwarding of packets The following diagram shows the data-flow through the application, using two client processes. -.. _figure_7: +.. _figure_client_svr_sym_multi_proc_app: -**Figure 7. Example Data Flow in a Client-Server Symmetric Multi-process Application** +.. figure:: img/client_svr_sym_multi_proc_app.* -.. image10_png has been renamed + Example Data Flow in a Client-Server Symmetric Multi-process Application -|client_svr_sym_multi_proc_app| Running the Application ^^^^^^^^^^^^^^^^^^^^^^^ @@ -395,13 +393,12 @@ Once the master process begins to run, it tries to initialize all the resources memory, CPU cores, driver, ports, and so on, as the other examples do. Thereafter, it creates slave processes, as shown in the following figure. -.. _figure_8: +.. _figure_master_slave_proc: -**Figure 8. Master-slave Process Workflow** +.. figure:: img/master_slave_proc.* -.. image11_png has been renamed + Master-slave Process Workflow -|master_slave_proc| The master process calls the rte_eal_mp_remote_launch() EAL function to launch an application function for each pinned thread through the pipe. Then, it waits to check if any slave processes have exited. @@ -475,13 +472,12 @@ Therefore, to provide the capability to resume the new slave instance if the pre The following diagram describes slave process recovery. -.. _figure_9: +.. _figure_slave_proc_recov: -**Figure 9. Slave Process Recovery Process Flow** +.. figure:: img/slave_proc_recov.* -.. image12_png has been renamed + Slave Process Recovery Process Flow -|slave_proc_recov| Floating Process Support ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -774,11 +770,3 @@ so it remaps the resource to the new core ID slot. } return 0; } - -.. |sym_multi_proc_app| image:: img/sym_multi_proc_app.* - -.. |client_svr_sym_multi_proc_app| image:: img/client_svr_sym_multi_proc_app.* - -.. |master_slave_proc| image:: img/master_slave_proc.* - -.. |slave_proc_recov| image:: img/slave_proc_recov.* diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index 56326df..66c261c 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -38,13 +38,12 @@ Overview The architecture of the QoS scheduler application is shown in the following figure. -.. _figure_10: +.. _figure_qos_sched_app_arch: -**Figure 10. QoS Scheduler Application Architecture** +.. figure:: img/qos_sched_app_arch.* -.. image13_png has been renamed + QoS Scheduler Application Architecture -|qos_sched_app_arch| There are two flavors of the runtime execution for this application, with two or three threads per each packet flow configuration being used. @@ -347,5 +346,3 @@ This application classifies based on the QinQ double VLAN tags and the IP destin +----------------+-------------------------+--------------------------------------------------+----------------------------------+ Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. - -.. |qos_sched_app_arch| image:: img/qos_sched_app_arch.* diff --git a/doc/guides/sample_app_ug/quota_watermark.rst b/doc/guides/sample_app_ug/quota_watermark.rst index e091ad9..de9e118 100644 --- a/doc/guides/sample_app_ug/quota_watermark.rst +++ b/doc/guides/sample_app_ug/quota_watermark.rst @@ -54,15 +54,14 @@ and ports 2 and 3 forward into each other. The MAC addresses of the forwarded Ethernet frames are not affected. Internally, packets are pulled from the ports by the master logical core and put on a variable length processing pipeline, -each stage of which being connected by rings, as shown in Figure 12. +each stage of which being connected by rings, as shown in :numref:`figure_pipeline_overview`. -.. _figure_12: +.. _figure_pipeline_overview: -**Figure 12. Pipeline Overview** +.. figure:: img/pipeline_overview.* -.. image15_png has been renamed + Pipeline Overview -|pipeline_overview| An adjustable quota value controls how many packets are being moved through the pipeline per enqueue and dequeue. Adjustable watermark values associated with the rings control a back-off mechanism that @@ -79,15 +78,14 @@ eventually lead to an Ethernet flow control frame being send to the source. On top of serving as an example of quota and watermark usage, this application can be used to benchmark ring based processing pipelines performance using a traffic- generator, -as shown in Figure 13. +as shown in :numref:`figure_ring_pipeline_perf_setup`. -.. _figure_13: +.. _figure_ring_pipeline_perf_setup: -**Figure 13. Ring-based Processing Pipeline Performance Setup** +.. figure:: img/ring_pipeline_perf_setup.* -.. image16_png has been renamed + Ring-based Processing Pipeline Performance Setup -|ring_pipeline_perf_setup| Compiling the Application ------------------------- @@ -311,7 +309,7 @@ Logical Cores Assignment The application uses the master logical core to poll all the ports for new packets and enqueue them on a ring associated with the port. Each logical core except the last runs pipeline_stage() after a ring for each used port is initialized on that core. -pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See Figure 14. +pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See :numref:`figure_threads_pipelines`. .. code-block:: c @@ -340,16 +338,12 @@ sending them out on the destination port setup by pair_ports(). Receive, Process and Transmit Packets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _figure_14: +.. _figure_threads_pipelines: -Figure 14 shows where each thread in the pipeline is. -It should be used as a reference while reading the rest of this section. +.. figure:: img/threads_pipelines.* -**Figure 14. Threads and Pipelines** + Threads and Pipelines -.. image17_png has been renamed - -|threads_pipelines| In the receive_stage() function running on the master logical core, the main task is to read ingress packets from the RX ports and enqueue them @@ -498,9 +492,3 @@ low_watermark from the rte_memzone previously created by qw. low_watermark = (unsigned int *) qw_memzone->addr + sizeof(int); } - -.. |pipeline_overview| image:: img/pipeline_overview.* - -.. |ring_pipeline_perf_setup| image:: img/ring_pipeline_perf_setup.* - -.. |threads_pipelines| image:: img/threads_pipelines.* diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index 0432942..fbc290c 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -49,13 +49,12 @@ The application uses three CPU cores: * Core C ("TX core") receives traffic from core B through software queues and sends it to the NIC ports for transmission. -.. _figure_21: +.. _figure_test_pipeline_app: -**Figure 21.Test Pipeline Application** +.. figure:: img/test_pipeline_app.* -.. image24_png has been renamed + Test Pipeline Application -|test_pipeline_app| Compiling the Application ------------------------- @@ -281,5 +280,3 @@ The profile for input traffic is TCP/IPv4 packets with: * destination TCP port fixed to 0 * source TCP port fixed to 0 - -.. |test_pipeline_app| image:: img/test_pipeline_app.* diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst index df8cd8c..1193f2b 100644 --- a/doc/guides/sample_app_ug/vhost.rst +++ b/doc/guides/sample_app_ug/vhost.rst @@ -48,13 +48,12 @@ between host and guest. It was found that virtio-net performance was poor due to context switching and packet copying between host, guest, and QEMU. The following figure shows the system architecture for a virtio-based networking (virtio-net). -.. _figure_16: +.. _figure_qemu_virtio_net: -**Figure16. QEMU Virtio-net (prior to vhost-net)** +.. figure:: img/qemu_virtio_net.* -.. image19_png has been renamed + System Architecture for Virtio-based Networking (virtio-net). -|qemu_virtio_net| The Linux* Kernel vhost-net module was developed as an offload mechanism for virtio-net. The vhost-net module enables KVM (QEMU) to offload the servicing of virtio-net devices to the vhost-net kernel module, @@ -76,13 +75,12 @@ This is achieved by QEMU sharing the following information with the vhost-net mo The following figure shows the system architecture for virtio-net networking with vhost-net offload. -.. _figure_17: +.. _figure_virtio_linux_vhost: -**Figure 17. Virtio with Linux* Kernel Vhost** +.. figure:: img/virtio_linux_vhost.* -.. image20_png has been renamed + Virtio with Linux -|virtio_linux_vhost| Sample Code Overview -------------------- @@ -119,23 +117,21 @@ The vhost sample code application is a simple packet switching application with The following figure shows the architecture of the Vhost sample application based on vhost-cuse. -.. _figure_18: +.. _figure_vhost_net_arch: -**Figure 18. Vhost-net Architectural Overview** +.. figure:: img/vhost_net_arch.* -.. image21_png has been renamed + Vhost-net Architectural Overview -|vhost_net_arch| The following figure shows the flow of packets through the vhost-net sample application. -.. _figure_19: +.. _figure_vhost_net_sample_app: -**Figure 19. Packet Flow Through the vhost-net Sample Application** +.. figure:: img/vhost_net_sample_app.* -.. image22_png has been renamed + Packet Flow Through the vhost-net Sample Application -|vhost_net_sample_app| Supported Distributions ----------------------- @@ -771,13 +767,12 @@ In the "wait and retry" mode if the virtqueue is found to be full, then testpmd The "wait and retry" algorithm is implemented in DPDK testpmd as a forwarding method call "mac_retry". The following sequence diagram describes the algorithm in detail. -.. _figure_20: +.. _figure_tx_dpdk_testpmd: -**Figure 20. Packet Flow on TX in DPDK-testpmd** +.. figure:: img/tx_dpdk_testpmd.* -.. image23_png has been renamed + Packet Flow on TX in DPDK-testpmd -|tx_dpdk_testpmd| Running Testpmd ~~~~~~~~~~~~~~~ @@ -838,13 +833,3 @@ For example: The above message indicates that device 0 has been registered with MAC address cc:bb:bb:bb:bb:bb and VLAN tag 1000. Any packets received on the NIC with these values is placed on the devices receive queue. When a virtio-net device transmits packets, the VLAN tag is added to the packet by the DPDK vhost sample code. - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |qemu_virtio_net| image:: img/qemu_virtio_net.* - -.. |tx_dpdk_testpmd| image:: img/tx_dpdk_testpmd.* - -.. |vhost_net_sample_app| image:: img/vhost_net_sample_app.* - -.. |virtio_linux_vhost| image:: img/virtio_linux_vhost.* diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst index 2a923d8..81db6ad 100644 --- a/doc/guides/sample_app_ug/vm_power_management.rst +++ b/doc/guides/sample_app_ug/vm_power_management.rst @@ -74,11 +74,12 @@ The solution is comprised of two high-level components: The l3fwd-power application will use this implementation when deployed on a VM (see Chapter 11 "L3 Forwarding with Power Management Application"). -.. _figure_24: +.. _figure_vm_power_mgr_highlevel: -**Figure 24. Highlevel Solution** +.. figure:: img/vm_power_mgr_highlevel.* + + Highlevel Solution -|vm_power_mgr_highlevel| Overview -------- @@ -105,11 +106,12 @@ at runtime based on the environment. Upon receiving a request, the host translates the vCPU to a pCPU via the libvirt API before forwarding to the host librte_power. -.. _figure_25: +.. _figure_vm_power_mgr_vm_request_seq: + +.. figure:: img/vm_power_mgr_vm_request_seq.* -**Figure 25. VM request to scale frequency** + VM request to scale frequency -|vm_power_mgr_vm_request_seq| Performance Considerations ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -355,7 +357,3 @@ Where {core_num} is the lcore and channel to change frequency by scaling up/down .. code-block:: console set_cpu_freq {core_num} up|down|min|max - -.. |vm_power_mgr_highlevel| image:: img/vm_power_mgr_highlevel.* - -.. |vm_power_mgr_vm_request_seq| image:: img/vm_power_mgr_vm_request_seq.* diff --git a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst index e5d34e1..49ec6ce 100644 --- a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst +++ b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst @@ -53,7 +53,7 @@ All traffic is read from a single incoming port (port 0) and output on port 1, w The traffic is split into 128 queues on input, where each thread of the application reads from multiple queues. For example, when run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. -As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in Figure 15. +As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in :numref:`figure_vmdq_dcb_example`. The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 32 pools of 4 queues each and this can be used by changing the NUM_POOLS parameter in the supplied code. The NUM_POOLS parameter can be passed on the command line, after the EAL parameters: @@ -64,13 +64,12 @@ The NUM_POOLS parameter can be passed on the command line, after the EAL paramet where, NP can be 16 or 32. -.. _figure_15: +.. _figure_vmdq_dcb_example: -**Figure 15. Packet Flow Through the VMDQ and DCB Sample Application** +.. figure:: img/vmdq_dcb_example.* -.. image18_png has been replaced + Packet Flow Through the VMDQ and DCB Sample Application -|vmdq_dcb_example| In Linux* user space, the application can display statistics with the number of packets received on each queue. To have the application display the statistics, send a SIGHUP signal to the running application process, as follows: @@ -247,5 +246,3 @@ To generate the statistics output, use the following command: Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, rather than the terminal from which the HUP signal was sent. - -.. |vmdq_dcb_example| image:: img/vmdq_dcb_example.* diff --git a/doc/guides/xen/pkt_switch.rst b/doc/guides/xen/pkt_switch.rst index a9eca52..3a6fc47 100644 --- a/doc/guides/xen/pkt_switch.rst +++ b/doc/guides/xen/pkt_switch.rst @@ -52,9 +52,13 @@ The switching back end maps those grant table references and creates shared ring The following diagram describes the functionality of the DPDK Xen Packet- Switching Solution. -.. image35_png has been renamed -|dpdk_xen_pkt_switch| +.. _figure_dpdk_xen_pkt_switch: + +.. figure:: img/dpdk_xen_pkt_switch.* + + Functionality of the DPDK Xen Packet Switching Solution. + Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory between domains (`http://wiki.xen.org/wiki/Grant Table <http://wiki.xen.org/wiki/Grant%20Table>`_). @@ -62,9 +66,13 @@ Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory A diagram of the design is shown below, where "gva" is the Guest Virtual Address, which is the data pointer of the mbuf, and "hva" is the Host Virtual Address: -.. image36_png has been renamed -|grant_table| +.. _figure_grant_table: + +.. figure:: img/grant_table.* + + DPDK Xen Layout + In this design, a Virtio ring is used as a para-virtualized interface for better performance over a Xen private ring when packet switching to and from a VM. @@ -139,9 +147,13 @@ Take idx#_mempool_gref node for example, the host maps those Grant references to The real Grant reference information is stored in this virtual address space, where (gref, pfn) pairs follow each other with -1 as the terminator. -.. image37_pnng has been renamed -|grant_refs| +.. _figure_grant_refs: + +.. figure:: img/grant_refs.* + + Mapping Grant references to a continuous virtual address space + After all gref# IDs are retrieved, the host maps them to a continuous virtual address space. With the guest mempool virtual address, the host establishes 1:1 address mapping. @@ -456,9 +468,3 @@ then sent out through hardware with destination MAC address 00:00:00:00:00:33. The packet flow is: packet generator->Virtio in guest VM1->switching backend->Virtio in guest VM2->switching backend->wire - -.. |grant_table| image:: img/grant_table.* - -.. |grant_refs| image:: img/grant_refs.* - -.. |dpdk_xen_pkt_switch| image:: img/dpdk_xen_pkt_switch.* -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH 2/2] doc: refactored table numbers into references 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 1/2] doc: refactored figure numbers " John McNamara @ 2015-04-24 13:11 ` John McNamara 2015-04-29 16:12 ` [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums " Thomas Monjalon ` (2 subsequent siblings) 4 siblings, 0 replies; 17+ messages in thread From: John McNamara @ 2015-04-24 13:11 UTC (permalink / raw) To: dev This change adds automatic table references to the docs. The table numbers in the generated Html and PDF docs are now automatically numbered based on section. Requires Sphinx >= 1.3. This change: * Adds a RST table:: directive to each table caption. * Indents the tables to the required directive level. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/prog_guide/index.rst | 70 +- doc/guides/prog_guide/packet_framework.rst | 1124 +++++++++++------------ doc/guides/prog_guide/qos_framework.rst | 1340 ++++++++++++++-------------- doc/guides/sample_app_ug/index.rst | 6 +- doc/guides/sample_app_ug/qos_metering.rst | 46 +- doc/guides/sample_app_ug/qos_scheduler.rst | 46 +- doc/guides/sample_app_ug/test_pipeline.rst | 302 +++---- 7 files changed, 1466 insertions(+), 1468 deletions(-) diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 84a657e..3295661 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -174,72 +174,70 @@ Programmer's Guide **Tables** -:ref:`Table 1. Packet Processing Pipeline Implementing QoS <pg_table_1>` +:numref:`table_qos_1` :ref:`table_qos_1` -:ref:`Table 2. Infrastructure Blocks Used by the Packet Processing Pipeline <pg_table_2>` +:numref:`table_qos_2` :ref:`table_qos_2` -:ref:`Table 3. Port Scheduling Hierarchy <pg_table_3>` +:numref:`table_qos_3` :ref:`table_qos_3` -:ref:`Table 4. Scheduler Internal Data Structures per Port <pg_table_4>` +:numref:`table_qos_4` :ref:`table_qos_4` -:ref:`Table 5. Ethernet Frame Overhead Fields <pg_table_5>` +:numref:`table_qos_5` :ref:`table_qos_5` -:ref:`Table 6. Token Bucket Generic Operations <pg_table_6>` +:numref:`table_qos_6` :ref:`table_qos_6` -:ref:`Table 7. Token Bucket Generic Parameters <pg_table_7>` +:numref:`table_qos_7` :ref:`table_qos_7` -:ref:`Table 8. Token Bucket Persistent Data Structure <pg_table_8>` +:numref:`table_qos_8` :ref:`table_qos_8` -:ref:`Table 9. Token Bucket Operations <pg_table_9>` +:numref:`table_qos_9` :ref:`table_qos_9` -:ref:`Table 10. Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure <pg_table_10>` +:numref:`table_qos_10` :ref:`table_qos_10` -:ref:`Table 11. Subport/Pipe Traffic Class Upper Limit Enforcement Operations <pg_table_11>` +:numref:`table_qos_11` :ref:`table_qos_11` -:ref:`Table 12. Weighted Round Robin (WRR) <pg_table_12>` +:numref:`table_qos_12` :ref:`table_qos_12` -:ref:`Table 13. Subport Traffic Class Oversubscription <pg_table_13>` +:numref:`table_qos_13` :ref:`table_qos_13` -:ref:`Table 14. Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period <pg_table_14>` +:numref:`table_qos_14` :ref:`table_qos_14` -:ref:`Table 15. Watermark Calculation <pg_table_15>` +:numref:`table_qos_15` :ref:`table_qos_15` -:ref:`Table 16. RED Configuration Parameters <pg_table_16>` +:numref:`table_qos_16` :ref:`table_qos_16` -:ref:`Table 17. Relative Performance of Alternative Approaches <pg_table_17>` +:numref:`table_qos_17` :ref:`table_qos_17` -:ref:`Table 18. RED Configuration Corresponding to RED Configuration File <pg_table_18>` +:numref:`table_qos_18` :ref:`table_qos_18` -:ref:`Table 19. Port types <pg_table_19>` +:numref:`table_qos_19` :ref:`table_qos_19` -:ref:`Table 20. Port abstract interface <pg_table_20>` +:numref:`table_qos_20` :ref:`table_qos_20` -:ref:`Table 21. Table types <pg_table_21>` +:numref:`table_qos_21` :ref:`table_qos_21` -:ref:`Table 29. Table Abstract Interface <pg_table_29_1>` +:numref:`table_qos_22` :ref:`table_qos_22` -:ref:`Table 22. Configuration parameters common for all hash table types <pg_table_22>` +:numref:`table_qos_23` :ref:`table_qos_23` -:ref:`Table 23. Configuration parameters specific to extendible bucket hash table <pg_table_23>` +:numref:`table_qos_24` :ref:`table_qos_24` -:ref:`Table 24. Configuration parameters specific to pre-computed key signature hash table <pg_table_24>` +:numref:`table_qos_25` :ref:`table_qos_25` -:ref:`Table 25. The main large data structures (arrays) used for configurable key size hash tables <pg_table_25>` +:numref:`table_qos_26` :ref:`table_qos_26` -:ref:`Table 26. Field description for bucket array entry (configurable key size hash tables) <pg_table_26>` +:numref:`table_qos_27` :ref:`table_qos_27` -:ref:`Table 27. Description of the bucket search pipeline stages (configurable key size hash tables) <pg_table_27>` +:numref:`table_qos_28` :ref:`table_qos_28` -:ref:`Table 28. Lookup tables for match, match_many, match_pos <pg_table_28>` +:numref:`table_qos_29` :ref:`table_qos_29` -:ref:`Table 29. Collapsed lookup tables for match, match_many and match_pos <pg_table_29>` +:numref:`table_qos_30` :ref:`table_qos_30` -:ref:`Table 30. The main large data structures (arrays) used for 8-byte and 16-byte key size hash tables <pg_table_30>` +:numref:`table_qos_31` :ref:`table_qos_31` -:ref:`Table 31. Field description for bucket array entry (8-byte and 16-byte key hash tables) <pg_table_31>` +:numref:`table_qos_32` :ref:`table_qos_32` -:ref:`Table 32. Description of the bucket search pipeline stages (8-byte and 16-byte key hash tables) <pg_table_32>` +:numref:`table_qos_33` :ref:`table_qos_33` -:ref:`Table 33. Next hop actions (reserved) <pg_table_33>` - -:ref:`Table 34. User action examples <pg_table_34>` +:numref:`table_qos_34` :ref:`table_qos_34` diff --git a/doc/guides/prog_guide/packet_framework.rst b/doc/guides/prog_guide/packet_framework.rst index 42bbbaa..1b88622 100644 --- a/doc/guides/prog_guide/packet_framework.rst +++ b/doc/guides/prog_guide/packet_framework.rst @@ -84,47 +84,47 @@ Port Types Table 19 is a non-exhaustive list of ports that can be implemented with the Packet Framework. -.. _pg_table_19: - -**Table 19 Port Types** - -+---+------------------+---------------------------------------------------------------------------------------+ -| # | Port type | Description | -| | | | -+===+==================+=======================================================================================+ -| 1 | SW ring | SW circular buffer used for message passing between the application threads. Uses | -| | | the DPDK rte_ring primitive. Expected to be the most commonly used type of | -| | | port. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 2 | HW ring | Queue of buffer descriptors used to interact with NIC, switch or accelerator ports. | -| | | For NIC ports, it uses the DPDK rte_eth_rx_queue or rte_eth_tx_queue | -| | | primitives. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 3 | IP reassembly | Input packets are either IP fragments or complete IP datagrams. Output packets are | -| | | complete IP datagrams. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 4 | IP fragmentation | Input packets are jumbo (IP datagrams with length bigger than MTU) or non-jumbo | -| | | packets. Output packets are non-jumbo packets. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 5 | Traffic manager | Traffic manager attached to a specific NIC output port, performing congestion | -| | | management and hierarchical scheduling according to pre-defined SLAs. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 6 | KNI | Send/receive packets to/from Linux kernel space. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 7 | Source | Input port used as packet generator. Similar to Linux kernel /dev/zero character | -| | | device. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 8 | Sink | Output port used to drop all input packets. Similar to Linux kernel /dev/null | -| | | character device. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ +.. _table_qos_19: + +.. table:: Port Types + + +---+------------------+---------------------------------------------------------------------------------------+ + | # | Port type | Description | + | | | | + +===+==================+=======================================================================================+ + | 1 | SW ring | SW circular buffer used for message passing between the application threads. Uses | + | | | the DPDK rte_ring primitive. Expected to be the most commonly used type of | + | | | port. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 2 | HW ring | Queue of buffer descriptors used to interact with NIC, switch or accelerator ports. | + | | | For NIC ports, it uses the DPDK rte_eth_rx_queue or rte_eth_tx_queue | + | | | primitives. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 3 | IP reassembly | Input packets are either IP fragments or complete IP datagrams. Output packets are | + | | | complete IP datagrams. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 4 | IP fragmentation | Input packets are jumbo (IP datagrams with length bigger than MTU) or non-jumbo | + | | | packets. Output packets are non-jumbo packets. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 5 | Traffic manager | Traffic manager attached to a specific NIC output port, performing congestion | + | | | management and hierarchical scheduling according to pre-defined SLAs. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 6 | KNI | Send/receive packets to/from Linux kernel space. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 7 | Source | Input port used as packet generator. Similar to Linux kernel /dev/zero character | + | | | device. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 8 | Sink | Output port used to drop all input packets. Similar to Linux kernel /dev/null | + | | | character device. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ Port Interface ~~~~~~~~~~~~~~ @@ -134,29 +134,29 @@ Each input/output port is required to implement an abstract interface that defines the initialization and run-time operation of the port. The port abstract interface is described in. -.. _pg_table_20: - -**Table 20 Port Abstract Interface** - -+---+----------------+-----------------------------------------------------------------------------------------+ -| # | Port Operation | Description | -| | | | -+===+================+=========================================================================================+ -| 1 | Create | Create the low-level port object (e.g. queue). Can internally allocate memory. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 2 | Free | Free the resources (e.g. memory) used by the low-level port object. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 3 | RX | Read a burst of input packets. Non-blocking operation. Only defined for input ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 4 | TX | Write a burst of input packets. Non-blocking operation. Only defined for output ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 5 | Flush | Flush the output buffer. Only defined for output ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ +.. _table_qos_20: + +.. table:: 20 Port Abstract Interface + + +---+----------------+-----------------------------------------------------------------------------------------+ + | # | Port Operation | Description | + | | | | + +===+================+=========================================================================================+ + | 1 | Create | Create the low-level port object (e.g. queue). Can internally allocate memory. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 2 | Free | Free the resources (e.g. memory) used by the low-level port object. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 3 | RX | Read a burst of input packets. Non-blocking operation. Only defined for input ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 4 | TX | Write a burst of input packets. Non-blocking operation. Only defined for output ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 5 | Flush | Flush the output buffer. Only defined for output ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ Table Library Design -------------------- @@ -164,71 +164,71 @@ Table Library Design Table Types ~~~~~~~~~~~ -.. _pg_table_21: - Table 21 is a non-exhaustive list of types of tables that can be implemented with the Packet Framework. -**Table 21 Table Types** - -+---+----------------------------+-----------------------------------------------------------------------------+ -| # | Table Type | Description | -| | | | -+===+============================+=============================================================================+ -| 1 | Hash table | Lookup key is n-tuple based. | -| | | | -| | | Typically, the lookup key is hashed to produce a signature that is used to | -| | | identify a bucket of entries where the lookup key is searched next. | -| | | | -| | | The signature associated with the lookup key of each input packet is either | -| | | read from the packet descriptor (pre-computed signature) or computed at | -| | | table lookup time. | -| | | | -| | | The table lookup, add entry and delete entry operations, as well as any | -| | | other pipeline block that pre-computes the signature all have to use the | -| | | same hashing algorithm to generate the signature. | -| | | | -| | | Typically used to implement flow classification tables, ARP caches, routing | -| | | table for tunnelling protocols, etc. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 2 | Longest Prefix Match (LPM) | Lookup key is the IP address. | -| | | | -| | | Each table entries has an associated IP prefix (IP and depth). | -| | | | -| | | The table lookup operation selects the IP prefix that is matched by the | -| | | lookup key; in case of multiple matches, the entry with the longest prefix | -| | | depth wins. | -| | | | -| | | Typically used to implement IP routing tables. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 3 | Access Control List (ACLs) | Lookup key is 7-tuple of two VLAN/MPLS labels, IP destination address, | -| | | IP source addresses, L4 protocol, L4 destination port, L4 source port. | -| | | | -| | | Each table entry has an associated ACL and priority. The ACL contains bit | -| | | masks for the VLAN/MPLS labels, IP prefix for IP destination address, IP | -| | | prefix for IP source addresses, L4 protocol and bitmask, L4 destination | -| | | port and bit mask, L4 source port and bit mask. | -| | | | -| | | The table lookup operation selects the ACL that is matched by the lookup | -| | | key; in case of multiple matches, the entry with the highest priority wins. | -| | | | -| | | Typically used to implement rule databases for firewalls, etc. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 4 | Pattern matching search | Lookup key is the packet payload. | -| | | | -| | | Table is a database of patterns, with each pattern having a priority | -| | | assigned. | -| | | | -| | | The table lookup operation selects the patterns that is matched by the | -| | | input packet; in case of multiple matches, the matching pattern with the | -| | | highest priority wins. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 5 | Array | Lookup key is the table entry index itself. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ +.. _table_qos_21: + +.. table:: Table Types + + +---+----------------------------+-----------------------------------------------------------------------------+ + | # | Table Type | Description | + | | | | + +===+============================+=============================================================================+ + | 1 | Hash table | Lookup key is n-tuple based. | + | | | | + | | | Typically, the lookup key is hashed to produce a signature that is used to | + | | | identify a bucket of entries where the lookup key is searched next. | + | | | | + | | | The signature associated with the lookup key of each input packet is either | + | | | read from the packet descriptor (pre-computed signature) or computed at | + | | | table lookup time. | + | | | | + | | | The table lookup, add entry and delete entry operations, as well as any | + | | | other pipeline block that pre-computes the signature all have to use the | + | | | same hashing algorithm to generate the signature. | + | | | | + | | | Typically used to implement flow classification tables, ARP caches, routing | + | | | table for tunnelling protocols, etc. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 2 | Longest Prefix Match (LPM) | Lookup key is the IP address. | + | | | | + | | | Each table entries has an associated IP prefix (IP and depth). | + | | | | + | | | The table lookup operation selects the IP prefix that is matched by the | + | | | lookup key; in case of multiple matches, the entry with the longest prefix | + | | | depth wins. | + | | | | + | | | Typically used to implement IP routing tables. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 3 | Access Control List (ACLs) | Lookup key is 7-tuple of two VLAN/MPLS labels, IP destination address, | + | | | IP source addresses, L4 protocol, L4 destination port, L4 source port. | + | | | | + | | | Each table entry has an associated ACL and priority. The ACL contains bit | + | | | masks for the VLAN/MPLS labels, IP prefix for IP destination address, IP | + | | | prefix for IP source addresses, L4 protocol and bitmask, L4 destination | + | | | port and bit mask, L4 source port and bit mask. | + | | | | + | | | The table lookup operation selects the ACL that is matched by the lookup | + | | | key; in case of multiple matches, the entry with the highest priority wins. | + | | | | + | | | Typically used to implement rule databases for firewalls, etc. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 4 | Pattern matching search | Lookup key is the packet payload. | + | | | | + | | | Table is a database of patterns, with each pattern having a priority | + | | | assigned. | + | | | | + | | | The table lookup operation selects the patterns that is matched by the | + | | | input packet; in case of multiple matches, the matching pattern with the | + | | | highest priority wins. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 5 | Array | Lookup key is the table entry index itself. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ Table Interface ~~~~~~~~~~~~~~~ @@ -237,40 +237,40 @@ Each table is required to implement an abstract interface that defines the initi and run-time operation of the table. The table abstract interface is described in Table 29. -.. _pg_table_29_1: - -**Table 29 Table Abstract Interface** - -+---+-----------------+----------------------------------------------------------------------------------------+ -| # | Table operation | Description | -| | | | -+===+=================+========================================================================================+ -| 1 | Create | Create the low-level data structures of the lookup table. Can internally allocate | -| | | memory. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 2 | Free | Free up all the resources used by the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 3 | Add entry | Add new entry to the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 4 | Delete entry | Delete specific entry from the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 5 | Lookup | Look up a burst of input packets and return a bit mask specifying the result of the | -| | | lookup operation for each packet: a set bit signifies lookup hit for the corresponding | -| | | packet, while a cleared bit a lookup miss. | -| | | | -| | | For each lookup hit packet, the lookup operation also returns a pointer to the table | -| | | entry that was hit, which contains the actions to be applied on the packet and any | -| | | associated metadata. | -| | | | -| | | For each lookup miss packet, the actions to be applied on the packet and any | -| | | associated metadata are specified by the default table entry preconfigured for lookup | -| | | miss. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ +.. _table_qos_29_1: + +.. table:: Table Abstract Interface + + +---+-----------------+----------------------------------------------------------------------------------------+ + | # | Table operation | Description | + | | | | + +===+=================+========================================================================================+ + | 1 | Create | Create the low-level data structures of the lookup table. Can internally allocate | + | | | memory. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 2 | Free | Free up all the resources used by the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 3 | Add entry | Add new entry to the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 4 | Delete entry | Delete specific entry from the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 5 | Lookup | Look up a burst of input packets and return a bit mask specifying the result of the | + | | | lookup operation for each packet: a set bit signifies lookup hit for the corresponding | + | | | packet, while a cleared bit a lookup miss. | + | | | | + | | | For each lookup hit packet, the lookup operation also returns a pointer to the table | + | | | entry that was hit, which contains the actions to be applied on the packet and any | + | | | associated metadata. | + | | | | + | | | For each lookup miss packet, the actions to be applied on the packet and any | + | | | associated metadata are specified by the default table entry preconfigured for lookup | + | | | miss. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ Hash Table Design @@ -391,38 +391,38 @@ The MAC address of the next hop station becomes the destination MAC address of t Hash Table Types ^^^^^^^^^^^^^^^^ -.. _pg_table_22: - Table 22 lists the hash table configuration parameters shared by all different hash table types. -**Table 22 Configuration Parameters Common for All Hash Table Types** - -+---+---------------------------+------------------------------------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+===========================+==============================================================================+ -| 1 | Key size | Measured as number of bytes. All keys have the same size. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 2 | Key value (key data) size | Measured as number of bytes. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 3 | Number of buckets | Needs to be a power of two. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 4 | Maximum number of keys | Needs to be a power of two. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 5 | Hash function | Examples: jhash, CRC hash, etc. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 6 | Hash function seed | Parameter to be passed to the hash function. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 7 | Key offset | Offset of the lookup key byte array within the packet meta-data stored in | -| | | the packet buffer. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ +.. _table_qos_22: + +.. table:: Configuration Parameters Common for All Hash Table Types + + +---+---------------------------+------------------------------------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+===========================+==============================================================================+ + | 1 | Key size | Measured as number of bytes. All keys have the same size. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 2 | Key value (key data) size | Measured as number of bytes. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 3 | Number of buckets | Needs to be a power of two. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 4 | Maximum number of keys | Needs to be a power of two. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 5 | Hash function | Examples: jhash, CRC hash, etc. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 6 | Hash function seed | Parameter to be passed to the hash function. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 7 | Key offset | Offset of the lookup key byte array within the packet meta-data stored in | + | | | the packet buffer. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ Bucket Full Problem """"""""""""""""""" @@ -452,17 +452,17 @@ The possible options are: the search continues beyond the first group of 4 keys, potentially until all keys in this bucket are examined. The extendible bucket logic requires maintaining specific data structures per table and per each bucket. -.. _pg_table_23: +.. _table_qos_23: -**Table 23 Configuration Parameters Specific to Extendible Bucket Hash Table** +.. table:: Configuration Parameters Specific to Extendible Bucket Hash Table -+---+---------------------------+--------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+===========================+==================================================+ -| 1 | Number of additional keys | Needs to be a power of two, at least equal to 4. | -| | | | -+---+---------------------------+--------------------------------------------------+ + +---+---------------------------+--------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+===========================+==================================================+ + | 1 | Number of additional keys | Needs to be a power of two, at least equal to 4. | + | | | | + +---+---------------------------+--------------------------------------------------+ Signature Computation @@ -481,17 +481,17 @@ The possible options for key signature computation are: The same CPU core reads the key from the packet meta-data, uses it to compute the key signature and also performs the bucket search step of the key lookup operation. -.. _pg_table_24: +.. _table_qos_24: -**Table 24 Configuration Parameters Specific to Pre-computed Key Signature Hash Table** +.. table:: Configuration Parameters Specific to Pre-computed Key Signature Hash Table -+---+------------------+-----------------------------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+==================+=======================================================================+ -| 1 | Signature offset | Offset of the pre-computed key signature within the packet meta-data. | -| | | | -+---+------------------+-----------------------------------------------------------------------+ + +---+------------------+-----------------------------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+==================+=======================================================================+ + | 1 | Signature offset | Offset of the pre-computed key signature within the packet meta-data. | + | | | | + +---+------------------+-----------------------------------------------------------------------+ Key Size Optimized Hash Tables """""""""""""""""""""""""""""" @@ -564,67 +564,67 @@ either with pre-computed signature or "do-sig"). Data Structures for Configurable Key Size Hash Tables -.. _pg_table_25: - -**Table 25 Main Large Data Structures (Arrays) used for Configurable Key Size Hash Tables** - -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| # | Array name | Number of entries | Entry size (bytes) | Description | -| | | | | | -+===+=========================+==============================+===========================+===============================+ -| 1 | Bucket array | n_buckets (configurable) | 32 | Buckets of the hash table. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 2 | Bucket extensions array | n_buckets_ext (configurable) | 32 | This array is only created | -| | | | | for extendible bucket tables. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 3 | Key array | n_keys | key_size (configurable) | Keys added to the hash table. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 4 | Data array | n_keys | entry_size (configurable) | Key values (key data) | -| | | | | associated with the hash | -| | | | | table keys. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ - -.. _pg_table_26: - -**Table 26 Field Description for Bucket Array Entry (Configurable Key Size Hash Tables)** - -+---+------------------+--------------------+------------------------------------------------------------------+ -| # | Field name | Field size (bytes) | Description | -| | | | | -+===+==================+====================+==================================================================+ -| 1 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the | -| | | | current bucket stored as array of 4 entries of 2 bytes each. | -| | | | Entry 0 stores the index (0 .. 3) of the MRU key, while entry 3 | -| | | | stores the index of the LRU key. | -| | | | | -| | | | For extendible bucket tables, this field represents the next | -| | | | pointer (i.e. the pointer to the next group of 4 keys linked to | -| | | | the current bucket). The next pointer is not NULL if the bucket | -| | | | is currently extended or NULL otherwise. | -| | | | To help the branchless implementation, bit 0 (least significant | -| | | | bit) of this field is set to 1 if the next pointer is not NULL | -| | | | and to 0 otherwise. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ -| 2 | Sig[0 .. 3] | 4 x 2 | If key X (X = 0 .. 3) is valid, then sig X bits 15 .. 1 store | -| | | | the most significant 15 bits of key X signature and sig X bit 0 | -| | | | is set to 1. | -| | | | | -| | | | If key X is not valid, then sig X is set to zero. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ -| 3 | Key Pos [0 .. 3] | 4 x 4 | If key X is valid (X = 0 .. 3), then Key Pos X represents the | -| | | | index into the key array where key X is stored, as well as the | -| | | | index into the data array where the value associated with key X | -| | | | is stored. | -| | | | | -| | | | If key X is not valid, then the value of Key Pos X is undefined. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ +.. _table_qos_25: + +.. table:: Main Large Data Structures (Arrays) used for Configurable Key Size Hash Tables + + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | # | Array name | Number of entries | Entry size (bytes) | Description | + | | | | | | + +===+=========================+==============================+===========================+===============================+ + | 1 | Bucket array | n_buckets (configurable) | 32 | Buckets of the hash table. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 2 | Bucket extensions array | n_buckets_ext (configurable) | 32 | This array is only created | + | | | | | for extendible bucket tables. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 3 | Key array | n_keys | key_size (configurable) | Keys added to the hash table. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 4 | Data array | n_keys | entry_size (configurable) | Key values (key data) | + | | | | | associated with the hash | + | | | | | table keys. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + +.. _table_qos_26: + +.. table:: Field Description for Bucket Array Entry (Configurable Key Size Hash Tables) + + +---+------------------+--------------------+------------------------------------------------------------------+ + | # | Field name | Field size (bytes) | Description | + | | | | | + +===+==================+====================+==================================================================+ + | 1 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the | + | | | | current bucket stored as array of 4 entries of 2 bytes each. | + | | | | Entry 0 stores the index (0 .. 3) of the MRU key, while entry 3 | + | | | | stores the index of the LRU key. | + | | | | | + | | | | For extendible bucket tables, this field represents the next | + | | | | pointer (i.e. the pointer to the next group of 4 keys linked to | + | | | | the current bucket). The next pointer is not NULL if the bucket | + | | | | is currently extended or NULL otherwise. | + | | | | To help the branchless implementation, bit 0 (least significant | + | | | | bit) of this field is set to 1 if the next pointer is not NULL | + | | | | and to 0 otherwise. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ + | 2 | Sig[0 .. 3] | 4 x 2 | If key X (X = 0 .. 3) is valid, then sig X bits 15 .. 1 store | + | | | | the most significant 15 bits of key X signature and sig X bit 0 | + | | | | is set to 1. | + | | | | | + | | | | If key X is not valid, then sig X is set to zero. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ + | 3 | Key Pos [0 .. 3] | 4 x 4 | If key X is valid (X = 0 .. 3), then Key Pos X represents the | + | | | | index into the key array where key X is stored, as well as the | + | | | | index into the data array where the value associated with key X | + | | | | is stored. | + | | | | | + | | | | If key X is not valid, then the value of Key Pos X is undefined. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ :numref:`figure_figure35` and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, @@ -639,80 +639,80 @@ For each pipeline stage, the described operations are applied to each of the two Tables) -.. _pg_table_27: - -**Table 27 Description of the Bucket Search Pipeline Stages (Configurable Key Size Hash Tables)** - -+---+---------------------------+------------------------------------------------------------------------------+ -| # | Stage name | Description | -| | | | -+===+===========================+==============================================================================+ -| 0 | Prefetch packet meta-data | Select next two packets from the burst of input packets. | -| | | | -| | | Prefetch packet meta-data containing the key and key signature. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 1 | Prefetch table bucket | Read the key signature from the packet meta-data (for extendable bucket hash | -| | | tables) or read the key from the packet meta-data and compute key signature | -| | | (for LRU tables). | -| | | | -| | | Identify the bucket ID using the key signature. | -| | | | -| | | Set bit 0 of the signature to 1 (to match only signatures of valid keys from | -| | | the table). | -| | | | -| | | Prefetch the bucket. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 2 | Prefetch table key | Read the key signatures from the bucket. | -| | | | -| | | Compare the signature of the input key against the 4 key signatures from the | -| | | packet. As result, the following is obtained: | -| | | | -| | | *match* | -| | | = equal to TRUE if there was at least one signature match and to FALSE in | -| | | the case of no signature match; | -| | | | -| | | *match_many* | -| | | = equal to TRUE is there were more than one signature matches (can be up to | -| | | 4 signature matches in the worst case scenario) and to FALSE otherwise; | -| | | | -| | | *match_pos* | -| | | = the index of the first key that produced signature match (only valid if | -| | | match is true). | -| | | | -| | | For extendable bucket hash tables only, set | -| | | *match_many* | -| | | to TRUE if next pointer is valid. | -| | | | -| | | Prefetch the bucket key indicated by | -| | | *match_pos* | -| | | (even if | -| | | *match_pos* | -| | | does not point to valid key valid). | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 3 | Prefetch table data | Read the bucket key indicated by | -| | | *match_pos*. | -| | | | -| | | Compare the bucket key against the input key. As result, the following is | -| | | obtained: | -| | | *match_key* | -| | | = equal to TRUE if the two keys match and to FALSE otherwise. | -| | | | -| | | Report input key as lookup hit only when both | -| | | *match* | -| | | and | -| | | *match_key* | -| | | are equal to TRUE and as lookup miss otherwise. | -| | | | -| | | For LRU tables only, use branchless logic to update the bucket LRU list | -| | | (the current key becomes the new MRU) only on lookup hit. | -| | | | -| | | Prefetch the key value (key data) associated with the current key (to avoid | -| | | branches, this is done on both lookup hit and miss). | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ +.. _table_qos_27: + +.. table:: Description of the Bucket Search Pipeline Stages (Configurable Key Size Hash Tables) + + +---+---------------------------+------------------------------------------------------------------------------+ + | # | Stage name | Description | + | | | | + +===+===========================+==============================================================================+ + | 0 | Prefetch packet meta-data | Select next two packets from the burst of input packets. | + | | | | + | | | Prefetch packet meta-data containing the key and key signature. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 1 | Prefetch table bucket | Read the key signature from the packet meta-data (for extendable bucket hash | + | | | tables) or read the key from the packet meta-data and compute key signature | + | | | (for LRU tables). | + | | | | + | | | Identify the bucket ID using the key signature. | + | | | | + | | | Set bit 0 of the signature to 1 (to match only signatures of valid keys from | + | | | the table). | + | | | | + | | | Prefetch the bucket. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 2 | Prefetch table key | Read the key signatures from the bucket. | + | | | | + | | | Compare the signature of the input key against the 4 key signatures from the | + | | | packet. As result, the following is obtained: | + | | | | + | | | *match* | + | | | = equal to TRUE if there was at least one signature match and to FALSE in | + | | | the case of no signature match; | + | | | | + | | | *match_many* | + | | | = equal to TRUE is there were more than one signature matches (can be up to | + | | | 4 signature matches in the worst case scenario) and to FALSE otherwise; | + | | | | + | | | *match_pos* | + | | | = the index of the first key that produced signature match (only valid if | + | | | match is true). | + | | | | + | | | For extendable bucket hash tables only, set | + | | | *match_many* | + | | | to TRUE if next pointer is valid. | + | | | | + | | | Prefetch the bucket key indicated by | + | | | *match_pos* | + | | | (even if | + | | | *match_pos* | + | | | does not point to valid key valid). | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 3 | Prefetch table data | Read the bucket key indicated by | + | | | *match_pos*. | + | | | | + | | | Compare the bucket key against the input key. As result, the following is | + | | | obtained: | + | | | *match_key* | + | | | = equal to TRUE if the two keys match and to FALSE otherwise. | + | | | | + | | | Report input key as lookup hit only when both | + | | | *match* | + | | | and | + | | | *match_key* | + | | | are equal to TRUE and as lookup miss otherwise. | + | | | | + | | | For LRU tables only, use branchless logic to update the bucket LRU list | + | | | (the current key becomes the new MRU) only on lookup hit. | + | | | | + | | | Prefetch the key value (key data) associated with the current key (to avoid | + | | | branches, this is done on both lookup hit and miss). | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ Additional notes: @@ -733,62 +733,62 @@ Additional notes: The key signature comparison logic is described in Table 28. -.. _pg_table_28: - -**Table 28 Lookup Tables for Match, Match_Many and Match_Pos** - -+----+------+---------------+--------------------+--------------------+ -| # | mask | match (1 bit) | match_many (1 bit) | match_pos (2 bits) | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 0 | 0000 | 0 | 0 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 1 | 0001 | 1 | 0 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 2 | 0010 | 1 | 0 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 3 | 0011 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 4 | 0100 | 1 | 0 | 10 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 5 | 0101 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 6 | 0110 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 7 | 0111 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 8 | 1000 | 1 | 0 | 11 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 9 | 1001 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 10 | 1010 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 11 | 1011 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 12 | 1100 | 1 | 1 | 10 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 13 | 1101 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 14 | 1110 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 15 | 1111 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ +.. _table_qos_28: + +.. table:: Lookup Tables for Match, Match_Many and Match_Pos + + +----+------+---------------+--------------------+--------------------+ + | # | mask | match (1 bit) | match_many (1 bit) | match_pos (2 bits) | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 0 | 0000 | 0 | 0 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 1 | 0001 | 1 | 0 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 2 | 0010 | 1 | 0 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 3 | 0011 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 4 | 0100 | 1 | 0 | 10 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 5 | 0101 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 6 | 0110 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 7 | 0111 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 8 | 1000 | 1 | 0 | 11 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 9 | 1001 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 10 | 1010 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 11 | 1011 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 12 | 1100 | 1 | 1 | 10 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 13 | 1101 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 14 | 1110 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 15 | 1111 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ The input *mask* hash bit X (X = 0 .. 3) set to 1 if input signature is equal to bucket signature X and set to 0 otherwise. The outputs *match*, *match_many* and *match_pos* are 1 bit, 1 bit and 2 bits in size respectively and their meaning has been explained above. @@ -798,23 +798,23 @@ As displayed in Table 29, the lookup tables for *match* and *match_many* can be Given the input *mask*, the values for *match*, *match_many* and *match_pos* can be obtained by indexing their respective bit array to extract 1 bit, 1 bit and 2 bits respectively with branchless logic. -.. _pg_table_29: +.. _table_qos_29: -**Table 29 Collapsed Lookup Tables for Match, Match_Many and Match_Pos** +.. table:: Collapsed Lookup Tables for Match, Match_Many and Match_Pos -+------------+------------------------------------------+-------------------+ -| | Bit array | Hexadecimal value | -| | | | -+------------+------------------------------------------+-------------------+ -| match | 1111_1111_1111_1110 | 0xFFFELLU | -| | | | -+------------+------------------------------------------+-------------------+ -| match_many | 1111_1110_1110_1000 | 0xFEE8LLU | -| | | | -+------------+------------------------------------------+-------------------+ -| match_pos | 0001_0010_0001_0011__0001_0010_0001_0000 | 0x12131210LLU | -| | | | -+------------+------------------------------------------+-------------------+ + +------------+------------------------------------------+-------------------+ + | | Bit array | Hexadecimal value | + | | | | + +------------+------------------------------------------+-------------------+ + | match | 1111_1111_1111_1110 | 0xFFFELLU | + | | | | + +------------+------------------------------------------+-------------------+ + | match_many | 1111_1110_1110_1000 | 0xFEE8LLU | + | | | | + +------------+------------------------------------------+-------------------+ + | match_pos | 0001_0010_0001_0011__0001_0010_0001_0000 | 0x12131210LLU | + | | | | + +------------+------------------------------------------+-------------------+ The pseudo-code for match, match_many and match_pos is:: @@ -845,67 +845,67 @@ Single Key Size Hash Tables Data Structures for 16-byte Key Hash Tables -.. _pg_table_30: - -**Table 30 Main Large Data Structures (Arrays) used for 8-byte and 16-byte Key Size Hash Tables** - -+---+-------------------------+------------------------------+----------------------+------------------------------------+ -| # | Array name | Number of entries | Entry size (bytes) | Description | -| | | | | | -+===+=========================+==============================+======================+====================================+ -| 1 | Bucket array | n_buckets (configurable) | *8-byte key size:* | Buckets of the hash table. | -| | | | | | -| | | | 64 + 4 x entry_size | | -| | | | | | -| | | | | | -| | | | *16-byte key size:* | | -| | | | | | -| | | | 128 + 4 x entry_size | | -| | | | | | -+---+-------------------------+------------------------------+----------------------+------------------------------------+ -| 2 | Bucket extensions array | n_buckets_ext (configurable) | *8-byte key size:* | This array is only created for | -| | | | | extendible bucket tables. | -| | | | | | -| | | | 64 + 4 x entry_size | | -| | | | | | -| | | | | | -| | | | *16-byte key size:* | | -| | | | | | -| | | | 128 + 4 x entry_size | | -| | | | | | -+---+-------------------------+------------------------------+----------------------+------------------------------------+ - -.. _pg_table_31: - -**Table 31 Field Description for Bucket Array Entry (8-byte and 16-byte Key Hash Tables)** - -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| # | Field name | Field size (bytes) | Description | -| | | | | -+===+===============+====================+===============================================================================+ -| 1 | Valid | 8 | Bit X (X = 0 .. 3) is set to 1 if key X is valid or to 0 otherwise. | -| | | | | -| | | | Bit 4 is only used for extendible bucket tables to help with the | -| | | | implementation of the branchless logic. In this case, bit 4 is set to 1 if | -| | | | next pointer is valid (not NULL) or to 0 otherwise. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 2 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the current bucket | -| | | | stored as array of 4 entries of 2 bytes each. Entry 0 stores the index | -| | | | (0 .. 3) of the MRU key, while entry 3 stores the index of the LRU key. | -| | | | | -| | | | For extendible bucket tables, this field represents the next pointer (i.e. | -| | | | the pointer to the next group of 4 keys linked to the current bucket). The | -| | | | next pointer is not NULL if the bucket is currently extended or NULL | -| | | | otherwise. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 3 | Key [0 .. 3] | 4 x key_size | Full keys. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 4 | Data [0 .. 3] | 4 x entry_size | Full key values (key data) associated with keys 0 .. 3. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ +.. _table_qos_30: + +.. table:: Main Large Data Structures (Arrays) used for 8-byte and 16-byte Key Size Hash Tables + + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + | # | Array name | Number of entries | Entry size (bytes) | Description | + | | | | | | + +===+=========================+==============================+======================+====================================+ + | 1 | Bucket array | n_buckets (configurable) | *8-byte key size:* | Buckets of the hash table. | + | | | | | | + | | | | 64 + 4 x entry_size | | + | | | | | | + | | | | | | + | | | | *16-byte key size:* | | + | | | | | | + | | | | 128 + 4 x entry_size | | + | | | | | | + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + | 2 | Bucket extensions array | n_buckets_ext (configurable) | *8-byte key size:* | This array is only created for | + | | | | | extendible bucket tables. | + | | | | | | + | | | | 64 + 4 x entry_size | | + | | | | | | + | | | | | | + | | | | *16-byte key size:* | | + | | | | | | + | | | | 128 + 4 x entry_size | | + | | | | | | + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + +.. _table_qos_31: + +.. table:: Field Description for Bucket Array Entry (8-byte and 16-byte Key Hash Tables) + + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | # | Field name | Field size (bytes) | Description | + | | | | | + +===+===============+====================+===============================================================================+ + | 1 | Valid | 8 | Bit X (X = 0 .. 3) is set to 1 if key X is valid or to 0 otherwise. | + | | | | | + | | | | Bit 4 is only used for extendible bucket tables to help with the | + | | | | implementation of the branchless logic. In this case, bit 4 is set to 1 if | + | | | | next pointer is valid (not NULL) or to 0 otherwise. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 2 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the current bucket | + | | | | stored as array of 4 entries of 2 bytes each. Entry 0 stores the index | + | | | | (0 .. 3) of the MRU key, while entry 3 stores the index of the LRU key. | + | | | | | + | | | | For extendible bucket tables, this field represents the next pointer (i.e. | + | | | | the pointer to the next group of 4 keys linked to the current bucket). The | + | | | | next pointer is not NULL if the bucket is currently extended or NULL | + | | | | otherwise. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 3 | Key [0 .. 3] | 4 x key_size | Full keys. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 4 | Data [0 .. 3] | 4 x entry_size | Full key values (key data) associated with keys 0 .. 3. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ and detail the bucket search pipeline used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). @@ -919,42 +919,42 @@ For each pipeline stage, the described operations are applied to each of the two Tables) -.. _pg_table_32: - -**Table 32 Description of the Bucket Search Pipeline Stages (8-byte and 16-byte Key Hash Tables)** - -+---+---------------------------+-----------------------------------------------------------------------------+ -| # | Stage name | Description | -| | | | -+===+===========================+=============================================================================+ -| 0 | Prefetch packet meta-data | #. Select next two packets from the burst of input packets. | -| | | | -| | | #. Prefetch packet meta-data containing the key and key signature. | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ -| 1 | Prefetch table bucket | #. Read the key signature from the packet meta-data (for extendable bucket | -| | | hash tables) or read the key from the packet meta-data and compute key | -| | | signature (for LRU tables). | -| | | | -| | | #. Identify the bucket ID using the key signature. | -| | | | -| | | #. Prefetch the bucket. | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ -| 2 | Prefetch table data | #. Read the bucket. | -| | | | -| | | #. Compare all 4 bucket keys against the input key. | -| | | | -| | | #. Report input key as lookup hit only when a match is identified (more | -| | | than one key match is not possible) | -| | | | -| | | #. For LRU tables only, use branchless logic to update the bucket LRU list | -| | | (the current key becomes the new MRU) only on lookup hit. | -| | | | -| | | #. Prefetch the key value (key data) associated with the matched key (to | -| | | avoid branches, this is done on both lookup hit and miss). | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ +.. _table_qos_32: + +.. table:: Description of the Bucket Search Pipeline Stages (8-byte and 16-byte Key Hash Tables) + + +---+---------------------------+-----------------------------------------------------------------------------+ + | # | Stage name | Description | + | | | | + +===+===========================+=============================================================================+ + | 0 | Prefetch packet meta-data | #. Select next two packets from the burst of input packets. | + | | | | + | | | #. Prefetch packet meta-data containing the key and key signature. | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ + | 1 | Prefetch table bucket | #. Read the key signature from the packet meta-data (for extendable bucket | + | | | hash tables) or read the key from the packet meta-data and compute key | + | | | signature (for LRU tables). | + | | | | + | | | #. Identify the bucket ID using the key signature. | + | | | | + | | | #. Prefetch the bucket. | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ + | 2 | Prefetch table data | #. Read the bucket. | + | | | | + | | | #. Compare all 4 bucket keys against the input key. | + | | | | + | | | #. Report input key as lookup hit only when a match is identified (more | + | | | than one key match is not possible) | + | | | | + | | | #. For LRU tables only, use branchless logic to update the bucket LRU list | + | | | (the current key becomes the new MRU) only on lookup hit. | + | | | | + | | | #. Prefetch the key value (key data) associated with the matched key (to | + | | | avoid branches, this is done on both lookup hit and miss). | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ Additional notes: @@ -1043,25 +1043,25 @@ A special category of the reserved actions is represented by the next hop action tables and output ports through the pipeline. Table 33 lists the next hop actions. -.. _pg_table_33: - -**Table 33 Next Hop Actions (Reserved)** - -+---+---------------------+-----------------------------------------------------------------------------------+ -| # | Next hop action | Description | -| | | | -+===+=====================+===================================================================================+ -| 1 | Drop | Drop the current packet. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ -| 2 | Send to output port | Send the current packet to specified output port. The output port ID is metadata | -| | | stored in the same table entry. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ -| 3 | Send to table | Send the current packet to specified table. The table ID is metadata stored in | -| | | the same table entry. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ +.. _table_qos_33: + +.. table:: Next Hop Actions (Reserved) + + +---+---------------------+-----------------------------------------------------------------------------------+ + | # | Next hop action | Description | + | | | | + +===+=====================+===================================================================================+ + | 1 | Drop | Drop the current packet. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ + | 2 | Send to output port | Send the current packet to specified output port. The output port ID is metadata | + | | | stored in the same table entry. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ + | 3 | Send to table | Send the current packet to specified table. The table ID is metadata stored in | + | | | the same table entry. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ User Actions ^^^^^^^^^^^^ @@ -1074,37 +1074,37 @@ for the user actions and their associated meta-data, with each table entry having its own set of enabled user actions and its own copy of the action meta-data. Table 34 contains a non-exhaustive list of user action examples. -.. _pg_table_34: - -**Table 34 User Action Examples** - -+---+-----------------------------------+---------------------------------------------------------------------+ -| # | User action | Description | -| | | | -+===+===================================+=====================================================================+ -| 1 | Metering | Per flow traffic metering using the srTCM and trTCM algorithms. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 2 | Statistics | Update the statistics counters maintained per flow. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 3 | App ID | Per flow state machine fed by variable length sequence of packets | -| | | at the flow initialization with the purpose of identifying the | -| | | traffic type and application. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 4 | Push/pop labels | Push/pop VLAN/MPLS labels to/from the current packet. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 5 | Network Address Translation (NAT) | Translate between the internal (LAN) and external (WAN) IP | -| | | destination/source address and/or L4 protocol destination/source | -| | | port. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 6 | TTL update | Decrement IP TTL and, in case of IPv4 packets, update the IP | -| | | checksum. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ +.. _table_qos_34: + +.. table:: User Action Examples + + +---+-----------------------------------+---------------------------------------------------------------------+ + | # | User action | Description | + | | | | + +===+===================================+=====================================================================+ + | 1 | Metering | Per flow traffic metering using the srTCM and trTCM algorithms. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 2 | Statistics | Update the statistics counters maintained per flow. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 3 | App ID | Per flow state machine fed by variable length sequence of packets | + | | | at the flow initialization with the purpose of identifying the | + | | | traffic type and application. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 4 | Push/pop labels | Push/pop VLAN/MPLS labels to/from the current packet. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 5 | Network Address Translation (NAT) | Translate between the internal (LAN) and external (WAN) IP | + | | | destination/source address and/or L4 protocol destination/source | + | | | port. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 6 | TTL update | Decrement IP TTL and, in case of IPv4 packets, update the IP | + | | | checksum. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ Multicore Scaling ----------------- diff --git a/doc/guides/prog_guide/qos_framework.rst b/doc/guides/prog_guide/qos_framework.rst index 59f7fb3..14c4f36 100644 --- a/doc/guides/prog_guide/qos_framework.rst +++ b/doc/guides/prog_guide/qos_framework.rst @@ -49,73 +49,73 @@ This pipeline can be built using reusable DPDK software libraries. The main blocks implementing QoS in this pipeline are: the policer, the dropper and the scheduler. A functional description of each block is provided in the following table. -.. _pg_table_1: - -**Table 1. Packet Processing Pipeline Implementing QoS** - -+---+------------------------+--------------------------------------------------------------------------------+ -| # | Block | Functional Description | -| | | | -+===+========================+================================================================================+ -| 1 | Packet I/O RX & TX | Packet reception/ transmission from/to multiple NIC ports. Poll mode drivers | -| | | (PMDs) for Intel 1 GbE/10 GbE NICs. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 2 | Packet parser | Identify the protocol stack of the input packet. Check the integrity of the | -| | | packet headers. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 3 | Flow classification | Map the input packet to one of the known traffic flows. Exact match table | -| | | lookup using configurable hash function (jhash, CRC and so on) and bucket | -| | | logic to handle collisions. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 4 | Policer | Packet metering using srTCM (RFC 2697) or trTCM (RFC2698) algorithms. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 5 | Load Balancer | Distribute the input packets to the application workers. Provide uniform load | -| | | to each worker. Preserve the affinity of traffic flows to workers and the | -| | | packet order within each flow. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 6 | Worker threads | Placeholders for the customer specific application workload (for example, IP | -| | | stack and so on). | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 7 | Dropper | Congestion management using the Random Early Detection (RED) algorithm | -| | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED). | -| | | Drop packets based on the current scheduler queue load level and packet | -| | | priority. When congestion is experienced, lower priority packets are dropped | -| | | first. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 8 | Hierarchical Scheduler | 5-level hierarchical scheduler (levels are: output port, subport, pipe, | -| | | traffic class and queue) with thousands (typically 64K) leaf nodes (queues). | -| | | Implements traffic shaping (for subport and pipe levels), strict priority | -| | | (for traffic class level) and Weighted Round Robin (WRR) (for queues within | -| | | each pipe traffic class). | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ +.. _table_qos_1: + +.. table:: Packet Processing Pipeline Implementing QoS + + +---+------------------------+--------------------------------------------------------------------------------+ + | # | Block | Functional Description | + | | | | + +===+========================+================================================================================+ + | 1 | Packet I/O RX & TX | Packet reception/ transmission from/to multiple NIC ports. Poll mode drivers | + | | | (PMDs) for Intel 1 GbE/10 GbE NICs. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 2 | Packet parser | Identify the protocol stack of the input packet. Check the integrity of the | + | | | packet headers. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 3 | Flow classification | Map the input packet to one of the known traffic flows. Exact match table | + | | | lookup using configurable hash function (jhash, CRC and so on) and bucket | + | | | logic to handle collisions. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 4 | Policer | Packet metering using srTCM (RFC 2697) or trTCM (RFC2698) algorithms. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 5 | Load Balancer | Distribute the input packets to the application workers. Provide uniform load | + | | | to each worker. Preserve the affinity of traffic flows to workers and the | + | | | packet order within each flow. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 6 | Worker threads | Placeholders for the customer specific application workload (for example, IP | + | | | stack and so on). | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 7 | Dropper | Congestion management using the Random Early Detection (RED) algorithm | + | | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED). | + | | | Drop packets based on the current scheduler queue load level and packet | + | | | priority. When congestion is experienced, lower priority packets are dropped | + | | | first. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 8 | Hierarchical Scheduler | 5-level hierarchical scheduler (levels are: output port, subport, pipe, | + | | | traffic class and queue) with thousands (typically 64K) leaf nodes (queues). | + | | | Implements traffic shaping (for subport and pipe levels), strict priority | + | | | (for traffic class level) and Weighted Round Robin (WRR) (for queues within | + | | | each pipe traffic class). | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ The infrastructure blocks used throughout the packet processing pipeline are listed in the following table. -.. _pg_table_2: +.. _table_qos_2: -**Table 2. Infrastructure Blocks Used by the Packet Processing Pipeline** +.. table:: Infrastructure Blocks Used by the Packet Processing Pipeline -+---+-----------------------+-----------------------------------------------------------------------+ -| # | Block | Functional Description | -| | | | -+===+=======================+=======================================================================+ -| 1 | Buffer manager | Support for global buffer pools and private per-thread buffer caches. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ -| 2 | Queue manager | Support for message passing between pipeline blocks. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ -| 3 | Power saving | Support for power saving during low activity periods. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ + +---+-----------------------+-----------------------------------------------------------------------+ + | # | Block | Functional Description | + | | | | + +===+=======================+=======================================================================+ + | 1 | Buffer manager | Support for global buffer pools and private per-thread buffer caches. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ + | 2 | Queue manager | Support for message passing between pipeline blocks. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ + | 3 | Power saving | Support for power saving during low activity periods. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ The mapping of pipeline blocks to CPU cores is configurable based on the performance level required by each specific application and the set of features enabled for each block. @@ -170,50 +170,50 @@ Each queue hosts packets from one or multiple connections of the same type belon The functionality of each hierarchical level is detailed in the following table. -.. _pg_table_3: - -**Table 3. Port Scheduling Hierarchy** - -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| # | Level | Siblings per Parent | Functional Description | -| | | | | -+===+====================+============================+===============================================================+ -| 1 | Port | - | #. Output Ethernet port 1/10/40 GbE. | -| | | | | -| | | | #. Multiple ports are scheduled in round robin order with | -| | | | all ports having equal priority. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 2 | Subport | Configurable (default: 8) | #. Traffic shaping using token bucket algorithm (one token | -| | | | bucket per subport). | -| | | | | -| | | | #. Upper limit enforced per Traffic Class (TC) at the | -| | | | subport level. | -| | | | | -| | | | #. Lower priority TCs able to reuse subport bandwidth | -| | | | currently unused by higher priority TCs. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 3 | Pipe | Configurable (default: 4K) | #. Traffic shaping using the token bucket algorithm (one | -| | | | token bucket per pipe. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 4 | Traffic Class (TC) | 4 | #. TCs of the same pipe handled in strict priority order. | -| | | | | -| | | | #. Upper limit enforced per TC at the pipe level. | -| | | | | -| | | | #. Lower priority TCs able to reuse pipe bandwidth currently | -| | | | unused by higher priority TCs. | -| | | | | -| | | | #. When subport TC is oversubscribed (configuration time | -| | | | event), pipe TC upper limit is capped to a dynamically | -| | | | adjusted value that is shared by all the subport pipes. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 5 | Queue | 4 | #. Queues of the same TC are serviced using Weighted Round | -| | | | Robin (WRR) according to predefined weights. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ +.. _table_qos_3: + +.. table:: Port Scheduling Hierarchy + + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | # | Level | Siblings per Parent | Functional Description | + | | | | | + +===+====================+============================+===============================================================+ + | 1 | Port | - | #. Output Ethernet port 1/10/40 GbE. | + | | | | | + | | | | #. Multiple ports are scheduled in round robin order with | + | | | | all ports having equal priority. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 2 | Subport | Configurable (default: 8) | #. Traffic shaping using token bucket algorithm (one token | + | | | | bucket per subport). | + | | | | | + | | | | #. Upper limit enforced per Traffic Class (TC) at the | + | | | | subport level. | + | | | | | + | | | | #. Lower priority TCs able to reuse subport bandwidth | + | | | | currently unused by higher priority TCs. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 3 | Pipe | Configurable (default: 4K) | #. Traffic shaping using the token bucket algorithm (one | + | | | | token bucket per pipe. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 4 | Traffic Class (TC) | 4 | #. TCs of the same pipe handled in strict priority order. | + | | | | | + | | | | #. Upper limit enforced per TC at the pipe level. | + | | | | | + | | | | #. Lower priority TCs able to reuse pipe bandwidth currently | + | | | | unused by higher priority TCs. | + | | | | | + | | | | #. When subport TC is oversubscribed (configuration time | + | | | | event), pipe TC upper limit is capped to a dynamically | + | | | | adjusted value that is shared by all the subport pipes. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 5 | Queue | 4 | #. Queues of the same TC are serviced using Weighted Round | + | | | | Robin (WRR) according to predefined weights. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ Application Programming Interface (API) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -297,65 +297,65 @@ A schematic of the internal data structures in shown in with details in. Internal Data Structures per Port -.. _pg_table_4: - -**Table 4. Scheduler Internal Data Structures per Port** - -+---+----------------------+-------------------------+---------------------+------------------------------+---------------------------------------------------+ -| # | Data structure | Size (bytes) | # per port | Access type | Description | -| | | | | | | -| | | | +-------------+----------------+---------------------------------------------------+ -| | | | | Enq | Deq | | -| | | | | | | | -+===+======================+=========================+=====================+=============+================+===================================================+ -| 1 | Subport table entry | 64 | # subports per port | - | Rd, Wr | Persistent subport data (credits, etc). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 2 | Pipe table entry | 64 | # pipes per port | - | Rd, Wr | Persistent data for pipe, its TCs and its queues | -| | | | | | | (credits, etc) that is updated during run-time. | -| | | | | | | | -| | | | | | | The pipe configuration parameters do not change | -| | | | | | | during run-time. The same pipe configuration | -| | | | | | | parameters are shared by multiple pipes, | -| | | | | | | therefore they are not part of pipe table entry. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 3 | Queue table entry | 4 | #queues per port | Rd, Wr | Rd, Wr | Persistent queue data (read and write pointers). | -| | | | | | | The queue size is the same per TC for all queues, | -| | | | | | | allowing the queue base address to be computed | -| | | | | | | using a fast formula, so these two parameters are | -| | | | | | | not part of queue table entry. | -| | | | | | | | -| | | | | | | The queue table entries for any given pipe are | -| | | | | | | stored in the same cache line. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 4 | Queue storage area | Config (default: 64 x8) | # queues per port | Wr | Rd | Array of elements per queue; each element is 8 | -| | | | | | | byte in size (mbuf pointer). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 5 | Active queues bitmap | 1 bit per queue | 1 | Wr (Set) | Rd, Wr (Clear) | The bitmap maintains one status bit per queue: | -| | | | | | | queue not active (queue is empty) or queue active | -| | | | | | | (queue is not empty). | -| | | | | | | | -| | | | | | | Queue bit is set by the scheduler enqueue and | -| | | | | | | cleared by the scheduler dequeue when queue | -| | | | | | | becomes empty. | -| | | | | | | | -| | | | | | | Bitmap scan operation returns the next non-empty | -| | | | | | | pipe and its status (16-bit mask of active queue | -| | | | | | | in the pipe). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 6 | Grinder | ~128 | Config (default: 8) | - | Rd, Wr | Short list of active pipes currently under | -| | | | | | | processing. The grinder contains temporary data | -| | | | | | | during pipe processing. | -| | | | | | | | -| | | | | | | Once the current pipe exhausts packets or | -| | | | | | | credits, it is replaced with another active pipe | -| | | | | | | from the bitmap. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ +.. _table_qos_4: + +.. table:: Scheduler Internal Data Structures per Port + + +---+----------------------+-------------------------+---------------------+------------------------------+---------------------------------------------------+ + | # | Data structure | Size (bytes) | # per port | Access type | Description | + | | | | | | | + | | | | +-------------+----------------+---------------------------------------------------+ + | | | | | Enq | Deq | | + | | | | | | | | + +===+======================+=========================+=====================+=============+================+===================================================+ + | 1 | Subport table entry | 64 | # subports per port | - | Rd, Wr | Persistent subport data (credits, etc). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 2 | Pipe table entry | 64 | # pipes per port | - | Rd, Wr | Persistent data for pipe, its TCs and its queues | + | | | | | | | (credits, etc) that is updated during run-time. | + | | | | | | | | + | | | | | | | The pipe configuration parameters do not change | + | | | | | | | during run-time. The same pipe configuration | + | | | | | | | parameters are shared by multiple pipes, | + | | | | | | | therefore they are not part of pipe table entry. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 3 | Queue table entry | 4 | #queues per port | Rd, Wr | Rd, Wr | Persistent queue data (read and write pointers). | + | | | | | | | The queue size is the same per TC for all queues, | + | | | | | | | allowing the queue base address to be computed | + | | | | | | | using a fast formula, so these two parameters are | + | | | | | | | not part of queue table entry. | + | | | | | | | | + | | | | | | | The queue table entries for any given pipe are | + | | | | | | | stored in the same cache line. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 4 | Queue storage area | Config (default: 64 x8) | # queues per port | Wr | Rd | Array of elements per queue; each element is 8 | + | | | | | | | byte in size (mbuf pointer). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 5 | Active queues bitmap | 1 bit per queue | 1 | Wr (Set) | Rd, Wr (Clear) | The bitmap maintains one status bit per queue: | + | | | | | | | queue not active (queue is empty) or queue active | + | | | | | | | (queue is not empty). | + | | | | | | | | + | | | | | | | Queue bit is set by the scheduler enqueue and | + | | | | | | | cleared by the scheduler dequeue when queue | + | | | | | | | becomes empty. | + | | | | | | | | + | | | | | | | Bitmap scan operation returns the next non-empty | + | | | | | | | pipe and its status (16-bit mask of active queue | + | | | | | | | in the pipe). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 6 | Grinder | ~128 | Config (default: 8) | - | Rd, Wr | Short list of active pipes currently under | + | | | | | | | processing. The grinder contains temporary data | + | | | | | | | during pipe processing. | + | | | | | | | | + | | | | | | | Once the current pipe exhausts packets or | + | | | | | | | credits, it is replaced with another active pipe | + | | | | | | | from the bitmap. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ Multicore Scaling Strategy ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -578,29 +578,29 @@ As the greatest common divisor for all packet lengths is one byte, the unit of c The number of credits required for the transmission of a packet of n bytes is equal to (n+h), where h is equal to the number of framing overhead bytes per packet. -.. _pg_table_5: - -**Table 5. Ethernet Frame Overhead Fields** - -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| # | Packet field | Length (bytes) | Comments | -| | | | | -+===+================================+================+===========================================================================+ -| 1 | Preamble | 7 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 2 | Start of Frame Delimiter (SFD) | 1 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 3 | Frame Check Sequence (FCS) | 4 | Considered overhead only if not included in the mbuf packet length field. | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 4 | Inter Frame Gap (IFG) | 12 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 5 | Total | 24 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ +.. _table_qos_5: + +.. table:: Ethernet Frame Overhead Fields + + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | # | Packet field | Length (bytes) | Comments | + | | | | | + +===+================================+================+===========================================================================+ + | 1 | Preamble | 7 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 2 | Start of Frame Delimiter (SFD) | 1 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 3 | Frame Check Sequence (FCS) | 4 | Considered overhead only if not included in the mbuf packet length field. | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 4 | Inter Frame Gap (IFG) | 12 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 5 | Total | 24 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ Traffic Shaping """"""""""""""" @@ -610,78 +610,78 @@ Each token bucket is implemented using one saturated counter that keeps track of The token bucket generic parameters and operations are presented in Table 6 and Table 7. -.. _pg_table_6: - -**Table 6. Token Bucket Generic Operations** - -+---+------------------------+--------------------+---------------------------------------------------------+ -| # | Token Bucket Parameter | Unit | Description | -| | | | | -+===+========================+====================+=========================================================+ -| 1 | bucket_rate | Credits per second | Rate of adding credits to the bucket. | -| | | | | -+---+------------------------+--------------------+---------------------------------------------------------+ -| 2 | bucket_size | Credits | Max number of credits that can be stored in the bucket. | -| | | | | -+---+------------------------+--------------------+---------------------------------------------------------+ - -.. _pg_table_7: - -**Table 7. Token Bucket Generic Parameters** - -+---+------------------------+------------------------------------------------------------------------------+ -| # | Token Bucket Operation | Description | -| | | | -+===+========================+==============================================================================+ -| 1 | Initialization | Bucket set to a predefined value, e.g. zero or half of the bucket size. | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ -| 2 | Credit update | Credits are added to the bucket on top of existing ones, either periodically | -| | | or on demand, based on the bucket_rate. Credits cannot exceed the upper | -| | | limit defined by the bucket_size, so any credits to be added to the bucket | -| | | while the bucket is full are dropped. | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | -| | | from the bucket. The packet can only be sent if enough credits are in the | -| | | bucket to send the full packet (packet bytes and framing overhead for the | -| | | packet). | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ +.. _table_qos_6: + +.. table:: Token Bucket Generic Operations + + +---+------------------------+--------------------+---------------------------------------------------------+ + | # | Token Bucket Parameter | Unit | Description | + | | | | | + +===+========================+====================+=========================================================+ + | 1 | bucket_rate | Credits per second | Rate of adding credits to the bucket. | + | | | | | + +---+------------------------+--------------------+---------------------------------------------------------+ + | 2 | bucket_size | Credits | Max number of credits that can be stored in the bucket. | + | | | | | + +---+------------------------+--------------------+---------------------------------------------------------+ + +.. _table_qos_7: + +.. table:: Token Bucket Generic Parameters + + +---+------------------------+------------------------------------------------------------------------------+ + | # | Token Bucket Operation | Description | + | | | | + +===+========================+==============================================================================+ + | 1 | Initialization | Bucket set to a predefined value, e.g. zero or half of the bucket size. | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ + | 2 | Credit update | Credits are added to the bucket on top of existing ones, either periodically | + | | | or on demand, based on the bucket_rate. Credits cannot exceed the upper | + | | | limit defined by the bucket_size, so any credits to be added to the bucket | + | | | while the bucket is full are dropped. | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | + | | | from the bucket. The packet can only be sent if enough credits are in the | + | | | bucket to send the full packet (packet bytes and framing overhead for the | + | | | packet). | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ To implement the token bucket generic operations described above, the current design uses the persistent data structure presented in, while the implementation of the token bucket operations is described in Table 9. -.. _pg_table_8: - -**Table 8. Token Bucket Persistent Data Structure** - -+---+------------------------+-------+----------------------------------------------------------------------+ -| # | Token bucket field | Unit | Description | -| | | | | -+===+========================+=======+======================================================================+ -| 1 | tb_time | Bytes | Time of the last credit update. Measured in bytes instead of seconds | -| | | | or CPU cycles for ease of credit consumption operation | -| | | | (as the current time is also maintained in bytes). | -| | | | | -| | | | See Section 26.2.4.5.1 "Internal Time Reference" for an | -| | | | explanation of why the time is maintained in byte units. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 2 | tb_period | Bytes | Time period that should elapse since the last credit update in order | -| | | | for the bucket to be awarded tb_credits_per_period worth or credits. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 3 | tb_credits_per_period | Bytes | Credit allowance per tb_period. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 4 | tb_size | Bytes | Bucket size, i.e. upper limit for the tb_credits. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 5 | tb_credits | Bytes | Number of credits currently in the bucket. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ +.. _table_qos_8: + +.. table:: Token Bucket Persistent Data Structure + + +---+------------------------+-------+----------------------------------------------------------------------+ + | # | Token bucket field | Unit | Description | + | | | | | + +===+========================+=======+======================================================================+ + | 1 | tb_time | Bytes | Time of the last credit update. Measured in bytes instead of seconds | + | | | | or CPU cycles for ease of credit consumption operation | + | | | | (as the current time is also maintained in bytes). | + | | | | | + | | | | See Section 26.2.4.5.1 "Internal Time Reference" for an | + | | | | explanation of why the time is maintained in byte units. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 2 | tb_period | Bytes | Time period that should elapse since the last credit update in order | + | | | | for the bucket to be awarded tb_credits_per_period worth or credits. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 3 | tb_credits_per_period | Bytes | Credit allowance per tb_period. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 4 | tb_size | Bytes | Bucket size, i.e. upper limit for the tb_credits. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 5 | tb_credits | Bytes | Number of credits currently in the bucket. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ The bucket rate (in bytes per second) can be computed with the following formula: @@ -689,65 +689,65 @@ The bucket rate (in bytes per second) can be computed with the following formula where, r = port line rate (in bytes per second). -.. _pg_table_9: - -**Table 9. Token Bucket Operations** - -+---+-------------------------+-----------------------------------------------------------------------------+ -| # | Token bucket operation | Description | -| | | | -+===+=========================+=============================================================================+ -| 1 | Initialization | *tb_credits = 0; or tb_credits = tb_size / 2;* | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ -| 2 | Credit update | Credit update options: | -| | | | -| | | * Every time a packet is sent for a port, update the credits of all the | -| | | the subports and pipes of that port. Not feasible. | -| | | | -| | | * Every time a packet is sent, update the credits for the pipe and | -| | | subport. Very accurate, but not needed (a lot of calculations). | -| | | | -| | | * Every time a pipe is selected (that is, picked by one | -| | | of the grinders), update the credits for the pipe and its subport. | -| | | | -| | | The current implementation is using option 3. According to Section | -| | | 26.2.4.4 "Dequeue State Machine", the pipe and subport credits are | -| | | updated every time a pipe is selected by the dequeue process before the | -| | | pipe and subport credits are actually used. | -| | | | -| | | The implementation uses a tradeoff between accuracy and speed by updating | -| | | the bucket credits only when at least a full *tb_period* has elapsed since | -| | | the last update. | -| | | | -| | | * Full accuracy can be achieved by selecting the value for *tb_period* | -| | | for which *tb_credits_per_period = 1*. | -| | | | -| | | * When full accuracy is not required, better performance is achieved by | -| | | setting *tb_credits* to a larger value. | -| | | | -| | | Update operations: | -| | | | -| | | * n_periods = (time - tb_time) / tb_period; | -| | | | -| | | * tb_credits += n_periods * tb_credits_per_period; | -| | | | -| | | * tb_credits = min(tb_credits, tb_size); | -| | | | -| | | * tb_time += n_periods * tb_period; | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | -| | (on packet scheduling) | from the bucket. The packet can only be sent if enough credits are in the | -| | | bucket to send the full packet (packet bytes and framing overhead for the | -| | | packet). | -| | | | -| | | Scheduling operations: | -| | | | -| | | pkt_credits = pkt_len + frame_overhead; | -| | | if (tb_credits >= pkt_credits){tb_credits -= pkt_credits;} | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ +.. _table_qos_9: + +.. table:: Token Bucket Operations + + +---+-------------------------+-----------------------------------------------------------------------------+ + | # | Token bucket operation | Description | + | | | | + +===+=========================+=============================================================================+ + | 1 | Initialization | *tb_credits = 0; or tb_credits = tb_size / 2;* | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ + | 2 | Credit update | Credit update options: | + | | | | + | | | * Every time a packet is sent for a port, update the credits of all the | + | | | the subports and pipes of that port. Not feasible. | + | | | | + | | | * Every time a packet is sent, update the credits for the pipe and | + | | | subport. Very accurate, but not needed (a lot of calculations). | + | | | | + | | | * Every time a pipe is selected (that is, picked by one | + | | | of the grinders), update the credits for the pipe and its subport. | + | | | | + | | | The current implementation is using option 3. According to Section | + | | | 26.2.4.4 "Dequeue State Machine", the pipe and subport credits are | + | | | updated every time a pipe is selected by the dequeue process before the | + | | | pipe and subport credits are actually used. | + | | | | + | | | The implementation uses a tradeoff between accuracy and speed by updating | + | | | the bucket credits only when at least a full *tb_period* has elapsed since | + | | | the last update. | + | | | | + | | | * Full accuracy can be achieved by selecting the value for *tb_period* | + | | | for which *tb_credits_per_period = 1*. | + | | | | + | | | * When full accuracy is not required, better performance is achieved by | + | | | setting *tb_credits* to a larger value. | + | | | | + | | | Update operations: | + | | | | + | | | * n_periods = (time - tb_time) / tb_period; | + | | | | + | | | * tb_credits += n_periods * tb_credits_per_period; | + | | | | + | | | * tb_credits = min(tb_credits, tb_size); | + | | | | + | | | * tb_time += n_periods * tb_period; | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | + | | (on packet scheduling) | from the bucket. The packet can only be sent if enough credits are in the | + | | | bucket to send the full packet (packet bytes and framing overhead for the | + | | | packet). | + | | | | + | | | Scheduling operations: | + | | | | + | | | pkt_credits = pkt_len + frame_overhead; | + | | | if (tb_credits >= pkt_credits){tb_credits -= pkt_credits;} | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ Traffic Classes """"""""""""""" @@ -772,175 +772,175 @@ pipe levels is enforced by periodically refilling the subport / pipe traffic cla out of which credits are consumed every time a packet is scheduled for that subport / pipe, as described in Table 10 and Table 11. -.. _pg_table_10: - -**Table 10. Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure** - -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| # | Subport or pipe field | Unit | Description | -| | | | | -+===+=======================+=======+=======================================================================+ -| 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the 4 TCs of the | -| | | | current subport / pipe. | -| | | | | -| | | | See Section 26.2.4.5.1, "Internal Time Reference" for the | -| | | | explanation of why the time is maintained in byte units. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 2 | tc_period | Bytes | Time between two consecutive updates for the 4 TCs of the current | -| | | | subport / pipe. This is expected to be many times bigger than the | -| | | | typical value of the token bucket tb_period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 3 | tc_credits_per_period | Bytes | Upper limit for the number of credits allowed to be consumed by the | -| | | | current TC during each enforcement period tc_period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 4 | tc_credits | Bytes | Current upper limit for the number of credits that can be consumed by | -| | | | the current traffic class for the remainder of the current | -| | | | enforcement period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ - -.. _pg_table_11: - -**Table 11. Subport/Pipe Traffic Class Upper Limit Enforcement Operations** - -+---+--------------------------+----------------------------------------------------------------------------+ -| # | Traffic Class Operation | Description | -| | | | -+===+==========================+============================================================================+ -| 1 | Initialization | tc_credits = tc_credits_per_period; | -| | | | -| | | tc_time = tc_period; | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ -| 2 | Credit update | Update operations: | -| | | | -| | | if (time >= tc_time) { | -| | | | -| | | tc_credits = tc_credits_per_period; | -| | | | -| | | tc_time = time + tc_period; | -| | | | -| | | } | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the TC limit is decreased with the | -| | (on packet scheduling) | necessary number of credits. The packet can only be sent if enough credits | -| | | are currently available in the TC limit to send the full packet | -| | | (packet bytes and framing overhead for the packet). | -| | | | -| | | Scheduling operations: | -| | | | -| | | pkt_credits = pk_len + frame_overhead; | -| | | | -| | | if (tc_credits >= pkt_credits) {tc_credits -= pkt_credits;} | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ +.. _table_qos_10: + +.. table:: Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure + + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | # | Subport or pipe field | Unit | Description | + | | | | | + +===+=======================+=======+=======================================================================+ + | 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the 4 TCs of the | + | | | | current subport / pipe. | + | | | | | + | | | | See Section 26.2.4.5.1, "Internal Time Reference" for the | + | | | | explanation of why the time is maintained in byte units. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 2 | tc_period | Bytes | Time between two consecutive updates for the 4 TCs of the current | + | | | | subport / pipe. This is expected to be many times bigger than the | + | | | | typical value of the token bucket tb_period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 3 | tc_credits_per_period | Bytes | Upper limit for the number of credits allowed to be consumed by the | + | | | | current TC during each enforcement period tc_period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 4 | tc_credits | Bytes | Current upper limit for the number of credits that can be consumed by | + | | | | the current traffic class for the remainder of the current | + | | | | enforcement period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + +.. _table_qos_11: + +.. table:: Subport/Pipe Traffic Class Upper Limit Enforcement Operations + + +---+--------------------------+----------------------------------------------------------------------------+ + | # | Traffic Class Operation | Description | + | | | | + +===+==========================+============================================================================+ + | 1 | Initialization | tc_credits = tc_credits_per_period; | + | | | | + | | | tc_time = tc_period; | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ + | 2 | Credit update | Update operations: | + | | | | + | | | if (time >= tc_time) { | + | | | | + | | | tc_credits = tc_credits_per_period; | + | | | | + | | | tc_time = time + tc_period; | + | | | | + | | | } | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the TC limit is decreased with the | + | | (on packet scheduling) | necessary number of credits. The packet can only be sent if enough credits | + | | | are currently available in the TC limit to send the full packet | + | | | (packet bytes and framing overhead for the packet). | + | | | | + | | | Scheduling operations: | + | | | | + | | | pkt_credits = pk_len + frame_overhead; | + | | | | + | | | if (tc_credits >= pkt_credits) {tc_credits -= pkt_credits;} | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ Weighted Round Robin (WRR) """""""""""""""""""""""""" The evolution of the WRR design solution from simple to complex is shown in Table 12. -.. _pg_table_12: - -**Table 12. Weighted Round Robin (WRR)** - -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| # | All Queues | Equal Weights | All Packets | Strategy | -| | Active? | for All Queues? | Equal? | | -+===+============+=================+=============+==========================================================+ -| 1 | Yes | Yes | Yes | **Byte level round robin** | -| | | | | | -| | | | | *Next queue* queue #i, i = *(i + 1) % n* | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 2 | Yes | Yes | No | **Packet level round robin** | -| | | | | | -| | | | | Consuming one byte from queue #i requires consuming | -| | | | | exactly one token for queue #i. | -| | | | | | -| | | | | T(i) = Accumulated number of tokens previously consumed | -| | | | | from queue #i. Every time a packet is consumed from | -| | | | | queue #i, T(i) is updated as: T(i) += *pkt_len*. | -| | | | | | -| | | | | *Next queue* : queue with the smallest T. | -| | | | | | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 3 | Yes | No | No | **Packet level weighted round robin** | -| | | | | | -| | | | | This case can be reduced to the previous case by | -| | | | | introducing a cost per byte that is different for each | -| | | | | queue. Queues with lower weights have a higher cost per | -| | | | | byte. This way, it is still meaningful to compare the | -| | | | | consumption amongst different queues in order to select | -| | | | | the next queue. | -| | | | | | -| | | | | w(i) = Weight of queue #i | -| | | | | | -| | | | | t(i) = Tokens per byte for queue #i, defined as the | -| | | | | inverse weight of queue #i. | -| | | | | For example, if w[0..3] = [1:2:4:8], | -| | | | | then t[0..3] = [8:4:2:1]; if w[0..3] = [1:4:15:20], | -| | | | | then t[0..3] = [60:15:4:3]. | -| | | | | Consuming one byte from queue #i requires consuming t(i) | -| | | | | tokens for queue #i. | -| | | | | | -| | | | | T(i) = Accumulated number of tokens previously consumed | -| | | | | from queue #i. Every time a packet is consumed from | -| | | | | queue #i, T(i) is updated as: *T(i) += pkt_len * t(i)*. | -| | | | | *Next queue* : queue with the smallest T. | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 4 | No | No | No | **Packet level weighted round robin with variable queue | -| | | | | status** | -| | | | | | -| | | | | Reduce this case to the previous case by setting the | -| | | | | consumption of inactive queues to a high number, so that | -| | | | | the inactive queues will never be selected by the | -| | | | | smallest T logic. | -| | | | | | -| | | | | To prevent T from overflowing as result of successive | -| | | | | accumulations, T(i) is truncated after each packet | -| | | | | consumption for all queues. | -| | | | | For example, T[0..3] = [1000, 1100, 1200, 1300] | -| | | | | is truncated to T[0..3] = [0, 100, 200, 300] | -| | | | | by subtracting the min T from T(i), i = 0..n. | -| | | | | | -| | | | | This requires having at least one active queue in the | -| | | | | set of input queues, which is guaranteed by the dequeue | -| | | | | state machine never selecting an inactive traffic class. | -| | | | | | -| | | | | *mask(i) = Saturation mask for queue #i, defined as:* | -| | | | | | -| | | | | mask(i) = (queue #i is active)? 0 : 0xFFFFFFFF; | -| | | | | | -| | | | | w(i) = Weight of queue #i | -| | | | | | -| | | | | t(i) = Tokens per byte for queue #i, defined as the | -| | | | | inverse weight of queue #i. | -| | | | | | -| | | | | T(i) = Accumulated numbers of tokens previously consumed | -| | | | | from queue #i. | -| | | | | | -| | | | | *Next queue* : queue with smallest T. | -| | | | | | -| | | | | Before packet consumption from queue #i: | -| | | | | | -| | | | | *T(i) |= mask(i)* | -| | | | | | -| | | | | After packet consumption from queue #i: | -| | | | | | -| | | | | T(j) -= T(i), j != i | -| | | | | | -| | | | | T(i) = pkt_len * t(i) | -| | | | | | -| | | | | Note: T(j) uses the T(i) value before T(i) is updated. | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ +.. _table_qos_12: + +.. table:: Weighted Round Robin (WRR) + + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | # | All Queues | Equal Weights | All Packets | Strategy | + | | Active? | for All Queues? | Equal? | | + +===+============+=================+=============+==========================================================+ + | 1 | Yes | Yes | Yes | **Byte level round robin** | + | | | | | | + | | | | | *Next queue* queue #i, i = *(i + 1) % n* | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 2 | Yes | Yes | No | **Packet level round robin** | + | | | | | | + | | | | | Consuming one byte from queue #i requires consuming | + | | | | | exactly one token for queue #i. | + | | | | | | + | | | | | T(i) = Accumulated number of tokens previously consumed | + | | | | | from queue #i. Every time a packet is consumed from | + | | | | | queue #i, T(i) is updated as: T(i) += *pkt_len*. | + | | | | | | + | | | | | *Next queue* : queue with the smallest T. | + | | | | | | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 3 | Yes | No | No | **Packet level weighted round robin** | + | | | | | | + | | | | | This case can be reduced to the previous case by | + | | | | | introducing a cost per byte that is different for each | + | | | | | queue. Queues with lower weights have a higher cost per | + | | | | | byte. This way, it is still meaningful to compare the | + | | | | | consumption amongst different queues in order to select | + | | | | | the next queue. | + | | | | | | + | | | | | w(i) = Weight of queue #i | + | | | | | | + | | | | | t(i) = Tokens per byte for queue #i, defined as the | + | | | | | inverse weight of queue #i. | + | | | | | For example, if w[0..3] = [1:2:4:8], | + | | | | | then t[0..3] = [8:4:2:1]; if w[0..3] = [1:4:15:20], | + | | | | | then t[0..3] = [60:15:4:3]. | + | | | | | Consuming one byte from queue #i requires consuming t(i) | + | | | | | tokens for queue #i. | + | | | | | | + | | | | | T(i) = Accumulated number of tokens previously consumed | + | | | | | from queue #i. Every time a packet is consumed from | + | | | | | queue #i, T(i) is updated as: *T(i) += pkt_len * t(i)*. | + | | | | | *Next queue* : queue with the smallest T. | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 4 | No | No | No | **Packet level weighted round robin with variable queue | + | | | | | status** | + | | | | | | + | | | | | Reduce this case to the previous case by setting the | + | | | | | consumption of inactive queues to a high number, so that | + | | | | | the inactive queues will never be selected by the | + | | | | | smallest T logic. | + | | | | | | + | | | | | To prevent T from overflowing as result of successive | + | | | | | accumulations, T(i) is truncated after each packet | + | | | | | consumption for all queues. | + | | | | | For example, T[0..3] = [1000, 1100, 1200, 1300] | + | | | | | is truncated to T[0..3] = [0, 100, 200, 300] | + | | | | | by subtracting the min T from T(i), i = 0..n. | + | | | | | | + | | | | | This requires having at least one active queue in the | + | | | | | set of input queues, which is guaranteed by the dequeue | + | | | | | state machine never selecting an inactive traffic class. | + | | | | | | + | | | | | *mask(i) = Saturation mask for queue #i, defined as:* | + | | | | | | + | | | | | mask(i) = (queue #i is active)? 0 : 0xFFFFFFFF; | + | | | | | | + | | | | | w(i) = Weight of queue #i | + | | | | | | + | | | | | t(i) = Tokens per byte for queue #i, defined as the | + | | | | | inverse weight of queue #i. | + | | | | | | + | | | | | T(i) = Accumulated numbers of tokens previously consumed | + | | | | | from queue #i. | + | | | | | | + | | | | | *Next queue* : queue with smallest T. | + | | | | | | + | | | | | Before packet consumption from queue #i: | + | | | | | | + | | | | | *T(i) |= mask(i)* | + | | | | | | + | | | | | After packet consumption from queue #i: | + | | | | | | + | | | | | T(j) -= T(i), j != i | + | | | | | | + | | | | | T(i) = pkt_len * t(i) | + | | | | | | + | | | | | Note: T(j) uses the T(i) value before T(i) is updated. | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ Subport Traffic Class Oversubscription """""""""""""""""""""""""""""""""""""" @@ -969,40 +969,40 @@ Solution Space summarizes some of the possible approaches for handling this problem, with the third approach selected for implementation. -.. _pg_table_13: - -**Table 13. Subport Traffic Class Oversubscription** - -+-----+---------------------------+-------------------------------------------------------------------------+ -| No. | Approach | Description | -| | | | -+=====+===========================+=========================================================================+ -| 1 | Don't care | First come, first served. | -| | | | -| | | This approach is not fair amongst subport member pipes, as pipes that | -| | | are served first will use up as much bandwidth for TC X as they need, | -| | | while pipes that are served later will receive poor service due to | -| | | bandwidth for TC X at the subport level being scarce. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ -| 2 | Scale down all pipes | All pipes within the subport have their bandwidth limit for TC X scaled | -| | | down by the same factor. | -| | | | -| | | This approach is not fair among subport member pipes, as the low end | -| | | pipes (that is, pipes configured with low bandwidth) can potentially | -| | | experience severe service degradation that might render their service | -| | | unusable (if available bandwidth for these pipes drops below the | -| | | minimum requirements for a workable service), while the service | -| | | degradation for high end pipes might not be noticeable at all. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ -| 3 | Cap the high demand pipes | Each subport member pipe receives an equal share of the bandwidth | -| | | available at run-time for TC X at the subport level. Any bandwidth left | -| | | unused by the low-demand pipes is redistributed in equal portions to | -| | | the high-demand pipes. This way, the high-demand pipes are truncated | -| | | while the low-demand pipes are not impacted. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ +.. _table_qos_13: + +.. table:: Subport Traffic Class Oversubscription + + +-----+---------------------------+-------------------------------------------------------------------------+ + | No. | Approach | Description | + | | | | + +=====+===========================+=========================================================================+ + | 1 | Don't care | First come, first served. | + | | | | + | | | This approach is not fair amongst subport member pipes, as pipes that | + | | | are served first will use up as much bandwidth for TC X as they need, | + | | | while pipes that are served later will receive poor service due to | + | | | bandwidth for TC X at the subport level being scarce. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ + | 2 | Scale down all pipes | All pipes within the subport have their bandwidth limit for TC X scaled | + | | | down by the same factor. | + | | | | + | | | This approach is not fair among subport member pipes, as the low end | + | | | pipes (that is, pipes configured with low bandwidth) can potentially | + | | | experience severe service degradation that might render their service | + | | | unusable (if available bandwidth for these pipes drops below the | + | | | minimum requirements for a workable service), while the service | + | | | degradation for high end pipes might not be noticeable at all. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ + | 3 | Cap the high demand pipes | Each subport member pipe receives an equal share of the bandwidth | + | | | available at run-time for TC X at the subport level. Any bandwidth left | + | | | unused by the low-demand pipes is redistributed in equal portions to | + | | | the high-demand pipes. This way, the high-demand pipes are truncated | + | | | while the low-demand pipes are not impacted. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ Typically, the subport TC oversubscription feature is enabled only for the lowest priority traffic class (TC 3), which is typically used for best effort traffic, @@ -1037,98 +1037,98 @@ When demand is low, the watermark is set high to prevent it from impeding the su The highest value for the watermark is picked as the highest rate configured for a subport member pipe. Table 15 illustrates the watermark operation. -.. _pg_table_14: - -**Table 14. Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period** - -+-----+---------------------------------+----------------------------------------------------+ -| No. | Subport Traffic Class Operation | Description | -| | | | -+=====+=================================+====================================================+ -| 1 | Initialization | **Subport level**: subport_period_id= 0 | -| | | | -| | | **Pipe level**: pipe_period_id = 0 | -| | | | -+-----+---------------------------------+----------------------------------------------------+ -| 2 | Credit update | **Subport Level**: | -| | | | -| | | if (time>=subport_tc_time) | -| | | | -| | | { | -| | | subport_wm = water_mark_update(); | -| | | | -| | | subport_tc_time = time + subport_tc_period; | -| | | | -| | | subport_period_id++; | -| | | | -| | | } | -| | | | -| | | **Pipelevel:** | -| | | | -| | | if(pipe_period_id != subport_period_id) | -| | | | -| | | { | -| | | | -| | | pipe_ov_credits = subport_wm \* pipe_weight; | -| | | | -| | | pipe_period_id = subport_period_id; | -| | | | -| | | } | -| | | | -+-----+---------------------------------+----------------------------------------------------+ -| 3 | Credit consumption | **Pipe level:** | -| | (on packet scheduling) | | -| | | pkt_credits = pk_len + frame_overhead; | -| | | | -| | | if(pipe_ov_credits >= pkt_credits{ | -| | | | -| | | pipe_ov_credits -= pkt_credits; | -| | | | -| | | } | -| | | | -+-----+---------------------------------+----------------------------------------------------+ - -.. _pg_table_15: - -**Table 15. Watermark Calculation** - -+-----+------------------+----------------------------------------------------------------------------------+ -| No. | Subport Traffic | Description | -| | Class Operation | | -+=====+==================+==================================================================================+ -| 1 | Initialization | **Subport level:** | -| | | | -| | | wm = WM_MAX | -| | | | -+-----+------------------+----------------------------------------------------------------------------------+ -| 2 | Credit update | **Subport level (water_mark_update):** | -| | | | -| | | tc0_cons = subport_tc0_credits_per_period - subport_tc0_credits; | -| | | | -| | | tc1_cons = subport_tc1_credits_per_period - subport_tc1_credits; | -| | | | -| | | tc2_cons = subport_tc2_credits_per_period - subport_tc2_credits; | -| | | | -| | | tc3_cons = subport_tc3_credits_per_period - subport_tc3_credits; | -| | | | -| | | tc3_cons_max = subport_tc3_credits_per_period - (tc0_cons + tc1_cons + | -| | | tc2_cons); | -| | | | -| | | if(tc3_consumption > (tc3_consumption_max - MTU)){ | -| | | | -| | | wm -= wm >> 7; | -| | | | -| | | if(wm < WM_MIN) wm = WM_MIN; | -| | | | -| | | } else { | -| | | | -| | | wm += (wm >> 7) + 1; | -| | | | -| | | if(wm > WM_MAX) wm = WM_MAX; | -| | | | -| | | } | -| | | | -+-----+------------------+----------------------------------------------------------------------------------+ +.. _table_qos_14: + +.. table:: Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period + + +-----+---------------------------------+----------------------------------------------------+ + | No. | Subport Traffic Class Operation | Description | + | | | | + +=====+=================================+====================================================+ + | 1 | Initialization | **Subport level**: subport_period_id= 0 | + | | | | + | | | **Pipe level**: pipe_period_id = 0 | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + | 2 | Credit update | **Subport Level**: | + | | | | + | | | if (time>=subport_tc_time) | + | | | | + | | | { | + | | | subport_wm = water_mark_update(); | + | | | | + | | | subport_tc_time = time + subport_tc_period; | + | | | | + | | | subport_period_id++; | + | | | | + | | | } | + | | | | + | | | **Pipelevel:** | + | | | | + | | | if(pipe_period_id != subport_period_id) | + | | | | + | | | { | + | | | | + | | | pipe_ov_credits = subport_wm \* pipe_weight; | + | | | | + | | | pipe_period_id = subport_period_id; | + | | | | + | | | } | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + | 3 | Credit consumption | **Pipe level:** | + | | (on packet scheduling) | | + | | | pkt_credits = pk_len + frame_overhead; | + | | | | + | | | if(pipe_ov_credits >= pkt_credits{ | + | | | | + | | | pipe_ov_credits -= pkt_credits; | + | | | | + | | | } | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + +.. _table_qos_15: + +.. table:: Watermark Calculation + + +-----+------------------+----------------------------------------------------------------------------------+ + | No. | Subport Traffic | Description | + | | Class Operation | | + +=====+==================+==================================================================================+ + | 1 | Initialization | **Subport level:** | + | | | | + | | | wm = WM_MAX | + | | | | + +-----+------------------+----------------------------------------------------------------------------------+ + | 2 | Credit update | **Subport level (water_mark_update):** | + | | | | + | | | tc0_cons = subport_tc0_credits_per_period - subport_tc0_credits; | + | | | | + | | | tc1_cons = subport_tc1_credits_per_period - subport_tc1_credits; | + | | | | + | | | tc2_cons = subport_tc2_credits_per_period - subport_tc2_credits; | + | | | | + | | | tc3_cons = subport_tc3_credits_per_period - subport_tc3_credits; | + | | | | + | | | tc3_cons_max = subport_tc3_credits_per_period - (tc0_cons + tc1_cons + | + | | | tc2_cons); | + | | | | + | | | if(tc3_consumption > (tc3_consumption_max - MTU)){ | + | | | | + | | | wm -= wm >> 7; | + | | | | + | | | if(wm < WM_MIN) wm = WM_MIN; | + | | | | + | | | } else { | + | | | | + | | | wm += (wm >> 7) + 1; | + | | | | + | | | if(wm > WM_MAX) wm = WM_MAX; | + | | | | + | | | } | + | | | | + +-----+------------------+----------------------------------------------------------------------------------+ Worst Case Scenarios for Performance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1227,26 +1227,26 @@ Configuration A RED configuration contains the parameters given in Table 16. -.. _pg_table_16: - -**Table 16. RED Configuration Parameters** - -+--------------------------+---------+---------+------------------+ -| Parameter | Minimum | Maximum | Typical | -| | | | | -+==========================+=========+=========+==================+ -| Minimum Threshold | 0 | 1022 | 1/4 x queue size | -| | | | | -+--------------------------+---------+---------+------------------+ -| Maximum Threshold | 1 | 1023 | 1/2 x queue size | -| | | | | -+--------------------------+---------+---------+------------------+ -| Inverse Mark Probability | 1 | 255 | 10 | -| | | | | -+--------------------------+---------+---------+------------------+ -| EWMA Filter Weight | 1 | 12 | 9 | -| | | | | -+--------------------------+---------+---------+------------------+ +.. _table_qos_16: + +.. table:: RED Configuration Parameters + + +--------------------------+---------+---------+------------------+ + | Parameter | Minimum | Maximum | Typical | + | | | | | + +==========================+=========+=========+==================+ + | Minimum Threshold | 0 | 1022 | 1/4 x queue size | + | | | | | + +--------------------------+---------+---------+------------------+ + | Maximum Threshold | 1 | 1023 | 1/2 x queue size | + | | | | | + +--------------------------+---------+---------+------------------+ + | Inverse Mark Probability | 1 | 255 | 10 | + | | | | | + +--------------------------+---------+---------+------------------+ + | EWMA Filter Weight | 1 | 12 | 9 | + | | | | | + +--------------------------+---------+---------+------------------+ The meaning of these parameters is explained in more detail in the following sections. The format of these parameters as specified to the dropper module API @@ -1393,33 +1393,33 @@ also achieves accuracy comparable to floating-point evaluation. Table 17 lists the performance of each of these alternative approaches relative to the method that is used in the dropper. As can be seen, the floating-point implementation achieved the worst performance. -.. _pg_table_17: - -**Table 17. Relative Performance of Alternative Approaches** - -+------------------------------------------------------------------------------------+----------------------+ -| Method | Relative Performance | -| | | -+====================================================================================+======================+ -| Current dropper method (see :ref:`Section 23.3.2.1.3 <Dropper>`) | 100% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Fixed-point method with small (512B) look-up table | 148% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| SSE method with small (512B) look-up table | 114% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Large (76KB) look-up table | 118% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Floating-point | 595% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| **Note**: In this case, since performance is expressed as time spent executing the operation in a | -| specific condition, any relative performance value above 100% runs slower than the reference method. | -| | -+-----------------------------------------------------------------------------------------------------------+ +.. _table_qos_17: + +.. table:: Relative Performance of Alternative Approaches + + +------------------------------------------------------------------------------------+----------------------+ + | Method | Relative Performance | + | | | + +====================================================================================+======================+ + | Current dropper method (see :ref:`Section 23.3.2.1.3 <Dropper>`) | 100% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Fixed-point method with small (512B) look-up table | 148% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | SSE method with small (512B) look-up table | 114% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Large (76KB) look-up table | 118% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Floating-point | 595% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | **Note**: In this case, since performance is expressed as time spent executing the operation in a | + | specific condition, any relative performance value above 100% runs slower than the reference method. | + | | + +-----------------------------------------------------------------------------------------------------------+ Drop Decision Block ^^^^^^^^^^^^^^^^^^^ @@ -1585,26 +1585,26 @@ A sample RED configuration is shown below. In this example, the queue size is 64 With this configuration file, the RED configuration that applies to green, yellow and red packets in traffic class 0 is shown in Table 18. -.. _pg_table_18: - -**Table 18. RED Configuration Corresponding to RED Configuration File** - -+--------------------+--------------------+-------+--------+-----+ -| RED Parameter | Configuration Name | Green | Yellow | Red | -| | | | | | -+====================+====================+=======+========+=====+ -| Minimum Threshold | tc 0 wred min | 28 | 22 | 16 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| Maximum Threshold | tc 0 wred max | 32 | 32 | 32 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| Mark Probability | tc 0 wred inv prob | 10 | 10 | 10 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| EWMA Filter Weight | tc 0 wred weight | 9 | 9 | 9 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ +.. _table_qos_18: + +.. table:: RED Configuration Corresponding to RED Configuration File + + +--------------------+--------------------+-------+--------+-----+ + | RED Parameter | Configuration Name | Green | Yellow | Red | + | | | | | | + +====================+====================+=======+========+=====+ + | Minimum Threshold | tc 0 wred min | 28 | 22 | 16 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | Maximum Threshold | tc 0 wred max | 32 | 32 | 32 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | Mark Probability | tc 0 wred inv prob | 10 | 10 | 10 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | EWMA Filter Weight | tc 0 wred weight | 9 | 9 | 9 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ Application Programming Interface (API) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 745a7ac..e1a0c56 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -134,8 +134,8 @@ Sample Applications User Guide **Tables** -:ref:`Table 1.Output Traffic Marking <table_1>` +:numref:`table_qos_metering_1` :ref:`table_qos_metering_1` -:ref:`Table 2.Entity Types <table_2>` +:numref:`table_qos_scheduler_1` :ref:`table_qos_scheduler_1` -:ref:`Table 3.Table Types <table_3>` +:numref:`table_test_pipeline_1` :ref:`table_test_pipeline_1` diff --git a/doc/guides/sample_app_ug/qos_metering.rst b/doc/guides/sample_app_ug/qos_metering.rst index 344faf7..e1a6ac7 100644 --- a/doc/guides/sample_app_ug/qos_metering.rst +++ b/doc/guides/sample_app_ug/qos_metering.rst @@ -140,29 +140,29 @@ The traffic meter parameters are configured in the application source code with Assuming the input traffic is generated at line rate and all packets are 64 bytes Ethernet frames (IPv4 packet size of 46 bytes) and green, the expected output traffic should be marked as shown in the following table: -.. _table_1: - -**Table 1. Output Traffic Marking** - -+-------------+------------------+-------------------+----------------+ -| **Mode** | **Green (Mpps)** | **Yellow (Mpps)** | **Red (Mpps)** | -| | | | | -+=============+==================+===================+================+ -| srTCM blind | 1 | 1 | 12.88 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| srTCM color | 1 | 1 | 12.88 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| trTCM blind | 1 | 0.5 | 13.38 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| trTCM color | 1 | 0.5 | 13.38 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| FWD | 14.88 | 0 | 0 | -| | | | | -+-------------+------------------+-------------------+----------------+ +.. _table_qos_metering_1: + +.. table:: Output Traffic Marking + + +-------------+------------------+-------------------+----------------+ + | **Mode** | **Green (Mpps)** | **Yellow (Mpps)** | **Red (Mpps)** | + | | | | | + +=============+==================+===================+================+ + | srTCM blind | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | srTCM color | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM blind | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM color | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | FWD | 14.88 | 0 | 0 | + | | | | | + +-------------+------------------+-------------------+----------------+ To set up the policing scheme as desired, it is necessary to modify the main.h source file, where this policy is implemented as a static structure, as follows: diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index 66c261c..ffa8ee2 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -321,28 +321,28 @@ The Port/Subport/Pipe/Traffic Class/Queue are the hierarchical entities in a typ The traffic flows that need to be configured are application dependent. This application classifies based on the QinQ double VLAN tags and the IP destination address as indicated in the following table. -.. _table_2: - -**Table 2. Entity Types** - -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | -| | | | | -+================+=========================+==================================================+==================================+ -| Port | - | Ethernet port | Physical port | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ +.. _table_qos_scheduler_1: + +.. table:: Entity Types + + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | + | | | | | + +================+=========================+==================================================+==================================+ + | Port | - | Ethernet port | Physical port | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index fbc290c..872f8d4 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -113,157 +113,157 @@ For hash tables, the following parameters can be selected: * **Table type (e.g. hash-spec-16-ext or hash-spec-16-lru).** The available options are ext (extendible bucket) or lru (least recently used). -.. _table_3: - -**Table 3. Table Types** - -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| **#** | **TABLE_TYPE** | **Description of Core B Table** | **Pre-added Table Entries** | -| | | | | -+=======+========================+==========================================================+=======================================================+ -| 1 | none | Core B is not implementing a DPDK pipeline. | N/A | -| | | Core B is implementing a pass-through from its input set | | -| | | of software queues to its output set of software queues. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 2 | stub | Stub table. Core B is implementing the same pass-through | N/A | -| | | functionality as described for the "none" option by | | -| | | using the DPDK Packet Framework by using one | | -| | | stub table for each input NIC port. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 3 | hash-[spec]-8-lru | LRU hash table with 8-byte key size and 16 million | 16 million entries are successfully added to the | -| | | entries. | hash table with the following key format: | -| | | | | -| | | | [4-byte index, 4 bytes of 0] | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Sendto output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for | -| | | | core B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 4 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 4 | hash-[spec]-8-ext | Extendible bucket hash table with 8-byte key size | Same as hash-[spec]-8-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 5 | hash-[spec]-16-lru | LRU hash table with 16-byte key size and 16 million | 16 million entries are successfully added to the hash | -| | | entries. | table with the following key format: | -| | | | | -| | | | [4-byte index, 12 bytes of 0] | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Send to output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for core | -| | | | B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 12 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 6 | hash-[spec]-16-ext | Extendible bucket hash table with 16-byte key size | Same as hash-[spec]-16-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 7 | hash-[spec]-32-lru | LRU hash table with 32-byte key size and 16 million | 16 million entries are successfully added to the hash | -| | | entries. | table with the following key format: | -| | | | | -| | | | [4-byte index, 28 bytes of 0]. | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Send to output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for | -| | | | Lpmcore B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 28 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 8 | hash-[spec]-32-ext | Extendible bucket hash table with 32-byte key size | Same as hash-[spec]-32-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 9 | lpm | Longest Prefix Match (LPM) IPv4 table. | In the case of two ports, two routes | -| | | | are added to the table: | -| | | | | -| | | | [0.0.0.0/9 => send to output port 0] | -| | | | | -| | | | [0.128.0.0/9 => send to output port 1] | -| | | | | -| | | | In case of four ports, four entries are added to the | -| | | | table: | -| | | | | -| | | | [0.0.0.0/10 => send to output port 0] | -| | | | | -| | | | [0.64.0.0/10 => send to output port 1] | -| | | | | -| | | | [0.128.0.0/10 => send to output port 2] | -| | | | | -| | | | [0.192.0.0/10 => send to output port 3] | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is storing the IPv4 destination | -| | | | within the packet meta data to be later used by core | -| | | | B as the lookup key. | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 10 | acl | Access Control List (ACL) table | In the case of two ports, two ACL rules are added to | -| | | | the table: | -| | | | | -| | | | [priority = 0 (highest), | -| | | | | -| | | | IPv4 source = ANY, | -| | | | | -| | | | IPv4 destination = 0.0.0.0/9, | -| | | | | -| | | | L4 protocol = ANY, | -| | | | | -| | | | TCP source port = ANY, | -| | | | | -| | | | TCP destination port = ANY | -| | | | | -| | | | => send to output port 0] | -| | | | | -| | | | | -| | | | [priority = 0 (highest), | -| | | | | -| | | | IPv4 source = ANY, | -| | | | | -| | | | IPv4 destination = 0.128.0.0/9, | -| | | | | -| | | | L4 protocol = ANY, | -| | | | | -| | | | TCP source port = ANY, | -| | | | | -| | | | TCP destination port = ANY | -| | | | | -| | | | => send to output port 0]. | -| | | | | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ +.. _table_test_pipeline_1: + +.. table:: Table Types + + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | **#** | **TABLE_TYPE** | **Description of Core B Table** | **Pre-added Table Entries** | + | | | | | + +=======+========================+==========================================================+=======================================================+ + | 1 | none | Core B is not implementing a DPDK pipeline. | N/A | + | | | Core B is implementing a pass-through from its input set | | + | | | of software queues to its output set of software queues. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 2 | stub | Stub table. Core B is implementing the same pass-through | N/A | + | | | functionality as described for the "none" option by | | + | | | using the DPDK Packet Framework by using one | | + | | | stub table for each input NIC port. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 3 | hash-[spec]-8-lru | LRU hash table with 8-byte key size and 16 million | 16 million entries are successfully added to the | + | | | entries. | hash table with the following key format: | + | | | | | + | | | | [4-byte index, 4 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Sendto output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | core B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 4 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 4 | hash-[spec]-8-ext | Extendible bucket hash table with 8-byte key size | Same as hash-[spec]-8-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 5 | hash-[spec]-16-lru | LRU hash table with 16-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 12 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for core | + | | | | B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 12 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 6 | hash-[spec]-16-ext | Extendible bucket hash table with 16-byte key size | Same as hash-[spec]-16-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 7 | hash-[spec]-32-lru | LRU hash table with 32-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 28 bytes of 0]. | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | Lpmcore B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 28 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 8 | hash-[spec]-32-ext | Extendible bucket hash table with 32-byte key size | Same as hash-[spec]-32-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 9 | lpm | Longest Prefix Match (LPM) IPv4 table. | In the case of two ports, two routes | + | | | | are added to the table: | + | | | | | + | | | | [0.0.0.0/9 => send to output port 0] | + | | | | | + | | | | [0.128.0.0/9 => send to output port 1] | + | | | | | + | | | | In case of four ports, four entries are added to the | + | | | | table: | + | | | | | + | | | | [0.0.0.0/10 => send to output port 0] | + | | | | | + | | | | [0.64.0.0/10 => send to output port 1] | + | | | | | + | | | | [0.128.0.0/10 => send to output port 2] | + | | | | | + | | | | [0.192.0.0/10 => send to output port 3] | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is storing the IPv4 destination | + | | | | within the packet meta data to be later used by core | + | | | | B as the lookup key. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 10 | acl | Access Control List (ACL) table | In the case of two ports, two ACL rules are added to | + | | | | the table: | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.0.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0] | + | | | | | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.128.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0]. | + | | | | | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ Input Traffic ~~~~~~~~~~~~~ -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 1/2] doc: refactored figure numbers " John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 2/2] doc: refactored table " John McNamara @ 2015-04-29 16:12 ` Thomas Monjalon 2015-04-30 13:34 ` Mcnamara, John 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 0/3] " John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara 4 siblings, 1 reply; 17+ messages in thread From: Thomas Monjalon @ 2015-04-29 16:12 UTC (permalink / raw) To: John McNamara; +Cc: dev Hi John, 2015-04-24 14:11, John McNamara: > This patch adds automatic figure and table references to the docs. The > figure and table numbers in the generated Html and PDF docs can now be > automatically numbered based on section. > > Requires Sphinx >= 1.3. > > Advantages: > > * Figues/tables are numbered automatically and adding/removing items > no longer requires manual fixing. > > Disadvantages: > > * Requires Sphinx >= 1.3. > > * Table captions are above tables while figure captions are below. This is > a current Sphinx/DocUtils limitation. > > * Both the :numref: and :ref: directives are required to link to a figure > or table number and caption. This is really a great work but I think it's not reasonnable to require sphinx 1.3. As almost nobody is using this version, it would be equivalent to prevent users and developers to generate the doc by themselves. It produces this error: ERROR: Unknown interpreted text role "numref". Do you think it's possible to implement a fallback in our conf.py in order to ignore this new role if not supported? ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references 2015-04-29 16:12 ` [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums " Thomas Monjalon @ 2015-04-30 13:34 ` Mcnamara, John 2015-05-13 19:07 ` Thomas Monjalon 0 siblings, 1 reply; 17+ messages in thread From: Mcnamara, John @ 2015-04-30 13:34 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, April 29, 2015 5:13 PM > To: Mcnamara, John > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums > into references > > is really a great work but I think it's not reasonnable to require > sphinx 1.3. As almost nobody is using this version, it would be equivalent > to prevent users and developers to generate the doc by themselves. Hi Thomas, Yes. That is probably right. > It produces this error: > ERROR: Unknown interpreted text role "numref". > > Do you think it's possible to implement a fallback in our conf.py in order > to ignore this new role if not supported? It would be possible but a full implementation probably wouldn't be worth it. We could add a workaround like the following to conf.py that would just render the figure/table ref numbers as the target name as a fallback. That would allow people to generate the docs with older versions of sphinx: $ git diff doc/guides/conf.py diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 1bc031f..bbf40f1 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -82,3 +82,16 @@ class CustomLatexFormatter(LatexFormatter): # Replace the default latex formatter. PygmentsBridge.latex_formatter = CustomLatexFormatter + +from docutils import nodes +from distutils.version import LooseVersion +from sphinx import __version__ as sphinx_version + +# Workaround to ignore :numref: in older versions of Sphinx. +def setup(app): + + if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): + print('[dpdk docs] Upgrade sphinx to version >= 1.3.1 for ' + 'improved Figure/Table number handling.') + app.add_generic_role('numref', nodes.emphasis) + That is just a workaround though, and maybe not worth it either. P.S. Also note, Sphinx 1.3.1 has a nice but very different default Html style. John ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references 2015-04-30 13:34 ` Mcnamara, John @ 2015-05-13 19:07 ` Thomas Monjalon 2015-05-14 10:41 ` Mcnamara, John 2015-05-18 10:48 ` Mcnamara, John 0 siblings, 2 replies; 17+ messages in thread From: Thomas Monjalon @ 2015-05-13 19:07 UTC (permalink / raw) To: Mcnamara, John; +Cc: dev 2015-04-30 13:34, Mcnamara, John: > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > It produces this error: > > ERROR: Unknown interpreted text role "numref". > > > > Do you think it's possible to implement a fallback in our conf.py in order > > to ignore this new role if not supported? > > It would be possible but a full implementation probably wouldn't be worth it. > We could add a workaround like the following to conf.py that would just > render the figure/table ref numbers as the target name as a fallback. > That would allow people to generate the docs with older versions of sphinx: > > +from docutils import nodes > +from distutils.version import LooseVersion > +from sphinx import __version__ as sphinx_version > + > +# Workaround to ignore :numref: in older versions of Sphinx. > +def setup(app): > + > + if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): > + print('[dpdk docs] Upgrade sphinx to version >= 1.3.1 for ' > + 'improved Figure/Table number handling.') > + app.add_generic_role('numref', nodes.emphasis) > > That is just a workaround though, and maybe not worth it either. The error is removed so it's better. With this patch, a figure reference looks like this: :numref:`figure_single_port_nic` Virtualization for a Single Port NIC in SR-IOV Mode The rst line is: :numref:`figure_single_port_nic` :ref:`figure_single_port_nic` I was trying to replace the numref output by a working link with "figure" as label. This is my trial to mimic :ref: as a first step: 8<------------------------ from docutils import nodes from distutils.version import LooseVersion from sphinx import __version__ as sphinx_version from sphinx import addnodes from sphinx.roles import XRefRole class XNumRefNode(addnodes.pending_xref): def __init__(self, rawsource='', *children, **attributes): attributes['reftype'] = 'ref' super(XNumRefNode, self).__init__(rawsource, *children, **attributes) class XNumRefRole(XRefRole): def __init__(self): super(XNumRefRole, self).__init__(nodeclass=XNumRefNode, innernodeclass=nodes.emphasis, warn_dangling=True) def setup(app): if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): print('Upgrade sphinx to version >= 1.3.1 for ' 'improved Figure/Table number handling.') app.add_node(XNumRefNode) app.add_role('numref', XNumRefRole()) 8<------------------------ Unfortunately it gives this error: _pickle.PicklingError: Can't pickle <class 'XNumRefNode'>: attribute lookup XNumRefNode on builtins failed Help of python experts is welcome. References: http://sphinx-doc.org/markup/inline.html#role-numref https://bitbucket.org/arjones6/sphinx-numfig/src/b2345f7d9fabefbd103c31cc0c4c26c68b29ac6a/numfig.py http://code.nabla.net/doc/sphinx/api/sphinx/addnodes/sphinx.addnodes.pending_xref.html ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references 2015-05-13 19:07 ` Thomas Monjalon @ 2015-05-14 10:41 ` Mcnamara, John 2015-05-18 10:48 ` Mcnamara, John 1 sibling, 0 replies; 17+ messages in thread From: Mcnamara, John @ 2015-05-14 10:41 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, May 13, 2015 8:08 PM > To: Mcnamara, John > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums > into references > > > The error is removed so it's better. > With this patch, a figure reference looks like this: > :numref:`figure_single_port_nic` Virtualization for a Single Port NIC in > SR-IOV Mode The rst line is: > :numref:`figure_single_port_nic` :ref:`figure_single_port_nic` > > I was trying to replace the numref output by a working link with "figure" > as label. > This is my trial to mimic :ref: as a first step: Hi Thomas, I'll take a look at it and see if I can get something working. John. -- ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references 2015-05-13 19:07 ` Thomas Monjalon 2015-05-14 10:41 ` Mcnamara, John @ 2015-05-18 10:48 ` Mcnamara, John 1 sibling, 0 replies; 17+ messages in thread From: Mcnamara, John @ 2015-05-18 10:48 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, May 13, 2015 8:08 PM > To: Mcnamara, John > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums > into references > > > Unfortunately it gives this error: > _pickle.PicklingError: Can't pickle <class 'XNumRefNode'>: attribute > lookup XNumRefNode on builtins failed > Hi Thomas, I ran into similar pickle.py issues trying to subclass the node.reference classes within sphinx. I was able to workaround it without a subclass. I'll submit a rebased V2 of the Figure/Table numbering patchset with updates to conf.py. You can review the workaround there. John. -- ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v2 0/3] doc: refactored fig and table nums into references 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara ` (2 preceding siblings ...) 2015-04-29 16:12 ` [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums " Thomas Monjalon @ 2015-05-18 11:15 ` John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH 1/3] doc: refactored figure numbers " John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 3/3] doc: add sphinx numref compatibility workaround John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara 4 siblings, 2 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:15 UTC (permalink / raw) To: dev This patchset adds automatic figure and table references to the docs. The figure and table numbers in the generated Html and PDF docs can now be automatically numbered by the build system. It replaces all hardcoded figure/table numbers and references. The numfig/numref feature requires Sphinx >= 1.3.1. For backward compatibility with older versions workaround handling is added to the sphinx conf.py file in patch 3/3. The workaround replaces the :numref: reference with a "Figure" or "Table" link to the target (for all Sphinx doc types). It doesn't number the figures or tables. This produces reasonable documentation links for users with older versions of sphinx while allowing automatic numbering support for newer versions. Tested with Sphinx 1.2.3 and 1.3.1. John McNamara (3): doc: refactored figure numbers into references doc: refactored table numbers into references doc: add sphinx numref compatibility workaround doc/guides/conf.py | 82 ++ doc/guides/nics/index.rst | 18 +- doc/guides/nics/intel_vf.rst | 37 +- doc/guides/nics/virtio.rst | 18 +- doc/guides/nics/vmxnet3.rst | 18 +- doc/guides/prog_guide/env_abstraction_layer.rst | 8 +- doc/guides/prog_guide/index.rst | 162 +- doc/guides/prog_guide/ivshmem_lib.rst | 8 +- doc/guides/prog_guide/kernel_nic_interface.rst | 40 +- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 +- doc/guides/prog_guide/lpm6_lib.rst | 8 +- doc/guides/prog_guide/lpm_lib.rst | 8 +- doc/guides/prog_guide/malloc_lib.rst | 9 +- doc/guides/prog_guide/mbuf_lib.rst | 20 +- doc/guides/prog_guide/mempool_lib.rst | 32 +- doc/guides/prog_guide/multi_proc_support.rst | 9 +- doc/guides/prog_guide/overview.rst | 9 +- doc/guides/prog_guide/packet_distrib_lib.rst | 15 +- doc/guides/prog_guide/packet_framework.rst | 1275 ++++++++-------- doc/guides/prog_guide/qos_framework.rst | 1543 ++++++++++---------- doc/guides/prog_guide/ring_lib.rst | 159 +- doc/guides/sample_app_ug/dist_app.rst | 20 +- doc/guides/sample_app_ug/exception_path.rst | 8 +- doc/guides/sample_app_ug/index.rst | 64 +- doc/guides/sample_app_ug/intel_quickassist.rst | 11 +- doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +- doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +- .../sample_app_ug/l2_forward_real_virtual.rst | 22 +- .../sample_app_ug/l3_forward_access_ctrl.rst | 21 +- doc/guides/sample_app_ug/load_balancer.rst | 9 +- doc/guides/sample_app_ug/multi_process.rst | 36 +- doc/guides/sample_app_ug/qos_metering.rst | 46 +- doc/guides/sample_app_ug/qos_scheduler.rst | 55 +- doc/guides/sample_app_ug/quota_watermark.rst | 36 +- doc/guides/sample_app_ug/test_pipeline.rst | 313 ++-- doc/guides/sample_app_ug/vhost.rst | 45 +- doc/guides/sample_app_ug/vm_power_management.rst | 18 +- doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +- doc/guides/xen/pkt_switch.rst | 30 +- 39 files changed, 2141 insertions(+), 2157 deletions(-) -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH 1/3] doc: refactored figure numbers into references 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 0/3] " John McNamara @ 2015-05-18 11:15 ` John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 3/3] doc: add sphinx numref compatibility workaround John McNamara 1 sibling, 0 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:15 UTC (permalink / raw) To: dev This change adds automatic figure references to the docs. The figure numbers in the generated Html and PDF docs are now automatically numbered based on section. Requires Sphinx >= 1.3.1. The patch makes the following changes. * Changes image:: tag to figure:: and moves image caption to the figure. * Adds captions to figures that didn't previously have any. * Un-templates the |image-name| substitution definitions into explicit figure:: tags. They weren't used more than once anyway and Sphinx doesn't support them for figure. * Adds a target to each image that didn't previously have one so that they can be cross-referenced. * Renamed existing image target to match the image name for consistency. * Replaces the Figures lists with automatic :numref: :ref: entries to generate automatic numbering and captions. * Replaces "Figure" references with automatic :numref: references. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/conf.py | 2 + doc/guides/nics/index.rst | 18 ++- doc/guides/nics/intel_vf.rst | 37 ++--- doc/guides/nics/virtio.rst | 18 ++- doc/guides/nics/vmxnet3.rst | 18 ++- doc/guides/prog_guide/env_abstraction_layer.rst | 8 +- doc/guides/prog_guide/index.rst | 92 +++++++----- doc/guides/prog_guide/ivshmem_lib.rst | 8 +- doc/guides/prog_guide/kernel_nic_interface.rst | 40 ++--- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 ++++-- doc/guides/prog_guide/lpm6_lib.rst | 8 +- doc/guides/prog_guide/lpm_lib.rst | 8 +- doc/guides/prog_guide/malloc_lib.rst | 9 +- doc/guides/prog_guide/mbuf_lib.rst | 20 +-- doc/guides/prog_guide/mempool_lib.rst | 32 ++-- doc/guides/prog_guide/multi_proc_support.rst | 9 +- doc/guides/prog_guide/overview.rst | 9 +- doc/guides/prog_guide/packet_distrib_lib.rst | 15 +- doc/guides/prog_guide/packet_framework.rst | 81 +++++----- doc/guides/prog_guide/qos_framework.rst | 163 +++++++-------------- doc/guides/prog_guide/ring_lib.rst | 159 +++++++++++--------- doc/guides/sample_app_ug/dist_app.rst | 20 ++- doc/guides/sample_app_ug/exception_path.rst | 8 +- doc/guides/sample_app_ug/index.rst | 58 ++++---- doc/guides/sample_app_ug/intel_quickassist.rst | 11 +- doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +- doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +-- .../sample_app_ug/l2_forward_real_virtual.rst | 22 +-- .../sample_app_ug/l3_forward_access_ctrl.rst | 21 ++- doc/guides/sample_app_ug/load_balancer.rst | 9 +- doc/guides/sample_app_ug/multi_process.rst | 36 ++--- doc/guides/sample_app_ug/qos_scheduler.rst | 9 +- doc/guides/sample_app_ug/quota_watermark.rst | 36 ++--- doc/guides/sample_app_ug/test_pipeline.rst | 9 +- doc/guides/sample_app_ug/vhost.rst | 45 ++---- doc/guides/sample_app_ug/vm_power_management.rst | 18 +-- doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +- doc/guides/xen/pkt_switch.rst | 30 ++-- 38 files changed, 539 insertions(+), 633 deletions(-) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index b1ef323..1bc031f 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -41,6 +41,8 @@ release = version master_doc = 'index' +numfig = True + latex_documents = [ ('index', 'doc.tex', diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index aadbae3..1ee67fa 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -50,14 +50,20 @@ Network Interface Controller Drivers **Figures** -:ref:`Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode <nic_figure_1>` +:numref:`figure_single_port_nic` :ref:`figure_single_port_nic` -:ref:`Figure 2. SR-IOV Performance Benchmark Setup <nic_figure_2>` +:numref:`figure_perf_benchmark` :ref:`figure_perf_benchmark` -:ref:`Figure 3. Fast Host-based Packet Processing <nic_figure_3>` +:numref:`figure_fast_pkt_proc` :ref:`figure_fast_pkt_proc` -:ref:`Figure 4. SR-IOV Inter-VM Communication <nic_figure_4>` +:numref:`figure_inter_vm_comms` :ref:`figure_inter_vm_comms` -:ref:`Figure 5. Virtio Host2VM Communication Example Using KNI vhost Back End <nic_figure_5>` +:numref:`figure_host_vm_comms` :ref:`figure_host_vm_comms` -:ref:`Figure 6. Virtio Host2VM Communication Example Using Qemu vhost Back End <nic_figure_6>` +:numref:`figure_host_vm_comms_qemu` :ref:`figure_host_vm_comms_qemu` + +:numref:`figure_vmxnet3_int` :ref:`figure_vmxnet3_int` + +:numref:`figure_vswitch_vm` :ref:`figure_vswitch_vm` + +:numref:`figure_vm_vm_comms` :ref:`figure_vm_vm_comms` diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst index eeca973..db86c64 100644 --- a/doc/guides/nics/intel_vf.rst +++ b/doc/guides/nics/intel_vf.rst @@ -49,9 +49,9 @@ SR-IOV Mode Utilization in a DPDK Environment The DPDK uses the SR-IOV feature for hardware-based I/O sharing in IOV mode. Therefore, it is possible to partition SR-IOV capability on Ethernet controller NIC resources logically and expose them to a virtual machine as a separate PCI function called a "Virtual Function". -Refer to Figure 10. +Refer to :numref:`figure_single_port_nic`. -Therefore, a NIC is logically distributed among multiple virtual machines (as shown in Figure 10), +Therefore, a NIC is logically distributed among multiple virtual machines (as shown in :numref:`figure_single_port_nic`), while still having global data in common to share with the Physical Function and other Virtual Functions. The DPDK fm10kvf, i40evf, igbvf or ixgbevf as a Poll Mode Driver (PMD) serves for the Intel® 82576 Gigabit Ethernet Controller, Intel® Ethernet Controller I350 family, Intel® 82599 10 Gigabit Ethernet Controller NIC, @@ -72,11 +72,12 @@ For more detail on SR-IOV, please refer to the following documents: * `Scalable I/O Virtualized Servers <http://www.intel.com/content/www/us/en/virtualization/server-virtualization/scalable-i-o-virtualized-servers-paper.html>`_ -.. _nic_figure_1: +.. _figure_single_port_nic: -**Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode** +.. figure:: img/single_port_nic.* + + Virtualization for a Single Port NIC in SR-IOV Mode -.. image:: img/single_port_nic.* Physical and Virtual Function Infrastructure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -548,13 +549,14 @@ The setup procedure is as follows: can also be used to bind and unbind devices to a virtual machine in Ubuntu. If this option is used, step 6 in the instructions provided will be different. - * The Virtual Machine Monitor (see Figure 11) is equivalent to a Host OS with KVM installed as described in the instructions. + * The Virtual Machine Monitor (see :numref:`figure_perf_benchmark`) is equivalent to a Host OS with KVM installed as described in the instructions. + +.. _figure_perf_benchmark: -.. _nic_figure_2: +.. figure:: img/perf_benchmark.* -**Figure 2. Performance Benchmark Setup** + Performance Benchmark Setup -.. image:: img/perf_benchmark.* DPDK SR-IOV PMD PF/VF Driver Usage Model ---------------------------------------- @@ -569,14 +571,15 @@ the DPDK VF PMD driver performs the same throughput result as a non-VT native en With such host instance fast packet processing, lots of services such as filtering, QoS, DPI can be offloaded on the host fast path. -Figure 12 shows the scenario where some VMs directly communicate externally via a VFs, +:numref:`figure_fast_pkt_proc` shows the scenario where some VMs directly communicate externally via a VFs, while others connect to a virtual switch and share the same uplink bandwidth. -.. _nic_figure_3: +.. _figure_fast_pkt_proc: + +.. figure:: img/fast_pkt_proc.* -**Figure 3. Fast Host-based Packet Processing** + Fast Host-based Packet Processing -.. image:: img/fast_pkt_proc.* SR-IOV (PF/VF) Approach for Inter-VM Communication -------------------------------------------------- @@ -587,7 +590,7 @@ So VF-to-VF traffic within the same physical port (VM0<->VM1) have hardware acce However, when VF crosses physical ports (VM0<->VM2), there is no such hardware bridge. In this case, the DPDK PMD PF driver provides host forwarding between such VMs. -Figure 13 shows an example. +:numref:`figure_inter_vm_comms` shows an example. In this case an update of the MAC address lookup tables in both the NIC and host DPDK application is required. In the NIC, writing the destination of a MAC address belongs to another cross device VM to the PF specific pool. @@ -598,8 +601,8 @@ that is, the packet is forwarded to the correct PF pool. The SR-IOV NIC switch forwards the packet to a specific VM according to the MAC destination address which belongs to the destination VF on the VM. -.. _nic_figure_4: +.. _figure_inter_vm_comms: -**Figure 4. Inter-VM Communication** +.. figure:: img/inter_vm_comms.* -.. image:: img/inter_vm_comms.* + Inter-VM Communication diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst index 073d980..9f18b3a 100644 --- a/doc/guides/nics/virtio.rst +++ b/doc/guides/nics/virtio.rst @@ -106,11 +106,12 @@ Virtio with kni vhost Back End This section demonstrates kni vhost back end example setup for Phy-VM Communication. -.. _nic_figure_5: +.. _figure_host_vm_comms: -**Figure 5. Host2VM Communication Example Using kni vhost Back End** +.. figure:: img/host_vm_comms.* + + Host2VM Communication Example Using kni vhost Back End -.. image:: img/host_vm_comms.* Host2VM communication example @@ -174,7 +175,9 @@ Host2VM communication example We use testpmd as the forwarding application in this example. - .. image:: img/console.* + .. figure:: img/console.* + + Running testpmd #. Use IXIA packet generator to inject a packet stream into the KNI physical port. @@ -185,11 +188,12 @@ Host2VM communication example Virtio with qemu virtio Back End -------------------------------- -.. _nic_figure_6: +.. _figure_host_vm_comms_qemu: + +.. figure:: img/host_vm_comms_qemu.* -**Figure 6. Host2VM Communication Example Using qemu vhost Back End** + Host2VM Communication Example Using qemu vhost Back End -.. image:: img/host_vm_comms_qemu.* .. code-block:: console diff --git a/doc/guides/nics/vmxnet3.rst b/doc/guides/nics/vmxnet3.rst index 830530d..e919088 100644 --- a/doc/guides/nics/vmxnet3.rst +++ b/doc/guides/nics/vmxnet3.rst @@ -121,7 +121,11 @@ The following prerequisites apply: * Before starting a VM, a VMXNET3 interface to a VM through VMware vSphere Client must be assigned. This is shown in the figure below. -.. image:: img/vmxnet3_int.* +.. _figure_vmxnet3_int: + +.. figure:: img/vmxnet3_int.* + + Assigning a VMXNET3 interface to a VM using VMware vSphere Client .. note:: @@ -142,7 +146,11 @@ VMXNET3 with a Native NIC Connected to a vSwitch This section describes an example setup for Phy-vSwitch-VM-Phy communication. -.. image:: img/vswitch_vm.* +.. _figure_vswitch_vm: + +.. figure:: img/vswitch_vm.* + + VMXNET3 with a Native NIC Connected to a vSwitch .. note:: @@ -164,7 +172,11 @@ VMXNET3 Chaining VMs Connected to a vSwitch The following figure shows an example VM-to-VM communication over a Phy-VM-vSwitch-VM-Phy communication channel. -.. image:: img/vm_vm_comms.* +.. _figure_vm_vm_comms: + +.. figure:: img/vm_vm_comms.* + + VMXNET3 Chaining VMs Connected to a vSwitch .. note:: diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 4ecbe6a..25eb281 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -85,13 +85,12 @@ A check is also performed at initialization time to ensure that the micro archit Then, the main() function is called. The core initialization and launch is done in rte_eal_init() (see the API documentation). It consist of calls to the pthread library (more specifically, pthread_self(), pthread_create(), and pthread_setaffinity_np()). -.. _pg_figure_2: +.. _figure_linuxapp_launch: -**Figure 2. EAL Initialization in a Linux Application Environment** +.. figure:: img/linuxapp_launch.* -.. image3_png has been replaced + EAL Initialization in a Linux Application Environment -|linuxapp_launch| .. note:: @@ -367,4 +366,3 @@ We expect only 50% of CPU spend on packet IO. echo 50000 > pkt_io/cpu.cfs_quota_us -.. |linuxapp_launch| image:: img/linuxapp_launch.* diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 57d516a..9a1e337 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -80,71 +80,97 @@ Programmer's Guide **Figures** -:ref:`Figure 1. Core Components Architecture <pg_figure_1>` +:numref:`figure_architecture-overview` :ref:`figure_architecture-overview` -:ref:`Figure 2. EAL Initialization in a Linux Application Environment <pg_figure_2>` +:numref:`figure_linuxapp_launch` :ref:`figure_linuxapp_launch` -:ref:`Figure 3. Example of a malloc heap and malloc elements within the malloc library <pg_figure_3>` +:numref:`figure_malloc_heap` :ref:`figure_malloc_heap` -:ref:`Figure 4. Ring Structure <pg_figure_4>` +:numref:`figure_ring1` :ref:`figure_ring1` -:ref:`Figure 5. Two Channels and Quad-ranked DIMM Example <pg_figure_5>` +:numref:`figure_ring-enqueue1` :ref:`figure_ring-enqueue1` -:ref:`Figure 6. Three Channels and Two Dual-ranked DIMM Example <pg_figure_6>` +:numref:`figure_ring-enqueue2` :ref:`figure_ring-enqueue2` -:ref:`Figure 7. A mempool in Memory with its Associated Ring <pg_figure_7>` +:numref:`figure_ring-enqueue3` :ref:`figure_ring-enqueue3` -:ref:`Figure 8. An mbuf with One Segment <pg_figure_8>` +:numref:`figure_ring-dequeue1` :ref:`figure_ring-dequeue1` -:ref:`Figure 9. An mbuf with Three Segments <pg_figure_9>` +:numref:`figure_ring-dequeue2` :ref:`figure_ring-dequeue2` -:ref:`Figure 16. Memory Sharing in the Intel® DPDK Multi-process Sample Application <pg_figure_16>` +:numref:`figure_ring-dequeue3` :ref:`figure_ring-dequeue3` -:ref:`Figure 17. Components of an Intel® DPDK KNI Application <pg_figure_17>` +:numref:`figure_ring-mp-enqueue1` :ref:`figure_ring-mp-enqueue1` -:ref:`Figure 18. Packet Flow via mbufs in the Intel DPDK® KNI <pg_figure_18>` +:numref:`figure_ring-mp-enqueue2` :ref:`figure_ring-mp-enqueue2` -:ref:`Figure 19. vHost-net Architecture Overview <pg_figure_19>` +:numref:`figure_ring-mp-enqueue3` :ref:`figure_ring-mp-enqueue3` -:ref:`Figure 20. KNI Traffic Flow <pg_figure_20>` +:numref:`figure_ring-mp-enqueue4` :ref:`figure_ring-mp-enqueue4` -:ref:`Figure 21. Complex Packet Processing Pipeline with QoS Support <pg_figure_21>` +:numref:`figure_ring-mp-enqueue5` :ref:`figure_ring-mp-enqueue5` -:ref:`Figure 22. Hierarchical Scheduler Block Internal Diagram <pg_figure_22>` +:numref:`figure_ring-modulo1` :ref:`figure_ring-modulo1` -:ref:`Figure 23. Scheduling Hierarchy per Port <pg_figure_23>` +:numref:`figure_ring-modulo2` :ref:`figure_ring-modulo2` -:ref:`Figure 24. Internal Data Structures per Port <pg_figure_24>` +:numref:`figure_memory-management` :ref:`figure_memory-management` -:ref:`Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation <pg_figure_25>` +:numref:`figure_memory-management2` :ref:`figure_memory-management2` -:ref:`Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation <pg_figure_26>` +:numref:`figure_mempool` :ref:`figure_mempool` -:ref:`Figure 27. High-level Block Diagram of the Intel® DPDK Dropper <pg_figure_27>` +:numref:`figure_mbuf1` :ref:`figure_mbuf1` -:ref:`Figure 28. Flow Through the Dropper <pg_figure_28>` +:numref:`figure_mbuf2` :ref:`figure_mbuf2` -:ref:`Figure 29. Example Data Flow Through Dropper <pg_figure_29>` +:numref:`figure_multi_process_memory` :ref:`figure_multi_process_memory` -:ref:`Figure 30. Packet Drop Probability for a Given RED Configuration <pg_figure_30>` +:numref:`figure_kernel_nic_intf` :ref:`figure_kernel_nic_intf` -:ref:`Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) <pg_figure_31>` +:numref:`figure_pkt_flow_kni` :ref:`figure_pkt_flow_kni` -:ref:`Figure 32. Example of packet processing pipeline. The input ports 0 and 1 are connected with the output ports 0, 1 and 2 through tables 0 and 1. <pg_figure_32>` +:numref:`figure_vhost_net_arch2` :ref:`figure_vhost_net_arch2` -:ref:`Figure 33. Sequence of steps for hash table operations in packet processing context <pg_figure_33>` +:numref:`figure_kni_traffic_flow` :ref:`figure_kni_traffic_flow` -:ref:`Figure 34. Data structures for configurable key size hash tables <pg_figure_34>` -:ref:`Figure 35. Bucket search pipeline for key lookup operation (configurable key size hash tables) <pg_figure_35>` +:numref:`figure_pkt_proc_pipeline_qos` :ref:`figure_pkt_proc_pipeline_qos` -:ref:`Figure 36. Pseudo-code for match, match_many and match_pos <pg_figure_36>` +:numref:`figure_hier_sched_blk` :ref:`figure_hier_sched_blk` -:ref:`Figure 37. Data structures for 8-byte key hash tables <pg_figure_37>` +:numref:`figure_sched_hier_per_port` :ref:`figure_sched_hier_per_port` -:ref:`Figure 38. Data structures for 16-byte key hash tables <pg_figure_38>` +:numref:`figure_data_struct_per_port` :ref:`figure_data_struct_per_port` + +:numref:`figure_prefetch_pipeline` :ref:`figure_prefetch_pipeline` + +:numref:`figure_pipe_prefetch_sm` :ref:`figure_pipe_prefetch_sm` + +:numref:`figure_blk_diag_dropper` :ref:`figure_blk_diag_dropper` + +:numref:`figure_flow_tru_droppper` :ref:`figure_flow_tru_droppper` + +:numref:`figure_ex_data_flow_tru_dropper` :ref:`figure_ex_data_flow_tru_dropper` + +:numref:`figure_pkt_drop_probability` :ref:`figure_pkt_drop_probability` + +:numref:`figure_drop_probability_graph` :ref:`figure_drop_probability_graph` + +:numref:`figure_figure32` :ref:`figure_figure32` + +:numref:`figure_figure33` :ref:`figure_figure33` + +:numref:`figure_figure34` :ref:`figure_figure34` + +:numref:`figure_figure35` :ref:`figure_figure35` + +:numref:`figure_figure37` :ref:`figure_figure37` + +:numref:`figure_figure38` :ref:`figure_figure38` + +:numref:`figure_figure39` :ref:`figure_figure39` -:ref:`Figure 39. Bucket search pipeline for key lookup operation (single key size hash tables) <pg_figure_39>` **Tables** diff --git a/doc/guides/prog_guide/ivshmem_lib.rst b/doc/guides/prog_guide/ivshmem_lib.rst index 75175fa..9401ccf 100644 --- a/doc/guides/prog_guide/ivshmem_lib.rst +++ b/doc/guides/prog_guide/ivshmem_lib.rst @@ -43,9 +43,11 @@ they are automatically recognized by the DPDK Environment Abstraction Layer (EAL A typical DPDK IVSHMEM use case looks like the following. -.. image28_png has been renamed -|ivshmem| +.. figure:: img/ivshmem.* + + Typical Ivshmem use case + The same could work with several virtual machines, providing host-to-VM or VM-to-VM communication. The maximum number of metadata files is 32 (by default) and each metadata file can contain different (or even the same) hugepages. @@ -154,5 +156,3 @@ It is important to note that once QEMU is started, it holds on to the hugepages As a result, if the user wishes to shut down or restart the IVSHMEM host application, it is not enough to simply shut the application down. The virtual machine must also be shut down (if not, it will hold onto outdated host data). - -.. |ivshmem| image:: img/ivshmem.* diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index bab376a..fb12d43 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -42,15 +42,14 @@ The benefits of using the DPDK KNI are: * Allows an interface with the kernel network stack. -The components of an application using the DPDK Kernel NIC Interface are shown in Figure 17. +The components of an application using the DPDK Kernel NIC Interface are shown in :numref:`figure_kernel_nic_intf`. -.. _pg_figure_17: +.. _figure_kernel_nic_intf: -**Figure 17. Components of a DPDK KNI Application** +.. figure:: img/kernel_nic_intf.* -.. image43_png has been renamed + Components of a DPDK KNI Application -|kernel_nic_intf| The DPDK KNI Kernel Module -------------------------- @@ -114,15 +113,14 @@ To minimize the amount of DPDK code running in kernel space, the mbuf mempool is The kernel module will be aware of mbufs, but all mbuf allocation and free operations will be handled by the DPDK application only. -Figure 18 shows a typical scenario with packets sent in both directions. +:numref:`figure_pkt_flow_kni` shows a typical scenario with packets sent in both directions. -.. _pg_figure_18: +.. _figure_pkt_flow_kni: -**Figure 18. Packet Flow via mbufs in the DPDK KNI** +.. figure:: img/pkt_flow_kni.* -.. image44_png has been renamed + Packet Flow via mbufs in the DPDK KNI -|pkt_flow_kni| Use Case: Ingress ----------------- @@ -189,13 +187,12 @@ it naturally supports both legacy virtio -net and the DPDK PMD virtio. There is a little penalty that comes from the non-polling mode of vhost. However, it scales throughput well when using KNI in multi-thread mode. -.. _pg_figure_19: +.. _figure_vhost_net_arch2: -**Figure 19. vHost-net Architecture Overview** +.. figure:: img/vhost_net_arch.* -.. image45_png has been renamed + vHost-net Architecture Overview -|vhost_net_arch| Packet Flow ~~~~~~~~~~~ @@ -208,13 +205,12 @@ All the packet copying, irrespective of whether it is on the transmit or receive happens in the context of vhost kthread. Every vhost-net device is exposed to a front end virtio device in the guest. -.. _pg_figure_20: +.. _figure_kni_traffic_flow: -**Figure 20. KNI Traffic Flow** +.. figure:: img/kni_traffic_flow.* -.. image46_png has been renamed + KNI Traffic Flow -|kni_traffic_flow| Sample Usage ~~~~~~~~~~~~ @@ -280,11 +276,3 @@ since the kni-vhost does not yet support those features. Even if the option is turned on, kni-vhost will ignore the information that the header contains. When working with legacy virtio on the guest, it is better to turn off unsupported offload features using ethtool -K. Otherwise, there may be problems such as an incorrect L4 checksum error. - -.. |kni_traffic_flow| image:: img/kni_traffic_flow.* - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |pkt_flow_kni| image:: img/pkt_flow_kni.* - -.. |kernel_nic_intf| image:: img/kernel_nic_intf.* diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index ae9b516..96e554f 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -35,7 +35,10 @@ In addition to Poll Mode Drivers (PMDs) for physical and virtual hardware, DPDK also includes a pure-software library that allows physical PMD's to be bonded together to create a single logical PMD. -|bond-overview| +.. figure:: img/bond-overview.* + + Bonded PMDs + The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups of ``rte_eth_dev`` ports of the same speed and duplex to provide @@ -62,7 +65,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Round-Robin (Mode 0):** -|bond-mode-0| +.. figure:: img/bond-mode-0.* + + Round-Robin (Mode 0) + This mode provides load balancing and fault tolerance by transmission of packets in sequential order from the first available slave device through @@ -72,7 +78,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Active Backup (Mode 1):** -|bond-mode-1| +.. figure:: img/bond-mode-1.* + + Active Backup (Mode 1) + In this mode only one slave in the bond is active at any time, a different slave becomes active if, and only if, the primary active slave fails, @@ -82,7 +91,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Balance XOR (Mode 2):** -|bond-mode-2| +.. figure:: img/bond-mode-2.* + + Balance XOR (Mode 2) + This mode provides transmit load balancing (based on the selected transmission policy) and fault tolerance. The default policy (layer2) uses @@ -101,14 +113,20 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Broadcast (Mode 3):** -|bond-mode-3| +.. figure:: img/bond-mode-3.* + + Broadcast (Mode 3) + This mode provides fault tolerance by transmission of packets on all slave ports. * **Link Aggregation 802.3AD (Mode 4):** -|bond-mode-4| +.. figure:: img/bond-mode-4.* + + Link Aggregation 802.3AD (Mode 4) + This mode provides dynamic link aggregation according to the 802.3ad specification. It negotiates and monitors aggregation groups that share the @@ -128,7 +146,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Transmit Load Balancing (Mode 5):** -|bond-mode-5| +.. figure:: img/bond-mode-5.* + + Transmit Load Balancing (Mode 5) + This mode provides an adaptive transmit load balancing. It dynamically changes the transmitting slave, according to the computed load. Statistics @@ -433,11 +454,3 @@ Create a bonded device in balance mode with two slaves specified by their PCI ad .. code-block:: console $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=2, slave=0000:00a:00.01,slave=0000:004:00.00,xmit_policy=l34' -- --port-topology=chained - -.. |bond-overview| image:: img/bond-overview.* -.. |bond-mode-0| image:: img/bond-mode-0.* -.. |bond-mode-1| image:: img/bond-mode-1.* -.. |bond-mode-2| image:: img/bond-mode-2.* -.. |bond-mode-3| image:: img/bond-mode-3.* -.. |bond-mode-4| image:: img/bond-mode-4.* -.. |bond-mode-5| image:: img/bond-mode-5.* diff --git a/doc/guides/prog_guide/lpm6_lib.rst b/doc/guides/prog_guide/lpm6_lib.rst index abc5adb..87f5066 100644 --- a/doc/guides/prog_guide/lpm6_lib.rst +++ b/doc/guides/prog_guide/lpm6_lib.rst @@ -108,9 +108,11 @@ This is not feasible due to resource restrictions. By splitting the process in different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access per level). -.. image40_png has been renamed -|tbl24_tbl8_tbl8| +.. figure:: img/tbl24_tbl8_tbl8.* + + Table split into different levels + An entry in a table contains the following fields: @@ -231,5 +233,3 @@ Use Case: IPv6 Forwarding ------------------------- The LPM algorithm is used to implement the Classless Inter-Domain Routing (CIDR) strategy used by routers implementing IP forwarding. - -.. |tbl24_tbl8_tbl8| image:: img/tbl24_tbl8_tbl8.* diff --git a/doc/guides/prog_guide/lpm_lib.rst b/doc/guides/prog_guide/lpm_lib.rst index 692e37f..c33e469 100644 --- a/doc/guides/prog_guide/lpm_lib.rst +++ b/doc/guides/prog_guide/lpm_lib.rst @@ -90,9 +90,11 @@ Instead, this approach takes advantage of the fact that rules longer than 24 bit By splitting the process in two different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access, most of the times). -.. image39 has been renamed -|tbl24_tbl8| +.. figure:: img/tbl24_tbl8.* + + Table split into different levels + An entry in tbl24 contains the following fields: @@ -219,5 +221,3 @@ References * Pankaj Gupta, Algorithms for Routing Lookups and Packet Classification, PhD Thesis, Stanford University, 2000 (`http://klamath.stanford.edu/~pankaj/thesis/ thesis_1sided.pdf <http://klamath.stanford.edu/~pankaj/thesis/%20thesis_1sided.pdf>`_ ) - -.. |tbl24_tbl8| image:: img/tbl24_tbl8.* diff --git a/doc/guides/prog_guide/malloc_lib.rst b/doc/guides/prog_guide/malloc_lib.rst index b9298f8..6418fab 100644 --- a/doc/guides/prog_guide/malloc_lib.rst +++ b/doc/guides/prog_guide/malloc_lib.rst @@ -117,13 +117,12 @@ The key fields of the heap structure and their function are described below (see since these are never touched except when they are to be freed again - at which point the pointer to the block is an input to the free() function. -.. _pg_figure_3: +.. _figure_malloc_heap: -**Figure 3. Example of a malloc heap and malloc elements within the malloc library** +.. figure:: img/malloc_heap.* -.. image4_png has been renamed + Example of a malloc heap and malloc elements within the malloc library -|malloc_heap| Structure: malloc_elem ^^^^^^^^^^^^^^^^^^^^^^ @@ -232,5 +231,3 @@ These next and previous elements are then checked to see if they too are free, and if so, they are merged with the current elements. This means that we can never have two free memory blocks adjacent to one another, they are always merged into a single block. - -.. |malloc_heap| image:: img/malloc_heap.* diff --git a/doc/guides/prog_guide/mbuf_lib.rst b/doc/guides/prog_guide/mbuf_lib.rst index 338f7da..32a041e 100644 --- a/doc/guides/prog_guide/mbuf_lib.rst +++ b/doc/guides/prog_guide/mbuf_lib.rst @@ -71,23 +71,21 @@ Message buffers may be used to carry control information, packets, events, and so on between different entities in the system. Message buffers may also use their buffer pointers to point to other message buffer data sections or other structures. -Figure 8 and Figure 9 show some of these scenarios. +:numref:`figure_mbuf1` and :numref:`figure_mbuf2` show some of these scenarios. -.. _pg_figure_8: +.. _figure_mbuf1: -**Figure 8. An mbuf with One Segment** +.. figure:: img/mbuf1.* -.. image22_png has been replaced + An mbuf with One Segment -|mbuf1| -.. _pg_figure_9: +.. _figure_mbuf2: -**Figure 9. An mbuf with Three Segments** +.. figure:: img/mbuf2.* -.. image23_png has been replaced + An mbuf with Three Segments -|mbuf2| The Buffer Manager implements a fairly standard set of buffer access functions to manipulate network packets. @@ -277,7 +275,3 @@ Use Cases --------- All networking application should use mbufs to transport network packets. - -.. |mbuf1| image:: img/mbuf1.* - -.. |mbuf2| image:: img/mbuf2.* diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f9b7cfe..f0ca06f 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -74,28 +74,27 @@ When running an application, the EAL command line options provide the ability to The command line must always have the number of memory channels specified for the processor. -Examples of alignment for different DIMM architectures are shown in Figure 5 and Figure 6. +Examples of alignment for different DIMM architectures are shown in +:numref:`figure_memory-management` and :numref:`figure_memory-management2`. -.. _pg_figure_5: +.. _figure_memory-management: -**Figure 5. Two Channels and Quad-ranked DIMM Example** +.. figure:: img/memory-management.* -.. image19_png has been replaced + Two Channels and Quad-ranked DIMM Example -|memory-management| In this case, the assumption is that a packet is 16 blocks of 64 bytes, which is not true. The Intel® 5520 chipset has three channels, so in most cases, no padding is required between objects (except for objects whose size are n x 3 x 64 bytes blocks). -.. _pg_figure_6: +.. _figure_memory-management2: -**Figure 6. Three Channels and Two Dual-ranked DIMM Example** +.. figure:: img/memory-management2.* -.. image20_png has been replaced + Three Channels and Two Dual-ranked DIMM Example -|memory-management2| When creating a new pool, the user can specify to use this feature or not. @@ -119,15 +118,14 @@ This cache can be enabled or disabled at creation of the pool. The maximum size of the cache is static and is defined at compilation time (CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE). -Figure 7 shows a cache in operation. +:numref:`figure_mempool` shows a cache in operation. -.. _pg_figure_7: +.. _figure_mempool: -**Figure 7. A mempool in Memory with its Associated Ring** +.. figure:: img/mempool.* -.. image21_png has been replaced + A mempool in Memory with its Associated Ring -|mempool| Use Cases --------- @@ -140,9 +138,3 @@ Below are some examples: * :ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` , for logging service * Any application that needs to allocate fixed-sized objects in the data plane and that will be continuously utilized by the system. - -.. |memory-management| image:: img/memory-management.* - -.. |memory-management2| image:: img/memory-management2.* - -.. |mempool| image:: img/mempool.* diff --git a/doc/guides/prog_guide/multi_proc_support.rst b/doc/guides/prog_guide/multi_proc_support.rst index 25a6056..6562f0d 100644 --- a/doc/guides/prog_guide/multi_proc_support.rst +++ b/doc/guides/prog_guide/multi_proc_support.rst @@ -83,13 +83,12 @@ and point to the same objects, in both processes. Refer to Section 23.3 "Multi-process Limitations" for details of how Linux kernel Address-Space Layout Randomization (ASLR) can affect memory sharing. -.. _pg_figure_16: +.. _figure_multi_process_memory: -**Figure 16. Memory Sharing in the DPDK Multi-process Sample Application** +.. figure:: img/multi_process_memory.* -.. image42_png has been replaced + Memory Sharing in the DPDK Multi-process Sample Application -|multi_process_memory| The EAL also supports an auto-detection mode (set by EAL --proc-type=auto flag ), whereby an DPDK process is started as a secondary instance if a primary instance is already running. @@ -199,5 +198,3 @@ instead of the functions which do the hashing internally, such as rte_hash_add() which means that only the first, primary DPDK process instance can open and mmap /dev/hpet. If the number of required DPDK processes exceeds that of the number of available HPET comparators, the TSC (which is the default timer in this release) must be used as a time source across all processes instead of the HPET. - -.. |multi_process_memory| image:: img/multi_process_memory.* diff --git a/doc/guides/prog_guide/overview.rst b/doc/guides/prog_guide/overview.rst index 062d923..cef6ca7 100644 --- a/doc/guides/prog_guide/overview.rst +++ b/doc/guides/prog_guide/overview.rst @@ -120,13 +120,12 @@ Core Components The *core components* are a set of libraries that provide all the elements needed for high-performance packet processing applications. -.. _pg_figure_1: +.. _figure_architecture-overview: -**Figure 1. Core Components Architecture** +.. figure:: img/architecture-overview.* -.. image2_png has been replaced + Core Components Architecture -|architecture-overview| Memory Manager (librte_malloc) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -203,5 +202,3 @@ librte_net The librte_net library is a collection of IP protocol definitions and convenience macros. It is based on code from the FreeBSD* IP stack and contains protocol numbers (for use in IP headers), IP-related macros, IPv4/IPv6 header structures and TCP, UDP and SCTP header structures. - -.. |architecture-overview| image:: img/architecture-overview.* diff --git a/doc/guides/prog_guide/packet_distrib_lib.rst b/doc/guides/prog_guide/packet_distrib_lib.rst index 767accc..b5bdabb 100644 --- a/doc/guides/prog_guide/packet_distrib_lib.rst +++ b/doc/guides/prog_guide/packet_distrib_lib.rst @@ -38,7 +38,10 @@ which is responsible for load balancing or distributing packets, and a set of worker lcores which are responsible for receiving the packets from the distributor and operating on them. The model of operation is shown in the diagram below. -|packet_distributor1| +.. figure:: img/packet_distributor1.* + + Packet Distributor mode of operation + Distributor Core Operation -------------------------- @@ -91,9 +94,11 @@ No packet ordering guarantees are made about packets which do not share a common Using the process and returned_pkts API, the following application workflow can be used, while allowing packet order within a packet flow -- identified by a tag -- to be maintained. -.. image41_png has been renamed -|packet_distributor2| +.. figure:: img/packet_distributor2.* + + Application workflow + The flush and clear_returns API calls, mentioned previously, are likely of less use that the process and returned_pkts APIS, and are principally provided to aid in unit testing of the library. @@ -110,7 +115,3 @@ Since it may be desirable to vary the number of worker cores, depending on the t i.e. to save power at times of lighter load, it is possible to have a worker stop processing packets by calling "rte_distributor_return_pkt()" to indicate that it has finished the current packet and does not want a new one. - -.. |packet_distributor1| image:: img/packet_distributor1.* - -.. |packet_distributor2| image:: img/packet_distributor2.* diff --git a/doc/guides/prog_guide/packet_framework.rst b/doc/guides/prog_guide/packet_framework.rst index 2056c4f..417f4db 100644 --- a/doc/guides/prog_guide/packet_framework.rst +++ b/doc/guides/prog_guide/packet_framework.rst @@ -66,15 +66,15 @@ one of the table entries (on lookup hit) or the default table entry (on lookup m provides the set of actions to be applied on the current packet, as well as the next hop for the packet, which can be either another table, an output port or packet drop. -An example of packet processing pipeline is presented in Figure 32: +An example of packet processing pipeline is presented in :numref:`figure_figure32`: -.. _pg_figure_32: +.. _figure_figure32: -**Figure 32 Example of Packet Processing Pipeline where Input Ports 0 and 1 are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1** +.. figure:: img/figure32.* -.. Object_1_png has been renamed + Example of Packet Processing Pipeline where Input Ports 0 and 1 + are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1 -|figure32| Port Library Design ------------------- @@ -344,13 +344,14 @@ considering *n_bits* as the number of bits set in *bucket_mask = n_buckets - 1*, this means that all the keys that end up in the same hash table bucket have the lower *n_bits* of their signature identical. In order to reduce the number of keys in the same bucket (collisions), the number of hash table buckets needs to be increased. -In packet processing context, the sequence of operations involved in hash table operations is described in Figure 33: +In packet processing context, the sequence of operations involved in hash table operations is described in :numref:`figure_figure33`: -.. _pg_figure_33: +.. _figure_figure33: -**Figure 33 Sequence of Steps for Hash Table Operations in a Packet Processing Context** +.. figure:: img/figure33.* + + Sequence of Steps for Hash Table Operations in a Packet Processing Context -|figure33| Hash Table Use Cases @@ -553,16 +554,15 @@ This avoids the important cost associated with flushing the CPU core execution p Configurable Key Size Hash Table """""""""""""""""""""""""""""""" -Figure 34, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, +:numref:`figure_figure34`, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_34: +.. _figure_figure34: -**Figure 34 Data Structures for Configurable Key Size Hash Tables** +.. figure:: img/figure34.* -.. image65_png has been renamed + Data Structures for Configurable Key Size Hash Tables -|figure34| .. _pg_table_25: @@ -627,15 +627,17 @@ either with pre-computed signature or "do-sig"). +---+------------------+--------------------+------------------------------------------------------------------+ -Figure 35 and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, +:numref:`figure_figure35` and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_35: +.. _figure_figure35: + +.. figure:: img/figure35.* -**Figure 35 Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash Tables)** + Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash + Tables) -|figure35| .. _pg_table_27: @@ -814,11 +816,8 @@ Given the input *mask*, the values for *match*, *match_many* and *match_pos* can | | | | +------------+------------------------------------------+-------------------+ -The pseudo-code is displayed in Figure 36. - -.. _pg_figure_36: -**Figure 36 Pseudo-code for match, match_many and match_pos** +The pseudo-code for match, match_many and match_pos is:: match = (0xFFFELLU >> mask) & 1; @@ -829,24 +828,22 @@ The pseudo-code is displayed in Figure 36. Single Key Size Hash Tables """"""""""""""""""""""""""" -Figure 37, Figure 38, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables +:numref:`figure_figure37`, :numref:`figure_figure38`, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_37: +.. _figure_figure37: -**Figure 37 Data Structures for 8-byte Key Hash Tables** +.. figure:: img/figure37.* -.. image66_png has been renamed + Data Structures for 8-byte Key Hash Tables -|figure37| -.. _pg_figure_38: +.. _figure_figure38: -**Figure 38 Data Structures for 16-byte Key Hash Tables** +.. figure:: img/figure38.* -.. image67_png has been renamed + Data Structures for 16-byte Key Hash Tables -|figure38| .. _pg_table_30: @@ -914,11 +911,13 @@ and detail the bucket search pipeline used to implement 8-byte and 16-byte key h either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_39: +.. _figure_figure39: -**Figure 39 Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash Tables)** +.. figure:: img/figure39.* + + Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash + Tables) -|figure39| .. _pg_table_32: @@ -1167,17 +1166,3 @@ Usually, to support a specific functional block, specific implementation of Pack with all the implementations sharing the same API: pure SW implementation (no acceleration), implementation using accelerator A, implementation using accelerator B, etc. The selection between these implementations could be done at build time or at run-time (recommended), based on which accelerators are present in the system, with no application changes required. - -.. |figure33| image:: img/figure33.* - -.. |figure35| image:: img/figure35.* - -.. |figure39| image:: img/figure39.* - -.. |figure34| image:: img/figure34.* - -.. |figure32| image:: img/figure32.* - -.. |figure37| image:: img/figure37.* - -.. |figure38| image:: img/figure38.* diff --git a/doc/guides/prog_guide/qos_framework.rst b/doc/guides/prog_guide/qos_framework.rst index 98d8714..72cfed9 100644 --- a/doc/guides/prog_guide/qos_framework.rst +++ b/doc/guides/prog_guide/qos_framework.rst @@ -38,13 +38,12 @@ Packet Pipeline with QoS Support An example of a complex packet processing pipeline with QoS support is shown in the following figure. -.. _pg_figure_21: +.. _figure_pkt_proc_pipeline_qos: -**Figure 21. Complex Packet Processing Pipeline with QoS Support** +.. figure:: img/pkt_proc_pipeline_qos.* -.. image47_png has been renamed + Complex Packet Processing Pipeline with QoS Support -|pkt_proc_pipeline_qos| This pipeline can be built using reusable DPDK software libraries. The main blocks implementing QoS in this pipeline are: the policer, the dropper and the scheduler. @@ -139,13 +138,12 @@ It typically acts like a buffer that is able to temporarily store a large number as the NIC TX is requesting more packets for transmission, these packets are later on removed and handed over to the NIC TX with the packet selection logic observing the predefined SLAs (dequeue operation). -.. _pg_figure_22: +.. _figure_hier_sched_blk: -**Figure 22. Hierarchical Scheduler Block Internal Diagram** +.. figure:: img/hier_sched_blk.* -.. image48_png has been renamed + Hierarchical Scheduler Block Internal Diagram -|hier_sched_blk| The hierarchical scheduler is optimized for a large number of packet queues. When only a small number of queues are needed, message passing queues should be used instead of this block. @@ -154,7 +152,7 @@ See Section 26.2.5 "Worst Case Scenarios for Performance" for a more detailed di Scheduling Hierarchy ~~~~~~~~~~~~~~~~~~~~ -The scheduling hierarchy is shown in Figure 23. +The scheduling hierarchy is shown in :numref:`figure_sched_hier_per_port`. The first level of the hierarchy is the Ethernet TX port 1/10/40 GbE, with subsequent hierarchy levels defined as subport, pipe, traffic class and queue. @@ -163,13 +161,12 @@ Each traffic class is the representation of a different traffic type with specif delay and jitter requirements, such as voice, video or data transfers. Each queue hosts packets from one or multiple connections of the same type belonging to the same user. -.. _pg_figure_23: +.. _figure_sched_hier_per_port: -**Figure 23. Scheduling Hierarchy per Port** +.. figure:: img/sched_hier_per_port.* -.. image49_png has been renamed + Scheduling Hierarchy per Port -|sched_hier_per_port| The functionality of each hierarchical level is detailed in the following table. @@ -293,13 +290,12 @@ Internal Data Structures per Port A schematic of the internal data structures in shown in with details in. -.. _pg_figure_24: +.. _figure_data_struct_per_port: -**Figure 24. Internal Data Structures per Port** +.. figure:: img/data_struct_per_port.* -.. image50_png has been renamed + Internal Data Structures per Port -|data_struct_per_port| .. _pg_table_4: @@ -434,16 +430,15 @@ the processor should not attempt to access the data structure currently under pr The only other work available is to execute different stages of the enqueue sequence of operations on other input packets, thus resulting in a pipelined implementation for the enqueue operation. -Figure 25 illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. +:numref:`figure_prefetch_pipeline` illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. No input packet can be part of more than one pipeline stage at a given time. -.. _pg_figure_25: +.. _figure_prefetch_pipeline: -**Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation** +.. figure:: img/prefetch_pipeline.* -.. image51 has been renamed + Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation -|prefetch_pipeline| The congestion management scheme implemented by the enqueue pipeline described above is very basic: packets are enqueued until a specific queue becomes full, @@ -478,13 +473,13 @@ The dequeue pipe state machine exploits the data presence into the processor cac therefore it tries to send as many packets from the same pipe TC and pipe as possible (up to the available packets and credits) before moving to the next active TC from the same pipe (if any) or to another active pipe. -.. _pg_figure_26: +.. _figure_pipe_prefetch_sm: -**Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation** +.. figure:: img/pipe_prefetch_sm.* -.. image52 has been renamed + Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue + Operation -|pipe_prefetch_sm| Timing and Synchronization ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1173,17 +1168,16 @@ Dropper The purpose of the DPDK dropper is to drop packets arriving at a packet scheduler to avoid congestion. The dropper supports the Random Early Detection (RED), Weighted Random Early Detection (WRED) and tail drop algorithms. -Figure 1 illustrates how the dropper integrates with the scheduler. +:numref:`figure_blk_diag_dropper` illustrates how the dropper integrates with the scheduler. The DPDK currently does not support congestion management so the dropper provides the only method for congestion avoidance. -.. _pg_figure_27: +.. _figure_blk_diag_dropper: -**Figure 27. High-level Block Diagram of the DPDK Dropper** +.. figure:: img/blk_diag_dropper.* -.. image53_png has been renamed + High-level Block Diagram of the DPDK Dropper -|blk_diag_dropper| The dropper uses the Random Early Detection (RED) congestion avoidance algorithm as documented in the reference publication. The purpose of the RED algorithm is to monitor a packet queue, @@ -1202,16 +1196,15 @@ In the case of severe congestion, the dropper resorts to tail drop. This occurs when a packet queue has reached maximum capacity and cannot store any more packets. In this situation, all arriving packets are dropped. -The flow through the dropper is illustrated in Figure 28. +The flow through the dropper is illustrated in :numref:`figure_flow_tru_droppper`. The RED/WRED algorithm is exercised first and tail drop second. -.. _pg_figure_28: +.. _figure_flow_tru_droppper: -**Figure 28. Flow Through the Dropper** +.. figure:: img/flow_tru_droppper.* -.. image54_png has been renamed + Flow Through the Dropper -|flow_tru_droppper| The use cases supported by the dropper are: @@ -1270,17 +1263,16 @@ for example, a filter weight parameter value of 9 corresponds to a filter weight Enqueue Operation ~~~~~~~~~~~~~~~~~ -In the example shown in Figure 29, q (actual queue size) is the input value, +In the example shown in :numref:`figure_ex_data_flow_tru_dropper`, q (actual queue size) is the input value, avg (average queue size) and count (number of packets since the last drop) are run-time values, decision is the output value and the remaining values are configuration parameters. -.. _pg_figure_29: +.. _figure_ex_data_flow_tru_dropper: -**Figure 29. Example Data Flow Through Dropper** +.. figure:: img/ex_data_flow_tru_dropper.* -.. image55_png has been renamed + Example Data Flow Through Dropper -|ex_data_flow_tru_dropper| EWMA Filter Microblock ^^^^^^^^^^^^^^^^^^^^^^ @@ -1298,11 +1290,7 @@ Average Queue Size Calculation when the Queue is not Empty The definition of the EWMA filter is given in the following equation. -**Equation 1.** - -.. image56_png has been renamed - -|ewma_filter_eq_1| +.. image:: img/ewma_filter_eq_1.* Where: @@ -1326,11 +1314,7 @@ When the queue becomes empty, average queue size should decay gradually to zero or remaining stagnant at the last computed value. When a packet is enqueued on an empty queue, the average queue size is computed using the following formula: -**Equation 2.** - -.. image57_png has been renamed - -|ewma_filter_eq_2| +.. image:: img/ewma_filter_eq_2.* Where: @@ -1338,9 +1322,7 @@ Where: In the dropper module, *m* is defined as: -.. image58_png has been renamed - -|m_definition| +.. image:: img/m_definition.* Where: @@ -1374,15 +1356,13 @@ A numerical method is used to compute the factor (1-wq)^m that appears in Equati This method is based on the following identity: -.. image59_png has been renamed +.. image:: img/eq2_factor.* -|eq2_factor| This allows us to express the following: -.. image60_png has been renamed +.. image:: img/eq2_expression.* -|eq2_expression| In the dropper module, a look-up table is used to compute log2(1-wq) for each value of wq supported by the dropper module. The factor (1-wq)^m can then be obtained by multiplying the table value by *m* and applying shift operations. @@ -1465,11 +1445,7 @@ Initial Packet Drop Probability The initial drop probability is calculated using the following equation. -**Equation 3.** - -.. image61_png has been renamed - -|drop_probability_eq3| +.. image:: img/drop_probability_eq3.* Where: @@ -1481,19 +1457,18 @@ Where: * *maxth* = maximum threshold -The calculation of the packet drop probability using Equation 3 is illustrated in Figure 30. +The calculation of the packet drop probability using Equation 3 is illustrated in :numref:`figure_pkt_drop_probability`. If the average queue size is below the minimum threshold, an arriving packet is enqueued. If the average queue size is at or above the maximum threshold, an arriving packet is dropped. If the average queue size is between the minimum and maximum thresholds, a drop probability is calculated to determine if the packet should be enqueued or dropped. -.. _pg_figure_30: +.. _figure_pkt_drop_probability: -**Figure 30. Packet Drop Probability for a Given RED Configuration** +.. figure:: img/pkt_drop_probability.* -.. image62_png has been renamed + Packet Drop Probability for a Given RED Configuration -|pkt_drop_probability| Actual Drop Probability """"""""""""""""""""""" @@ -1501,11 +1476,7 @@ Actual Drop Probability If the average queue size is between the minimum and maximum thresholds, then the actual drop probability is calculated from the following equation. -**Equation 4.** - -.. image63_png has been renamed - -|drop_probability_eq4| +.. image:: img/drop_probability_eq4.* Where: @@ -1518,7 +1489,7 @@ given in the reference document where a value of 1 is used instead. It should be noted that the value pa computed from can be negative or greater than 1. If this is the case, then a value of 1 should be used instead. -The initial and actual drop probabilities are shown in Figure 31. +The initial and actual drop probabilities are shown in :numref:`figure_drop_probability_graph`. The actual drop probability is shown for the case where the formula given in the reference document1 is used (blue curve) and also for the case where the formula implemented in the dropper module, @@ -1528,13 +1499,13 @@ compared to the mark probability configuration parameter specified by the user. The choice to deviate from the reference document is simply a design decision and one that has been taken by other RED implementations, for example, FreeBSD* ALTQ RED. -.. _pg_figure_31: +.. _figure_drop_probability_graph: -**Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve)** +.. figure:: img/drop_probability_graph.* -.. image64_png has been renamed + Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using + a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) -|drop_probability_graph| .. _Queue_Empty_Operation: @@ -1727,39 +1698,3 @@ For each input packet, the steps for the srTCM / trTCM algorithms are: the input color of the packet is also considered. When the output color is not red, a number of tokens equal to the length of the IP packet are subtracted from the C or E /P or both buckets, depending on the algorithm and the output color of the packet. - -.. |flow_tru_droppper| image:: img/flow_tru_droppper.* - -.. |drop_probability_graph| image:: img/drop_probability_graph.* - -.. |drop_probability_eq3| image:: img/drop_probability_eq3.* - -.. |eq2_expression| image:: img/eq2_expression.* - -.. |drop_probability_eq4| image:: img/drop_probability_eq4.* - -.. |pkt_drop_probability| image:: img/pkt_drop_probability.* - -.. |pkt_proc_pipeline_qos| image:: img/pkt_proc_pipeline_qos.* - -.. |ex_data_flow_tru_dropper| image:: img/ex_data_flow_tru_dropper.* - -.. |ewma_filter_eq_1| image:: img/ewma_filter_eq_1.* - -.. |ewma_filter_eq_2| image:: img/ewma_filter_eq_2.* - -.. |data_struct_per_port| image:: img/data_struct_per_port.* - -.. |prefetch_pipeline| image:: img/prefetch_pipeline.* - -.. |pipe_prefetch_sm| image:: img/pipe_prefetch_sm.* - -.. |blk_diag_dropper| image:: img/blk_diag_dropper.* - -.. |m_definition| image:: img/m_definition.* - -.. |eq2_factor| image:: img/eq2_factor.* - -.. |sched_hier_per_port| image:: img/sched_hier_per_port.* - -.. |hier_sched_blk| image:: img/hier_sched_blk.* diff --git a/doc/guides/prog_guide/ring_lib.rst b/doc/guides/prog_guide/ring_lib.rst index 8547b38..3b92a8f 100644 --- a/doc/guides/prog_guide/ring_lib.rst +++ b/doc/guides/prog_guide/ring_lib.rst @@ -72,13 +72,12 @@ The disadvantages: A simplified representation of a Ring is shown in with consumer and producer head and tail pointers to objects stored in the data structure. -.. _pg_figure_4: +.. _figure_ring1: -**Figure 4. Ring Structure** +.. figure:: img/ring1.* -.. image5_png has been replaced + Ring Structure -|ring1| References for Ring Implementation in FreeBSD* ---------------------------------------------- @@ -155,9 +154,13 @@ The prod_next local variable points to the next element of the table, or several If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image6_png has been replaced -|ring-enqueue1| +.. _figure_ring-enqueue1: + +.. figure:: img/ring-enqueue1.* + + Enqueue first step + Enqueue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -166,9 +169,13 @@ The second step is to modify *ring->prod_head* in ring structure to point to the A pointer to the added object is copied in the ring (obj4). -.. image7_png has been replaced -|ring-enqueue2| +.. _figure_ring-enqueue2: + +.. figure:: img/ring-enqueue2.* + + Enqueue second step + Enqueue Last Step ^^^^^^^^^^^^^^^^^ @@ -176,9 +183,13 @@ Enqueue Last Step Once the object is added in the ring, ring->prod_tail in the ring structure is modified to point to the same location as *ring->prod_head*. The enqueue operation is finished. -.. image8_png has been replaced -|ring-enqueue3| +.. _figure_ring-enqueue3: + +.. figure:: img/ring-enqueue3.* + + Enqueue last step + Single Consumer Dequeue ~~~~~~~~~~~~~~~~~~~~~~~ @@ -196,9 +207,13 @@ The cons_next local variable points to the next element of the table, or several If there are not enough objects in the ring (this is detected by checking prod_tail), it returns an error. -.. image9_png has been replaced -|ring-dequeue1| +.. _figure_ring-dequeue1: + +.. figure:: img/ring-dequeue1.* + + Dequeue last step + Dequeue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -207,9 +222,13 @@ The second step is to modify ring->cons_head in the ring structure to point to t The pointer to the dequeued object (obj1) is copied in the pointer given by the user. -.. image10_png has been replaced -|ring-dequeue2| +.. _figure_ring-dequeue2: + +.. figure:: img/ring-dequeue2.* + + Dequeue second step + Dequeue Last Step ^^^^^^^^^^^^^^^^^ @@ -217,9 +236,13 @@ Dequeue Last Step Finally, ring->cons_tail in the ring structure is modified to point to the same location as ring->cons_head. The dequeue operation is finished. -.. image11_png has been replaced -|ring-dequeue3| +.. _figure_ring-dequeue3: + +.. figure:: img/ring-dequeue3.* + + Dequeue last step + Multiple Producers Enqueue ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -229,8 +252,8 @@ In this example, only the producer head and tail (prod_head and prod_tail) are m The initial state is to have a prod_head and prod_tail pointing at the same location. -MC Enqueue First Step -^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue First Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On both cores, *ring->prod_head* and ring->cons_tail are copied in local variables. The prod_next local variable points to the next element of the table, @@ -238,12 +261,16 @@ or several elements after in the case of bulk enqueue. If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image12_png has been replaced -|ring-mp-enqueue1| +.. _figure_ring-mp-enqueue1: + +.. figure:: img/ring-mp-enqueue1.* + + Multiple consumer enqueue first step + -MC Enqueue Second Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Second Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The second step is to modify ring->prod_head in the ring structure to point to the same location as prod_next. This operation is done using a Compare And Swap (CAS) instruction, which does the following operations atomically: @@ -256,41 +283,57 @@ This operation is done using a Compare And Swap (CAS) instruction, which does th In the figure, the operation succeeded on core 1, and step one restarted on core 2. -.. image13_png has been replaced -|ring-mp-enqueue2| +.. _figure_ring-mp-enqueue2: -MC Enqueue Third Step -^^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue2.* + + Multiple consumer enqueue second step + + +Multiple Consumer Enqueue Third Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The CAS operation is retried on core 2 with success. The core 1 updates one element of the ring(obj4), and the core 2 updates another one (obj5). -.. image14_png has been replaced -|ring-mp-enqueue3| +.. _figure_ring-mp-enqueue3: + +.. figure:: img/ring-mp-enqueue3.* + + Multiple consumer enqueue third step + -MC Enqueue Fourth Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Fourth Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Each core now wants to update ring->prod_tail. A core can only update it if ring->prod_tail is equal to the prod_head local variable. This is only true on core 1. The operation is finished on core 1. -.. image15_png has been replaced -|ring-mp-enqueue4| +.. _figure_ring-mp-enqueue4: -MC Enqueue Last Step -^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue4.* + + Multiple consumer enqueue fourth step + + +Multiple Consumer Enqueue Last Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Once ring->prod_tail is updated by core 1, core 2 is allowed to update it too. The operation is also finished on core 2. -.. image16_png has been replaced -|ring-mp-enqueue5| +.. _figure_ring-mp-enqueue5: + +.. figure:: img/ring-mp-enqueue5.* + + Multiple consumer enqueue last step + Modulo 32-bit Indexes ~~~~~~~~~~~~~~~~~~~~~ @@ -309,15 +352,23 @@ The following are two examples that help to explain how indexes are used in a ri In addition, the four indexes are defined as unsigned 16-bit integers, as opposed to unsigned 32-bit integers in the more realistic case. -.. image17_png has been replaced -|ring-modulo1| +.. _figure_ring-modulo1: + +.. figure:: img/ring-modulo1.* + + Modulo 32-bit indexes - Example 1 + This ring contains 11000 entries. -.. image18_png has been replaced -|ring-modulo2| +.. _figure_ring-modulo2: + +.. figure:: img/ring-modulo2.* + + Modulo 32-bit indexes - Example 2 + This ring contains 12536 entries. @@ -346,31 +397,3 @@ References * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_ (version 8) * `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_ - -.. |ring1| image:: img/ring1.* - -.. |ring-enqueue1| image:: img/ring-enqueue1.* - -.. |ring-enqueue2| image:: img/ring-enqueue2.* - -.. |ring-enqueue3| image:: img/ring-enqueue3.* - -.. |ring-dequeue1| image:: img/ring-dequeue1.* - -.. |ring-dequeue2| image:: img/ring-dequeue2.* - -.. |ring-dequeue3| image:: img/ring-dequeue3.* - -.. |ring-mp-enqueue1| image:: img/ring-mp-enqueue1.* - -.. |ring-mp-enqueue2| image:: img/ring-mp-enqueue2.* - -.. |ring-mp-enqueue3| image:: img/ring-mp-enqueue3.* - -.. |ring-mp-enqueue4| image:: img/ring-mp-enqueue4.* - -.. |ring-mp-enqueue5| image:: img/ring-mp-enqueue5.* - -.. |ring-modulo1| image:: img/ring-modulo1.* - -.. |ring-modulo2| image:: img/ring-modulo2.* diff --git a/doc/guides/sample_app_ug/dist_app.rst b/doc/guides/sample_app_ug/dist_app.rst index 56195bb..3c9a991 100644 --- a/doc/guides/sample_app_ug/dist_app.rst +++ b/doc/guides/sample_app_ug/dist_app.rst @@ -47,11 +47,12 @@ into each other. This application can be used to benchmark performance using the traffic generator as shown in the figure below. -.. _figure_22: +.. _figure_dist_perf: -**Figure 22. Performance Benchmarking Setup (Basic Environment)** +.. figure:: img/dist_perf.* + + Performance Benchmarking Setup (Basic Environment) -|dist_perf| Compiling the Application ------------------------- @@ -106,7 +107,7 @@ Explanation The distributor application consists of three types of threads: a receive thread (lcore_rx()), a set of worker threads(lcore_worker()) and a transmit thread(lcore_tx()). How these threads work together is shown -in Fig2 below. The main() function launches threads of these three types. +in :numref:`figure_dist_app` below. The main() function launches threads of these three types. Each thread has a while loop which will be doing processing and which is terminated only upon SIGINT or ctrl+C. The receive and transmit threads communicate using a software ring (rte_ring structure). @@ -136,11 +137,12 @@ Users who wish to terminate the running of the application have to press ctrl+C in the application will terminate all running threads gracefully and print final statistics to the user. -.. _figure_23: +.. _figure_dist_app: + +.. figure:: img/dist_app.* -**Figure 23. Distributor Sample Application Layout** + Distributor Sample Application Layout -|dist_app| Debug Logging Support --------------------- @@ -171,7 +173,3 @@ Sample Application. See Section 9.4.4, "RX Queue Initialization". TX queue initialization is done in the same way as it is done in the L2 Forwarding Sample Application. See Section 9.4.5, "TX Queue Initialization". - -.. |dist_perf| image:: img/dist_perf.* - -.. |dist_app| image:: img/dist_app.* diff --git a/doc/guides/sample_app_ug/exception_path.rst b/doc/guides/sample_app_ug/exception_path.rst index 6c06959..3cc7cbe 100644 --- a/doc/guides/sample_app_ug/exception_path.rst +++ b/doc/guides/sample_app_ug/exception_path.rst @@ -46,13 +46,12 @@ The second thread reads from a TAP interface and writes the data unmodified to t The packet flow through the exception path application is as shown in the following figure. -.. _figure_1: +.. _figure_exception_path_example: -**Figure 1. Packet Flow** +.. figure:: img/exception_path_example.* -.. image2_png has been replaced + Packet Flow -|exception_path_example| To make throughput measurements, kernel bridges must be setup to forward data between the bridges appropriately. @@ -327,4 +326,3 @@ To remove bridges and persistent TAP interfaces, the following commands are used brctl delbr br0 openvpn --rmtun --dev tap_dpdk_00 -.. |exception_path_example| image:: img/exception_path_example.* diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index aaa95ef..745a7ac 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -74,57 +74,63 @@ Sample Applications User Guide **Figures** -:ref:`Figure 1.Packet Flow <figure_1>` +:numref:`figure_exception_path_example` :ref:`figure_exception_path_example` -:ref:`Figure 2.Kernel NIC Application Packet Flow <figure_2>` +:numref:`figure_kernel_nic` :ref:`figure_kernel_nic` -:ref:`Figure 3.Performance Benchmark Setup (Basic Environment) <figure_3>` +:numref:`figure_l2_fwd_benchmark_setup_jobstats` :ref:`figure_l2_fwd_benchmark_setup_jobstats` -:ref:`Figure 4.Performance Benchmark Setup (Virtualized Environment) <figure_4>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` :ref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` -:ref:`Figure 5.Load Balancer Application Architecture <figure_5>` +:numref:`figure_l2_fwd_benchmark_setup` :ref:`figure_l2_fwd_benchmark_setup` -:ref:`Figure 5.Example Rules File <figure_5_1>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup` :ref:`figure_l2_fwd_virtenv_benchmark_setup` -:ref:`Figure 6.Example Data Flow in a Symmetric Multi-process Application <figure_6>` +:numref:`figure_ipv4_acl_rule` :ref:`figure_ipv4_acl_rule` -:ref:`Figure 7.Example Data Flow in a Client-Server Symmetric Multi-process Application <figure_7>` +:numref:`figure_example_rules` :ref:`figure_example_rules` -:ref:`Figure 8.Master-slave Process Workflow <figure_8>` +:numref:`figure_load_bal_app_arch` :ref:`figure_load_bal_app_arch` -:ref:`Figure 9.Slave Process Recovery Process Flow <figure_9>` +:numref:`figure_sym_multi_proc_app` :ref:`figure_sym_multi_proc_app` -:ref:`Figure 10.QoS Scheduler Application Architecture <figure_10>` +:numref:`figure_client_svr_sym_multi_proc_app` :ref:`figure_client_svr_sym_multi_proc_app` -:ref:`Figure 11.Intel®QuickAssist Technology Application Block Diagram <figure_11>` +:numref:`figure_master_slave_proc` :ref:`figure_master_slave_proc` -:ref:`Figure 12.Pipeline Overview <figure_12>` +:numref:`figure_slave_proc_recov` :ref:`figure_slave_proc_recov` -:ref:`Figure 13.Ring-based Processing Pipeline Performance Setup <figure_13>` +:numref:`figure_qos_sched_app_arch` :ref:`figure_qos_sched_app_arch` -:ref:`Figure 14.Threads and Pipelines <figure_14>` +:numref:`figure_quickassist_block_diagram` :ref:`figure_quickassist_block_diagram` -:ref:`Figure 15.Packet Flow Through the VMDQ and DCB Sample Application <figure_15>` +:numref:`figure_pipeline_overview` :ref:`figure_pipeline_overview` -:ref:`Figure 16.QEMU Virtio-net (prior to vhost-net) <figure_16>` +:numref:`figure_ring_pipeline_perf_setup` :ref:`figure_ring_pipeline_perf_setup` -:ref:`Figure 17.Virtio with Linux* Kernel Vhost <figure_17>` +:numref:`figure_threads_pipelines` :ref:`figure_threads_pipelines` -:ref:`Figure 18.Vhost-net Architectural Overview <figure_18>` +:numref:`figure_vmdq_dcb_example` :ref:`figure_vmdq_dcb_example` -:ref:`Figure 19.Packet Flow Through the vhost-net Sample Application <figure_19>` +:numref:`figure_qemu_virtio_net` :ref:`figure_qemu_virtio_net` -:ref:`Figure 20.Packet Flow on TX in DPDK-testpmd <figure_20>` +:numref:`figure_virtio_linux_vhost` :ref:`figure_virtio_linux_vhost` -:ref:`Figure 21.Test Pipeline Application <figure_21>` +:numref:`figure_vhost_net_arch` :ref:`figure_vhost_net_arch` -:ref:`Figure 22.Performance Benchmarking Setup (Basic Environment) <figure_22>` +:numref:`figure_vhost_net_sample_app` :ref:`figure_vhost_net_sample_app` -:ref:`Figure 23.Distributor Sample Application Layout <figure_23>` +:numref:`figure_tx_dpdk_testpmd` :ref:`figure_tx_dpdk_testpmd` -:ref:`Figure 24.High level Solution <figure_24>` +:numref:`figure_test_pipeline_app` :ref:`figure_test_pipeline_app` -:ref:`Figure 25.VM request to scale frequency <figure_25>` +:numref:`figure_dist_perf` :ref:`figure_dist_perf` + +:numref:`figure_dist_app` :ref:`figure_dist_app` + +:numref:`figure_vm_power_mgr_highlevel` :ref:`figure_vm_power_mgr_highlevel` + +:numref:`figure_vm_power_mgr_vm_request_seq` :ref:`figure_vm_power_mgr_vm_request_seq` **Tables** diff --git a/doc/guides/sample_app_ug/intel_quickassist.rst b/doc/guides/sample_app_ug/intel_quickassist.rst index 7f55282..a80d4ca 100644 --- a/doc/guides/sample_app_ug/intel_quickassist.rst +++ b/doc/guides/sample_app_ug/intel_quickassist.rst @@ -46,17 +46,16 @@ For this sample application, there is a dependency on either of: Overview -------- -An overview of the application is provided in Figure 11. +An overview of the application is provided in :numref:`figure_quickassist_block_diagram`. For simplicity, only two NIC ports and one Intel® QuickAssist Technology device are shown in this diagram, although the number of NIC ports and Intel® QuickAssist Technology devices can be different. -.. _figure_11: +.. _figure_quickassist_block_diagram: -**Figure 11. Intel® QuickAssist Technology Application Block Diagram** +.. figure:: img/quickassist_block_diagram.* -.. image14_png has been renamed + Intel® QuickAssist Technology Application Block Diagram -|quickassist_block_diagram| The application allows the configuration of the following items: @@ -220,5 +219,3 @@ performing AES-CBC-128 encryption with AES-XCBC-MAC-96 hash, the following setti Refer to the *DPDK Test Report* for more examples of traffic generator setup and the application startup command lines. If no errors are generated in response to the startup commands, the application is running correctly. - -.. |quickassist_block_diagram| image:: img/quickassist_block_diagram.* diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index d6876e2..02dde59 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -71,13 +71,12 @@ it is just for performance testing, or it can work together with VMDq support in The packet flow through the Kernel NIC Interface application is as shown in the following figure. -.. _figure_2: +.. _figure_kernel_nic: -**Figure 2. Kernel NIC Application Packet Flow** +.. figure:: img/kernel_nic.* -.. image3_png has been renamed to kernel_nic.* + Kernel NIC Application Packet Flow -|kernel_nic| Compiling the Application ------------------------- @@ -616,5 +615,3 @@ Currently, setting a new MTU and configuring the network interface (up/ down) ar RTE_LOG(ERR, APP, "Failed to start port %d\n", port_id); return ret; } - -.. |kernel_nic| image:: img/kernel_nic.* diff --git a/doc/guides/sample_app_ug/l2_forward_job_stats.rst b/doc/guides/sample_app_ug/l2_forward_job_stats.rst index 10dfecb..6cbf627 100644 --- a/doc/guides/sample_app_ug/l2_forward_job_stats.rst +++ b/doc/guides/sample_app_ug/l2_forward_job_stats.rst @@ -55,27 +55,24 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup_jobstats`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup_jobstats: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| +.. _figure_l2_fwd_virtenv_benchmark_setup_jobstats: -.. _figure_4: +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** + Performance Benchmark Setup (Virtualized Environment) -.. image5_png has been renamed - -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -631,7 +628,3 @@ however it improves performance: * in which it was called. */ rte_jobstats_finish(&qconf->flush_job, qconf->flush_job.target); } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst index 234d71d..9334e75 100644 --- a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst +++ b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst @@ -54,27 +54,25 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| -.. _figure_4: +.. _figure_l2_fwd_virtenv_benchmark_setup: -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -.. image5_png has been renamed + Performance Benchmark Setup (Virtualized Environment) -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -526,7 +524,3 @@ however it improves performance: prev_tsc = cur_tsc; } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst index 73fa4df..dbf47c7 100644 --- a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst +++ b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst @@ -142,9 +142,13 @@ Other lines types are considered invalid. * A typical IPv4 ACL rule line should have a format as shown below: -.. image6_png has been renamed -|ipv4_acl_rule| +.. _figure_ipv4_acl_rule: + +.. figure:: img/ipv4_acl_rule.* + + A typical IPv4 ACL rule + IPv4 addresses are specified in CIDR format as specified in RFC 4632. They consist of the dot notation for the address and a prefix length separated by '/'. @@ -164,15 +168,12 @@ For example: 6/0xfe matches protocol values 6 and 7. Rules File Example ~~~~~~~~~~~~~~~~~~ -.. _figure_5_1: +.. _figure_example_rules: -Figure 5 is an example of a rules file. This file has three rules, one for ACL and two for route information. +.. figure:: img/example_rules.* -**Figure 5.Example Rules File** + Rules example -.. image7_png has been renamed - -|example_rules| Each rule is explained as follows: @@ -397,7 +398,3 @@ Finally, the application creates contexts handler from the ACL library, adds rules parsed from the file into the database and build an ACL trie. It is important to note that the application creates an independent copy of each database for each socket CPU involved in the task to reduce the time for remote memory access. - -.. |ipv4_acl_rule| image:: img/ipv4_acl_rule.* - -.. |example_rules| image:: img/example_rules.* diff --git a/doc/guides/sample_app_ug/load_balancer.rst b/doc/guides/sample_app_ug/load_balancer.rst index 3b32bdc..615826a 100644 --- a/doc/guides/sample_app_ug/load_balancer.rst +++ b/doc/guides/sample_app_ug/load_balancer.rst @@ -44,13 +44,12 @@ Overview The architecture of the Load Balance application is presented in the following figure. -.. _figure_5: +.. _figure_load_bal_app_arch: -**Figure 5. Load Balancer Application Architecture** +.. figure:: img/load_bal_app_arch.* -.. image8_png has been renamed + Load Balancer Application Architecture -|load_bal_app_arch| For the sake of simplicity, the diagram illustrates a specific case of two I/O RX and two I/O TX lcores off loading the packet I/O overhead incurred by four NIC ports from four worker cores, with each I/O lcore handling RX/TX for two NIC ports. @@ -241,5 +240,3 @@ are on the same or different CPU sockets, the following run-time scenarios are p #. ABC: The packet is received on socket A, it is processed by an lcore on socket B, then it has to be transmitted out by a NIC connected to socket C. The performance price for crossing the CPU socket boundary is paid twice for this packet. - -.. |load_bal_app_arch| image:: img/load_bal_app_arch.* diff --git a/doc/guides/sample_app_ug/multi_process.rst b/doc/guides/sample_app_ug/multi_process.rst index 9ed450b..dc287af 100644 --- a/doc/guides/sample_app_ug/multi_process.rst +++ b/doc/guides/sample_app_ug/multi_process.rst @@ -190,13 +190,12 @@ such as a client-server mode of operation seen in the next example, where different processes perform different tasks, yet co-operate to form a packet-processing system.) The following diagram shows the data-flow through the application, using two processes. -.. _figure_6: +.. _figure_sym_multi_proc_app: -**Figure 6. Example Data Flow in a Symmetric Multi-process Application** +.. figure:: img/sym_multi_proc_app.* -.. image9_png has been renamed + Example Data Flow in a Symmetric Multi-process Application -|sym_multi_proc_app| As the diagram shows, each process reads packets from each of the network ports in use. RSS is used to distribute incoming packets on each port to different hardware RX queues. @@ -296,13 +295,12 @@ In this case, the client applications just perform level-2 forwarding of packets The following diagram shows the data-flow through the application, using two client processes. -.. _figure_7: +.. _figure_client_svr_sym_multi_proc_app: -**Figure 7. Example Data Flow in a Client-Server Symmetric Multi-process Application** +.. figure:: img/client_svr_sym_multi_proc_app.* -.. image10_png has been renamed + Example Data Flow in a Client-Server Symmetric Multi-process Application -|client_svr_sym_multi_proc_app| Running the Application ^^^^^^^^^^^^^^^^^^^^^^^ @@ -395,13 +393,12 @@ Once the master process begins to run, it tries to initialize all the resources memory, CPU cores, driver, ports, and so on, as the other examples do. Thereafter, it creates slave processes, as shown in the following figure. -.. _figure_8: +.. _figure_master_slave_proc: -**Figure 8. Master-slave Process Workflow** +.. figure:: img/master_slave_proc.* -.. image11_png has been renamed + Master-slave Process Workflow -|master_slave_proc| The master process calls the rte_eal_mp_remote_launch() EAL function to launch an application function for each pinned thread through the pipe. Then, it waits to check if any slave processes have exited. @@ -475,13 +472,12 @@ Therefore, to provide the capability to resume the new slave instance if the pre The following diagram describes slave process recovery. -.. _figure_9: +.. _figure_slave_proc_recov: -**Figure 9. Slave Process Recovery Process Flow** +.. figure:: img/slave_proc_recov.* -.. image12_png has been renamed + Slave Process Recovery Process Flow -|slave_proc_recov| Floating Process Support ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -774,11 +770,3 @@ so it remaps the resource to the new core ID slot. } return 0; } - -.. |sym_multi_proc_app| image:: img/sym_multi_proc_app.* - -.. |client_svr_sym_multi_proc_app| image:: img/client_svr_sym_multi_proc_app.* - -.. |master_slave_proc| image:: img/master_slave_proc.* - -.. |slave_proc_recov| image:: img/slave_proc_recov.* diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index 56326df..66c261c 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -38,13 +38,12 @@ Overview The architecture of the QoS scheduler application is shown in the following figure. -.. _figure_10: +.. _figure_qos_sched_app_arch: -**Figure 10. QoS Scheduler Application Architecture** +.. figure:: img/qos_sched_app_arch.* -.. image13_png has been renamed + QoS Scheduler Application Architecture -|qos_sched_app_arch| There are two flavors of the runtime execution for this application, with two or three threads per each packet flow configuration being used. @@ -347,5 +346,3 @@ This application classifies based on the QinQ double VLAN tags and the IP destin +----------------+-------------------------+--------------------------------------------------+----------------------------------+ Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. - -.. |qos_sched_app_arch| image:: img/qos_sched_app_arch.* diff --git a/doc/guides/sample_app_ug/quota_watermark.rst b/doc/guides/sample_app_ug/quota_watermark.rst index 4274223..c56683a 100644 --- a/doc/guides/sample_app_ug/quota_watermark.rst +++ b/doc/guides/sample_app_ug/quota_watermark.rst @@ -54,15 +54,14 @@ and ports 2 and 3 forward into each other. The MAC addresses of the forwarded Ethernet frames are not affected. Internally, packets are pulled from the ports by the master logical core and put on a variable length processing pipeline, -each stage of which being connected by rings, as shown in Figure 12. +each stage of which being connected by rings, as shown in :numref:`figure_pipeline_overview`. -.. _figure_12: +.. _figure_pipeline_overview: -**Figure 12. Pipeline Overview** +.. figure:: img/pipeline_overview.* -.. image15_png has been renamed + Pipeline Overview -|pipeline_overview| An adjustable quota value controls how many packets are being moved through the pipeline per enqueue and dequeue. Adjustable watermark values associated with the rings control a back-off mechanism that @@ -79,15 +78,14 @@ eventually lead to an Ethernet flow control frame being send to the source. On top of serving as an example of quota and watermark usage, this application can be used to benchmark ring based processing pipelines performance using a traffic- generator, -as shown in Figure 13. +as shown in :numref:`figure_ring_pipeline_perf_setup`. -.. _figure_13: +.. _figure_ring_pipeline_perf_setup: -**Figure 13. Ring-based Processing Pipeline Performance Setup** +.. figure:: img/ring_pipeline_perf_setup.* -.. image16_png has been renamed + Ring-based Processing Pipeline Performance Setup -|ring_pipeline_perf_setup| Compiling the Application ------------------------- @@ -311,7 +309,7 @@ Logical Cores Assignment The application uses the master logical core to poll all the ports for new packets and enqueue them on a ring associated with the port. Each logical core except the last runs pipeline_stage() after a ring for each used port is initialized on that core. -pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See Figure 14. +pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See :numref:`figure_threads_pipelines`. .. code-block:: c @@ -340,16 +338,12 @@ sending them out on the destination port setup by pair_ports(). Receive, Process and Transmit Packets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _figure_14: +.. _figure_threads_pipelines: -Figure 14 shows where each thread in the pipeline is. -It should be used as a reference while reading the rest of this section. +.. figure:: img/threads_pipelines.* -**Figure 14. Threads and Pipelines** + Threads and Pipelines -.. image17_png has been renamed - -|threads_pipelines| In the receive_stage() function running on the master logical core, the main task is to read ingress packets from the RX ports and enqueue them @@ -498,9 +492,3 @@ low_watermark from the rte_memzone previously created by qw. low_watermark = (unsigned int *) qw_memzone->addr + sizeof(int); } - -.. |pipeline_overview| image:: img/pipeline_overview.* - -.. |ring_pipeline_perf_setup| image:: img/ring_pipeline_perf_setup.* - -.. |threads_pipelines| image:: img/threads_pipelines.* diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index 46aa6d5..3c2c25b 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -49,13 +49,12 @@ The application uses three CPU cores: * Core C ("TX core") receives traffic from core B through software queues and sends it to the NIC ports for transmission. -.. _figure_21: +.. _figure_test_pipeline_app: -**Figure 21.Test Pipeline Application** +.. figure:: img/test_pipeline_app.* -.. image24_png has been renamed + Test Pipeline Application -|test_pipeline_app| Compiling the Application ------------------------- @@ -281,5 +280,3 @@ The profile for input traffic is TCP/IPv4 packets with: * destination TCP port fixed to 0 * source TCP port fixed to 0 - -.. |test_pipeline_app| image:: img/test_pipeline_app.* diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst index ca9390d..730b9da 100644 --- a/doc/guides/sample_app_ug/vhost.rst +++ b/doc/guides/sample_app_ug/vhost.rst @@ -48,13 +48,12 @@ between host and guest. It was found that virtio-net performance was poor due to context switching and packet copying between host, guest, and QEMU. The following figure shows the system architecture for a virtio-based networking (virtio-net). -.. _figure_16: +.. _figure_qemu_virtio_net: -**Figure16. QEMU Virtio-net (prior to vhost-net)** +.. figure:: img/qemu_virtio_net.* -.. image19_png has been renamed + System Architecture for Virtio-based Networking (virtio-net). -|qemu_virtio_net| The Linux* Kernel vhost-net module was developed as an offload mechanism for virtio-net. The vhost-net module enables KVM (QEMU) to offload the servicing of virtio-net devices to the vhost-net kernel module, @@ -76,13 +75,12 @@ This is achieved by QEMU sharing the following information with the vhost-net mo The following figure shows the system architecture for virtio-net networking with vhost-net offload. -.. _figure_17: +.. _figure_virtio_linux_vhost: -**Figure 17. Virtio with Linux* Kernel Vhost** +.. figure:: img/virtio_linux_vhost.* -.. image20_png has been renamed + Virtio with Linux -|virtio_linux_vhost| Sample Code Overview -------------------- @@ -119,23 +117,21 @@ The vhost sample code application is a simple packet switching application with The following figure shows the architecture of the Vhost sample application based on vhost-cuse. -.. _figure_18: +.. _figure_vhost_net_arch: -**Figure 18. Vhost-net Architectural Overview** +.. figure:: img/vhost_net_arch.* -.. image21_png has been renamed + Vhost-net Architectural Overview -|vhost_net_arch| The following figure shows the flow of packets through the vhost-net sample application. -.. _figure_19: +.. _figure_vhost_net_sample_app: -**Figure 19. Packet Flow Through the vhost-net Sample Application** +.. figure:: img/vhost_net_sample_app.* -.. image22_png has been renamed + Packet Flow Through the vhost-net Sample Application -|vhost_net_sample_app| Supported Distributions ----------------------- @@ -771,13 +767,12 @@ In the "wait and retry" mode if the virtqueue is found to be full, then testpmd The "wait and retry" algorithm is implemented in DPDK testpmd as a forwarding method call "mac_retry". The following sequence diagram describes the algorithm in detail. -.. _figure_20: +.. _figure_tx_dpdk_testpmd: -**Figure 20. Packet Flow on TX in DPDK-testpmd** +.. figure:: img/tx_dpdk_testpmd.* -.. image23_png has been renamed + Packet Flow on TX in DPDK-testpmd -|tx_dpdk_testpmd| Running Testpmd ~~~~~~~~~~~~~~~ @@ -838,13 +833,3 @@ For example: The above message indicates that device 0 has been registered with MAC address cc:bb:bb:bb:bb:bb and VLAN tag 1000. Any packets received on the NIC with these values is placed on the devices receive queue. When a virtio-net device transmits packets, the VLAN tag is added to the packet by the DPDK vhost sample code. - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |qemu_virtio_net| image:: img/qemu_virtio_net.* - -.. |tx_dpdk_testpmd| image:: img/tx_dpdk_testpmd.* - -.. |vhost_net_sample_app| image:: img/vhost_net_sample_app.* - -.. |virtio_linux_vhost| image:: img/virtio_linux_vhost.* diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst index dd6e1e8..da8be12 100644 --- a/doc/guides/sample_app_ug/vm_power_management.rst +++ b/doc/guides/sample_app_ug/vm_power_management.rst @@ -74,11 +74,12 @@ The solution is comprised of two high-level components: The l3fwd-power application will use this implementation when deployed on a VM (see Chapter 11 "L3 Forwarding with Power Management Application"). -.. _figure_24: +.. _figure_vm_power_mgr_highlevel: -**Figure 24. Highlevel Solution** +.. figure:: img/vm_power_mgr_highlevel.* + + Highlevel Solution -|vm_power_mgr_highlevel| Overview -------- @@ -105,11 +106,12 @@ at runtime based on the environment. Upon receiving a request, the host translates the vCPU to a pCPU via the libvirt API before forwarding to the host librte_power. -.. _figure_25: +.. _figure_vm_power_mgr_vm_request_seq: + +.. figure:: img/vm_power_mgr_vm_request_seq.* -**Figure 25. VM request to scale frequency** + VM request to scale frequency -|vm_power_mgr_vm_request_seq| Performance Considerations ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -355,7 +357,3 @@ Where {core_num} is the lcore and channel to change frequency by scaling up/down .. code-block:: console set_cpu_freq {core_num} up|down|min|max - -.. |vm_power_mgr_highlevel| image:: img/vm_power_mgr_highlevel.* - -.. |vm_power_mgr_vm_request_seq| image:: img/vm_power_mgr_vm_request_seq.* diff --git a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst index 9fc1fd5..6fb0f0b 100644 --- a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst +++ b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst @@ -53,7 +53,7 @@ All traffic is read from a single incoming port (port 0) and output on port 1, w The traffic is split into 128 queues on input, where each thread of the application reads from multiple queues. For example, when run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. -As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in Figure 15. +As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in :numref:`figure_vmdq_dcb_example`. The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 32 pools of 4 queues each and this can be used by changing the NUM_POOLS parameter in the supplied code. The NUM_POOLS parameter can be passed on the command line, after the EAL parameters: @@ -64,13 +64,12 @@ The NUM_POOLS parameter can be passed on the command line, after the EAL paramet where, NP can be 16 or 32. -.. _figure_15: +.. _figure_vmdq_dcb_example: -**Figure 15. Packet Flow Through the VMDQ and DCB Sample Application** +.. figure:: img/vmdq_dcb_example.* -.. image18_png has been replaced + Packet Flow Through the VMDQ and DCB Sample Application -|vmdq_dcb_example| In Linux* user space, the application can display statistics with the number of packets received on each queue. To have the application display the statistics, send a SIGHUP signal to the running application process, as follows: @@ -247,5 +246,3 @@ To generate the statistics output, use the following command: Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, rather than the terminal from which the HUP signal was sent. - -.. |vmdq_dcb_example| image:: img/vmdq_dcb_example.* diff --git a/doc/guides/xen/pkt_switch.rst b/doc/guides/xen/pkt_switch.rst index a9eca52..3a6fc47 100644 --- a/doc/guides/xen/pkt_switch.rst +++ b/doc/guides/xen/pkt_switch.rst @@ -52,9 +52,13 @@ The switching back end maps those grant table references and creates shared ring The following diagram describes the functionality of the DPDK Xen Packet- Switching Solution. -.. image35_png has been renamed -|dpdk_xen_pkt_switch| +.. _figure_dpdk_xen_pkt_switch: + +.. figure:: img/dpdk_xen_pkt_switch.* + + Functionality of the DPDK Xen Packet Switching Solution. + Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory between domains (`http://wiki.xen.org/wiki/Grant Table <http://wiki.xen.org/wiki/Grant%20Table>`_). @@ -62,9 +66,13 @@ Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory A diagram of the design is shown below, where "gva" is the Guest Virtual Address, which is the data pointer of the mbuf, and "hva" is the Host Virtual Address: -.. image36_png has been renamed -|grant_table| +.. _figure_grant_table: + +.. figure:: img/grant_table.* + + DPDK Xen Layout + In this design, a Virtio ring is used as a para-virtualized interface for better performance over a Xen private ring when packet switching to and from a VM. @@ -139,9 +147,13 @@ Take idx#_mempool_gref node for example, the host maps those Grant references to The real Grant reference information is stored in this virtual address space, where (gref, pfn) pairs follow each other with -1 as the terminator. -.. image37_pnng has been renamed -|grant_refs| +.. _figure_grant_refs: + +.. figure:: img/grant_refs.* + + Mapping Grant references to a continuous virtual address space + After all gref# IDs are retrieved, the host maps them to a continuous virtual address space. With the guest mempool virtual address, the host establishes 1:1 address mapping. @@ -456,9 +468,3 @@ then sent out through hardware with destination MAC address 00:00:00:00:00:33. The packet flow is: packet generator->Virtio in guest VM1->switching backend->Virtio in guest VM2->switching backend->wire - -.. |grant_table| image:: img/grant_table.* - -.. |grant_refs| image:: img/grant_refs.* - -.. |dpdk_xen_pkt_switch| image:: img/dpdk_xen_pkt_switch.* -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v2 3/3] doc: add sphinx numref compatibility workaround 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 0/3] " John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH 1/3] doc: refactored figure numbers " John McNamara @ 2015-05-18 11:15 ` John McNamara 1 sibling, 0 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:15 UTC (permalink / raw) To: dev; +Cc: John McNamara From: John McNamara <jmcnamara@cpan.org> This change adds some simple handling for the :numref: directive for Sphinx versions prior to 1.3.1. This allows the Guides documentation to be built with older versions of Sphinx and still produce reasonable results. The patch replaces the :numref: reference with a link to the target (for all Sphinx doc types). It doesn't try to label figures/tables. Full numref support with automatic figure/table numbering and links can be obtained by upgrading to Sphinx 1.3.1 or later. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/conf.py | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 1bc031f..cab97ac 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -31,6 +31,9 @@ import subprocess from sphinx.highlighting import PygmentsBridge from pygments.formatters.latex import LatexFormatter +from docutils import nodes +from distutils.version import LooseVersion +from sphinx import __version__ as sphinx_version project = 'DPDK' @@ -72,6 +75,7 @@ latex_elements = { 'preamble': latex_preamble } + # Override the default Latex formatter in order to modify the # code/verbatim blocks. class CustomLatexFormatter(LatexFormatter): @@ -82,3 +86,79 @@ class CustomLatexFormatter(LatexFormatter): # Replace the default latex formatter. PygmentsBridge.latex_formatter = CustomLatexFormatter + + +# The following hook functions add some simple handling for the :numref: +# directive for Sphinx versions prior to 1.3.1. The functions replace the +# :numref: reference with a link to the target (for all Sphinx doc types). It +# doesn't try to label figures/tables. + +def numref_role(reftype, rawtext, text, lineno, inliner): + """ + Add a Sphinx role to handle numref references. Note, we can't convert the + link here because the doctree isn't build and the target information isn't + available. + + """ + + # Add an identifier to distinguish numref from other references. + newnode = nodes.reference('', + '', + refuri='_local_numref_#%s' % text, + internal=True) + + return [newnode], [] + + +def process_numref(app, doctree, from_docname): + """ + Process the numref nodes once the doctree has been built and prior to + writing the files. The processing involves replacing the numref with a + link plus text to indicate if it is a Figure or Table link. + + """ + env = app.builder.env + + # Iterate over the reference nodes in the doctree. + for node in doctree.traverse(nodes.reference): + target = node.get('refuri', '') + + # Look for numref nodes. + if target.startswith('_local_numref_#'): + target = target.replace('_local_numref_#', '') + + # Get the target label and link information from the Sphinx env. + data = env.domains['std'].data + docname, label, _ = data['labels'].get(target, ('', '', '')) + relative_url = app.builder.get_relative_uri(from_docname, docname) + + # Add a text label to the link. + if target.startswith('figure'): + caption = 'Figure' + elif target.startswith('table'): + caption = 'Table' + else: + caption = 'Link' + + # Create a new reference node with the updated link information. + newnode = nodes.reference('', + caption, + refuri='%s#%s' % (relative_url, label), + internal=True) + + # Replace the node. + node.replace_self(newnode) + + +def setup(app): + + if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): + + print('[dpdk docs] Upgrade sphinx to version >= 1.3.1 for ' + 'improved Figure/Table number handling.') + + # Add a role to handle :numref: references. + app.add_role('numref', numref_role) + + # Process the numref references once the doctree has been created. + app.connect('doctree-resolved', process_numref) -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara ` (3 preceding siblings ...) 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 0/3] " John McNamara @ 2015-05-18 11:34 ` John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 1/3] doc: refactored figure numbers " John McNamara ` (3 more replies) 4 siblings, 4 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:34 UTC (permalink / raw) To: dev This patchset adds automatic figure and table references to the docs. The figure and table numbers in the generated Html and PDF docs can now be automatically numbered. It replaces all hardcoded figure/table numbers and references. The numfig/numref feature requires Sphinx >= 1.3.1. For backward compatibility with older versions workaround handling is added to the sphinx conf.py file in patch 3/3. The workaround replaces the :numref: reference with a "Figure" or "Table" link to the target (for all Sphinx doc types). It doesn't number the figures or tables. This produces reasonable documentation links for users with older versions of sphinx while allowing automatic numbering support for newer versions. Tested with Sphinx 1.2.3 and 1.3.1. John McNamara (3): doc: refactored figure numbers into references doc: refactored table numbers into references doc: add sphinx numref compatibility workaround doc/guides/conf.py | 82 ++ doc/guides/nics/index.rst | 18 +- doc/guides/nics/intel_vf.rst | 37 +- doc/guides/nics/virtio.rst | 18 +- doc/guides/nics/vmxnet3.rst | 18 +- doc/guides/prog_guide/env_abstraction_layer.rst | 8 +- doc/guides/prog_guide/index.rst | 162 +- doc/guides/prog_guide/ivshmem_lib.rst | 8 +- doc/guides/prog_guide/kernel_nic_interface.rst | 40 +- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 +- doc/guides/prog_guide/lpm6_lib.rst | 8 +- doc/guides/prog_guide/lpm_lib.rst | 8 +- doc/guides/prog_guide/malloc_lib.rst | 9 +- doc/guides/prog_guide/mbuf_lib.rst | 20 +- doc/guides/prog_guide/mempool_lib.rst | 32 +- doc/guides/prog_guide/multi_proc_support.rst | 9 +- doc/guides/prog_guide/overview.rst | 9 +- doc/guides/prog_guide/packet_distrib_lib.rst | 15 +- doc/guides/prog_guide/packet_framework.rst | 1275 ++++++++-------- doc/guides/prog_guide/qos_framework.rst | 1543 ++++++++++---------- doc/guides/prog_guide/ring_lib.rst | 159 +- doc/guides/sample_app_ug/dist_app.rst | 20 +- doc/guides/sample_app_ug/exception_path.rst | 8 +- doc/guides/sample_app_ug/index.rst | 64 +- doc/guides/sample_app_ug/intel_quickassist.rst | 11 +- doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +- doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +- .../sample_app_ug/l2_forward_real_virtual.rst | 22 +- .../sample_app_ug/l3_forward_access_ctrl.rst | 21 +- doc/guides/sample_app_ug/load_balancer.rst | 9 +- doc/guides/sample_app_ug/multi_process.rst | 36 +- doc/guides/sample_app_ug/qos_metering.rst | 46 +- doc/guides/sample_app_ug/qos_scheduler.rst | 55 +- doc/guides/sample_app_ug/quota_watermark.rst | 36 +- doc/guides/sample_app_ug/test_pipeline.rst | 313 ++-- doc/guides/sample_app_ug/vhost.rst | 45 +- doc/guides/sample_app_ug/vm_power_management.rst | 18 +- doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +- doc/guides/xen/pkt_switch.rst | 30 +- 39 files changed, 2141 insertions(+), 2157 deletions(-) -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v3 1/3] doc: refactored figure numbers into references 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara @ 2015-05-18 11:34 ` John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 2/3] doc: refactored table " John McNamara ` (2 subsequent siblings) 3 siblings, 0 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:34 UTC (permalink / raw) To: dev This change adds automatic figure references to the docs. The figure numbers in the generated Html and PDF docs are now automatically numbered based on section. Requires Sphinx >= 1.3.1. The patch makes the following changes. * Changes image:: tag to figure:: and moves image caption to the figure. * Adds captions to figures that didn't previously have any. * Un-templates the |image-name| substitution definitions into explicit figure:: tags. They weren't used more than once anyway and Sphinx doesn't support them for figure. * Adds a target to each image that didn't previously have one so that they can be cross-referenced. * Renamed existing image target to match the image name for consistency. * Replaces the Figures lists with automatic :numref: :ref: entries to generate automatic numbering and captions. * Replaces "Figure" references with automatic :numref: references. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/conf.py | 2 + doc/guides/nics/index.rst | 18 ++- doc/guides/nics/intel_vf.rst | 37 ++--- doc/guides/nics/virtio.rst | 18 ++- doc/guides/nics/vmxnet3.rst | 18 ++- doc/guides/prog_guide/env_abstraction_layer.rst | 8 +- doc/guides/prog_guide/index.rst | 92 +++++++----- doc/guides/prog_guide/ivshmem_lib.rst | 8 +- doc/guides/prog_guide/kernel_nic_interface.rst | 40 ++--- .../prog_guide/link_bonding_poll_mode_drv_lib.rst | 43 ++++-- doc/guides/prog_guide/lpm6_lib.rst | 8 +- doc/guides/prog_guide/lpm_lib.rst | 8 +- doc/guides/prog_guide/malloc_lib.rst | 9 +- doc/guides/prog_guide/mbuf_lib.rst | 20 +-- doc/guides/prog_guide/mempool_lib.rst | 32 ++-- doc/guides/prog_guide/multi_proc_support.rst | 9 +- doc/guides/prog_guide/overview.rst | 9 +- doc/guides/prog_guide/packet_distrib_lib.rst | 15 +- doc/guides/prog_guide/packet_framework.rst | 81 +++++----- doc/guides/prog_guide/qos_framework.rst | 163 +++++++-------------- doc/guides/prog_guide/ring_lib.rst | 159 +++++++++++--------- doc/guides/sample_app_ug/dist_app.rst | 20 ++- doc/guides/sample_app_ug/exception_path.rst | 8 +- doc/guides/sample_app_ug/index.rst | 58 ++++---- doc/guides/sample_app_ug/intel_quickassist.rst | 11 +- doc/guides/sample_app_ug/kernel_nic_interface.rst | 9 +- doc/guides/sample_app_ug/l2_forward_job_stats.rst | 23 +-- .../sample_app_ug/l2_forward_real_virtual.rst | 22 +-- .../sample_app_ug/l3_forward_access_ctrl.rst | 21 ++- doc/guides/sample_app_ug/load_balancer.rst | 9 +- doc/guides/sample_app_ug/multi_process.rst | 36 ++--- doc/guides/sample_app_ug/qos_scheduler.rst | 9 +- doc/guides/sample_app_ug/quota_watermark.rst | 36 ++--- doc/guides/sample_app_ug/test_pipeline.rst | 9 +- doc/guides/sample_app_ug/vhost.rst | 45 ++---- doc/guides/sample_app_ug/vm_power_management.rst | 18 +-- doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst | 11 +- doc/guides/xen/pkt_switch.rst | 30 ++-- 38 files changed, 539 insertions(+), 633 deletions(-) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index b1ef323..1bc031f 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -41,6 +41,8 @@ release = version master_doc = 'index' +numfig = True + latex_documents = [ ('index', 'doc.tex', diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index aadbae3..1ee67fa 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -50,14 +50,20 @@ Network Interface Controller Drivers **Figures** -:ref:`Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode <nic_figure_1>` +:numref:`figure_single_port_nic` :ref:`figure_single_port_nic` -:ref:`Figure 2. SR-IOV Performance Benchmark Setup <nic_figure_2>` +:numref:`figure_perf_benchmark` :ref:`figure_perf_benchmark` -:ref:`Figure 3. Fast Host-based Packet Processing <nic_figure_3>` +:numref:`figure_fast_pkt_proc` :ref:`figure_fast_pkt_proc` -:ref:`Figure 4. SR-IOV Inter-VM Communication <nic_figure_4>` +:numref:`figure_inter_vm_comms` :ref:`figure_inter_vm_comms` -:ref:`Figure 5. Virtio Host2VM Communication Example Using KNI vhost Back End <nic_figure_5>` +:numref:`figure_host_vm_comms` :ref:`figure_host_vm_comms` -:ref:`Figure 6. Virtio Host2VM Communication Example Using Qemu vhost Back End <nic_figure_6>` +:numref:`figure_host_vm_comms_qemu` :ref:`figure_host_vm_comms_qemu` + +:numref:`figure_vmxnet3_int` :ref:`figure_vmxnet3_int` + +:numref:`figure_vswitch_vm` :ref:`figure_vswitch_vm` + +:numref:`figure_vm_vm_comms` :ref:`figure_vm_vm_comms` diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst index eeca973..db86c64 100644 --- a/doc/guides/nics/intel_vf.rst +++ b/doc/guides/nics/intel_vf.rst @@ -49,9 +49,9 @@ SR-IOV Mode Utilization in a DPDK Environment The DPDK uses the SR-IOV feature for hardware-based I/O sharing in IOV mode. Therefore, it is possible to partition SR-IOV capability on Ethernet controller NIC resources logically and expose them to a virtual machine as a separate PCI function called a "Virtual Function". -Refer to Figure 10. +Refer to :numref:`figure_single_port_nic`. -Therefore, a NIC is logically distributed among multiple virtual machines (as shown in Figure 10), +Therefore, a NIC is logically distributed among multiple virtual machines (as shown in :numref:`figure_single_port_nic`), while still having global data in common to share with the Physical Function and other Virtual Functions. The DPDK fm10kvf, i40evf, igbvf or ixgbevf as a Poll Mode Driver (PMD) serves for the Intel® 82576 Gigabit Ethernet Controller, Intel® Ethernet Controller I350 family, Intel® 82599 10 Gigabit Ethernet Controller NIC, @@ -72,11 +72,12 @@ For more detail on SR-IOV, please refer to the following documents: * `Scalable I/O Virtualized Servers <http://www.intel.com/content/www/us/en/virtualization/server-virtualization/scalable-i-o-virtualized-servers-paper.html>`_ -.. _nic_figure_1: +.. _figure_single_port_nic: -**Figure 1. Virtualization for a Single Port NIC in SR-IOV Mode** +.. figure:: img/single_port_nic.* + + Virtualization for a Single Port NIC in SR-IOV Mode -.. image:: img/single_port_nic.* Physical and Virtual Function Infrastructure ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -548,13 +549,14 @@ The setup procedure is as follows: can also be used to bind and unbind devices to a virtual machine in Ubuntu. If this option is used, step 6 in the instructions provided will be different. - * The Virtual Machine Monitor (see Figure 11) is equivalent to a Host OS with KVM installed as described in the instructions. + * The Virtual Machine Monitor (see :numref:`figure_perf_benchmark`) is equivalent to a Host OS with KVM installed as described in the instructions. + +.. _figure_perf_benchmark: -.. _nic_figure_2: +.. figure:: img/perf_benchmark.* -**Figure 2. Performance Benchmark Setup** + Performance Benchmark Setup -.. image:: img/perf_benchmark.* DPDK SR-IOV PMD PF/VF Driver Usage Model ---------------------------------------- @@ -569,14 +571,15 @@ the DPDK VF PMD driver performs the same throughput result as a non-VT native en With such host instance fast packet processing, lots of services such as filtering, QoS, DPI can be offloaded on the host fast path. -Figure 12 shows the scenario where some VMs directly communicate externally via a VFs, +:numref:`figure_fast_pkt_proc` shows the scenario where some VMs directly communicate externally via a VFs, while others connect to a virtual switch and share the same uplink bandwidth. -.. _nic_figure_3: +.. _figure_fast_pkt_proc: + +.. figure:: img/fast_pkt_proc.* -**Figure 3. Fast Host-based Packet Processing** + Fast Host-based Packet Processing -.. image:: img/fast_pkt_proc.* SR-IOV (PF/VF) Approach for Inter-VM Communication -------------------------------------------------- @@ -587,7 +590,7 @@ So VF-to-VF traffic within the same physical port (VM0<->VM1) have hardware acce However, when VF crosses physical ports (VM0<->VM2), there is no such hardware bridge. In this case, the DPDK PMD PF driver provides host forwarding between such VMs. -Figure 13 shows an example. +:numref:`figure_inter_vm_comms` shows an example. In this case an update of the MAC address lookup tables in both the NIC and host DPDK application is required. In the NIC, writing the destination of a MAC address belongs to another cross device VM to the PF specific pool. @@ -598,8 +601,8 @@ that is, the packet is forwarded to the correct PF pool. The SR-IOV NIC switch forwards the packet to a specific VM according to the MAC destination address which belongs to the destination VF on the VM. -.. _nic_figure_4: +.. _figure_inter_vm_comms: -**Figure 4. Inter-VM Communication** +.. figure:: img/inter_vm_comms.* -.. image:: img/inter_vm_comms.* + Inter-VM Communication diff --git a/doc/guides/nics/virtio.rst b/doc/guides/nics/virtio.rst index 073d980..9f18b3a 100644 --- a/doc/guides/nics/virtio.rst +++ b/doc/guides/nics/virtio.rst @@ -106,11 +106,12 @@ Virtio with kni vhost Back End This section demonstrates kni vhost back end example setup for Phy-VM Communication. -.. _nic_figure_5: +.. _figure_host_vm_comms: -**Figure 5. Host2VM Communication Example Using kni vhost Back End** +.. figure:: img/host_vm_comms.* + + Host2VM Communication Example Using kni vhost Back End -.. image:: img/host_vm_comms.* Host2VM communication example @@ -174,7 +175,9 @@ Host2VM communication example We use testpmd as the forwarding application in this example. - .. image:: img/console.* + .. figure:: img/console.* + + Running testpmd #. Use IXIA packet generator to inject a packet stream into the KNI physical port. @@ -185,11 +188,12 @@ Host2VM communication example Virtio with qemu virtio Back End -------------------------------- -.. _nic_figure_6: +.. _figure_host_vm_comms_qemu: + +.. figure:: img/host_vm_comms_qemu.* -**Figure 6. Host2VM Communication Example Using qemu vhost Back End** + Host2VM Communication Example Using qemu vhost Back End -.. image:: img/host_vm_comms_qemu.* .. code-block:: console diff --git a/doc/guides/nics/vmxnet3.rst b/doc/guides/nics/vmxnet3.rst index 830530d..e919088 100644 --- a/doc/guides/nics/vmxnet3.rst +++ b/doc/guides/nics/vmxnet3.rst @@ -121,7 +121,11 @@ The following prerequisites apply: * Before starting a VM, a VMXNET3 interface to a VM through VMware vSphere Client must be assigned. This is shown in the figure below. -.. image:: img/vmxnet3_int.* +.. _figure_vmxnet3_int: + +.. figure:: img/vmxnet3_int.* + + Assigning a VMXNET3 interface to a VM using VMware vSphere Client .. note:: @@ -142,7 +146,11 @@ VMXNET3 with a Native NIC Connected to a vSwitch This section describes an example setup for Phy-vSwitch-VM-Phy communication. -.. image:: img/vswitch_vm.* +.. _figure_vswitch_vm: + +.. figure:: img/vswitch_vm.* + + VMXNET3 with a Native NIC Connected to a vSwitch .. note:: @@ -164,7 +172,11 @@ VMXNET3 Chaining VMs Connected to a vSwitch The following figure shows an example VM-to-VM communication over a Phy-VM-vSwitch-VM-Phy communication channel. -.. image:: img/vm_vm_comms.* +.. _figure_vm_vm_comms: + +.. figure:: img/vm_vm_comms.* + + VMXNET3 Chaining VMs Connected to a vSwitch .. note:: diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 4ecbe6a..25eb281 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -85,13 +85,12 @@ A check is also performed at initialization time to ensure that the micro archit Then, the main() function is called. The core initialization and launch is done in rte_eal_init() (see the API documentation). It consist of calls to the pthread library (more specifically, pthread_self(), pthread_create(), and pthread_setaffinity_np()). -.. _pg_figure_2: +.. _figure_linuxapp_launch: -**Figure 2. EAL Initialization in a Linux Application Environment** +.. figure:: img/linuxapp_launch.* -.. image3_png has been replaced + EAL Initialization in a Linux Application Environment -|linuxapp_launch| .. note:: @@ -367,4 +366,3 @@ We expect only 50% of CPU spend on packet IO. echo 50000 > pkt_io/cpu.cfs_quota_us -.. |linuxapp_launch| image:: img/linuxapp_launch.* diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 57d516a..9a1e337 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -80,71 +80,97 @@ Programmer's Guide **Figures** -:ref:`Figure 1. Core Components Architecture <pg_figure_1>` +:numref:`figure_architecture-overview` :ref:`figure_architecture-overview` -:ref:`Figure 2. EAL Initialization in a Linux Application Environment <pg_figure_2>` +:numref:`figure_linuxapp_launch` :ref:`figure_linuxapp_launch` -:ref:`Figure 3. Example of a malloc heap and malloc elements within the malloc library <pg_figure_3>` +:numref:`figure_malloc_heap` :ref:`figure_malloc_heap` -:ref:`Figure 4. Ring Structure <pg_figure_4>` +:numref:`figure_ring1` :ref:`figure_ring1` -:ref:`Figure 5. Two Channels and Quad-ranked DIMM Example <pg_figure_5>` +:numref:`figure_ring-enqueue1` :ref:`figure_ring-enqueue1` -:ref:`Figure 6. Three Channels and Two Dual-ranked DIMM Example <pg_figure_6>` +:numref:`figure_ring-enqueue2` :ref:`figure_ring-enqueue2` -:ref:`Figure 7. A mempool in Memory with its Associated Ring <pg_figure_7>` +:numref:`figure_ring-enqueue3` :ref:`figure_ring-enqueue3` -:ref:`Figure 8. An mbuf with One Segment <pg_figure_8>` +:numref:`figure_ring-dequeue1` :ref:`figure_ring-dequeue1` -:ref:`Figure 9. An mbuf with Three Segments <pg_figure_9>` +:numref:`figure_ring-dequeue2` :ref:`figure_ring-dequeue2` -:ref:`Figure 16. Memory Sharing in the Intel® DPDK Multi-process Sample Application <pg_figure_16>` +:numref:`figure_ring-dequeue3` :ref:`figure_ring-dequeue3` -:ref:`Figure 17. Components of an Intel® DPDK KNI Application <pg_figure_17>` +:numref:`figure_ring-mp-enqueue1` :ref:`figure_ring-mp-enqueue1` -:ref:`Figure 18. Packet Flow via mbufs in the Intel DPDK® KNI <pg_figure_18>` +:numref:`figure_ring-mp-enqueue2` :ref:`figure_ring-mp-enqueue2` -:ref:`Figure 19. vHost-net Architecture Overview <pg_figure_19>` +:numref:`figure_ring-mp-enqueue3` :ref:`figure_ring-mp-enqueue3` -:ref:`Figure 20. KNI Traffic Flow <pg_figure_20>` +:numref:`figure_ring-mp-enqueue4` :ref:`figure_ring-mp-enqueue4` -:ref:`Figure 21. Complex Packet Processing Pipeline with QoS Support <pg_figure_21>` +:numref:`figure_ring-mp-enqueue5` :ref:`figure_ring-mp-enqueue5` -:ref:`Figure 22. Hierarchical Scheduler Block Internal Diagram <pg_figure_22>` +:numref:`figure_ring-modulo1` :ref:`figure_ring-modulo1` -:ref:`Figure 23. Scheduling Hierarchy per Port <pg_figure_23>` +:numref:`figure_ring-modulo2` :ref:`figure_ring-modulo2` -:ref:`Figure 24. Internal Data Structures per Port <pg_figure_24>` +:numref:`figure_memory-management` :ref:`figure_memory-management` -:ref:`Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation <pg_figure_25>` +:numref:`figure_memory-management2` :ref:`figure_memory-management2` -:ref:`Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation <pg_figure_26>` +:numref:`figure_mempool` :ref:`figure_mempool` -:ref:`Figure 27. High-level Block Diagram of the Intel® DPDK Dropper <pg_figure_27>` +:numref:`figure_mbuf1` :ref:`figure_mbuf1` -:ref:`Figure 28. Flow Through the Dropper <pg_figure_28>` +:numref:`figure_mbuf2` :ref:`figure_mbuf2` -:ref:`Figure 29. Example Data Flow Through Dropper <pg_figure_29>` +:numref:`figure_multi_process_memory` :ref:`figure_multi_process_memory` -:ref:`Figure 30. Packet Drop Probability for a Given RED Configuration <pg_figure_30>` +:numref:`figure_kernel_nic_intf` :ref:`figure_kernel_nic_intf` -:ref:`Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) <pg_figure_31>` +:numref:`figure_pkt_flow_kni` :ref:`figure_pkt_flow_kni` -:ref:`Figure 32. Example of packet processing pipeline. The input ports 0 and 1 are connected with the output ports 0, 1 and 2 through tables 0 and 1. <pg_figure_32>` +:numref:`figure_vhost_net_arch2` :ref:`figure_vhost_net_arch2` -:ref:`Figure 33. Sequence of steps for hash table operations in packet processing context <pg_figure_33>` +:numref:`figure_kni_traffic_flow` :ref:`figure_kni_traffic_flow` -:ref:`Figure 34. Data structures for configurable key size hash tables <pg_figure_34>` -:ref:`Figure 35. Bucket search pipeline for key lookup operation (configurable key size hash tables) <pg_figure_35>` +:numref:`figure_pkt_proc_pipeline_qos` :ref:`figure_pkt_proc_pipeline_qos` -:ref:`Figure 36. Pseudo-code for match, match_many and match_pos <pg_figure_36>` +:numref:`figure_hier_sched_blk` :ref:`figure_hier_sched_blk` -:ref:`Figure 37. Data structures for 8-byte key hash tables <pg_figure_37>` +:numref:`figure_sched_hier_per_port` :ref:`figure_sched_hier_per_port` -:ref:`Figure 38. Data structures for 16-byte key hash tables <pg_figure_38>` +:numref:`figure_data_struct_per_port` :ref:`figure_data_struct_per_port` + +:numref:`figure_prefetch_pipeline` :ref:`figure_prefetch_pipeline` + +:numref:`figure_pipe_prefetch_sm` :ref:`figure_pipe_prefetch_sm` + +:numref:`figure_blk_diag_dropper` :ref:`figure_blk_diag_dropper` + +:numref:`figure_flow_tru_droppper` :ref:`figure_flow_tru_droppper` + +:numref:`figure_ex_data_flow_tru_dropper` :ref:`figure_ex_data_flow_tru_dropper` + +:numref:`figure_pkt_drop_probability` :ref:`figure_pkt_drop_probability` + +:numref:`figure_drop_probability_graph` :ref:`figure_drop_probability_graph` + +:numref:`figure_figure32` :ref:`figure_figure32` + +:numref:`figure_figure33` :ref:`figure_figure33` + +:numref:`figure_figure34` :ref:`figure_figure34` + +:numref:`figure_figure35` :ref:`figure_figure35` + +:numref:`figure_figure37` :ref:`figure_figure37` + +:numref:`figure_figure38` :ref:`figure_figure38` + +:numref:`figure_figure39` :ref:`figure_figure39` -:ref:`Figure 39. Bucket search pipeline for key lookup operation (single key size hash tables) <pg_figure_39>` **Tables** diff --git a/doc/guides/prog_guide/ivshmem_lib.rst b/doc/guides/prog_guide/ivshmem_lib.rst index 75175fa..9401ccf 100644 --- a/doc/guides/prog_guide/ivshmem_lib.rst +++ b/doc/guides/prog_guide/ivshmem_lib.rst @@ -43,9 +43,11 @@ they are automatically recognized by the DPDK Environment Abstraction Layer (EAL A typical DPDK IVSHMEM use case looks like the following. -.. image28_png has been renamed -|ivshmem| +.. figure:: img/ivshmem.* + + Typical Ivshmem use case + The same could work with several virtual machines, providing host-to-VM or VM-to-VM communication. The maximum number of metadata files is 32 (by default) and each metadata file can contain different (or even the same) hugepages. @@ -154,5 +156,3 @@ It is important to note that once QEMU is started, it holds on to the hugepages As a result, if the user wishes to shut down or restart the IVSHMEM host application, it is not enough to simply shut the application down. The virtual machine must also be shut down (if not, it will hold onto outdated host data). - -.. |ivshmem| image:: img/ivshmem.* diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index bab376a..fb12d43 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -42,15 +42,14 @@ The benefits of using the DPDK KNI are: * Allows an interface with the kernel network stack. -The components of an application using the DPDK Kernel NIC Interface are shown in Figure 17. +The components of an application using the DPDK Kernel NIC Interface are shown in :numref:`figure_kernel_nic_intf`. -.. _pg_figure_17: +.. _figure_kernel_nic_intf: -**Figure 17. Components of a DPDK KNI Application** +.. figure:: img/kernel_nic_intf.* -.. image43_png has been renamed + Components of a DPDK KNI Application -|kernel_nic_intf| The DPDK KNI Kernel Module -------------------------- @@ -114,15 +113,14 @@ To minimize the amount of DPDK code running in kernel space, the mbuf mempool is The kernel module will be aware of mbufs, but all mbuf allocation and free operations will be handled by the DPDK application only. -Figure 18 shows a typical scenario with packets sent in both directions. +:numref:`figure_pkt_flow_kni` shows a typical scenario with packets sent in both directions. -.. _pg_figure_18: +.. _figure_pkt_flow_kni: -**Figure 18. Packet Flow via mbufs in the DPDK KNI** +.. figure:: img/pkt_flow_kni.* -.. image44_png has been renamed + Packet Flow via mbufs in the DPDK KNI -|pkt_flow_kni| Use Case: Ingress ----------------- @@ -189,13 +187,12 @@ it naturally supports both legacy virtio -net and the DPDK PMD virtio. There is a little penalty that comes from the non-polling mode of vhost. However, it scales throughput well when using KNI in multi-thread mode. -.. _pg_figure_19: +.. _figure_vhost_net_arch2: -**Figure 19. vHost-net Architecture Overview** +.. figure:: img/vhost_net_arch.* -.. image45_png has been renamed + vHost-net Architecture Overview -|vhost_net_arch| Packet Flow ~~~~~~~~~~~ @@ -208,13 +205,12 @@ All the packet copying, irrespective of whether it is on the transmit or receive happens in the context of vhost kthread. Every vhost-net device is exposed to a front end virtio device in the guest. -.. _pg_figure_20: +.. _figure_kni_traffic_flow: -**Figure 20. KNI Traffic Flow** +.. figure:: img/kni_traffic_flow.* -.. image46_png has been renamed + KNI Traffic Flow -|kni_traffic_flow| Sample Usage ~~~~~~~~~~~~ @@ -280,11 +276,3 @@ since the kni-vhost does not yet support those features. Even if the option is turned on, kni-vhost will ignore the information that the header contains. When working with legacy virtio on the guest, it is better to turn off unsupported offload features using ethtool -K. Otherwise, there may be problems such as an incorrect L4 checksum error. - -.. |kni_traffic_flow| image:: img/kni_traffic_flow.* - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |pkt_flow_kni| image:: img/pkt_flow_kni.* - -.. |kernel_nic_intf| image:: img/kernel_nic_intf.* diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst index ae9b516..96e554f 100644 --- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst +++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst @@ -35,7 +35,10 @@ In addition to Poll Mode Drivers (PMDs) for physical and virtual hardware, DPDK also includes a pure-software library that allows physical PMD's to be bonded together to create a single logical PMD. -|bond-overview| +.. figure:: img/bond-overview.* + + Bonded PMDs + The Link Bonding PMD library(librte_pmd_bond) supports bonding of groups of ``rte_eth_dev`` ports of the same speed and duplex to provide @@ -62,7 +65,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Round-Robin (Mode 0):** -|bond-mode-0| +.. figure:: img/bond-mode-0.* + + Round-Robin (Mode 0) + This mode provides load balancing and fault tolerance by transmission of packets in sequential order from the first available slave device through @@ -72,7 +78,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Active Backup (Mode 1):** -|bond-mode-1| +.. figure:: img/bond-mode-1.* + + Active Backup (Mode 1) + In this mode only one slave in the bond is active at any time, a different slave becomes active if, and only if, the primary active slave fails, @@ -82,7 +91,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Balance XOR (Mode 2):** -|bond-mode-2| +.. figure:: img/bond-mode-2.* + + Balance XOR (Mode 2) + This mode provides transmit load balancing (based on the selected transmission policy) and fault tolerance. The default policy (layer2) uses @@ -101,14 +113,20 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Broadcast (Mode 3):** -|bond-mode-3| +.. figure:: img/bond-mode-3.* + + Broadcast (Mode 3) + This mode provides fault tolerance by transmission of packets on all slave ports. * **Link Aggregation 802.3AD (Mode 4):** -|bond-mode-4| +.. figure:: img/bond-mode-4.* + + Link Aggregation 802.3AD (Mode 4) + This mode provides dynamic link aggregation according to the 802.3ad specification. It negotiates and monitors aggregation groups that share the @@ -128,7 +146,10 @@ Currently the Link Bonding PMD library supports 4 modes of operation: * **Transmit Load Balancing (Mode 5):** -|bond-mode-5| +.. figure:: img/bond-mode-5.* + + Transmit Load Balancing (Mode 5) + This mode provides an adaptive transmit load balancing. It dynamically changes the transmitting slave, according to the computed load. Statistics @@ -433,11 +454,3 @@ Create a bonded device in balance mode with two slaves specified by their PCI ad .. code-block:: console $RTE_TARGET/app/testpmd -c '0xf' -n 4 --vdev 'eth_bond0,mode=2, slave=0000:00a:00.01,slave=0000:004:00.00,xmit_policy=l34' -- --port-topology=chained - -.. |bond-overview| image:: img/bond-overview.* -.. |bond-mode-0| image:: img/bond-mode-0.* -.. |bond-mode-1| image:: img/bond-mode-1.* -.. |bond-mode-2| image:: img/bond-mode-2.* -.. |bond-mode-3| image:: img/bond-mode-3.* -.. |bond-mode-4| image:: img/bond-mode-4.* -.. |bond-mode-5| image:: img/bond-mode-5.* diff --git a/doc/guides/prog_guide/lpm6_lib.rst b/doc/guides/prog_guide/lpm6_lib.rst index abc5adb..87f5066 100644 --- a/doc/guides/prog_guide/lpm6_lib.rst +++ b/doc/guides/prog_guide/lpm6_lib.rst @@ -108,9 +108,11 @@ This is not feasible due to resource restrictions. By splitting the process in different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access per level). -.. image40_png has been renamed -|tbl24_tbl8_tbl8| +.. figure:: img/tbl24_tbl8_tbl8.* + + Table split into different levels + An entry in a table contains the following fields: @@ -231,5 +233,3 @@ Use Case: IPv6 Forwarding ------------------------- The LPM algorithm is used to implement the Classless Inter-Domain Routing (CIDR) strategy used by routers implementing IP forwarding. - -.. |tbl24_tbl8_tbl8| image:: img/tbl24_tbl8_tbl8.* diff --git a/doc/guides/prog_guide/lpm_lib.rst b/doc/guides/prog_guide/lpm_lib.rst index 692e37f..c33e469 100644 --- a/doc/guides/prog_guide/lpm_lib.rst +++ b/doc/guides/prog_guide/lpm_lib.rst @@ -90,9 +90,11 @@ Instead, this approach takes advantage of the fact that rules longer than 24 bit By splitting the process in two different tables/levels and limiting the number of tbl8s, we can greatly reduce memory consumption while maintaining a very good lookup speed (one memory access, most of the times). -.. image39 has been renamed -|tbl24_tbl8| +.. figure:: img/tbl24_tbl8.* + + Table split into different levels + An entry in tbl24 contains the following fields: @@ -219,5 +221,3 @@ References * Pankaj Gupta, Algorithms for Routing Lookups and Packet Classification, PhD Thesis, Stanford University, 2000 (`http://klamath.stanford.edu/~pankaj/thesis/ thesis_1sided.pdf <http://klamath.stanford.edu/~pankaj/thesis/%20thesis_1sided.pdf>`_ ) - -.. |tbl24_tbl8| image:: img/tbl24_tbl8.* diff --git a/doc/guides/prog_guide/malloc_lib.rst b/doc/guides/prog_guide/malloc_lib.rst index b9298f8..6418fab 100644 --- a/doc/guides/prog_guide/malloc_lib.rst +++ b/doc/guides/prog_guide/malloc_lib.rst @@ -117,13 +117,12 @@ The key fields of the heap structure and their function are described below (see since these are never touched except when they are to be freed again - at which point the pointer to the block is an input to the free() function. -.. _pg_figure_3: +.. _figure_malloc_heap: -**Figure 3. Example of a malloc heap and malloc elements within the malloc library** +.. figure:: img/malloc_heap.* -.. image4_png has been renamed + Example of a malloc heap and malloc elements within the malloc library -|malloc_heap| Structure: malloc_elem ^^^^^^^^^^^^^^^^^^^^^^ @@ -232,5 +231,3 @@ These next and previous elements are then checked to see if they too are free, and if so, they are merged with the current elements. This means that we can never have two free memory blocks adjacent to one another, they are always merged into a single block. - -.. |malloc_heap| image:: img/malloc_heap.* diff --git a/doc/guides/prog_guide/mbuf_lib.rst b/doc/guides/prog_guide/mbuf_lib.rst index 338f7da..32a041e 100644 --- a/doc/guides/prog_guide/mbuf_lib.rst +++ b/doc/guides/prog_guide/mbuf_lib.rst @@ -71,23 +71,21 @@ Message buffers may be used to carry control information, packets, events, and so on between different entities in the system. Message buffers may also use their buffer pointers to point to other message buffer data sections or other structures. -Figure 8 and Figure 9 show some of these scenarios. +:numref:`figure_mbuf1` and :numref:`figure_mbuf2` show some of these scenarios. -.. _pg_figure_8: +.. _figure_mbuf1: -**Figure 8. An mbuf with One Segment** +.. figure:: img/mbuf1.* -.. image22_png has been replaced + An mbuf with One Segment -|mbuf1| -.. _pg_figure_9: +.. _figure_mbuf2: -**Figure 9. An mbuf with Three Segments** +.. figure:: img/mbuf2.* -.. image23_png has been replaced + An mbuf with Three Segments -|mbuf2| The Buffer Manager implements a fairly standard set of buffer access functions to manipulate network packets. @@ -277,7 +275,3 @@ Use Cases --------- All networking application should use mbufs to transport network packets. - -.. |mbuf1| image:: img/mbuf1.* - -.. |mbuf2| image:: img/mbuf2.* diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst index f9b7cfe..f0ca06f 100644 --- a/doc/guides/prog_guide/mempool_lib.rst +++ b/doc/guides/prog_guide/mempool_lib.rst @@ -74,28 +74,27 @@ When running an application, the EAL command line options provide the ability to The command line must always have the number of memory channels specified for the processor. -Examples of alignment for different DIMM architectures are shown in Figure 5 and Figure 6. +Examples of alignment for different DIMM architectures are shown in +:numref:`figure_memory-management` and :numref:`figure_memory-management2`. -.. _pg_figure_5: +.. _figure_memory-management: -**Figure 5. Two Channels and Quad-ranked DIMM Example** +.. figure:: img/memory-management.* -.. image19_png has been replaced + Two Channels and Quad-ranked DIMM Example -|memory-management| In this case, the assumption is that a packet is 16 blocks of 64 bytes, which is not true. The Intel® 5520 chipset has three channels, so in most cases, no padding is required between objects (except for objects whose size are n x 3 x 64 bytes blocks). -.. _pg_figure_6: +.. _figure_memory-management2: -**Figure 6. Three Channels and Two Dual-ranked DIMM Example** +.. figure:: img/memory-management2.* -.. image20_png has been replaced + Three Channels and Two Dual-ranked DIMM Example -|memory-management2| When creating a new pool, the user can specify to use this feature or not. @@ -119,15 +118,14 @@ This cache can be enabled or disabled at creation of the pool. The maximum size of the cache is static and is defined at compilation time (CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE). -Figure 7 shows a cache in operation. +:numref:`figure_mempool` shows a cache in operation. -.. _pg_figure_7: +.. _figure_mempool: -**Figure 7. A mempool in Memory with its Associated Ring** +.. figure:: img/mempool.* -.. image21_png has been replaced + A mempool in Memory with its Associated Ring -|mempool| Use Cases --------- @@ -140,9 +138,3 @@ Below are some examples: * :ref:`Environment Abstraction Layer <Environment_Abstraction_Layer>` , for logging service * Any application that needs to allocate fixed-sized objects in the data plane and that will be continuously utilized by the system. - -.. |memory-management| image:: img/memory-management.* - -.. |memory-management2| image:: img/memory-management2.* - -.. |mempool| image:: img/mempool.* diff --git a/doc/guides/prog_guide/multi_proc_support.rst b/doc/guides/prog_guide/multi_proc_support.rst index 25a6056..6562f0d 100644 --- a/doc/guides/prog_guide/multi_proc_support.rst +++ b/doc/guides/prog_guide/multi_proc_support.rst @@ -83,13 +83,12 @@ and point to the same objects, in both processes. Refer to Section 23.3 "Multi-process Limitations" for details of how Linux kernel Address-Space Layout Randomization (ASLR) can affect memory sharing. -.. _pg_figure_16: +.. _figure_multi_process_memory: -**Figure 16. Memory Sharing in the DPDK Multi-process Sample Application** +.. figure:: img/multi_process_memory.* -.. image42_png has been replaced + Memory Sharing in the DPDK Multi-process Sample Application -|multi_process_memory| The EAL also supports an auto-detection mode (set by EAL --proc-type=auto flag ), whereby an DPDK process is started as a secondary instance if a primary instance is already running. @@ -199,5 +198,3 @@ instead of the functions which do the hashing internally, such as rte_hash_add() which means that only the first, primary DPDK process instance can open and mmap /dev/hpet. If the number of required DPDK processes exceeds that of the number of available HPET comparators, the TSC (which is the default timer in this release) must be used as a time source across all processes instead of the HPET. - -.. |multi_process_memory| image:: img/multi_process_memory.* diff --git a/doc/guides/prog_guide/overview.rst b/doc/guides/prog_guide/overview.rst index 062d923..cef6ca7 100644 --- a/doc/guides/prog_guide/overview.rst +++ b/doc/guides/prog_guide/overview.rst @@ -120,13 +120,12 @@ Core Components The *core components* are a set of libraries that provide all the elements needed for high-performance packet processing applications. -.. _pg_figure_1: +.. _figure_architecture-overview: -**Figure 1. Core Components Architecture** +.. figure:: img/architecture-overview.* -.. image2_png has been replaced + Core Components Architecture -|architecture-overview| Memory Manager (librte_malloc) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -203,5 +202,3 @@ librte_net The librte_net library is a collection of IP protocol definitions and convenience macros. It is based on code from the FreeBSD* IP stack and contains protocol numbers (for use in IP headers), IP-related macros, IPv4/IPv6 header structures and TCP, UDP and SCTP header structures. - -.. |architecture-overview| image:: img/architecture-overview.* diff --git a/doc/guides/prog_guide/packet_distrib_lib.rst b/doc/guides/prog_guide/packet_distrib_lib.rst index 767accc..b5bdabb 100644 --- a/doc/guides/prog_guide/packet_distrib_lib.rst +++ b/doc/guides/prog_guide/packet_distrib_lib.rst @@ -38,7 +38,10 @@ which is responsible for load balancing or distributing packets, and a set of worker lcores which are responsible for receiving the packets from the distributor and operating on them. The model of operation is shown in the diagram below. -|packet_distributor1| +.. figure:: img/packet_distributor1.* + + Packet Distributor mode of operation + Distributor Core Operation -------------------------- @@ -91,9 +94,11 @@ No packet ordering guarantees are made about packets which do not share a common Using the process and returned_pkts API, the following application workflow can be used, while allowing packet order within a packet flow -- identified by a tag -- to be maintained. -.. image41_png has been renamed -|packet_distributor2| +.. figure:: img/packet_distributor2.* + + Application workflow + The flush and clear_returns API calls, mentioned previously, are likely of less use that the process and returned_pkts APIS, and are principally provided to aid in unit testing of the library. @@ -110,7 +115,3 @@ Since it may be desirable to vary the number of worker cores, depending on the t i.e. to save power at times of lighter load, it is possible to have a worker stop processing packets by calling "rte_distributor_return_pkt()" to indicate that it has finished the current packet and does not want a new one. - -.. |packet_distributor1| image:: img/packet_distributor1.* - -.. |packet_distributor2| image:: img/packet_distributor2.* diff --git a/doc/guides/prog_guide/packet_framework.rst b/doc/guides/prog_guide/packet_framework.rst index 2056c4f..417f4db 100644 --- a/doc/guides/prog_guide/packet_framework.rst +++ b/doc/guides/prog_guide/packet_framework.rst @@ -66,15 +66,15 @@ one of the table entries (on lookup hit) or the default table entry (on lookup m provides the set of actions to be applied on the current packet, as well as the next hop for the packet, which can be either another table, an output port or packet drop. -An example of packet processing pipeline is presented in Figure 32: +An example of packet processing pipeline is presented in :numref:`figure_figure32`: -.. _pg_figure_32: +.. _figure_figure32: -**Figure 32 Example of Packet Processing Pipeline where Input Ports 0 and 1 are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1** +.. figure:: img/figure32.* -.. Object_1_png has been renamed + Example of Packet Processing Pipeline where Input Ports 0 and 1 + are Connected with Output Ports 0, 1 and 2 through Tables 0 and 1 -|figure32| Port Library Design ------------------- @@ -344,13 +344,14 @@ considering *n_bits* as the number of bits set in *bucket_mask = n_buckets - 1*, this means that all the keys that end up in the same hash table bucket have the lower *n_bits* of their signature identical. In order to reduce the number of keys in the same bucket (collisions), the number of hash table buckets needs to be increased. -In packet processing context, the sequence of operations involved in hash table operations is described in Figure 33: +In packet processing context, the sequence of operations involved in hash table operations is described in :numref:`figure_figure33`: -.. _pg_figure_33: +.. _figure_figure33: -**Figure 33 Sequence of Steps for Hash Table Operations in a Packet Processing Context** +.. figure:: img/figure33.* + + Sequence of Steps for Hash Table Operations in a Packet Processing Context -|figure33| Hash Table Use Cases @@ -553,16 +554,15 @@ This avoids the important cost associated with flushing the CPU core execution p Configurable Key Size Hash Table """""""""""""""""""""""""""""""" -Figure 34, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, +:numref:`figure_figure34`, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_34: +.. _figure_figure34: -**Figure 34 Data Structures for Configurable Key Size Hash Tables** +.. figure:: img/figure34.* -.. image65_png has been renamed + Data Structures for Configurable Key Size Hash Tables -|figure34| .. _pg_table_25: @@ -627,15 +627,17 @@ either with pre-computed signature or "do-sig"). +---+------------------+--------------------+------------------------------------------------------------------+ -Figure 35 and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, +:numref:`figure_figure35` and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_35: +.. _figure_figure35: + +.. figure:: img/figure35.* -**Figure 35 Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash Tables)** + Bucket Search Pipeline for Key Lookup Operation (Configurable Key Size Hash + Tables) -|figure35| .. _pg_table_27: @@ -814,11 +816,8 @@ Given the input *mask*, the values for *match*, *match_many* and *match_pos* can | | | | +------------+------------------------------------------+-------------------+ -The pseudo-code is displayed in Figure 36. - -.. _pg_figure_36: -**Figure 36 Pseudo-code for match, match_many and match_pos** +The pseudo-code for match, match_many and match_pos is:: match = (0xFFFELLU >> mask) & 1; @@ -829,24 +828,22 @@ The pseudo-code is displayed in Figure 36. Single Key Size Hash Tables """"""""""""""""""""""""""" -Figure 37, Figure 38, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables +:numref:`figure_figure37`, :numref:`figure_figure38`, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). -.. _pg_figure_37: +.. _figure_figure37: -**Figure 37 Data Structures for 8-byte Key Hash Tables** +.. figure:: img/figure37.* -.. image66_png has been renamed + Data Structures for 8-byte Key Hash Tables -|figure37| -.. _pg_figure_38: +.. _figure_figure38: -**Figure 38 Data Structures for 16-byte Key Hash Tables** +.. figure:: img/figure38.* -.. image67_png has been renamed + Data Structures for 16-byte Key Hash Tables -|figure38| .. _pg_table_30: @@ -914,11 +911,13 @@ and detail the bucket search pipeline used to implement 8-byte and 16-byte key h either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. -.. _pg_figure_39: +.. _figure_figure39: -**Figure 39 Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash Tables)** +.. figure:: img/figure39.* + + Bucket Search Pipeline for Key Lookup Operation (Single Key Size Hash + Tables) -|figure39| .. _pg_table_32: @@ -1167,17 +1166,3 @@ Usually, to support a specific functional block, specific implementation of Pack with all the implementations sharing the same API: pure SW implementation (no acceleration), implementation using accelerator A, implementation using accelerator B, etc. The selection between these implementations could be done at build time or at run-time (recommended), based on which accelerators are present in the system, with no application changes required. - -.. |figure33| image:: img/figure33.* - -.. |figure35| image:: img/figure35.* - -.. |figure39| image:: img/figure39.* - -.. |figure34| image:: img/figure34.* - -.. |figure32| image:: img/figure32.* - -.. |figure37| image:: img/figure37.* - -.. |figure38| image:: img/figure38.* diff --git a/doc/guides/prog_guide/qos_framework.rst b/doc/guides/prog_guide/qos_framework.rst index 98d8714..72cfed9 100644 --- a/doc/guides/prog_guide/qos_framework.rst +++ b/doc/guides/prog_guide/qos_framework.rst @@ -38,13 +38,12 @@ Packet Pipeline with QoS Support An example of a complex packet processing pipeline with QoS support is shown in the following figure. -.. _pg_figure_21: +.. _figure_pkt_proc_pipeline_qos: -**Figure 21. Complex Packet Processing Pipeline with QoS Support** +.. figure:: img/pkt_proc_pipeline_qos.* -.. image47_png has been renamed + Complex Packet Processing Pipeline with QoS Support -|pkt_proc_pipeline_qos| This pipeline can be built using reusable DPDK software libraries. The main blocks implementing QoS in this pipeline are: the policer, the dropper and the scheduler. @@ -139,13 +138,12 @@ It typically acts like a buffer that is able to temporarily store a large number as the NIC TX is requesting more packets for transmission, these packets are later on removed and handed over to the NIC TX with the packet selection logic observing the predefined SLAs (dequeue operation). -.. _pg_figure_22: +.. _figure_hier_sched_blk: -**Figure 22. Hierarchical Scheduler Block Internal Diagram** +.. figure:: img/hier_sched_blk.* -.. image48_png has been renamed + Hierarchical Scheduler Block Internal Diagram -|hier_sched_blk| The hierarchical scheduler is optimized for a large number of packet queues. When only a small number of queues are needed, message passing queues should be used instead of this block. @@ -154,7 +152,7 @@ See Section 26.2.5 "Worst Case Scenarios for Performance" for a more detailed di Scheduling Hierarchy ~~~~~~~~~~~~~~~~~~~~ -The scheduling hierarchy is shown in Figure 23. +The scheduling hierarchy is shown in :numref:`figure_sched_hier_per_port`. The first level of the hierarchy is the Ethernet TX port 1/10/40 GbE, with subsequent hierarchy levels defined as subport, pipe, traffic class and queue. @@ -163,13 +161,12 @@ Each traffic class is the representation of a different traffic type with specif delay and jitter requirements, such as voice, video or data transfers. Each queue hosts packets from one or multiple connections of the same type belonging to the same user. -.. _pg_figure_23: +.. _figure_sched_hier_per_port: -**Figure 23. Scheduling Hierarchy per Port** +.. figure:: img/sched_hier_per_port.* -.. image49_png has been renamed + Scheduling Hierarchy per Port -|sched_hier_per_port| The functionality of each hierarchical level is detailed in the following table. @@ -293,13 +290,12 @@ Internal Data Structures per Port A schematic of the internal data structures in shown in with details in. -.. _pg_figure_24: +.. _figure_data_struct_per_port: -**Figure 24. Internal Data Structures per Port** +.. figure:: img/data_struct_per_port.* -.. image50_png has been renamed + Internal Data Structures per Port -|data_struct_per_port| .. _pg_table_4: @@ -434,16 +430,15 @@ the processor should not attempt to access the data structure currently under pr The only other work available is to execute different stages of the enqueue sequence of operations on other input packets, thus resulting in a pipelined implementation for the enqueue operation. -Figure 25 illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. +:numref:`figure_prefetch_pipeline` illustrates a pipelined implementation for the enqueue operation with 4 pipeline stages and each stage executing 2 different input packets. No input packet can be part of more than one pipeline stage at a given time. -.. _pg_figure_25: +.. _figure_prefetch_pipeline: -**Figure 25. Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation** +.. figure:: img/prefetch_pipeline.* -.. image51 has been renamed + Prefetch Pipeline for the Hierarchical Scheduler Enqueue Operation -|prefetch_pipeline| The congestion management scheme implemented by the enqueue pipeline described above is very basic: packets are enqueued until a specific queue becomes full, @@ -478,13 +473,13 @@ The dequeue pipe state machine exploits the data presence into the processor cac therefore it tries to send as many packets from the same pipe TC and pipe as possible (up to the available packets and credits) before moving to the next active TC from the same pipe (if any) or to another active pipe. -.. _pg_figure_26: +.. _figure_pipe_prefetch_sm: -**Figure 26. Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue Operation** +.. figure:: img/pipe_prefetch_sm.* -.. image52 has been renamed + Pipe Prefetch State Machine for the Hierarchical Scheduler Dequeue + Operation -|pipe_prefetch_sm| Timing and Synchronization ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1173,17 +1168,16 @@ Dropper The purpose of the DPDK dropper is to drop packets arriving at a packet scheduler to avoid congestion. The dropper supports the Random Early Detection (RED), Weighted Random Early Detection (WRED) and tail drop algorithms. -Figure 1 illustrates how the dropper integrates with the scheduler. +:numref:`figure_blk_diag_dropper` illustrates how the dropper integrates with the scheduler. The DPDK currently does not support congestion management so the dropper provides the only method for congestion avoidance. -.. _pg_figure_27: +.. _figure_blk_diag_dropper: -**Figure 27. High-level Block Diagram of the DPDK Dropper** +.. figure:: img/blk_diag_dropper.* -.. image53_png has been renamed + High-level Block Diagram of the DPDK Dropper -|blk_diag_dropper| The dropper uses the Random Early Detection (RED) congestion avoidance algorithm as documented in the reference publication. The purpose of the RED algorithm is to monitor a packet queue, @@ -1202,16 +1196,15 @@ In the case of severe congestion, the dropper resorts to tail drop. This occurs when a packet queue has reached maximum capacity and cannot store any more packets. In this situation, all arriving packets are dropped. -The flow through the dropper is illustrated in Figure 28. +The flow through the dropper is illustrated in :numref:`figure_flow_tru_droppper`. The RED/WRED algorithm is exercised first and tail drop second. -.. _pg_figure_28: +.. _figure_flow_tru_droppper: -**Figure 28. Flow Through the Dropper** +.. figure:: img/flow_tru_droppper.* -.. image54_png has been renamed + Flow Through the Dropper -|flow_tru_droppper| The use cases supported by the dropper are: @@ -1270,17 +1263,16 @@ for example, a filter weight parameter value of 9 corresponds to a filter weight Enqueue Operation ~~~~~~~~~~~~~~~~~ -In the example shown in Figure 29, q (actual queue size) is the input value, +In the example shown in :numref:`figure_ex_data_flow_tru_dropper`, q (actual queue size) is the input value, avg (average queue size) and count (number of packets since the last drop) are run-time values, decision is the output value and the remaining values are configuration parameters. -.. _pg_figure_29: +.. _figure_ex_data_flow_tru_dropper: -**Figure 29. Example Data Flow Through Dropper** +.. figure:: img/ex_data_flow_tru_dropper.* -.. image55_png has been renamed + Example Data Flow Through Dropper -|ex_data_flow_tru_dropper| EWMA Filter Microblock ^^^^^^^^^^^^^^^^^^^^^^ @@ -1298,11 +1290,7 @@ Average Queue Size Calculation when the Queue is not Empty The definition of the EWMA filter is given in the following equation. -**Equation 1.** - -.. image56_png has been renamed - -|ewma_filter_eq_1| +.. image:: img/ewma_filter_eq_1.* Where: @@ -1326,11 +1314,7 @@ When the queue becomes empty, average queue size should decay gradually to zero or remaining stagnant at the last computed value. When a packet is enqueued on an empty queue, the average queue size is computed using the following formula: -**Equation 2.** - -.. image57_png has been renamed - -|ewma_filter_eq_2| +.. image:: img/ewma_filter_eq_2.* Where: @@ -1338,9 +1322,7 @@ Where: In the dropper module, *m* is defined as: -.. image58_png has been renamed - -|m_definition| +.. image:: img/m_definition.* Where: @@ -1374,15 +1356,13 @@ A numerical method is used to compute the factor (1-wq)^m that appears in Equati This method is based on the following identity: -.. image59_png has been renamed +.. image:: img/eq2_factor.* -|eq2_factor| This allows us to express the following: -.. image60_png has been renamed +.. image:: img/eq2_expression.* -|eq2_expression| In the dropper module, a look-up table is used to compute log2(1-wq) for each value of wq supported by the dropper module. The factor (1-wq)^m can then be obtained by multiplying the table value by *m* and applying shift operations. @@ -1465,11 +1445,7 @@ Initial Packet Drop Probability The initial drop probability is calculated using the following equation. -**Equation 3.** - -.. image61_png has been renamed - -|drop_probability_eq3| +.. image:: img/drop_probability_eq3.* Where: @@ -1481,19 +1457,18 @@ Where: * *maxth* = maximum threshold -The calculation of the packet drop probability using Equation 3 is illustrated in Figure 30. +The calculation of the packet drop probability using Equation 3 is illustrated in :numref:`figure_pkt_drop_probability`. If the average queue size is below the minimum threshold, an arriving packet is enqueued. If the average queue size is at or above the maximum threshold, an arriving packet is dropped. If the average queue size is between the minimum and maximum thresholds, a drop probability is calculated to determine if the packet should be enqueued or dropped. -.. _pg_figure_30: +.. _figure_pkt_drop_probability: -**Figure 30. Packet Drop Probability for a Given RED Configuration** +.. figure:: img/pkt_drop_probability.* -.. image62_png has been renamed + Packet Drop Probability for a Given RED Configuration -|pkt_drop_probability| Actual Drop Probability """"""""""""""""""""""" @@ -1501,11 +1476,7 @@ Actual Drop Probability If the average queue size is between the minimum and maximum thresholds, then the actual drop probability is calculated from the following equation. -**Equation 4.** - -.. image63_png has been renamed - -|drop_probability_eq4| +.. image:: img/drop_probability_eq4.* Where: @@ -1518,7 +1489,7 @@ given in the reference document where a value of 1 is used instead. It should be noted that the value pa computed from can be negative or greater than 1. If this is the case, then a value of 1 should be used instead. -The initial and actual drop probabilities are shown in Figure 31. +The initial and actual drop probabilities are shown in :numref:`figure_drop_probability_graph`. The actual drop probability is shown for the case where the formula given in the reference document1 is used (blue curve) and also for the case where the formula implemented in the dropper module, @@ -1528,13 +1499,13 @@ compared to the mark probability configuration parameter specified by the user. The choice to deviate from the reference document is simply a design decision and one that has been taken by other RED implementations, for example, FreeBSD* ALTQ RED. -.. _pg_figure_31: +.. _figure_drop_probability_graph: -**Figure 31. Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using a Factor 1 (Blue Curve) and a Factor 2 (Red Curve)** +.. figure:: img/drop_probability_graph.* -.. image64_png has been renamed + Initial Drop Probability (pb), Actual Drop probability (pa) Computed Using + a Factor 1 (Blue Curve) and a Factor 2 (Red Curve) -|drop_probability_graph| .. _Queue_Empty_Operation: @@ -1727,39 +1698,3 @@ For each input packet, the steps for the srTCM / trTCM algorithms are: the input color of the packet is also considered. When the output color is not red, a number of tokens equal to the length of the IP packet are subtracted from the C or E /P or both buckets, depending on the algorithm and the output color of the packet. - -.. |flow_tru_droppper| image:: img/flow_tru_droppper.* - -.. |drop_probability_graph| image:: img/drop_probability_graph.* - -.. |drop_probability_eq3| image:: img/drop_probability_eq3.* - -.. |eq2_expression| image:: img/eq2_expression.* - -.. |drop_probability_eq4| image:: img/drop_probability_eq4.* - -.. |pkt_drop_probability| image:: img/pkt_drop_probability.* - -.. |pkt_proc_pipeline_qos| image:: img/pkt_proc_pipeline_qos.* - -.. |ex_data_flow_tru_dropper| image:: img/ex_data_flow_tru_dropper.* - -.. |ewma_filter_eq_1| image:: img/ewma_filter_eq_1.* - -.. |ewma_filter_eq_2| image:: img/ewma_filter_eq_2.* - -.. |data_struct_per_port| image:: img/data_struct_per_port.* - -.. |prefetch_pipeline| image:: img/prefetch_pipeline.* - -.. |pipe_prefetch_sm| image:: img/pipe_prefetch_sm.* - -.. |blk_diag_dropper| image:: img/blk_diag_dropper.* - -.. |m_definition| image:: img/m_definition.* - -.. |eq2_factor| image:: img/eq2_factor.* - -.. |sched_hier_per_port| image:: img/sched_hier_per_port.* - -.. |hier_sched_blk| image:: img/hier_sched_blk.* diff --git a/doc/guides/prog_guide/ring_lib.rst b/doc/guides/prog_guide/ring_lib.rst index 8547b38..3b92a8f 100644 --- a/doc/guides/prog_guide/ring_lib.rst +++ b/doc/guides/prog_guide/ring_lib.rst @@ -72,13 +72,12 @@ The disadvantages: A simplified representation of a Ring is shown in with consumer and producer head and tail pointers to objects stored in the data structure. -.. _pg_figure_4: +.. _figure_ring1: -**Figure 4. Ring Structure** +.. figure:: img/ring1.* -.. image5_png has been replaced + Ring Structure -|ring1| References for Ring Implementation in FreeBSD* ---------------------------------------------- @@ -155,9 +154,13 @@ The prod_next local variable points to the next element of the table, or several If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image6_png has been replaced -|ring-enqueue1| +.. _figure_ring-enqueue1: + +.. figure:: img/ring-enqueue1.* + + Enqueue first step + Enqueue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -166,9 +169,13 @@ The second step is to modify *ring->prod_head* in ring structure to point to the A pointer to the added object is copied in the ring (obj4). -.. image7_png has been replaced -|ring-enqueue2| +.. _figure_ring-enqueue2: + +.. figure:: img/ring-enqueue2.* + + Enqueue second step + Enqueue Last Step ^^^^^^^^^^^^^^^^^ @@ -176,9 +183,13 @@ Enqueue Last Step Once the object is added in the ring, ring->prod_tail in the ring structure is modified to point to the same location as *ring->prod_head*. The enqueue operation is finished. -.. image8_png has been replaced -|ring-enqueue3| +.. _figure_ring-enqueue3: + +.. figure:: img/ring-enqueue3.* + + Enqueue last step + Single Consumer Dequeue ~~~~~~~~~~~~~~~~~~~~~~~ @@ -196,9 +207,13 @@ The cons_next local variable points to the next element of the table, or several If there are not enough objects in the ring (this is detected by checking prod_tail), it returns an error. -.. image9_png has been replaced -|ring-dequeue1| +.. _figure_ring-dequeue1: + +.. figure:: img/ring-dequeue1.* + + Dequeue last step + Dequeue Second Step ^^^^^^^^^^^^^^^^^^^ @@ -207,9 +222,13 @@ The second step is to modify ring->cons_head in the ring structure to point to t The pointer to the dequeued object (obj1) is copied in the pointer given by the user. -.. image10_png has been replaced -|ring-dequeue2| +.. _figure_ring-dequeue2: + +.. figure:: img/ring-dequeue2.* + + Dequeue second step + Dequeue Last Step ^^^^^^^^^^^^^^^^^ @@ -217,9 +236,13 @@ Dequeue Last Step Finally, ring->cons_tail in the ring structure is modified to point to the same location as ring->cons_head. The dequeue operation is finished. -.. image11_png has been replaced -|ring-dequeue3| +.. _figure_ring-dequeue3: + +.. figure:: img/ring-dequeue3.* + + Dequeue last step + Multiple Producers Enqueue ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -229,8 +252,8 @@ In this example, only the producer head and tail (prod_head and prod_tail) are m The initial state is to have a prod_head and prod_tail pointing at the same location. -MC Enqueue First Step -^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue First Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ On both cores, *ring->prod_head* and ring->cons_tail are copied in local variables. The prod_next local variable points to the next element of the table, @@ -238,12 +261,16 @@ or several elements after in the case of bulk enqueue. If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error. -.. image12_png has been replaced -|ring-mp-enqueue1| +.. _figure_ring-mp-enqueue1: + +.. figure:: img/ring-mp-enqueue1.* + + Multiple consumer enqueue first step + -MC Enqueue Second Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Second Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The second step is to modify ring->prod_head in the ring structure to point to the same location as prod_next. This operation is done using a Compare And Swap (CAS) instruction, which does the following operations atomically: @@ -256,41 +283,57 @@ This operation is done using a Compare And Swap (CAS) instruction, which does th In the figure, the operation succeeded on core 1, and step one restarted on core 2. -.. image13_png has been replaced -|ring-mp-enqueue2| +.. _figure_ring-mp-enqueue2: -MC Enqueue Third Step -^^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue2.* + + Multiple consumer enqueue second step + + +Multiple Consumer Enqueue Third Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The CAS operation is retried on core 2 with success. The core 1 updates one element of the ring(obj4), and the core 2 updates another one (obj5). -.. image14_png has been replaced -|ring-mp-enqueue3| +.. _figure_ring-mp-enqueue3: + +.. figure:: img/ring-mp-enqueue3.* + + Multiple consumer enqueue third step + -MC Enqueue Fourth Step -^^^^^^^^^^^^^^^^^^^^^^ +Multiple Consumer Enqueue Fourth Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Each core now wants to update ring->prod_tail. A core can only update it if ring->prod_tail is equal to the prod_head local variable. This is only true on core 1. The operation is finished on core 1. -.. image15_png has been replaced -|ring-mp-enqueue4| +.. _figure_ring-mp-enqueue4: -MC Enqueue Last Step -^^^^^^^^^^^^^^^^^^^^ +.. figure:: img/ring-mp-enqueue4.* + + Multiple consumer enqueue fourth step + + +Multiple Consumer Enqueue Last Step +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Once ring->prod_tail is updated by core 1, core 2 is allowed to update it too. The operation is also finished on core 2. -.. image16_png has been replaced -|ring-mp-enqueue5| +.. _figure_ring-mp-enqueue5: + +.. figure:: img/ring-mp-enqueue5.* + + Multiple consumer enqueue last step + Modulo 32-bit Indexes ~~~~~~~~~~~~~~~~~~~~~ @@ -309,15 +352,23 @@ The following are two examples that help to explain how indexes are used in a ri In addition, the four indexes are defined as unsigned 16-bit integers, as opposed to unsigned 32-bit integers in the more realistic case. -.. image17_png has been replaced -|ring-modulo1| +.. _figure_ring-modulo1: + +.. figure:: img/ring-modulo1.* + + Modulo 32-bit indexes - Example 1 + This ring contains 11000 entries. -.. image18_png has been replaced -|ring-modulo2| +.. _figure_ring-modulo2: + +.. figure:: img/ring-modulo2.* + + Modulo 32-bit indexes - Example 2 + This ring contains 12536 entries. @@ -346,31 +397,3 @@ References * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_ (version 8) * `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_ - -.. |ring1| image:: img/ring1.* - -.. |ring-enqueue1| image:: img/ring-enqueue1.* - -.. |ring-enqueue2| image:: img/ring-enqueue2.* - -.. |ring-enqueue3| image:: img/ring-enqueue3.* - -.. |ring-dequeue1| image:: img/ring-dequeue1.* - -.. |ring-dequeue2| image:: img/ring-dequeue2.* - -.. |ring-dequeue3| image:: img/ring-dequeue3.* - -.. |ring-mp-enqueue1| image:: img/ring-mp-enqueue1.* - -.. |ring-mp-enqueue2| image:: img/ring-mp-enqueue2.* - -.. |ring-mp-enqueue3| image:: img/ring-mp-enqueue3.* - -.. |ring-mp-enqueue4| image:: img/ring-mp-enqueue4.* - -.. |ring-mp-enqueue5| image:: img/ring-mp-enqueue5.* - -.. |ring-modulo1| image:: img/ring-modulo1.* - -.. |ring-modulo2| image:: img/ring-modulo2.* diff --git a/doc/guides/sample_app_ug/dist_app.rst b/doc/guides/sample_app_ug/dist_app.rst index 56195bb..3c9a991 100644 --- a/doc/guides/sample_app_ug/dist_app.rst +++ b/doc/guides/sample_app_ug/dist_app.rst @@ -47,11 +47,12 @@ into each other. This application can be used to benchmark performance using the traffic generator as shown in the figure below. -.. _figure_22: +.. _figure_dist_perf: -**Figure 22. Performance Benchmarking Setup (Basic Environment)** +.. figure:: img/dist_perf.* + + Performance Benchmarking Setup (Basic Environment) -|dist_perf| Compiling the Application ------------------------- @@ -106,7 +107,7 @@ Explanation The distributor application consists of three types of threads: a receive thread (lcore_rx()), a set of worker threads(lcore_worker()) and a transmit thread(lcore_tx()). How these threads work together is shown -in Fig2 below. The main() function launches threads of these three types. +in :numref:`figure_dist_app` below. The main() function launches threads of these three types. Each thread has a while loop which will be doing processing and which is terminated only upon SIGINT or ctrl+C. The receive and transmit threads communicate using a software ring (rte_ring structure). @@ -136,11 +137,12 @@ Users who wish to terminate the running of the application have to press ctrl+C in the application will terminate all running threads gracefully and print final statistics to the user. -.. _figure_23: +.. _figure_dist_app: + +.. figure:: img/dist_app.* -**Figure 23. Distributor Sample Application Layout** + Distributor Sample Application Layout -|dist_app| Debug Logging Support --------------------- @@ -171,7 +173,3 @@ Sample Application. See Section 9.4.4, "RX Queue Initialization". TX queue initialization is done in the same way as it is done in the L2 Forwarding Sample Application. See Section 9.4.5, "TX Queue Initialization". - -.. |dist_perf| image:: img/dist_perf.* - -.. |dist_app| image:: img/dist_app.* diff --git a/doc/guides/sample_app_ug/exception_path.rst b/doc/guides/sample_app_ug/exception_path.rst index 6c06959..3cc7cbe 100644 --- a/doc/guides/sample_app_ug/exception_path.rst +++ b/doc/guides/sample_app_ug/exception_path.rst @@ -46,13 +46,12 @@ The second thread reads from a TAP interface and writes the data unmodified to t The packet flow through the exception path application is as shown in the following figure. -.. _figure_1: +.. _figure_exception_path_example: -**Figure 1. Packet Flow** +.. figure:: img/exception_path_example.* -.. image2_png has been replaced + Packet Flow -|exception_path_example| To make throughput measurements, kernel bridges must be setup to forward data between the bridges appropriately. @@ -327,4 +326,3 @@ To remove bridges and persistent TAP interfaces, the following commands are used brctl delbr br0 openvpn --rmtun --dev tap_dpdk_00 -.. |exception_path_example| image:: img/exception_path_example.* diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index aaa95ef..745a7ac 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -74,57 +74,63 @@ Sample Applications User Guide **Figures** -:ref:`Figure 1.Packet Flow <figure_1>` +:numref:`figure_exception_path_example` :ref:`figure_exception_path_example` -:ref:`Figure 2.Kernel NIC Application Packet Flow <figure_2>` +:numref:`figure_kernel_nic` :ref:`figure_kernel_nic` -:ref:`Figure 3.Performance Benchmark Setup (Basic Environment) <figure_3>` +:numref:`figure_l2_fwd_benchmark_setup_jobstats` :ref:`figure_l2_fwd_benchmark_setup_jobstats` -:ref:`Figure 4.Performance Benchmark Setup (Virtualized Environment) <figure_4>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` :ref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats` -:ref:`Figure 5.Load Balancer Application Architecture <figure_5>` +:numref:`figure_l2_fwd_benchmark_setup` :ref:`figure_l2_fwd_benchmark_setup` -:ref:`Figure 5.Example Rules File <figure_5_1>` +:numref:`figure_l2_fwd_virtenv_benchmark_setup` :ref:`figure_l2_fwd_virtenv_benchmark_setup` -:ref:`Figure 6.Example Data Flow in a Symmetric Multi-process Application <figure_6>` +:numref:`figure_ipv4_acl_rule` :ref:`figure_ipv4_acl_rule` -:ref:`Figure 7.Example Data Flow in a Client-Server Symmetric Multi-process Application <figure_7>` +:numref:`figure_example_rules` :ref:`figure_example_rules` -:ref:`Figure 8.Master-slave Process Workflow <figure_8>` +:numref:`figure_load_bal_app_arch` :ref:`figure_load_bal_app_arch` -:ref:`Figure 9.Slave Process Recovery Process Flow <figure_9>` +:numref:`figure_sym_multi_proc_app` :ref:`figure_sym_multi_proc_app` -:ref:`Figure 10.QoS Scheduler Application Architecture <figure_10>` +:numref:`figure_client_svr_sym_multi_proc_app` :ref:`figure_client_svr_sym_multi_proc_app` -:ref:`Figure 11.Intel®QuickAssist Technology Application Block Diagram <figure_11>` +:numref:`figure_master_slave_proc` :ref:`figure_master_slave_proc` -:ref:`Figure 12.Pipeline Overview <figure_12>` +:numref:`figure_slave_proc_recov` :ref:`figure_slave_proc_recov` -:ref:`Figure 13.Ring-based Processing Pipeline Performance Setup <figure_13>` +:numref:`figure_qos_sched_app_arch` :ref:`figure_qos_sched_app_arch` -:ref:`Figure 14.Threads and Pipelines <figure_14>` +:numref:`figure_quickassist_block_diagram` :ref:`figure_quickassist_block_diagram` -:ref:`Figure 15.Packet Flow Through the VMDQ and DCB Sample Application <figure_15>` +:numref:`figure_pipeline_overview` :ref:`figure_pipeline_overview` -:ref:`Figure 16.QEMU Virtio-net (prior to vhost-net) <figure_16>` +:numref:`figure_ring_pipeline_perf_setup` :ref:`figure_ring_pipeline_perf_setup` -:ref:`Figure 17.Virtio with Linux* Kernel Vhost <figure_17>` +:numref:`figure_threads_pipelines` :ref:`figure_threads_pipelines` -:ref:`Figure 18.Vhost-net Architectural Overview <figure_18>` +:numref:`figure_vmdq_dcb_example` :ref:`figure_vmdq_dcb_example` -:ref:`Figure 19.Packet Flow Through the vhost-net Sample Application <figure_19>` +:numref:`figure_qemu_virtio_net` :ref:`figure_qemu_virtio_net` -:ref:`Figure 20.Packet Flow on TX in DPDK-testpmd <figure_20>` +:numref:`figure_virtio_linux_vhost` :ref:`figure_virtio_linux_vhost` -:ref:`Figure 21.Test Pipeline Application <figure_21>` +:numref:`figure_vhost_net_arch` :ref:`figure_vhost_net_arch` -:ref:`Figure 22.Performance Benchmarking Setup (Basic Environment) <figure_22>` +:numref:`figure_vhost_net_sample_app` :ref:`figure_vhost_net_sample_app` -:ref:`Figure 23.Distributor Sample Application Layout <figure_23>` +:numref:`figure_tx_dpdk_testpmd` :ref:`figure_tx_dpdk_testpmd` -:ref:`Figure 24.High level Solution <figure_24>` +:numref:`figure_test_pipeline_app` :ref:`figure_test_pipeline_app` -:ref:`Figure 25.VM request to scale frequency <figure_25>` +:numref:`figure_dist_perf` :ref:`figure_dist_perf` + +:numref:`figure_dist_app` :ref:`figure_dist_app` + +:numref:`figure_vm_power_mgr_highlevel` :ref:`figure_vm_power_mgr_highlevel` + +:numref:`figure_vm_power_mgr_vm_request_seq` :ref:`figure_vm_power_mgr_vm_request_seq` **Tables** diff --git a/doc/guides/sample_app_ug/intel_quickassist.rst b/doc/guides/sample_app_ug/intel_quickassist.rst index 7f55282..a80d4ca 100644 --- a/doc/guides/sample_app_ug/intel_quickassist.rst +++ b/doc/guides/sample_app_ug/intel_quickassist.rst @@ -46,17 +46,16 @@ For this sample application, there is a dependency on either of: Overview -------- -An overview of the application is provided in Figure 11. +An overview of the application is provided in :numref:`figure_quickassist_block_diagram`. For simplicity, only two NIC ports and one Intel® QuickAssist Technology device are shown in this diagram, although the number of NIC ports and Intel® QuickAssist Technology devices can be different. -.. _figure_11: +.. _figure_quickassist_block_diagram: -**Figure 11. Intel® QuickAssist Technology Application Block Diagram** +.. figure:: img/quickassist_block_diagram.* -.. image14_png has been renamed + Intel® QuickAssist Technology Application Block Diagram -|quickassist_block_diagram| The application allows the configuration of the following items: @@ -220,5 +219,3 @@ performing AES-CBC-128 encryption with AES-XCBC-MAC-96 hash, the following setti Refer to the *DPDK Test Report* for more examples of traffic generator setup and the application startup command lines. If no errors are generated in response to the startup commands, the application is running correctly. - -.. |quickassist_block_diagram| image:: img/quickassist_block_diagram.* diff --git a/doc/guides/sample_app_ug/kernel_nic_interface.rst b/doc/guides/sample_app_ug/kernel_nic_interface.rst index d6876e2..02dde59 100644 --- a/doc/guides/sample_app_ug/kernel_nic_interface.rst +++ b/doc/guides/sample_app_ug/kernel_nic_interface.rst @@ -71,13 +71,12 @@ it is just for performance testing, or it can work together with VMDq support in The packet flow through the Kernel NIC Interface application is as shown in the following figure. -.. _figure_2: +.. _figure_kernel_nic: -**Figure 2. Kernel NIC Application Packet Flow** +.. figure:: img/kernel_nic.* -.. image3_png has been renamed to kernel_nic.* + Kernel NIC Application Packet Flow -|kernel_nic| Compiling the Application ------------------------- @@ -616,5 +615,3 @@ Currently, setting a new MTU and configuring the network interface (up/ down) ar RTE_LOG(ERR, APP, "Failed to start port %d\n", port_id); return ret; } - -.. |kernel_nic| image:: img/kernel_nic.* diff --git a/doc/guides/sample_app_ug/l2_forward_job_stats.rst b/doc/guides/sample_app_ug/l2_forward_job_stats.rst index 10dfecb..6cbf627 100644 --- a/doc/guides/sample_app_ug/l2_forward_job_stats.rst +++ b/doc/guides/sample_app_ug/l2_forward_job_stats.rst @@ -55,27 +55,24 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup_jobstats`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup_jobstats`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup_jobstats: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| +.. _figure_l2_fwd_virtenv_benchmark_setup_jobstats: -.. _figure_4: +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** + Performance Benchmark Setup (Virtualized Environment) -.. image5_png has been renamed - -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -631,7 +628,3 @@ however it improves performance: * in which it was called. */ rte_jobstats_finish(&qconf->flush_job, qconf->flush_job.target); } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst index 234d71d..9334e75 100644 --- a/doc/guides/sample_app_ug/l2_forward_real_virtual.rst +++ b/doc/guides/sample_app_ug/l2_forward_real_virtual.rst @@ -54,27 +54,25 @@ Also, the MAC addresses are affected as follows: * The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID -This application can be used to benchmark performance using a traffic-generator, as shown in the Figure 3. +This application can be used to benchmark performance using a traffic-generator, as shown in the :numref:`figure_l2_fwd_benchmark_setup`. -The application can also be used in a virtualized environment as shown in Figure 4. +The application can also be used in a virtualized environment as shown in :numref:`figure_l2_fwd_virtenv_benchmark_setup`. The L2 Forwarding application can also be used as a starting point for developing a new application based on the DPDK. -.. _figure_3: +.. _figure_l2_fwd_benchmark_setup: -**Figure 3. Performance Benchmark Setup (Basic Environment)** +.. figure:: img/l2_fwd_benchmark_setup.* -.. image4_png has been replaced + Performance Benchmark Setup (Basic Environment) -|l2_fwd_benchmark_setup| -.. _figure_4: +.. _figure_l2_fwd_virtenv_benchmark_setup: -**Figure 4. Performance Benchmark Setup (Virtualized Environment)** +.. figure:: img/l2_fwd_virtenv_benchmark_setup.* -.. image5_png has been renamed + Performance Benchmark Setup (Virtualized Environment) -|l2_fwd_virtenv_benchmark_setup| Virtual Function Setup Instructions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -526,7 +524,3 @@ however it improves performance: prev_tsc = cur_tsc; } - -.. |l2_fwd_benchmark_setup| image:: img/l2_fwd_benchmark_setup.* - -.. |l2_fwd_virtenv_benchmark_setup| image:: img/l2_fwd_virtenv_benchmark_setup.* diff --git a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst index 73fa4df..dbf47c7 100644 --- a/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst +++ b/doc/guides/sample_app_ug/l3_forward_access_ctrl.rst @@ -142,9 +142,13 @@ Other lines types are considered invalid. * A typical IPv4 ACL rule line should have a format as shown below: -.. image6_png has been renamed -|ipv4_acl_rule| +.. _figure_ipv4_acl_rule: + +.. figure:: img/ipv4_acl_rule.* + + A typical IPv4 ACL rule + IPv4 addresses are specified in CIDR format as specified in RFC 4632. They consist of the dot notation for the address and a prefix length separated by '/'. @@ -164,15 +168,12 @@ For example: 6/0xfe matches protocol values 6 and 7. Rules File Example ~~~~~~~~~~~~~~~~~~ -.. _figure_5_1: +.. _figure_example_rules: -Figure 5 is an example of a rules file. This file has three rules, one for ACL and two for route information. +.. figure:: img/example_rules.* -**Figure 5.Example Rules File** + Rules example -.. image7_png has been renamed - -|example_rules| Each rule is explained as follows: @@ -397,7 +398,3 @@ Finally, the application creates contexts handler from the ACL library, adds rules parsed from the file into the database and build an ACL trie. It is important to note that the application creates an independent copy of each database for each socket CPU involved in the task to reduce the time for remote memory access. - -.. |ipv4_acl_rule| image:: img/ipv4_acl_rule.* - -.. |example_rules| image:: img/example_rules.* diff --git a/doc/guides/sample_app_ug/load_balancer.rst b/doc/guides/sample_app_ug/load_balancer.rst index 3b32bdc..615826a 100644 --- a/doc/guides/sample_app_ug/load_balancer.rst +++ b/doc/guides/sample_app_ug/load_balancer.rst @@ -44,13 +44,12 @@ Overview The architecture of the Load Balance application is presented in the following figure. -.. _figure_5: +.. _figure_load_bal_app_arch: -**Figure 5. Load Balancer Application Architecture** +.. figure:: img/load_bal_app_arch.* -.. image8_png has been renamed + Load Balancer Application Architecture -|load_bal_app_arch| For the sake of simplicity, the diagram illustrates a specific case of two I/O RX and two I/O TX lcores off loading the packet I/O overhead incurred by four NIC ports from four worker cores, with each I/O lcore handling RX/TX for two NIC ports. @@ -241,5 +240,3 @@ are on the same or different CPU sockets, the following run-time scenarios are p #. ABC: The packet is received on socket A, it is processed by an lcore on socket B, then it has to be transmitted out by a NIC connected to socket C. The performance price for crossing the CPU socket boundary is paid twice for this packet. - -.. |load_bal_app_arch| image:: img/load_bal_app_arch.* diff --git a/doc/guides/sample_app_ug/multi_process.rst b/doc/guides/sample_app_ug/multi_process.rst index 9ed450b..dc287af 100644 --- a/doc/guides/sample_app_ug/multi_process.rst +++ b/doc/guides/sample_app_ug/multi_process.rst @@ -190,13 +190,12 @@ such as a client-server mode of operation seen in the next example, where different processes perform different tasks, yet co-operate to form a packet-processing system.) The following diagram shows the data-flow through the application, using two processes. -.. _figure_6: +.. _figure_sym_multi_proc_app: -**Figure 6. Example Data Flow in a Symmetric Multi-process Application** +.. figure:: img/sym_multi_proc_app.* -.. image9_png has been renamed + Example Data Flow in a Symmetric Multi-process Application -|sym_multi_proc_app| As the diagram shows, each process reads packets from each of the network ports in use. RSS is used to distribute incoming packets on each port to different hardware RX queues. @@ -296,13 +295,12 @@ In this case, the client applications just perform level-2 forwarding of packets The following diagram shows the data-flow through the application, using two client processes. -.. _figure_7: +.. _figure_client_svr_sym_multi_proc_app: -**Figure 7. Example Data Flow in a Client-Server Symmetric Multi-process Application** +.. figure:: img/client_svr_sym_multi_proc_app.* -.. image10_png has been renamed + Example Data Flow in a Client-Server Symmetric Multi-process Application -|client_svr_sym_multi_proc_app| Running the Application ^^^^^^^^^^^^^^^^^^^^^^^ @@ -395,13 +393,12 @@ Once the master process begins to run, it tries to initialize all the resources memory, CPU cores, driver, ports, and so on, as the other examples do. Thereafter, it creates slave processes, as shown in the following figure. -.. _figure_8: +.. _figure_master_slave_proc: -**Figure 8. Master-slave Process Workflow** +.. figure:: img/master_slave_proc.* -.. image11_png has been renamed + Master-slave Process Workflow -|master_slave_proc| The master process calls the rte_eal_mp_remote_launch() EAL function to launch an application function for each pinned thread through the pipe. Then, it waits to check if any slave processes have exited. @@ -475,13 +472,12 @@ Therefore, to provide the capability to resume the new slave instance if the pre The following diagram describes slave process recovery. -.. _figure_9: +.. _figure_slave_proc_recov: -**Figure 9. Slave Process Recovery Process Flow** +.. figure:: img/slave_proc_recov.* -.. image12_png has been renamed + Slave Process Recovery Process Flow -|slave_proc_recov| Floating Process Support ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -774,11 +770,3 @@ so it remaps the resource to the new core ID slot. } return 0; } - -.. |sym_multi_proc_app| image:: img/sym_multi_proc_app.* - -.. |client_svr_sym_multi_proc_app| image:: img/client_svr_sym_multi_proc_app.* - -.. |master_slave_proc| image:: img/master_slave_proc.* - -.. |slave_proc_recov| image:: img/slave_proc_recov.* diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index 56326df..66c261c 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -38,13 +38,12 @@ Overview The architecture of the QoS scheduler application is shown in the following figure. -.. _figure_10: +.. _figure_qos_sched_app_arch: -**Figure 10. QoS Scheduler Application Architecture** +.. figure:: img/qos_sched_app_arch.* -.. image13_png has been renamed + QoS Scheduler Application Architecture -|qos_sched_app_arch| There are two flavors of the runtime execution for this application, with two or three threads per each packet flow configuration being used. @@ -347,5 +346,3 @@ This application classifies based on the QinQ double VLAN tags and the IP destin +----------------+-------------------------+--------------------------------------------------+----------------------------------+ Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. - -.. |qos_sched_app_arch| image:: img/qos_sched_app_arch.* diff --git a/doc/guides/sample_app_ug/quota_watermark.rst b/doc/guides/sample_app_ug/quota_watermark.rst index 4274223..c56683a 100644 --- a/doc/guides/sample_app_ug/quota_watermark.rst +++ b/doc/guides/sample_app_ug/quota_watermark.rst @@ -54,15 +54,14 @@ and ports 2 and 3 forward into each other. The MAC addresses of the forwarded Ethernet frames are not affected. Internally, packets are pulled from the ports by the master logical core and put on a variable length processing pipeline, -each stage of which being connected by rings, as shown in Figure 12. +each stage of which being connected by rings, as shown in :numref:`figure_pipeline_overview`. -.. _figure_12: +.. _figure_pipeline_overview: -**Figure 12. Pipeline Overview** +.. figure:: img/pipeline_overview.* -.. image15_png has been renamed + Pipeline Overview -|pipeline_overview| An adjustable quota value controls how many packets are being moved through the pipeline per enqueue and dequeue. Adjustable watermark values associated with the rings control a back-off mechanism that @@ -79,15 +78,14 @@ eventually lead to an Ethernet flow control frame being send to the source. On top of serving as an example of quota and watermark usage, this application can be used to benchmark ring based processing pipelines performance using a traffic- generator, -as shown in Figure 13. +as shown in :numref:`figure_ring_pipeline_perf_setup`. -.. _figure_13: +.. _figure_ring_pipeline_perf_setup: -**Figure 13. Ring-based Processing Pipeline Performance Setup** +.. figure:: img/ring_pipeline_perf_setup.* -.. image16_png has been renamed + Ring-based Processing Pipeline Performance Setup -|ring_pipeline_perf_setup| Compiling the Application ------------------------- @@ -311,7 +309,7 @@ Logical Cores Assignment The application uses the master logical core to poll all the ports for new packets and enqueue them on a ring associated with the port. Each logical core except the last runs pipeline_stage() after a ring for each used port is initialized on that core. -pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See Figure 14. +pipeline_stage() on core X dequeues packets from core X-1's rings and enqueue them on its own rings. See :numref:`figure_threads_pipelines`. .. code-block:: c @@ -340,16 +338,12 @@ sending them out on the destination port setup by pair_ports(). Receive, Process and Transmit Packets ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _figure_14: +.. _figure_threads_pipelines: -Figure 14 shows where each thread in the pipeline is. -It should be used as a reference while reading the rest of this section. +.. figure:: img/threads_pipelines.* -**Figure 14. Threads and Pipelines** + Threads and Pipelines -.. image17_png has been renamed - -|threads_pipelines| In the receive_stage() function running on the master logical core, the main task is to read ingress packets from the RX ports and enqueue them @@ -498,9 +492,3 @@ low_watermark from the rte_memzone previously created by qw. low_watermark = (unsigned int *) qw_memzone->addr + sizeof(int); } - -.. |pipeline_overview| image:: img/pipeline_overview.* - -.. |ring_pipeline_perf_setup| image:: img/ring_pipeline_perf_setup.* - -.. |threads_pipelines| image:: img/threads_pipelines.* diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index 46aa6d5..3c2c25b 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -49,13 +49,12 @@ The application uses three CPU cores: * Core C ("TX core") receives traffic from core B through software queues and sends it to the NIC ports for transmission. -.. _figure_21: +.. _figure_test_pipeline_app: -**Figure 21.Test Pipeline Application** +.. figure:: img/test_pipeline_app.* -.. image24_png has been renamed + Test Pipeline Application -|test_pipeline_app| Compiling the Application ------------------------- @@ -281,5 +280,3 @@ The profile for input traffic is TCP/IPv4 packets with: * destination TCP port fixed to 0 * source TCP port fixed to 0 - -.. |test_pipeline_app| image:: img/test_pipeline_app.* diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst index ca9390d..730b9da 100644 --- a/doc/guides/sample_app_ug/vhost.rst +++ b/doc/guides/sample_app_ug/vhost.rst @@ -48,13 +48,12 @@ between host and guest. It was found that virtio-net performance was poor due to context switching and packet copying between host, guest, and QEMU. The following figure shows the system architecture for a virtio-based networking (virtio-net). -.. _figure_16: +.. _figure_qemu_virtio_net: -**Figure16. QEMU Virtio-net (prior to vhost-net)** +.. figure:: img/qemu_virtio_net.* -.. image19_png has been renamed + System Architecture for Virtio-based Networking (virtio-net). -|qemu_virtio_net| The Linux* Kernel vhost-net module was developed as an offload mechanism for virtio-net. The vhost-net module enables KVM (QEMU) to offload the servicing of virtio-net devices to the vhost-net kernel module, @@ -76,13 +75,12 @@ This is achieved by QEMU sharing the following information with the vhost-net mo The following figure shows the system architecture for virtio-net networking with vhost-net offload. -.. _figure_17: +.. _figure_virtio_linux_vhost: -**Figure 17. Virtio with Linux* Kernel Vhost** +.. figure:: img/virtio_linux_vhost.* -.. image20_png has been renamed + Virtio with Linux -|virtio_linux_vhost| Sample Code Overview -------------------- @@ -119,23 +117,21 @@ The vhost sample code application is a simple packet switching application with The following figure shows the architecture of the Vhost sample application based on vhost-cuse. -.. _figure_18: +.. _figure_vhost_net_arch: -**Figure 18. Vhost-net Architectural Overview** +.. figure:: img/vhost_net_arch.* -.. image21_png has been renamed + Vhost-net Architectural Overview -|vhost_net_arch| The following figure shows the flow of packets through the vhost-net sample application. -.. _figure_19: +.. _figure_vhost_net_sample_app: -**Figure 19. Packet Flow Through the vhost-net Sample Application** +.. figure:: img/vhost_net_sample_app.* -.. image22_png has been renamed + Packet Flow Through the vhost-net Sample Application -|vhost_net_sample_app| Supported Distributions ----------------------- @@ -771,13 +767,12 @@ In the "wait and retry" mode if the virtqueue is found to be full, then testpmd The "wait and retry" algorithm is implemented in DPDK testpmd as a forwarding method call "mac_retry". The following sequence diagram describes the algorithm in detail. -.. _figure_20: +.. _figure_tx_dpdk_testpmd: -**Figure 20. Packet Flow on TX in DPDK-testpmd** +.. figure:: img/tx_dpdk_testpmd.* -.. image23_png has been renamed + Packet Flow on TX in DPDK-testpmd -|tx_dpdk_testpmd| Running Testpmd ~~~~~~~~~~~~~~~ @@ -838,13 +833,3 @@ For example: The above message indicates that device 0 has been registered with MAC address cc:bb:bb:bb:bb:bb and VLAN tag 1000. Any packets received on the NIC with these values is placed on the devices receive queue. When a virtio-net device transmits packets, the VLAN tag is added to the packet by the DPDK vhost sample code. - -.. |vhost_net_arch| image:: img/vhost_net_arch.* - -.. |qemu_virtio_net| image:: img/qemu_virtio_net.* - -.. |tx_dpdk_testpmd| image:: img/tx_dpdk_testpmd.* - -.. |vhost_net_sample_app| image:: img/vhost_net_sample_app.* - -.. |virtio_linux_vhost| image:: img/virtio_linux_vhost.* diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst index dd6e1e8..da8be12 100644 --- a/doc/guides/sample_app_ug/vm_power_management.rst +++ b/doc/guides/sample_app_ug/vm_power_management.rst @@ -74,11 +74,12 @@ The solution is comprised of two high-level components: The l3fwd-power application will use this implementation when deployed on a VM (see Chapter 11 "L3 Forwarding with Power Management Application"). -.. _figure_24: +.. _figure_vm_power_mgr_highlevel: -**Figure 24. Highlevel Solution** +.. figure:: img/vm_power_mgr_highlevel.* + + Highlevel Solution -|vm_power_mgr_highlevel| Overview -------- @@ -105,11 +106,12 @@ at runtime based on the environment. Upon receiving a request, the host translates the vCPU to a pCPU via the libvirt API before forwarding to the host librte_power. -.. _figure_25: +.. _figure_vm_power_mgr_vm_request_seq: + +.. figure:: img/vm_power_mgr_vm_request_seq.* -**Figure 25. VM request to scale frequency** + VM request to scale frequency -|vm_power_mgr_vm_request_seq| Performance Considerations ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -355,7 +357,3 @@ Where {core_num} is the lcore and channel to change frequency by scaling up/down .. code-block:: console set_cpu_freq {core_num} up|down|min|max - -.. |vm_power_mgr_highlevel| image:: img/vm_power_mgr_highlevel.* - -.. |vm_power_mgr_vm_request_seq| image:: img/vm_power_mgr_vm_request_seq.* diff --git a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst index 9fc1fd5..6fb0f0b 100644 --- a/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst +++ b/doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst @@ -53,7 +53,7 @@ All traffic is read from a single incoming port (port 0) and output on port 1, w The traffic is split into 128 queues on input, where each thread of the application reads from multiple queues. For example, when run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. -As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in Figure 15. +As supplied, the sample application configures the VMDQ feature to have 16 pools with 8 queues each as indicated in :numref:`figure_vmdq_dcb_example`. The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 32 pools of 4 queues each and this can be used by changing the NUM_POOLS parameter in the supplied code. The NUM_POOLS parameter can be passed on the command line, after the EAL parameters: @@ -64,13 +64,12 @@ The NUM_POOLS parameter can be passed on the command line, after the EAL paramet where, NP can be 16 or 32. -.. _figure_15: +.. _figure_vmdq_dcb_example: -**Figure 15. Packet Flow Through the VMDQ and DCB Sample Application** +.. figure:: img/vmdq_dcb_example.* -.. image18_png has been replaced + Packet Flow Through the VMDQ and DCB Sample Application -|vmdq_dcb_example| In Linux* user space, the application can display statistics with the number of packets received on each queue. To have the application display the statistics, send a SIGHUP signal to the running application process, as follows: @@ -247,5 +246,3 @@ To generate the statistics output, use the following command: Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, rather than the terminal from which the HUP signal was sent. - -.. |vmdq_dcb_example| image:: img/vmdq_dcb_example.* diff --git a/doc/guides/xen/pkt_switch.rst b/doc/guides/xen/pkt_switch.rst index a9eca52..3a6fc47 100644 --- a/doc/guides/xen/pkt_switch.rst +++ b/doc/guides/xen/pkt_switch.rst @@ -52,9 +52,13 @@ The switching back end maps those grant table references and creates shared ring The following diagram describes the functionality of the DPDK Xen Packet- Switching Solution. -.. image35_png has been renamed -|dpdk_xen_pkt_switch| +.. _figure_dpdk_xen_pkt_switch: + +.. figure:: img/dpdk_xen_pkt_switch.* + + Functionality of the DPDK Xen Packet Switching Solution. + Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory between domains (`http://wiki.xen.org/wiki/Grant Table <http://wiki.xen.org/wiki/Grant%20Table>`_). @@ -62,9 +66,13 @@ Note 1 The Xen hypervisor uses a mechanism called a Grant Table to share memory A diagram of the design is shown below, where "gva" is the Guest Virtual Address, which is the data pointer of the mbuf, and "hva" is the Host Virtual Address: -.. image36_png has been renamed -|grant_table| +.. _figure_grant_table: + +.. figure:: img/grant_table.* + + DPDK Xen Layout + In this design, a Virtio ring is used as a para-virtualized interface for better performance over a Xen private ring when packet switching to and from a VM. @@ -139,9 +147,13 @@ Take idx#_mempool_gref node for example, the host maps those Grant references to The real Grant reference information is stored in this virtual address space, where (gref, pfn) pairs follow each other with -1 as the terminator. -.. image37_pnng has been renamed -|grant_refs| +.. _figure_grant_refs: + +.. figure:: img/grant_refs.* + + Mapping Grant references to a continuous virtual address space + After all gref# IDs are retrieved, the host maps them to a continuous virtual address space. With the guest mempool virtual address, the host establishes 1:1 address mapping. @@ -456,9 +468,3 @@ then sent out through hardware with destination MAC address 00:00:00:00:00:33. The packet flow is: packet generator->Virtio in guest VM1->switching backend->Virtio in guest VM2->switching backend->wire - -.. |grant_table| image:: img/grant_table.* - -.. |grant_refs| image:: img/grant_refs.* - -.. |dpdk_xen_pkt_switch| image:: img/dpdk_xen_pkt_switch.* -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v3 2/3] doc: refactored table numbers into references 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 1/3] doc: refactored figure numbers " John McNamara @ 2015-05-18 11:34 ` John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 3/3] doc: add sphinx numref compatibility workaround John McNamara 2015-05-25 14:05 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references Thomas Monjalon 3 siblings, 0 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:34 UTC (permalink / raw) To: dev This change adds automatic table references to the docs. The table numbers in the generated Html and PDF docs are now automatically numbered based on section. Requires Sphinx >= 1.3.1. This change: * Adds a RST table:: directive to each table caption. * Indents the tables to the required directive level. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/prog_guide/index.rst | 70 +- doc/guides/prog_guide/packet_framework.rst | 1164 +++++++++++------------ doc/guides/prog_guide/qos_framework.rst | 1372 ++++++++++++++-------------- doc/guides/sample_app_ug/index.rst | 6 +- doc/guides/sample_app_ug/qos_metering.rst | 46 +- doc/guides/sample_app_ug/qos_scheduler.rst | 46 +- doc/guides/sample_app_ug/test_pipeline.rst | 304 +++--- 7 files changed, 1503 insertions(+), 1505 deletions(-) diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 9a1e337..3295661 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -174,72 +174,70 @@ Programmer's Guide **Tables** -:ref:`Table 1. Packet Processing Pipeline Implementing QoS <pg_table_1>` +:numref:`table_qos_1` :ref:`table_qos_1` -:ref:`Table 2. Infrastructure Blocks Used by the Packet Processing Pipeline <pg_table_2>` +:numref:`table_qos_2` :ref:`table_qos_2` -:ref:`Table 3. Port Scheduling Hierarchy <pg_table_3>` +:numref:`table_qos_3` :ref:`table_qos_3` -:ref:`Table 4. Scheduler Internal Data Structures per Port <pg_table_4>` +:numref:`table_qos_4` :ref:`table_qos_4` -:ref:`Table 5. Ethernet Frame Overhead Fields <pg_table_5>` +:numref:`table_qos_5` :ref:`table_qos_5` -:ref:`Table 6. Token Bucket Generic Operations <pg_table_6>` +:numref:`table_qos_6` :ref:`table_qos_6` -:ref:`Table 7. Token Bucket Generic Parameters <pg_table_7>` +:numref:`table_qos_7` :ref:`table_qos_7` -:ref:`Table 8. Token Bucket Persistent Data Structure <pg_table_8>` +:numref:`table_qos_8` :ref:`table_qos_8` -:ref:`Table 9. Token Bucket Operations <pg_table_9>` +:numref:`table_qos_9` :ref:`table_qos_9` -:ref:`Table 10. Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure <pg_table_10>` +:numref:`table_qos_10` :ref:`table_qos_10` -:ref:`Table 11. Subport/Pipe Traffic Class Upper Limit Enforcement Operations <pg_table_11>` +:numref:`table_qos_11` :ref:`table_qos_11` -:ref:`Table 12. Weighted Round Robin (WRR) <pg_table_12>` +:numref:`table_qos_12` :ref:`table_qos_12` -:ref:`Table 13. Subport Traffic Class Oversubscription <pg_table_13>` +:numref:`table_qos_13` :ref:`table_qos_13` -:ref:`Table 14. Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period <pg_table_14>` +:numref:`table_qos_14` :ref:`table_qos_14` -:ref:`Table 15. Watermark Calculation <pg_table_15>` +:numref:`table_qos_15` :ref:`table_qos_15` -:ref:`Table 16. RED Configuration Parameters <pg_table_16>` +:numref:`table_qos_16` :ref:`table_qos_16` -:ref:`Table 17. Relative Performance of Alternative Approaches <pg_table_17>` +:numref:`table_qos_17` :ref:`table_qos_17` -:ref:`Table 18. RED Configuration Corresponding to RED Configuration File <pg_table_18>` +:numref:`table_qos_18` :ref:`table_qos_18` -:ref:`Table 19. Port types <pg_table_19>` +:numref:`table_qos_19` :ref:`table_qos_19` -:ref:`Table 20. Port abstract interface <pg_table_20>` +:numref:`table_qos_20` :ref:`table_qos_20` -:ref:`Table 21. Table types <pg_table_21>` +:numref:`table_qos_21` :ref:`table_qos_21` -:ref:`Table 29. Table Abstract Interface <pg_table_29_1>` +:numref:`table_qos_22` :ref:`table_qos_22` -:ref:`Table 22. Configuration parameters common for all hash table types <pg_table_22>` +:numref:`table_qos_23` :ref:`table_qos_23` -:ref:`Table 23. Configuration parameters specific to extendable bucket hash table <pg_table_23>` +:numref:`table_qos_24` :ref:`table_qos_24` -:ref:`Table 24. Configuration parameters specific to pre-computed key signature hash table <pg_table_24>` +:numref:`table_qos_25` :ref:`table_qos_25` -:ref:`Table 25. The main large data structures (arrays) used for configurable key size hash tables <pg_table_25>` +:numref:`table_qos_26` :ref:`table_qos_26` -:ref:`Table 26. Field description for bucket array entry (configurable key size hash tables) <pg_table_26>` +:numref:`table_qos_27` :ref:`table_qos_27` -:ref:`Table 27. Description of the bucket search pipeline stages (configurable key size hash tables) <pg_table_27>` +:numref:`table_qos_28` :ref:`table_qos_28` -:ref:`Table 28. Lookup tables for match, match_many, match_pos <pg_table_28>` +:numref:`table_qos_29` :ref:`table_qos_29` -:ref:`Table 29. Collapsed lookup tables for match, match_many and match_pos <pg_table_29>` +:numref:`table_qos_30` :ref:`table_qos_30` -:ref:`Table 30. The main large data structures (arrays) used for 8-byte and 16-byte key size hash tables <pg_table_30>` +:numref:`table_qos_31` :ref:`table_qos_31` -:ref:`Table 31. Field description for bucket array entry (8-byte and 16-byte key hash tables) <pg_table_31>` +:numref:`table_qos_32` :ref:`table_qos_32` -:ref:`Table 32. Description of the bucket search pipeline stages (8-byte and 16-byte key hash tables) <pg_table_32>` +:numref:`table_qos_33` :ref:`table_qos_33` -:ref:`Table 33. Next hop actions (reserved) <pg_table_33>` - -:ref:`Table 34. User action examples <pg_table_34>` +:numref:`table_qos_34` :ref:`table_qos_34` diff --git a/doc/guides/prog_guide/packet_framework.rst b/doc/guides/prog_guide/packet_framework.rst index 417f4db..381f320 100644 --- a/doc/guides/prog_guide/packet_framework.rst +++ b/doc/guides/prog_guide/packet_framework.rst @@ -82,49 +82,49 @@ Port Library Design Port Types ~~~~~~~~~~ -Table 19 is a non-exhaustive list of ports that can be implemented with the Packet Framework. - -.. _pg_table_19: - -**Table 19 Port Types** - -+---+------------------+---------------------------------------------------------------------------------------+ -| # | Port type | Description | -| | | | -+===+==================+=======================================================================================+ -| 1 | SW ring | SW circular buffer used for message passing between the application threads. Uses | -| | | the DPDK rte_ring primitive. Expected to be the most commonly used type of | -| | | port. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 2 | HW ring | Queue of buffer descriptors used to interact with NIC, switch or accelerator ports. | -| | | For NIC ports, it uses the DPDK rte_eth_rx_queue or rte_eth_tx_queue | -| | | primitives. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 3 | IP reassembly | Input packets are either IP fragments or complete IP datagrams. Output packets are | -| | | complete IP datagrams. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 4 | IP fragmentation | Input packets are jumbo (IP datagrams with length bigger than MTU) or non-jumbo | -| | | packets. Output packets are non-jumbo packets. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 5 | Traffic manager | Traffic manager attached to a specific NIC output port, performing congestion | -| | | management and hierarchical scheduling according to pre-defined SLAs. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 6 | KNI | Send/receive packets to/from Linux kernel space. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 7 | Source | Input port used as packet generator. Similar to Linux kernel /dev/zero character | -| | | device. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ -| 8 | Sink | Output port used to drop all input packets. Similar to Linux kernel /dev/null | -| | | character device. | -| | | | -+---+------------------+---------------------------------------------------------------------------------------+ +:numref:`table_qos_19` is a non-exhaustive list of ports that can be implemented with the Packet Framework. + +.. _table_qos_19: + +.. table:: Port Types + + +---+------------------+---------------------------------------------------------------------------------------+ + | # | Port type | Description | + | | | | + +===+==================+=======================================================================================+ + | 1 | SW ring | SW circular buffer used for message passing between the application threads. Uses | + | | | the DPDK rte_ring primitive. Expected to be the most commonly used type of | + | | | port. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 2 | HW ring | Queue of buffer descriptors used to interact with NIC, switch or accelerator ports. | + | | | For NIC ports, it uses the DPDK rte_eth_rx_queue or rte_eth_tx_queue | + | | | primitives. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 3 | IP reassembly | Input packets are either IP fragments or complete IP datagrams. Output packets are | + | | | complete IP datagrams. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 4 | IP fragmentation | Input packets are jumbo (IP datagrams with length bigger than MTU) or non-jumbo | + | | | packets. Output packets are non-jumbo packets. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 5 | Traffic manager | Traffic manager attached to a specific NIC output port, performing congestion | + | | | management and hierarchical scheduling according to pre-defined SLAs. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 6 | KNI | Send/receive packets to/from Linux kernel space. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 7 | Source | Input port used as packet generator. Similar to Linux kernel /dev/zero character | + | | | device. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ + | 8 | Sink | Output port used to drop all input packets. Similar to Linux kernel /dev/null | + | | | character device. | + | | | | + +---+------------------+---------------------------------------------------------------------------------------+ Port Interface ~~~~~~~~~~~~~~ @@ -134,29 +134,29 @@ Each input/output port is required to implement an abstract interface that defines the initialization and run-time operation of the port. The port abstract interface is described in. -.. _pg_table_20: - -**Table 20 Port Abstract Interface** - -+---+----------------+-----------------------------------------------------------------------------------------+ -| # | Port Operation | Description | -| | | | -+===+================+=========================================================================================+ -| 1 | Create | Create the low-level port object (e.g. queue). Can internally allocate memory. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 2 | Free | Free the resources (e.g. memory) used by the low-level port object. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 3 | RX | Read a burst of input packets. Non-blocking operation. Only defined for input ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 4 | TX | Write a burst of input packets. Non-blocking operation. Only defined for output ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ -| 5 | Flush | Flush the output buffer. Only defined for output ports. | -| | | | -+---+----------------+-----------------------------------------------------------------------------------------+ +.. _table_qos_20: + +.. table:: 20 Port Abstract Interface + + +---+----------------+-----------------------------------------------------------------------------------------+ + | # | Port Operation | Description | + | | | | + +===+================+=========================================================================================+ + | 1 | Create | Create the low-level port object (e.g. queue). Can internally allocate memory. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 2 | Free | Free the resources (e.g. memory) used by the low-level port object. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 3 | RX | Read a burst of input packets. Non-blocking operation. Only defined for input ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 4 | TX | Write a burst of input packets. Non-blocking operation. Only defined for output ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ + | 5 | Flush | Flush the output buffer. Only defined for output ports. | + | | | | + +---+----------------+-----------------------------------------------------------------------------------------+ Table Library Design -------------------- @@ -164,113 +164,113 @@ Table Library Design Table Types ~~~~~~~~~~~ -.. _pg_table_21: - -Table 21 is a non-exhaustive list of types of tables that can be implemented with the Packet Framework. - -**Table 21 Table Types** - -+---+----------------------------+-----------------------------------------------------------------------------+ -| # | Table Type | Description | -| | | | -+===+============================+=============================================================================+ -| 1 | Hash table | Lookup key is n-tuple based. | -| | | | -| | | Typically, the lookup key is hashed to produce a signature that is used to | -| | | identify a bucket of entries where the lookup key is searched next. | -| | | | -| | | The signature associated with the lookup key of each input packet is either | -| | | read from the packet descriptor (pre-computed signature) or computed at | -| | | table lookup time. | -| | | | -| | | The table lookup, add entry and delete entry operations, as well as any | -| | | other pipeline block that pre-computes the signature all have to use the | -| | | same hashing algorithm to generate the signature. | -| | | | -| | | Typically used to implement flow classification tables, ARP caches, routing | -| | | table for tunnelling protocols, etc. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 2 | Longest Prefix Match (LPM) | Lookup key is the IP address. | -| | | | -| | | Each table entries has an associated IP prefix (IP and depth). | -| | | | -| | | The table lookup operation selects the IP prefix that is matched by the | -| | | lookup key; in case of multiple matches, the entry with the longest prefix | -| | | depth wins. | -| | | | -| | | Typically used to implement IP routing tables. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 3 | Access Control List (ACLs) | Lookup key is 7-tuple of two VLAN/MPLS labels, IP destination address, | -| | | IP source addresses, L4 protocol, L4 destination port, L4 source port. | -| | | | -| | | Each table entry has an associated ACL and priority. The ACL contains bit | -| | | masks for the VLAN/MPLS labels, IP prefix for IP destination address, IP | -| | | prefix for IP source addresses, L4 protocol and bitmask, L4 destination | -| | | port and bit mask, L4 source port and bit mask. | -| | | | -| | | The table lookup operation selects the ACL that is matched by the lookup | -| | | key; in case of multiple matches, the entry with the highest priority wins. | -| | | | -| | | Typically used to implement rule databases for firewalls, etc. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 4 | Pattern matching search | Lookup key is the packet payload. | -| | | | -| | | Table is a database of patterns, with each pattern having a priority | -| | | assigned. | -| | | | -| | | The table lookup operation selects the patterns that is matched by the | -| | | input packet; in case of multiple matches, the matching pattern with the | -| | | highest priority wins. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ -| 5 | Array | Lookup key is the table entry index itself. | -| | | | -+---+----------------------------+-----------------------------------------------------------------------------+ +:numref:`table_qos_21` is a non-exhaustive list of types of tables that can be implemented with the Packet Framework. + +.. _table_qos_21: + +.. table:: Table Types + + +---+----------------------------+-----------------------------------------------------------------------------+ + | # | Table Type | Description | + | | | | + +===+============================+=============================================================================+ + | 1 | Hash table | Lookup key is n-tuple based. | + | | | | + | | | Typically, the lookup key is hashed to produce a signature that is used to | + | | | identify a bucket of entries where the lookup key is searched next. | + | | | | + | | | The signature associated with the lookup key of each input packet is either | + | | | read from the packet descriptor (pre-computed signature) or computed at | + | | | table lookup time. | + | | | | + | | | The table lookup, add entry and delete entry operations, as well as any | + | | | other pipeline block that pre-computes the signature all have to use the | + | | | same hashing algorithm to generate the signature. | + | | | | + | | | Typically used to implement flow classification tables, ARP caches, routing | + | | | table for tunnelling protocols, etc. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 2 | Longest Prefix Match (LPM) | Lookup key is the IP address. | + | | | | + | | | Each table entries has an associated IP prefix (IP and depth). | + | | | | + | | | The table lookup operation selects the IP prefix that is matched by the | + | | | lookup key; in case of multiple matches, the entry with the longest prefix | + | | | depth wins. | + | | | | + | | | Typically used to implement IP routing tables. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 3 | Access Control List (ACLs) | Lookup key is 7-tuple of two VLAN/MPLS labels, IP destination address, | + | | | IP source addresses, L4 protocol, L4 destination port, L4 source port. | + | | | | + | | | Each table entry has an associated ACL and priority. The ACL contains bit | + | | | masks for the VLAN/MPLS labels, IP prefix for IP destination address, IP | + | | | prefix for IP source addresses, L4 protocol and bitmask, L4 destination | + | | | port and bit mask, L4 source port and bit mask. | + | | | | + | | | The table lookup operation selects the ACL that is matched by the lookup | + | | | key; in case of multiple matches, the entry with the highest priority wins. | + | | | | + | | | Typically used to implement rule databases for firewalls, etc. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 4 | Pattern matching search | Lookup key is the packet payload. | + | | | | + | | | Table is a database of patterns, with each pattern having a priority | + | | | assigned. | + | | | | + | | | The table lookup operation selects the patterns that is matched by the | + | | | input packet; in case of multiple matches, the matching pattern with the | + | | | highest priority wins. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ + | 5 | Array | Lookup key is the table entry index itself. | + | | | | + +---+----------------------------+-----------------------------------------------------------------------------+ Table Interface ~~~~~~~~~~~~~~~ Each table is required to implement an abstract interface that defines the initialization and run-time operation of the table. -The table abstract interface is described in Table 29. - -.. _pg_table_29_1: - -**Table 29 Table Abstract Interface** - -+---+-----------------+----------------------------------------------------------------------------------------+ -| # | Table operation | Description | -| | | | -+===+=================+========================================================================================+ -| 1 | Create | Create the low-level data structures of the lookup table. Can internally allocate | -| | | memory. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 2 | Free | Free up all the resources used by the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 3 | Add entry | Add new entry to the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 4 | Delete entry | Delete specific entry from the lookup table. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ -| 5 | Lookup | Look up a burst of input packets and return a bit mask specifying the result of the | -| | | lookup operation for each packet: a set bit signifies lookup hit for the corresponding | -| | | packet, while a cleared bit a lookup miss. | -| | | | -| | | For each lookup hit packet, the lookup operation also returns a pointer to the table | -| | | entry that was hit, which contains the actions to be applied on the packet and any | -| | | associated metadata. | -| | | | -| | | For each lookup miss packet, the actions to be applied on the packet and any | -| | | associated metadata are specified by the default table entry preconfigured for lookup | -| | | miss. | -| | | | -+---+-----------------+----------------------------------------------------------------------------------------+ +The table abstract interface is described in :numref:`table_qos_29_1`. + +.. _table_qos_29_1: + +.. table:: Table Abstract Interface + + +---+-----------------+----------------------------------------------------------------------------------------+ + | # | Table operation | Description | + | | | | + +===+=================+========================================================================================+ + | 1 | Create | Create the low-level data structures of the lookup table. Can internally allocate | + | | | memory. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 2 | Free | Free up all the resources used by the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 3 | Add entry | Add new entry to the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 4 | Delete entry | Delete specific entry from the lookup table. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ + | 5 | Lookup | Look up a burst of input packets and return a bit mask specifying the result of the | + | | | lookup operation for each packet: a set bit signifies lookup hit for the corresponding | + | | | packet, while a cleared bit a lookup miss. | + | | | | + | | | For each lookup hit packet, the lookup operation also returns a pointer to the table | + | | | entry that was hit, which contains the actions to be applied on the packet and any | + | | | associated metadata. | + | | | | + | | | For each lookup miss packet, the actions to be applied on the packet and any | + | | | associated metadata are specified by the default table entry preconfigured for lookup | + | | | miss. | + | | | | + +---+-----------------+----------------------------------------------------------------------------------------+ Hash Table Design @@ -391,38 +391,38 @@ The MAC address of the next hop station becomes the destination MAC address of t Hash Table Types ^^^^^^^^^^^^^^^^ -.. _pg_table_22: - -Table 22 lists the hash table configuration parameters shared by all different hash table types. - -**Table 22 Configuration Parameters Common for All Hash Table Types** - -+---+---------------------------+------------------------------------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+===========================+==============================================================================+ -| 1 | Key size | Measured as number of bytes. All keys have the same size. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 2 | Key value (key data) size | Measured as number of bytes. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 3 | Number of buckets | Needs to be a power of two. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 4 | Maximum number of keys | Needs to be a power of two. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 5 | Hash function | Examples: jhash, CRC hash, etc. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 6 | Hash function seed | Parameter to be passed to the hash function. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 7 | Key offset | Offset of the lookup key byte array within the packet meta-data stored in | -| | | the packet buffer. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ +:numref:`table_qos_22` lists the hash table configuration parameters shared by all different hash table types. + +.. _table_qos_22: + +.. table:: Configuration Parameters Common for All Hash Table Types + + +---+---------------------------+------------------------------------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+===========================+==============================================================================+ + | 1 | Key size | Measured as number of bytes. All keys have the same size. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 2 | Key value (key data) size | Measured as number of bytes. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 3 | Number of buckets | Needs to be a power of two. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 4 | Maximum number of keys | Needs to be a power of two. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 5 | Hash function | Examples: jhash, CRC hash, etc. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 6 | Hash function seed | Parameter to be passed to the hash function. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 7 | Key offset | Offset of the lookup key byte array within the packet meta-data stored in | + | | | the packet buffer. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ Bucket Full Problem """"""""""""""""""" @@ -452,17 +452,17 @@ The possible options are: the search continues beyond the first group of 4 keys, potentially until all keys in this bucket are examined. The extendable bucket logic requires maintaining specific data structures per table and per each bucket. -.. _pg_table_23: +.. _table_qos_23: -**Table 23 Configuration Parameters Specific to Extendable Bucket Hash Table** +.. table:: Configuration Parameters Specific to Extendible Bucket Hash Table -+---+---------------------------+--------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+===========================+==================================================+ -| 1 | Number of additional keys | Needs to be a power of two, at least equal to 4. | -| | | | -+---+---------------------------+--------------------------------------------------+ + +---+---------------------------+--------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+===========================+==================================================+ + | 1 | Number of additional keys | Needs to be a power of two, at least equal to 4. | + | | | | + +---+---------------------------+--------------------------------------------------+ Signature Computation @@ -481,17 +481,17 @@ The possible options for key signature computation are: The same CPU core reads the key from the packet meta-data, uses it to compute the key signature and also performs the bucket search step of the key lookup operation. -.. _pg_table_24: +.. _table_qos_24: -**Table 24 Configuration Parameters Specific to Pre-computed Key Signature Hash Table** +.. table:: Configuration Parameters Specific to Pre-computed Key Signature Hash Table -+---+------------------+-----------------------------------------------------------------------+ -| # | Parameter | Details | -| | | | -+===+==================+=======================================================================+ -| 1 | Signature offset | Offset of the pre-computed key signature within the packet meta-data. | -| | | | -+---+------------------+-----------------------------------------------------------------------+ + +---+------------------+-----------------------------------------------------------------------+ + | # | Parameter | Details | + | | | | + +===+==================+=======================================================================+ + | 1 | Signature offset | Offset of the pre-computed key signature within the packet meta-data. | + | | | | + +---+------------------+-----------------------------------------------------------------------+ Key Size Optimized Hash Tables """""""""""""""""""""""""""""" @@ -554,7 +554,7 @@ This avoids the important cost associated with flushing the CPU core execution p Configurable Key Size Hash Table """""""""""""""""""""""""""""""" -:numref:`figure_figure34`, Table 25 and Table 26 detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, +:numref:`figure_figure34`, :numref:`table_qos_25` and :numref:`table_qos_26` detail the main data structures used to implement configurable key size hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). .. _figure_figure34: @@ -564,70 +564,70 @@ either with pre-computed signature or "do-sig"). Data Structures for Configurable Key Size Hash Tables -.. _pg_table_25: - -**Table 25 Main Large Data Structures (Arrays) used for Configurable Key Size Hash Tables** - -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| # | Array name | Number of entries | Entry size (bytes) | Description | -| | | | | | -+===+=========================+==============================+===========================+===============================+ -| 1 | Bucket array | n_buckets (configurable) | 32 | Buckets of the hash table. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 2 | Bucket extensions array | n_buckets_ext (configurable) | 32 | This array is only created | -| | | | | for extendable bucket tables. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 3 | Key array | n_keys | key_size (configurable) | Keys added to the hash table. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ -| 4 | Data array | n_keys | entry_size (configurable) | Key values (key data) | -| | | | | associated with the hash | -| | | | | table keys. | -| | | | | | -+---+-------------------------+------------------------------+---------------------------+-------------------------------+ - -.. _pg_table_26: - -**Table 26 Field Description for Bucket Array Entry (Configurable Key Size Hash Tables)** - -+---+------------------+--------------------+------------------------------------------------------------------+ -| # | Field name | Field size (bytes) | Description | -| | | | | -+===+==================+====================+==================================================================+ -| 1 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the | -| | | | current bucket stored as array of 4 entries of 2 bytes each. | -| | | | Entry 0 stores the index (0 .. 3) of the MRU key, while entry 3 | -| | | | stores the index of the LRU key. | -| | | | | -| | | | For extendable bucket tables, this field represents the next | -| | | | pointer (i.e. the pointer to the next group of 4 keys linked to | -| | | | the current bucket). The next pointer is not NULL if the bucket | -| | | | is currently extended or NULL otherwise. | -| | | | To help the branchless implementation, bit 0 (least significant | -| | | | bit) of this field is set to 1 if the next pointer is not NULL | -| | | | and to 0 otherwise. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ -| 2 | Sig[0 .. 3] | 4 x 2 | If key X (X = 0 .. 3) is valid, then sig X bits 15 .. 1 store | -| | | | the most significant 15 bits of key X signature and sig X bit 0 | -| | | | is set to 1. | -| | | | | -| | | | If key X is not valid, then sig X is set to zero. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ -| 3 | Key Pos [0 .. 3] | 4 x 4 | If key X is valid (X = 0 .. 3), then Key Pos X represents the | -| | | | index into the key array where key X is stored, as well as the | -| | | | index into the data array where the value associated with key X | -| | | | is stored. | -| | | | | -| | | | If key X is not valid, then the value of Key Pos X is undefined. | -| | | | | -+---+------------------+--------------------+------------------------------------------------------------------+ - - -:numref:`figure_figure35` and Table 27 detail the bucket search pipeline stages (either LRU or extendable bucket, +.. _table_qos_25: + +.. table:: Main Large Data Structures (Arrays) used for Configurable Key Size Hash Tables + + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | # | Array name | Number of entries | Entry size (bytes) | Description | + | | | | | | + +===+=========================+==============================+===========================+===============================+ + | 1 | Bucket array | n_buckets (configurable) | 32 | Buckets of the hash table. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 2 | Bucket extensions array | n_buckets_ext (configurable) | 32 | This array is only created | + | | | | | for extendible bucket tables. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 3 | Key array | n_keys | key_size (configurable) | Keys added to the hash table. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + | 4 | Data array | n_keys | entry_size (configurable) | Key values (key data) | + | | | | | associated with the hash | + | | | | | table keys. | + | | | | | | + +---+-------------------------+------------------------------+---------------------------+-------------------------------+ + +.. _table_qos_26: + +.. table:: Field Description for Bucket Array Entry (Configurable Key Size Hash Tables) + + +---+------------------+--------------------+------------------------------------------------------------------+ + | # | Field name | Field size (bytes) | Description | + | | | | | + +===+==================+====================+==================================================================+ + | 1 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the | + | | | | current bucket stored as array of 4 entries of 2 bytes each. | + | | | | Entry 0 stores the index (0 .. 3) of the MRU key, while entry 3 | + | | | | stores the index of the LRU key. | + | | | | | + | | | | For extendible bucket tables, this field represents the next | + | | | | pointer (i.e. the pointer to the next group of 4 keys linked to | + | | | | the current bucket). The next pointer is not NULL if the bucket | + | | | | is currently extended or NULL otherwise. | + | | | | To help the branchless implementation, bit 0 (least significant | + | | | | bit) of this field is set to 1 if the next pointer is not NULL | + | | | | and to 0 otherwise. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ + | 2 | Sig[0 .. 3] | 4 x 2 | If key X (X = 0 .. 3) is valid, then sig X bits 15 .. 1 store | + | | | | the most significant 15 bits of key X signature and sig X bit 0 | + | | | | is set to 1. | + | | | | | + | | | | If key X is not valid, then sig X is set to zero. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ + | 3 | Key Pos [0 .. 3] | 4 x 4 | If key X is valid (X = 0 .. 3), then Key Pos X represents the | + | | | | index into the key array where key X is stored, as well as the | + | | | | index into the data array where the value associated with key X | + | | | | is stored. | + | | | | | + | | | | If key X is not valid, then the value of Key Pos X is undefined. | + | | | | | + +---+------------------+--------------------+------------------------------------------------------------------+ + + +:numref:`figure_figure35` and :numref:`table_qos_27` detail the bucket search pipeline stages (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). For each pipeline stage, the described operations are applied to each of the two packets handled by that stage. @@ -639,80 +639,80 @@ For each pipeline stage, the described operations are applied to each of the two Tables) -.. _pg_table_27: - -**Table 27 Description of the Bucket Search Pipeline Stages (Configurable Key Size Hash Tables)** - -+---+---------------------------+------------------------------------------------------------------------------+ -| # | Stage name | Description | -| | | | -+===+===========================+==============================================================================+ -| 0 | Prefetch packet meta-data | Select next two packets from the burst of input packets. | -| | | | -| | | Prefetch packet meta-data containing the key and key signature. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 1 | Prefetch table bucket | Read the key signature from the packet meta-data (for extendable bucket hash | -| | | tables) or read the key from the packet meta-data and compute key signature | -| | | (for LRU tables). | -| | | | -| | | Identify the bucket ID using the key signature. | -| | | | -| | | Set bit 0 of the signature to 1 (to match only signatures of valid keys from | -| | | the table). | -| | | | -| | | Prefetch the bucket. | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 2 | Prefetch table key | Read the key signatures from the bucket. | -| | | | -| | | Compare the signature of the input key against the 4 key signatures from the | -| | | packet. As result, the following is obtained: | -| | | | -| | | *match* | -| | | = equal to TRUE if there was at least one signature match and to FALSE in | -| | | the case of no signature match; | -| | | | -| | | *match_many* | -| | | = equal to TRUE is there were more than one signature matches (can be up to | -| | | 4 signature matches in the worst case scenario) and to FALSE otherwise; | -| | | | -| | | *match_pos* | -| | | = the index of the first key that produced signature match (only valid if | -| | | match is true). | -| | | | -| | | For extendable bucket hash tables only, set | -| | | *match_many* | -| | | to TRUE if next pointer is valid. | -| | | | -| | | Prefetch the bucket key indicated by | -| | | *match_pos* | -| | | (even if | -| | | *match_pos* | -| | | does not point to valid key valid). | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ -| 3 | Prefetch table data | Read the bucket key indicated by | -| | | *match_pos*. | -| | | | -| | | Compare the bucket key against the input key. As result, the following is | -| | | obtained: | -| | | *match_key* | -| | | = equal to TRUE if the two keys match and to FALSE otherwise. | -| | | | -| | | Report input key as lookup hit only when both | -| | | *match* | -| | | and | -| | | *match_key* | -| | | are equal to TRUE and as lookup miss otherwise. | -| | | | -| | | For LRU tables only, use branchless logic to update the bucket LRU list | -| | | (the current key becomes the new MRU) only on lookup hit. | -| | | | -| | | Prefetch the key value (key data) associated with the current key (to avoid | -| | | branches, this is done on both lookup hit and miss). | -| | | | -+---+---------------------------+------------------------------------------------------------------------------+ +.. _table_qos_27: + +.. table:: Description of the Bucket Search Pipeline Stages (Configurable Key Size Hash Tables) + + +---+---------------------------+------------------------------------------------------------------------------+ + | # | Stage name | Description | + | | | | + +===+===========================+==============================================================================+ + | 0 | Prefetch packet meta-data | Select next two packets from the burst of input packets. | + | | | | + | | | Prefetch packet meta-data containing the key and key signature. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 1 | Prefetch table bucket | Read the key signature from the packet meta-data (for extendable bucket hash | + | | | tables) or read the key from the packet meta-data and compute key signature | + | | | (for LRU tables). | + | | | | + | | | Identify the bucket ID using the key signature. | + | | | | + | | | Set bit 0 of the signature to 1 (to match only signatures of valid keys from | + | | | the table). | + | | | | + | | | Prefetch the bucket. | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 2 | Prefetch table key | Read the key signatures from the bucket. | + | | | | + | | | Compare the signature of the input key against the 4 key signatures from the | + | | | packet. As result, the following is obtained: | + | | | | + | | | *match* | + | | | = equal to TRUE if there was at least one signature match and to FALSE in | + | | | the case of no signature match; | + | | | | + | | | *match_many* | + | | | = equal to TRUE is there were more than one signature matches (can be up to | + | | | 4 signature matches in the worst case scenario) and to FALSE otherwise; | + | | | | + | | | *match_pos* | + | | | = the index of the first key that produced signature match (only valid if | + | | | match is true). | + | | | | + | | | For extendable bucket hash tables only, set | + | | | *match_many* | + | | | to TRUE if next pointer is valid. | + | | | | + | | | Prefetch the bucket key indicated by | + | | | *match_pos* | + | | | (even if | + | | | *match_pos* | + | | | does not point to valid key valid). | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ + | 3 | Prefetch table data | Read the bucket key indicated by | + | | | *match_pos*. | + | | | | + | | | Compare the bucket key against the input key. As result, the following is | + | | | obtained: | + | | | *match_key* | + | | | = equal to TRUE if the two keys match and to FALSE otherwise. | + | | | | + | | | Report input key as lookup hit only when both | + | | | *match* | + | | | and | + | | | *match_key* | + | | | are equal to TRUE and as lookup miss otherwise. | + | | | | + | | | For LRU tables only, use branchless logic to update the bucket LRU list | + | | | (the current key becomes the new MRU) only on lookup hit. | + | | | | + | | | Prefetch the key value (key data) associated with the current key (to avoid | + | | | branches, this is done on both lookup hit and miss). | + | | | | + +---+---------------------------+------------------------------------------------------------------------------+ Additional notes: @@ -731,90 +731,90 @@ Additional notes: **Key Signature Comparison Logic** -The key signature comparison logic is described in Table 28. - -.. _pg_table_28: - -**Table 28 Lookup Tables for Match, Match_Many and Match_Pos** - -+----+------+---------------+--------------------+--------------------+ -| # | mask | match (1 bit) | match_many (1 bit) | match_pos (2 bits) | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 0 | 0000 | 0 | 0 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 1 | 0001 | 1 | 0 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 2 | 0010 | 1 | 0 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 3 | 0011 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 4 | 0100 | 1 | 0 | 10 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 5 | 0101 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 6 | 0110 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 7 | 0111 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 8 | 1000 | 1 | 0 | 11 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 9 | 1001 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 10 | 1010 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 11 | 1011 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 12 | 1100 | 1 | 1 | 10 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 13 | 1101 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 14 | 1110 | 1 | 1 | 01 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ -| 15 | 1111 | 1 | 1 | 00 | -| | | | | | -+----+------+---------------+--------------------+--------------------+ +The key signature comparison logic is described in :numref:`table_qos_28`. + +.. _table_qos_28: + +.. table:: Lookup Tables for Match, Match_Many and Match_Pos + + +----+------+---------------+--------------------+--------------------+ + | # | mask | match (1 bit) | match_many (1 bit) | match_pos (2 bits) | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 0 | 0000 | 0 | 0 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 1 | 0001 | 1 | 0 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 2 | 0010 | 1 | 0 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 3 | 0011 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 4 | 0100 | 1 | 0 | 10 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 5 | 0101 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 6 | 0110 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 7 | 0111 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 8 | 1000 | 1 | 0 | 11 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 9 | 1001 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 10 | 1010 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 11 | 1011 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 12 | 1100 | 1 | 1 | 10 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 13 | 1101 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 14 | 1110 | 1 | 1 | 01 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ + | 15 | 1111 | 1 | 1 | 00 | + | | | | | | + +----+------+---------------+--------------------+--------------------+ The input *mask* hash bit X (X = 0 .. 3) set to 1 if input signature is equal to bucket signature X and set to 0 otherwise. The outputs *match*, *match_many* and *match_pos* are 1 bit, 1 bit and 2 bits in size respectively and their meaning has been explained above. -As displayed in Table 29, the lookup tables for *match* and *match_many* can be collapsed into a single 32-bit value and the lookup table for +As displayed in :numref:`table_qos_29`, the lookup tables for *match* and *match_many* can be collapsed into a single 32-bit value and the lookup table for *match_pos* can be collapsed into a 64-bit value. Given the input *mask*, the values for *match*, *match_many* and *match_pos* can be obtained by indexing their respective bit array to extract 1 bit, 1 bit and 2 bits respectively with branchless logic. -.. _pg_table_29: +.. _table_qos_29: -**Table 29 Collapsed Lookup Tables for Match, Match_Many and Match_Pos** +.. table:: Collapsed Lookup Tables for Match, Match_Many and Match_Pos -+------------+------------------------------------------+-------------------+ -| | Bit array | Hexadecimal value | -| | | | -+------------+------------------------------------------+-------------------+ -| match | 1111_1111_1111_1110 | 0xFFFELLU | -| | | | -+------------+------------------------------------------+-------------------+ -| match_many | 1111_1110_1110_1000 | 0xFEE8LLU | -| | | | -+------------+------------------------------------------+-------------------+ -| match_pos | 0001_0010_0001_0011__0001_0010_0001_0000 | 0x12131210LLU | -| | | | -+------------+------------------------------------------+-------------------+ + +------------+------------------------------------------+-------------------+ + | | Bit array | Hexadecimal value | + | | | | + +------------+------------------------------------------+-------------------+ + | match | 1111_1111_1111_1110 | 0xFFFELLU | + | | | | + +------------+------------------------------------------+-------------------+ + | match_many | 1111_1110_1110_1000 | 0xFEE8LLU | + | | | | + +------------+------------------------------------------+-------------------+ + | match_pos | 0001_0010_0001_0011__0001_0010_0001_0000 | 0x12131210LLU | + | | | | + +------------+------------------------------------------+-------------------+ The pseudo-code for match, match_many and match_pos is:: @@ -828,7 +828,7 @@ The pseudo-code for match, match_many and match_pos is:: Single Key Size Hash Tables """"""""""""""""""""""""""" -:numref:`figure_figure37`, :numref:`figure_figure38`, Table 30 and 31 detail the main data structures used to implement 8-byte and 16-byte key hash tables +:numref:`figure_figure37`, :numref:`figure_figure38`, :numref:`table_qos_30` and :numref:`table_qos_31` detail the main data structures used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). .. _figure_figure37: @@ -845,67 +845,67 @@ Single Key Size Hash Tables Data Structures for 16-byte Key Hash Tables -.. _pg_table_30: - -**Table 30 Main Large Data Structures (Arrays) used for 8-byte and 16-byte Key Size Hash Tables** - -+---+-------------------------+------------------------------+----------------------+------------------------------------+ -| # | Array name | Number of entries | Entry size (bytes) | Description | -| | | | | | -+===+=========================+==============================+======================+====================================+ -| 1 | Bucket array | n_buckets (configurable) | *8-byte key size:* | Buckets of the hash table. | -| | | | | | -| | | | 64 + 4 x entry_size | | -| | | | | | -| | | | | | -| | | | *16-byte key size:* | | -| | | | | | -| | | | 128 + 4 x entry_size | | -| | | | | | -+---+-------------------------+------------------------------+----------------------+------------------------------------+ -| 2 | Bucket extensions array | n_buckets_ext (configurable) | *8-byte key size:* | This array is only created for | -| | | | | extendable bucket tables. | -| | | | | | -| | | | 64 + 4 x entry_size | | -| | | | | | -| | | | | | -| | | | *16-byte key size:* | | -| | | | | | -| | | | 128 + 4 x entry_size | | -| | | | | | -+---+-------------------------+------------------------------+----------------------+------------------------------------+ - -.. _pg_table_31: - -**Table 31 Field Description for Bucket Array Entry (8-byte and 16-byte Key Hash Tables)** - -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| # | Field name | Field size (bytes) | Description | -| | | | | -+===+===============+====================+===============================================================================+ -| 1 | Valid | 8 | Bit X (X = 0 .. 3) is set to 1 if key X is valid or to 0 otherwise. | -| | | | | -| | | | Bit 4 is only used for extendable bucket tables to help with the | -| | | | implementation of the branchless logic. In this case, bit 4 is set to 1 if | -| | | | next pointer is valid (not NULL) or to 0 otherwise. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 2 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the current bucket | -| | | | stored as array of 4 entries of 2 bytes each. Entry 0 stores the index | -| | | | (0 .. 3) of the MRU key, while entry 3 stores the index of the LRU key. | -| | | | | -| | | | For extendable bucket tables, this field represents the next pointer (i.e. | -| | | | the pointer to the next group of 4 keys linked to the current bucket). The | -| | | | next pointer is not NULL if the bucket is currently extended or NULL | -| | | | otherwise. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 3 | Key [0 .. 3] | 4 x key_size | Full keys. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ -| 4 | Data [0 .. 3] | 4 x entry_size | Full key values (key data) associated with keys 0 .. 3. | -| | | | | -+---+---------------+--------------------+-------------------------------------------------------------------------------+ +.. _table_qos_30: + +.. table:: Main Large Data Structures (Arrays) used for 8-byte and 16-byte Key Size Hash Tables + + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + | # | Array name | Number of entries | Entry size (bytes) | Description | + | | | | | | + +===+=========================+==============================+======================+====================================+ + | 1 | Bucket array | n_buckets (configurable) | *8-byte key size:* | Buckets of the hash table. | + | | | | | | + | | | | 64 + 4 x entry_size | | + | | | | | | + | | | | | | + | | | | *16-byte key size:* | | + | | | | | | + | | | | 128 + 4 x entry_size | | + | | | | | | + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + | 2 | Bucket extensions array | n_buckets_ext (configurable) | *8-byte key size:* | This array is only created for | + | | | | | extendible bucket tables. | + | | | | | | + | | | | 64 + 4 x entry_size | | + | | | | | | + | | | | | | + | | | | *16-byte key size:* | | + | | | | | | + | | | | 128 + 4 x entry_size | | + | | | | | | + +---+-------------------------+------------------------------+----------------------+------------------------------------+ + +.. _table_qos_31: + +.. table:: Field Description for Bucket Array Entry (8-byte and 16-byte Key Hash Tables) + + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | # | Field name | Field size (bytes) | Description | + | | | | | + +===+===============+====================+===============================================================================+ + | 1 | Valid | 8 | Bit X (X = 0 .. 3) is set to 1 if key X is valid or to 0 otherwise. | + | | | | | + | | | | Bit 4 is only used for extendible bucket tables to help with the | + | | | | implementation of the branchless logic. In this case, bit 4 is set to 1 if | + | | | | next pointer is valid (not NULL) or to 0 otherwise. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 2 | Next Ptr/LRU | 8 | For LRU tables, this fields represents the LRU list for the current bucket | + | | | | stored as array of 4 entries of 2 bytes each. Entry 0 stores the index | + | | | | (0 .. 3) of the MRU key, while entry 3 stores the index of the LRU key. | + | | | | | + | | | | For extendible bucket tables, this field represents the next pointer (i.e. | + | | | | the pointer to the next group of 4 keys linked to the current bucket). The | + | | | | next pointer is not NULL if the bucket is currently extended or NULL | + | | | | otherwise. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 3 | Key [0 .. 3] | 4 x key_size | Full keys. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ + | 4 | Data [0 .. 3] | 4 x entry_size | Full key values (key data) associated with keys 0 .. 3. | + | | | | | + +---+---------------+--------------------+-------------------------------------------------------------------------------+ and detail the bucket search pipeline used to implement 8-byte and 16-byte key hash tables (either LRU or extendable bucket, either with pre-computed signature or "do-sig"). @@ -919,42 +919,42 @@ For each pipeline stage, the described operations are applied to each of the two Tables) -.. _pg_table_32: - -**Table 32 Description of the Bucket Search Pipeline Stages (8-byte and 16-byte Key Hash Tables)** - -+---+---------------------------+-----------------------------------------------------------------------------+ -| # | Stage name | Description | -| | | | -+===+===========================+=============================================================================+ -| 0 | Prefetch packet meta-data | #. Select next two packets from the burst of input packets. | -| | | | -| | | #. Prefetch packet meta-data containing the key and key signature. | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ -| 1 | Prefetch table bucket | #. Read the key signature from the packet meta-data (for extendable bucket | -| | | hash tables) or read the key from the packet meta-data and compute key | -| | | signature (for LRU tables). | -| | | | -| | | #. Identify the bucket ID using the key signature. | -| | | | -| | | #. Prefetch the bucket. | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ -| 2 | Prefetch table data | #. Read the bucket. | -| | | | -| | | #. Compare all 4 bucket keys against the input key. | -| | | | -| | | #. Report input key as lookup hit only when a match is identified (more | -| | | than one key match is not possible) | -| | | | -| | | #. For LRU tables only, use branchless logic to update the bucket LRU list | -| | | (the current key becomes the new MRU) only on lookup hit. | -| | | | -| | | #. Prefetch the key value (key data) associated with the matched key (to | -| | | avoid branches, this is done on both lookup hit and miss). | -| | | | -+---+---------------------------+-----------------------------------------------------------------------------+ +.. _table_qos_32: + +.. table:: Description of the Bucket Search Pipeline Stages (8-byte and 16-byte Key Hash Tables) + + +---+---------------------------+-----------------------------------------------------------------------------+ + | # | Stage name | Description | + | | | | + +===+===========================+=============================================================================+ + | 0 | Prefetch packet meta-data | #. Select next two packets from the burst of input packets. | + | | | | + | | | #. Prefetch packet meta-data containing the key and key signature. | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ + | 1 | Prefetch table bucket | #. Read the key signature from the packet meta-data (for extendable bucket | + | | | hash tables) or read the key from the packet meta-data and compute key | + | | | signature (for LRU tables). | + | | | | + | | | #. Identify the bucket ID using the key signature. | + | | | | + | | | #. Prefetch the bucket. | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ + | 2 | Prefetch table data | #. Read the bucket. | + | | | | + | | | #. Compare all 4 bucket keys against the input key. | + | | | | + | | | #. Report input key as lookup hit only when a match is identified (more | + | | | than one key match is not possible) | + | | | | + | | | #. For LRU tables only, use branchless logic to update the bucket LRU list | + | | | (the current key becomes the new MRU) only on lookup hit. | + | | | | + | | | #. Prefetch the key value (key data) associated with the matched key (to | + | | | avoid branches, this is done on both lookup hit and miss). | + | | | | + +---+---------------------------+-----------------------------------------------------------------------------+ Additional notes: @@ -1041,27 +1041,27 @@ The reserved actions are handled directly by the Packet Framework without the us through the table action handler configuration. A special category of the reserved actions is represented by the next hop actions, which regulate the packet flow between input ports, tables and output ports through the pipeline. -Table 33 lists the next hop actions. - -.. _pg_table_33: - -**Table 33 Next Hop Actions (Reserved)** - -+---+---------------------+-----------------------------------------------------------------------------------+ -| # | Next hop action | Description | -| | | | -+===+=====================+===================================================================================+ -| 1 | Drop | Drop the current packet. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ -| 2 | Send to output port | Send the current packet to specified output port. The output port ID is metadata | -| | | stored in the same table entry. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ -| 3 | Send to table | Send the current packet to specified table. The table ID is metadata stored in | -| | | the same table entry. | -| | | | -+---+---------------------+-----------------------------------------------------------------------------------+ +:numref:`table_qos_33` lists the next hop actions. + +.. _table_qos_33: + +.. table:: Next Hop Actions (Reserved) + + +---+---------------------+-----------------------------------------------------------------------------------+ + | # | Next hop action | Description | + | | | | + +===+=====================+===================================================================================+ + | 1 | Drop | Drop the current packet. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ + | 2 | Send to output port | Send the current packet to specified output port. The output port ID is metadata | + | | | stored in the same table entry. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ + | 3 | Send to table | Send the current packet to specified table. The table ID is metadata stored in | + | | | the same table entry. | + | | | | + +---+---------------------+-----------------------------------------------------------------------------------+ User Actions ^^^^^^^^^^^^ @@ -1072,39 +1072,39 @@ and their associated meta-data is private to each table. Within the same table, all the table entries (including the table default entry) share the same definition for the user actions and their associated meta-data, with each table entry having its own set of enabled user actions and its own copy of the action meta-data. -Table 34 contains a non-exhaustive list of user action examples. - -.. _pg_table_34: - -**Table 34 User Action Examples** - -+---+-----------------------------------+---------------------------------------------------------------------+ -| # | User action | Description | -| | | | -+===+===================================+=====================================================================+ -| 1 | Metering | Per flow traffic metering using the srTCM and trTCM algorithms. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 2 | Statistics | Update the statistics counters maintained per flow. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 3 | App ID | Per flow state machine fed by variable length sequence of packets | -| | | at the flow initialization with the purpose of identifying the | -| | | traffic type and application. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 4 | Push/pop labels | Push/pop VLAN/MPLS labels to/from the current packet. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 5 | Network Address Translation (NAT) | Translate between the internal (LAN) and external (WAN) IP | -| | | destination/source address and/or L4 protocol destination/source | -| | | port. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ -| 6 | TTL update | Decrement IP TTL and, in case of IPv4 packets, update the IP | -| | | checksum. | -| | | | -+---+-----------------------------------+---------------------------------------------------------------------+ +:numref:`table_qos_34` contains a non-exhaustive list of user action examples. + +.. _table_qos_34: + +.. table:: User Action Examples + + +---+-----------------------------------+---------------------------------------------------------------------+ + | # | User action | Description | + | | | | + +===+===================================+=====================================================================+ + | 1 | Metering | Per flow traffic metering using the srTCM and trTCM algorithms. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 2 | Statistics | Update the statistics counters maintained per flow. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 3 | App ID | Per flow state machine fed by variable length sequence of packets | + | | | at the flow initialization with the purpose of identifying the | + | | | traffic type and application. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 4 | Push/pop labels | Push/pop VLAN/MPLS labels to/from the current packet. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 5 | Network Address Translation (NAT) | Translate between the internal (LAN) and external (WAN) IP | + | | | destination/source address and/or L4 protocol destination/source | + | | | port. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ + | 6 | TTL update | Decrement IP TTL and, in case of IPv4 packets, update the IP | + | | | checksum. | + | | | | + +---+-----------------------------------+---------------------------------------------------------------------+ Multicore Scaling ----------------- diff --git a/doc/guides/prog_guide/qos_framework.rst b/doc/guides/prog_guide/qos_framework.rst index 72cfed9..c4e390c 100644 --- a/doc/guides/prog_guide/qos_framework.rst +++ b/doc/guides/prog_guide/qos_framework.rst @@ -49,73 +49,73 @@ This pipeline can be built using reusable DPDK software libraries. The main blocks implementing QoS in this pipeline are: the policer, the dropper and the scheduler. A functional description of each block is provided in the following table. -.. _pg_table_1: - -**Table 1. Packet Processing Pipeline Implementing QoS** - -+---+------------------------+--------------------------------------------------------------------------------+ -| # | Block | Functional Description | -| | | | -+===+========================+================================================================================+ -| 1 | Packet I/O RX & TX | Packet reception/ transmission from/to multiple NIC ports. Poll mode drivers | -| | | (PMDs) for Intel 1 GbE/10 GbE NICs. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 2 | Packet parser | Identify the protocol stack of the input packet. Check the integrity of the | -| | | packet headers. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 3 | Flow classification | Map the input packet to one of the known traffic flows. Exact match table | -| | | lookup using configurable hash function (jhash, CRC and so on) and bucket | -| | | logic to handle collisions. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 4 | Policer | Packet metering using srTCM (RFC 2697) or trTCM (RFC2698) algorithms. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 5 | Load Balancer | Distribute the input packets to the application workers. Provide uniform load | -| | | to each worker. Preserve the affinity of traffic flows to workers and the | -| | | packet order within each flow. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 6 | Worker threads | Placeholders for the customer specific application workload (for example, IP | -| | | stack and so on). | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 7 | Dropper | Congestion management using the Random Early Detection (RED) algorithm | -| | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED). | -| | | Drop packets based on the current scheduler queue load level and packet | -| | | priority. When congestion is experienced, lower priority packets are dropped | -| | | first. | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ -| 8 | Hierarchical Scheduler | 5-level hierarchical scheduler (levels are: output port, subport, pipe, | -| | | traffic class and queue) with thousands (typically 64K) leaf nodes (queues). | -| | | Implements traffic shaping (for subport and pipe levels), strict priority | -| | | (for traffic class level) and Weighted Round Robin (WRR) (for queues within | -| | | each pipe traffic class). | -| | | | -+---+------------------------+--------------------------------------------------------------------------------+ +.. _table_qos_1: + +.. table:: Packet Processing Pipeline Implementing QoS + + +---+------------------------+--------------------------------------------------------------------------------+ + | # | Block | Functional Description | + | | | | + +===+========================+================================================================================+ + | 1 | Packet I/O RX & TX | Packet reception/ transmission from/to multiple NIC ports. Poll mode drivers | + | | | (PMDs) for Intel 1 GbE/10 GbE NICs. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 2 | Packet parser | Identify the protocol stack of the input packet. Check the integrity of the | + | | | packet headers. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 3 | Flow classification | Map the input packet to one of the known traffic flows. Exact match table | + | | | lookup using configurable hash function (jhash, CRC and so on) and bucket | + | | | logic to handle collisions. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 4 | Policer | Packet metering using srTCM (RFC 2697) or trTCM (RFC2698) algorithms. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 5 | Load Balancer | Distribute the input packets to the application workers. Provide uniform load | + | | | to each worker. Preserve the affinity of traffic flows to workers and the | + | | | packet order within each flow. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 6 | Worker threads | Placeholders for the customer specific application workload (for example, IP | + | | | stack and so on). | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 7 | Dropper | Congestion management using the Random Early Detection (RED) algorithm | + | | | (specified by the Sally Floyd - Van Jacobson paper) or Weighted RED (WRED). | + | | | Drop packets based on the current scheduler queue load level and packet | + | | | priority. When congestion is experienced, lower priority packets are dropped | + | | | first. | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ + | 8 | Hierarchical Scheduler | 5-level hierarchical scheduler (levels are: output port, subport, pipe, | + | | | traffic class and queue) with thousands (typically 64K) leaf nodes (queues). | + | | | Implements traffic shaping (for subport and pipe levels), strict priority | + | | | (for traffic class level) and Weighted Round Robin (WRR) (for queues within | + | | | each pipe traffic class). | + | | | | + +---+------------------------+--------------------------------------------------------------------------------+ The infrastructure blocks used throughout the packet processing pipeline are listed in the following table. -.. _pg_table_2: +.. _table_qos_2: -**Table 2. Infrastructure Blocks Used by the Packet Processing Pipeline** +.. table:: Infrastructure Blocks Used by the Packet Processing Pipeline -+---+-----------------------+-----------------------------------------------------------------------+ -| # | Block | Functional Description | -| | | | -+===+=======================+=======================================================================+ -| 1 | Buffer manager | Support for global buffer pools and private per-thread buffer caches. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ -| 2 | Queue manager | Support for message passing between pipeline blocks. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ -| 3 | Power saving | Support for power saving during low activity periods. | -| | | | -+---+-----------------------+-----------------------------------------------------------------------+ + +---+-----------------------+-----------------------------------------------------------------------+ + | # | Block | Functional Description | + | | | | + +===+=======================+=======================================================================+ + | 1 | Buffer manager | Support for global buffer pools and private per-thread buffer caches. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ + | 2 | Queue manager | Support for message passing between pipeline blocks. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ + | 3 | Power saving | Support for power saving during low activity periods. | + | | | | + +---+-----------------------+-----------------------------------------------------------------------+ The mapping of pipeline blocks to CPU cores is configurable based on the performance level required by each specific application and the set of features enabled for each block. @@ -170,50 +170,50 @@ Each queue hosts packets from one or multiple connections of the same type belon The functionality of each hierarchical level is detailed in the following table. -.. _pg_table_3: - -**Table 3. Port Scheduling Hierarchy** - -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| # | Level | Siblings per Parent | Functional Description | -| | | | | -+===+====================+============================+===============================================================+ -| 1 | Port | - | #. Output Ethernet port 1/10/40 GbE. | -| | | | | -| | | | #. Multiple ports are scheduled in round robin order with | -| | | | all ports having equal priority. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 2 | Subport | Configurable (default: 8) | #. Traffic shaping using token bucket algorithm (one token | -| | | | bucket per subport). | -| | | | | -| | | | #. Upper limit enforced per Traffic Class (TC) at the | -| | | | subport level. | -| | | | | -| | | | #. Lower priority TCs able to reuse subport bandwidth | -| | | | currently unused by higher priority TCs. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 3 | Pipe | Configurable (default: 4K) | #. Traffic shaping using the token bucket algorithm (one | -| | | | token bucket per pipe. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 4 | Traffic Class (TC) | 4 | #. TCs of the same pipe handled in strict priority order. | -| | | | | -| | | | #. Upper limit enforced per TC at the pipe level. | -| | | | | -| | | | #. Lower priority TCs able to reuse pipe bandwidth currently | -| | | | unused by higher priority TCs. | -| | | | | -| | | | #. When subport TC is oversubscribed (configuration time | -| | | | event), pipe TC upper limit is capped to a dynamically | -| | | | adjusted value that is shared by all the subport pipes. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ -| 5 | Queue | 4 | #. Queues of the same TC are serviced using Weighted Round | -| | | | Robin (WRR) according to predefined weights. | -| | | | | -+---+--------------------+----------------------------+---------------------------------------------------------------+ +.. _table_qos_3: + +.. table:: Port Scheduling Hierarchy + + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | # | Level | Siblings per Parent | Functional Description | + | | | | | + +===+====================+============================+===============================================================+ + | 1 | Port | - | #. Output Ethernet port 1/10/40 GbE. | + | | | | | + | | | | #. Multiple ports are scheduled in round robin order with | + | | | | all ports having equal priority. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 2 | Subport | Configurable (default: 8) | #. Traffic shaping using token bucket algorithm (one token | + | | | | bucket per subport). | + | | | | | + | | | | #. Upper limit enforced per Traffic Class (TC) at the | + | | | | subport level. | + | | | | | + | | | | #. Lower priority TCs able to reuse subport bandwidth | + | | | | currently unused by higher priority TCs. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 3 | Pipe | Configurable (default: 4K) | #. Traffic shaping using the token bucket algorithm (one | + | | | | token bucket per pipe. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 4 | Traffic Class (TC) | 4 | #. TCs of the same pipe handled in strict priority order. | + | | | | | + | | | | #. Upper limit enforced per TC at the pipe level. | + | | | | | + | | | | #. Lower priority TCs able to reuse pipe bandwidth currently | + | | | | unused by higher priority TCs. | + | | | | | + | | | | #. When subport TC is oversubscribed (configuration time | + | | | | event), pipe TC upper limit is capped to a dynamically | + | | | | adjusted value that is shared by all the subport pipes. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ + | 5 | Queue | 4 | #. Queues of the same TC are serviced using Weighted Round | + | | | | Robin (WRR) according to predefined weights. | + | | | | | + +---+--------------------+----------------------------+---------------------------------------------------------------+ Application Programming Interface (API) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -297,65 +297,65 @@ A schematic of the internal data structures in shown in with details in. Internal Data Structures per Port -.. _pg_table_4: - -**Table 4. Scheduler Internal Data Structures per Port** - -+---+----------------------+-------------------------+---------------------+------------------------------+---------------------------------------------------+ -| # | Data structure | Size (bytes) | # per port | Access type | Description | -| | | | | | | -| | | | +-------------+----------------+---------------------------------------------------+ -| | | | | Enq | Deq | | -| | | | | | | | -+===+======================+=========================+=====================+=============+================+===================================================+ -| 1 | Subport table entry | 64 | # subports per port | - | Rd, Wr | Persistent subport data (credits, etc). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 2 | Pipe table entry | 64 | # pipes per port | - | Rd, Wr | Persistent data for pipe, its TCs and its queues | -| | | | | | | (credits, etc) that is updated during run-time. | -| | | | | | | | -| | | | | | | The pipe configuration parameters do not change | -| | | | | | | during run-time. The same pipe configuration | -| | | | | | | parameters are shared by multiple pipes, | -| | | | | | | therefore they are not part of pipe table entry. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 3 | Queue table entry | 4 | #queues per port | Rd, Wr | Rd, Wr | Persistent queue data (read and write pointers). | -| | | | | | | The queue size is the same per TC for all queues, | -| | | | | | | allowing the queue base address to be computed | -| | | | | | | using a fast formula, so these two parameters are | -| | | | | | | not part of queue table entry. | -| | | | | | | | -| | | | | | | The queue table entries for any given pipe are | -| | | | | | | stored in the same cache line. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 4 | Queue storage area | Config (default: 64 x8) | # queues per port | Wr | Rd | Array of elements per queue; each element is 8 | -| | | | | | | byte in size (mbuf pointer). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 5 | Active queues bitmap | 1 bit per queue | 1 | Wr (Set) | Rd, Wr (Clear) | The bitmap maintains one status bit per queue: | -| | | | | | | queue not active (queue is empty) or queue active | -| | | | | | | (queue is not empty). | -| | | | | | | | -| | | | | | | Queue bit is set by the scheduler enqueue and | -| | | | | | | cleared by the scheduler dequeue when queue | -| | | | | | | becomes empty. | -| | | | | | | | -| | | | | | | Bitmap scan operation returns the next non-empty | -| | | | | | | pipe and its status (16-bit mask of active queue | -| | | | | | | in the pipe). | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ -| 6 | Grinder | ~128 | Config (default: 8) | - | Rd, Wr | Short list of active pipes currently under | -| | | | | | | processing. The grinder contains temporary data | -| | | | | | | during pipe processing. | -| | | | | | | | -| | | | | | | Once the current pipe exhausts packets or | -| | | | | | | credits, it is replaced with another active pipe | -| | | | | | | from the bitmap. | -| | | | | | | | -+---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ +.. _table_qos_4: + +.. table:: Scheduler Internal Data Structures per Port + + +---+----------------------+-------------------------+---------------------+------------------------------+---------------------------------------------------+ + | # | Data structure | Size (bytes) | # per port | Access type | Description | + | | | | | | | + | | | | +-------------+----------------+---------------------------------------------------+ + | | | | | Enq | Deq | | + | | | | | | | | + +===+======================+=========================+=====================+=============+================+===================================================+ + | 1 | Subport table entry | 64 | # subports per port | - | Rd, Wr | Persistent subport data (credits, etc). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 2 | Pipe table entry | 64 | # pipes per port | - | Rd, Wr | Persistent data for pipe, its TCs and its queues | + | | | | | | | (credits, etc) that is updated during run-time. | + | | | | | | | | + | | | | | | | The pipe configuration parameters do not change | + | | | | | | | during run-time. The same pipe configuration | + | | | | | | | parameters are shared by multiple pipes, | + | | | | | | | therefore they are not part of pipe table entry. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 3 | Queue table entry | 4 | #queues per port | Rd, Wr | Rd, Wr | Persistent queue data (read and write pointers). | + | | | | | | | The queue size is the same per TC for all queues, | + | | | | | | | allowing the queue base address to be computed | + | | | | | | | using a fast formula, so these two parameters are | + | | | | | | | not part of queue table entry. | + | | | | | | | | + | | | | | | | The queue table entries for any given pipe are | + | | | | | | | stored in the same cache line. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 4 | Queue storage area | Config (default: 64 x8) | # queues per port | Wr | Rd | Array of elements per queue; each element is 8 | + | | | | | | | byte in size (mbuf pointer). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 5 | Active queues bitmap | 1 bit per queue | 1 | Wr (Set) | Rd, Wr (Clear) | The bitmap maintains one status bit per queue: | + | | | | | | | queue not active (queue is empty) or queue active | + | | | | | | | (queue is not empty). | + | | | | | | | | + | | | | | | | Queue bit is set by the scheduler enqueue and | + | | | | | | | cleared by the scheduler dequeue when queue | + | | | | | | | becomes empty. | + | | | | | | | | + | | | | | | | Bitmap scan operation returns the next non-empty | + | | | | | | | pipe and its status (16-bit mask of active queue | + | | | | | | | in the pipe). | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ + | 6 | Grinder | ~128 | Config (default: 8) | - | Rd, Wr | Short list of active pipes currently under | + | | | | | | | processing. The grinder contains temporary data | + | | | | | | | during pipe processing. | + | | | | | | | | + | | | | | | | Once the current pipe exhausts packets or | + | | | | | | | credits, it is replaced with another active pipe | + | | | | | | | from the bitmap. | + | | | | | | | | + +---+----------------------+-------------------------+---------------------+-------------+----------------+---------------------------------------------------+ Multicore Scaling Strategy ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -578,29 +578,29 @@ As the greatest common divisor for all packet lengths is one byte, the unit of c The number of credits required for the transmission of a packet of n bytes is equal to (n+h), where h is equal to the number of framing overhead bytes per packet. -.. _pg_table_5: - -**Table 5. Ethernet Frame Overhead Fields** - -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| # | Packet field | Length (bytes) | Comments | -| | | | | -+===+================================+================+===========================================================================+ -| 1 | Preamble | 7 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 2 | Start of Frame Delimiter (SFD) | 1 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 3 | Frame Check Sequence (FCS) | 4 | Considered overhead only if not included in the mbuf packet length field. | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 4 | Inter Frame Gap (IFG) | 12 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ -| 5 | Total | 24 | | -| | | | | -+---+--------------------------------+----------------+---------------------------------------------------------------------------+ +.. _table_qos_5: + +.. table:: Ethernet Frame Overhead Fields + + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | # | Packet field | Length (bytes) | Comments | + | | | | | + +===+================================+================+===========================================================================+ + | 1 | Preamble | 7 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 2 | Start of Frame Delimiter (SFD) | 1 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 3 | Frame Check Sequence (FCS) | 4 | Considered overhead only if not included in the mbuf packet length field. | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 4 | Inter Frame Gap (IFG) | 12 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ + | 5 | Total | 24 | | + | | | | | + +---+--------------------------------+----------------+---------------------------------------------------------------------------+ Traffic Shaping """"""""""""""" @@ -608,80 +608,80 @@ Traffic Shaping The traffic shaping for subport and pipe is implemented using a token bucket per subport/per pipe. Each token bucket is implemented using one saturated counter that keeps track of the number of available credits. -The token bucket generic parameters and operations are presented in Table 6 and Table 7. - -.. _pg_table_6: - -**Table 6. Token Bucket Generic Operations** - -+---+------------------------+--------------------+---------------------------------------------------------+ -| # | Token Bucket Parameter | Unit | Description | -| | | | | -+===+========================+====================+=========================================================+ -| 1 | bucket_rate | Credits per second | Rate of adding credits to the bucket. | -| | | | | -+---+------------------------+--------------------+---------------------------------------------------------+ -| 2 | bucket_size | Credits | Max number of credits that can be stored in the bucket. | -| | | | | -+---+------------------------+--------------------+---------------------------------------------------------+ - -.. _pg_table_7: - -**Table 7. Token Bucket Generic Parameters** - -+---+------------------------+------------------------------------------------------------------------------+ -| # | Token Bucket Operation | Description | -| | | | -+===+========================+==============================================================================+ -| 1 | Initialization | Bucket set to a predefined value, e.g. zero or half of the bucket size. | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ -| 2 | Credit update | Credits are added to the bucket on top of existing ones, either periodically | -| | | or on demand, based on the bucket_rate. Credits cannot exceed the upper | -| | | limit defined by the bucket_size, so any credits to be added to the bucket | -| | | while the bucket is full are dropped. | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | -| | | from the bucket. The packet can only be sent if enough credits are in the | -| | | bucket to send the full packet (packet bytes and framing overhead for the | -| | | packet). | -| | | | -+---+------------------------+------------------------------------------------------------------------------+ +The token bucket generic parameters and operations are presented in :numref:`table_qos_6` and :numref:`table_qos_7`. + +.. _table_qos_6: + +.. table:: Token Bucket Generic Operations + + +---+------------------------+--------------------+---------------------------------------------------------+ + | # | Token Bucket Parameter | Unit | Description | + | | | | | + +===+========================+====================+=========================================================+ + | 1 | bucket_rate | Credits per second | Rate of adding credits to the bucket. | + | | | | | + +---+------------------------+--------------------+---------------------------------------------------------+ + | 2 | bucket_size | Credits | Max number of credits that can be stored in the bucket. | + | | | | | + +---+------------------------+--------------------+---------------------------------------------------------+ + +.. _table_qos_7: + +.. table:: Token Bucket Generic Parameters + + +---+------------------------+------------------------------------------------------------------------------+ + | # | Token Bucket Operation | Description | + | | | | + +===+========================+==============================================================================+ + | 1 | Initialization | Bucket set to a predefined value, e.g. zero or half of the bucket size. | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ + | 2 | Credit update | Credits are added to the bucket on top of existing ones, either periodically | + | | | or on demand, based on the bucket_rate. Credits cannot exceed the upper | + | | | limit defined by the bucket_size, so any credits to be added to the bucket | + | | | while the bucket is full are dropped. | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | + | | | from the bucket. The packet can only be sent if enough credits are in the | + | | | bucket to send the full packet (packet bytes and framing overhead for the | + | | | packet). | + | | | | + +---+------------------------+------------------------------------------------------------------------------+ To implement the token bucket generic operations described above, -the current design uses the persistent data structure presented in, -while the implementation of the token bucket operations is described in Table 9. - -.. _pg_table_8: - -**Table 8. Token Bucket Persistent Data Structure** - -+---+------------------------+-------+----------------------------------------------------------------------+ -| # | Token bucket field | Unit | Description | -| | | | | -+===+========================+=======+======================================================================+ -| 1 | tb_time | Bytes | Time of the last credit update. Measured in bytes instead of seconds | -| | | | or CPU cycles for ease of credit consumption operation | -| | | | (as the current time is also maintained in bytes). | -| | | | | -| | | | See Section 26.2.4.5.1 "Internal Time Reference" for an | -| | | | explanation of why the time is maintained in byte units. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 2 | tb_period | Bytes | Time period that should elapse since the last credit update in order | -| | | | for the bucket to be awarded tb_credits_per_period worth or credits. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 3 | tb_credits_per_period | Bytes | Credit allowance per tb_period. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 4 | tb_size | Bytes | Bucket size, i.e. upper limit for the tb_credits. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ -| 5 | tb_credits | Bytes | Number of credits currently in the bucket. | -| | | | | -+---+------------------------+-------+----------------------------------------------------------------------+ +the current design uses the persistent data structure presented in :numref:`table_qos_8`, +while the implementation of the token bucket operations is described in :numref:`table_qos_9`. + +.. _table_qos_8: + +.. table:: Token Bucket Persistent Data Structure + + +---+------------------------+-------+----------------------------------------------------------------------+ + | # | Token bucket field | Unit | Description | + | | | | | + +===+========================+=======+======================================================================+ + | 1 | tb_time | Bytes | Time of the last credit update. Measured in bytes instead of seconds | + | | | | or CPU cycles for ease of credit consumption operation | + | | | | (as the current time is also maintained in bytes). | + | | | | | + | | | | See Section 26.2.4.5.1 "Internal Time Reference" for an | + | | | | explanation of why the time is maintained in byte units. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 2 | tb_period | Bytes | Time period that should elapse since the last credit update in order | + | | | | for the bucket to be awarded tb_credits_per_period worth or credits. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 3 | tb_credits_per_period | Bytes | Credit allowance per tb_period. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 4 | tb_size | Bytes | Bucket size, i.e. upper limit for the tb_credits. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ + | 5 | tb_credits | Bytes | Number of credits currently in the bucket. | + | | | | | + +---+------------------------+-------+----------------------------------------------------------------------+ The bucket rate (in bytes per second) can be computed with the following formula: @@ -689,65 +689,65 @@ The bucket rate (in bytes per second) can be computed with the following formula where, r = port line rate (in bytes per second). -.. _pg_table_9: - -**Table 9. Token Bucket Operations** - -+---+-------------------------+-----------------------------------------------------------------------------+ -| # | Token bucket operation | Description | -| | | | -+===+=========================+=============================================================================+ -| 1 | Initialization | *tb_credits = 0; or tb_credits = tb_size / 2;* | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ -| 2 | Credit update | Credit update options: | -| | | | -| | | * Every time a packet is sent for a port, update the credits of all the | -| | | the subports and pipes of that port. Not feasible. | -| | | | -| | | * Every time a packet is sent, update the credits for the pipe and | -| | | subport. Very accurate, but not needed (a lot of calculations). | -| | | | -| | | * Every time a pipe is selected (that is, picked by one | -| | | of the grinders), update the credits for the pipe and its subport. | -| | | | -| | | The current implementation is using option 3. According to Section | -| | | 26.2.4.4 "Dequeue State Machine", the pipe and subport credits are | -| | | updated every time a pipe is selected by the dequeue process before the | -| | | pipe and subport credits are actually used. | -| | | | -| | | The implementation uses a tradeoff between accuracy and speed by updating | -| | | the bucket credits only when at least a full *tb_period* has elapsed since | -| | | the last update. | -| | | | -| | | * Full accuracy can be achieved by selecting the value for *tb_period* | -| | | for which *tb_credits_per_period = 1*. | -| | | | -| | | * When full accuracy is not required, better performance is achieved by | -| | | setting *tb_credits* to a larger value. | -| | | | -| | | Update operations: | -| | | | -| | | * n_periods = (time - tb_time) / tb_period; | -| | | | -| | | * tb_credits += n_periods * tb_credits_per_period; | -| | | | -| | | * tb_credits = min(tb_credits, tb_size); | -| | | | -| | | * tb_time += n_periods * tb_period; | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | -| | (on packet scheduling) | from the bucket. The packet can only be sent if enough credits are in the | -| | | bucket to send the full packet (packet bytes and framing overhead for the | -| | | packet). | -| | | | -| | | Scheduling operations: | -| | | | -| | | pkt_credits = pkt_len + frame_overhead; | -| | | if (tb_credits >= pkt_credits){tb_credits -= pkt_credits;} | -| | | | -+---+-------------------------+-----------------------------------------------------------------------------+ +.. _table_qos_9: + +.. table:: Token Bucket Operations + + +---+-------------------------+-----------------------------------------------------------------------------+ + | # | Token bucket operation | Description | + | | | | + +===+=========================+=============================================================================+ + | 1 | Initialization | *tb_credits = 0; or tb_credits = tb_size / 2;* | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ + | 2 | Credit update | Credit update options: | + | | | | + | | | * Every time a packet is sent for a port, update the credits of all the | + | | | the subports and pipes of that port. Not feasible. | + | | | | + | | | * Every time a packet is sent, update the credits for the pipe and | + | | | subport. Very accurate, but not needed (a lot of calculations). | + | | | | + | | | * Every time a pipe is selected (that is, picked by one | + | | | of the grinders), update the credits for the pipe and its subport. | + | | | | + | | | The current implementation is using option 3. According to Section | + | | | 26.2.4.4 "Dequeue State Machine", the pipe and subport credits are | + | | | updated every time a pipe is selected by the dequeue process before the | + | | | pipe and subport credits are actually used. | + | | | | + | | | The implementation uses a tradeoff between accuracy and speed by updating | + | | | the bucket credits only when at least a full *tb_period* has elapsed since | + | | | the last update. | + | | | | + | | | * Full accuracy can be achieved by selecting the value for *tb_period* | + | | | for which *tb_credits_per_period = 1*. | + | | | | + | | | * When full accuracy is not required, better performance is achieved by | + | | | setting *tb_credits* to a larger value. | + | | | | + | | | Update operations: | + | | | | + | | | * n_periods = (time - tb_time) / tb_period; | + | | | | + | | | * tb_credits += n_periods * tb_credits_per_period; | + | | | | + | | | * tb_credits = min(tb_credits, tb_size); | + | | | | + | | | * tb_time += n_periods * tb_period; | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the necessary number of credits is removed | + | | (on packet scheduling) | from the bucket. The packet can only be sent if enough credits are in the | + | | | bucket to send the full packet (packet bytes and framing overhead for the | + | | | packet). | + | | | | + | | | Scheduling operations: | + | | | | + | | | pkt_credits = pkt_len + frame_overhead; | + | | | if (tb_credits >= pkt_credits){tb_credits -= pkt_credits;} | + | | | | + +---+-------------------------+-----------------------------------------------------------------------------+ Traffic Classes """"""""""""""" @@ -770,177 +770,177 @@ so there is no token bucket maintained in this context. The upper limit for the traffic classes at the subport and pipe levels is enforced by periodically refilling the subport / pipe traffic class credit counter, out of which credits are consumed every time a packet is scheduled for that subport / pipe, -as described in Table 10 and Table 11. - -.. _pg_table_10: - -**Table 10. Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure** - -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| # | Subport or pipe field | Unit | Description | -| | | | | -+===+=======================+=======+=======================================================================+ -| 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the 4 TCs of the | -| | | | current subport / pipe. | -| | | | | -| | | | See Section 26.2.4.5.1, "Internal Time Reference" for the | -| | | | explanation of why the time is maintained in byte units. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 2 | tc_period | Bytes | Time between two consecutive updates for the 4 TCs of the current | -| | | | subport / pipe. This is expected to be many times bigger than the | -| | | | typical value of the token bucket tb_period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 3 | tc_credits_per_period | Bytes | Upper limit for the number of credits allowed to be consumed by the | -| | | | current TC during each enforcement period tc_period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ -| 4 | tc_credits | Bytes | Current upper limit for the number of credits that can be consumed by | -| | | | the current traffic class for the remainder of the current | -| | | | enforcement period. | -| | | | | -+---+-----------------------+-------+-----------------------------------------------------------------------+ - -.. _pg_table_11: - -**Table 11. Subport/Pipe Traffic Class Upper Limit Enforcement Operations** - -+---+--------------------------+----------------------------------------------------------------------------+ -| # | Traffic Class Operation | Description | -| | | | -+===+==========================+============================================================================+ -| 1 | Initialization | tc_credits = tc_credits_per_period; | -| | | | -| | | tc_time = tc_period; | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ -| 2 | Credit update | Update operations: | -| | | | -| | | if (time >= tc_time) { | -| | | | -| | | tc_credits = tc_credits_per_period; | -| | | | -| | | tc_time = time + tc_period; | -| | | | -| | | } | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ -| 3 | Credit consumption | As result of packet scheduling, the TC limit is decreased with the | -| | (on packet scheduling) | necessary number of credits. The packet can only be sent if enough credits | -| | | are currently available in the TC limit to send the full packet | -| | | (packet bytes and framing overhead for the packet). | -| | | | -| | | Scheduling operations: | -| | | | -| | | pkt_credits = pk_len + frame_overhead; | -| | | | -| | | if (tc_credits >= pkt_credits) {tc_credits -= pkt_credits;} | -| | | | -+---+--------------------------+----------------------------------------------------------------------------+ +as described in :numref:`table_qos_10` and :numref:`table_qos_11`. + +.. _table_qos_10: + +.. table:: Subport/Pipe Traffic Class Upper Limit Enforcement Persistent Data Structure + + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | # | Subport or pipe field | Unit | Description | + | | | | | + +===+=======================+=======+=======================================================================+ + | 1 | tc_time | Bytes | Time of the next update (upper limit refill) for the 4 TCs of the | + | | | | current subport / pipe. | + | | | | | + | | | | See Section 26.2.4.5.1, "Internal Time Reference" for the | + | | | | explanation of why the time is maintained in byte units. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 2 | tc_period | Bytes | Time between two consecutive updates for the 4 TCs of the current | + | | | | subport / pipe. This is expected to be many times bigger than the | + | | | | typical value of the token bucket tb_period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 3 | tc_credits_per_period | Bytes | Upper limit for the number of credits allowed to be consumed by the | + | | | | current TC during each enforcement period tc_period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + | 4 | tc_credits | Bytes | Current upper limit for the number of credits that can be consumed by | + | | | | the current traffic class for the remainder of the current | + | | | | enforcement period. | + | | | | | + +---+-----------------------+-------+-----------------------------------------------------------------------+ + +.. _table_qos_11: + +.. table:: Subport/Pipe Traffic Class Upper Limit Enforcement Operations + + +---+--------------------------+----------------------------------------------------------------------------+ + | # | Traffic Class Operation | Description | + | | | | + +===+==========================+============================================================================+ + | 1 | Initialization | tc_credits = tc_credits_per_period; | + | | | | + | | | tc_time = tc_period; | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ + | 2 | Credit update | Update operations: | + | | | | + | | | if (time >= tc_time) { | + | | | | + | | | tc_credits = tc_credits_per_period; | + | | | | + | | | tc_time = time + tc_period; | + | | | | + | | | } | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ + | 3 | Credit consumption | As result of packet scheduling, the TC limit is decreased with the | + | | (on packet scheduling) | necessary number of credits. The packet can only be sent if enough credits | + | | | are currently available in the TC limit to send the full packet | + | | | (packet bytes and framing overhead for the packet). | + | | | | + | | | Scheduling operations: | + | | | | + | | | pkt_credits = pk_len + frame_overhead; | + | | | | + | | | if (tc_credits >= pkt_credits) {tc_credits -= pkt_credits;} | + | | | | + +---+--------------------------+----------------------------------------------------------------------------+ Weighted Round Robin (WRR) """""""""""""""""""""""""" -The evolution of the WRR design solution from simple to complex is shown in Table 12. - -.. _pg_table_12: - -**Table 12. Weighted Round Robin (WRR)** - -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| # | All Queues | Equal Weights | All Packets | Strategy | -| | Active? | for All Queues? | Equal? | | -+===+============+=================+=============+==========================================================+ -| 1 | Yes | Yes | Yes | **Byte level round robin** | -| | | | | | -| | | | | *Next queue* queue #i, i = *(i + 1) % n* | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 2 | Yes | Yes | No | **Packet level round robin** | -| | | | | | -| | | | | Consuming one byte from queue #i requires consuming | -| | | | | exactly one token for queue #i. | -| | | | | | -| | | | | T(i) = Accumulated number of tokens previously consumed | -| | | | | from queue #i. Every time a packet is consumed from | -| | | | | queue #i, T(i) is updated as: T(i) += *pkt_len*. | -| | | | | | -| | | | | *Next queue* : queue with the smallest T. | -| | | | | | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 3 | Yes | No | No | **Packet level weighted round robin** | -| | | | | | -| | | | | This case can be reduced to the previous case by | -| | | | | introducing a cost per byte that is different for each | -| | | | | queue. Queues with lower weights have a higher cost per | -| | | | | byte. This way, it is still meaningful to compare the | -| | | | | consumption among different queues in order to select | -| | | | | the next queue. | -| | | | | | -| | | | | w(i) = Weight of queue #i | -| | | | | | -| | | | | t(i) = Tokens per byte for queue #i, defined as the | -| | | | | inverse weight of queue #i. | -| | | | | For example, if w[0..3] = [1:2:4:8], | -| | | | | then t[0..3] = [8:4:2:1]; if w[0..3] = [1:4:15:20], | -| | | | | then t[0..3] = [60:15:4:3]. | -| | | | | Consuming one byte from queue #i requires consuming t(i) | -| | | | | tokens for queue #i. | -| | | | | | -| | | | | T(i) = Accumulated number of tokens previously consumed | -| | | | | from queue #i. Every time a packet is consumed from | -| | | | | queue #i, T(i) is updated as: *T(i) += pkt_len * t(i)*. | -| | | | | *Next queue* : queue with the smallest T. | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ -| 4 | No | No | No | **Packet level weighted round robin with variable queue | -| | | | | status** | -| | | | | | -| | | | | Reduce this case to the previous case by setting the | -| | | | | consumption of inactive queues to a high number, so that | -| | | | | the inactive queues will never be selected by the | -| | | | | smallest T logic. | -| | | | | | -| | | | | To prevent T from overflowing as result of successive | -| | | | | accumulations, T(i) is truncated after each packet | -| | | | | consumption for all queues. | -| | | | | For example, T[0..3] = [1000, 1100, 1200, 1300] | -| | | | | is truncated to T[0..3] = [0, 100, 200, 300] | -| | | | | by subtracting the min T from T(i), i = 0..n. | -| | | | | | -| | | | | This requires having at least one active queue in the | -| | | | | set of input queues, which is guaranteed by the dequeue | -| | | | | state machine never selecting an inactive traffic class. | -| | | | | | -| | | | | *mask(i) = Saturation mask for queue #i, defined as:* | -| | | | | | -| | | | | mask(i) = (queue #i is active)? 0 : 0xFFFFFFFF; | -| | | | | | -| | | | | w(i) = Weight of queue #i | -| | | | | | -| | | | | t(i) = Tokens per byte for queue #i, defined as the | -| | | | | inverse weight of queue #i. | -| | | | | | -| | | | | T(i) = Accumulated numbers of tokens previously consumed | -| | | | | from queue #i. | -| | | | | | -| | | | | *Next queue* : queue with smallest T. | -| | | | | | -| | | | | Before packet consumption from queue #i: | -| | | | | | -| | | | | *T(i) |= mask(i)* | -| | | | | | -| | | | | After packet consumption from queue #i: | -| | | | | | -| | | | | T(j) -= T(i), j != i | -| | | | | | -| | | | | T(i) = pkt_len * t(i) | -| | | | | | -| | | | | Note: T(j) uses the T(i) value before T(i) is updated. | -| | | | | | -+---+------------+-----------------+-------------+----------------------------------------------------------+ +The evolution of the WRR design solution from simple to complex is shown in :numref:`table_qos_12`. + +.. _table_qos_12: + +.. table:: Weighted Round Robin (WRR) + + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | # | All Queues | Equal Weights | All Packets | Strategy | + | | Active? | for All Queues? | Equal? | | + +===+============+=================+=============+==========================================================+ + | 1 | Yes | Yes | Yes | **Byte level round robin** | + | | | | | | + | | | | | *Next queue* queue #i, i = *(i + 1) % n* | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 2 | Yes | Yes | No | **Packet level round robin** | + | | | | | | + | | | | | Consuming one byte from queue #i requires consuming | + | | | | | exactly one token for queue #i. | + | | | | | | + | | | | | T(i) = Accumulated number of tokens previously consumed | + | | | | | from queue #i. Every time a packet is consumed from | + | | | | | queue #i, T(i) is updated as: T(i) += *pkt_len*. | + | | | | | | + | | | | | *Next queue* : queue with the smallest T. | + | | | | | | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 3 | Yes | No | No | **Packet level weighted round robin** | + | | | | | | + | | | | | This case can be reduced to the previous case by | + | | | | | introducing a cost per byte that is different for each | + | | | | | queue. Queues with lower weights have a higher cost per | + | | | | | byte. This way, it is still meaningful to compare the | + | | | | | consumption amongst different queues in order to select | + | | | | | the next queue. | + | | | | | | + | | | | | w(i) = Weight of queue #i | + | | | | | | + | | | | | t(i) = Tokens per byte for queue #i, defined as the | + | | | | | inverse weight of queue #i. | + | | | | | For example, if w[0..3] = [1:2:4:8], | + | | | | | then t[0..3] = [8:4:2:1]; if w[0..3] = [1:4:15:20], | + | | | | | then t[0..3] = [60:15:4:3]. | + | | | | | Consuming one byte from queue #i requires consuming t(i) | + | | | | | tokens for queue #i. | + | | | | | | + | | | | | T(i) = Accumulated number of tokens previously consumed | + | | | | | from queue #i. Every time a packet is consumed from | + | | | | | queue #i, T(i) is updated as: *T(i) += pkt_len * t(i)*. | + | | | | | *Next queue* : queue with the smallest T. | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ + | 4 | No | No | No | **Packet level weighted round robin with variable queue | + | | | | | status** | + | | | | | | + | | | | | Reduce this case to the previous case by setting the | + | | | | | consumption of inactive queues to a high number, so that | + | | | | | the inactive queues will never be selected by the | + | | | | | smallest T logic. | + | | | | | | + | | | | | To prevent T from overflowing as result of successive | + | | | | | accumulations, T(i) is truncated after each packet | + | | | | | consumption for all queues. | + | | | | | For example, T[0..3] = [1000, 1100, 1200, 1300] | + | | | | | is truncated to T[0..3] = [0, 100, 200, 300] | + | | | | | by subtracting the min T from T(i), i = 0..n. | + | | | | | | + | | | | | This requires having at least one active queue in the | + | | | | | set of input queues, which is guaranteed by the dequeue | + | | | | | state machine never selecting an inactive traffic class. | + | | | | | | + | | | | | *mask(i) = Saturation mask for queue #i, defined as:* | + | | | | | | + | | | | | mask(i) = (queue #i is active)? 0 : 0xFFFFFFFF; | + | | | | | | + | | | | | w(i) = Weight of queue #i | + | | | | | | + | | | | | t(i) = Tokens per byte for queue #i, defined as the | + | | | | | inverse weight of queue #i. | + | | | | | | + | | | | | T(i) = Accumulated numbers of tokens previously consumed | + | | | | | from queue #i. | + | | | | | | + | | | | | *Next queue* : queue with smallest T. | + | | | | | | + | | | | | Before packet consumption from queue #i: | + | | | | | | + | | | | | *T(i) |= mask(i)* | + | | | | | | + | | | | | After packet consumption from queue #i: | + | | | | | | + | | | | | T(j) -= T(i), j != i | + | | | | | | + | | | | | T(i) = pkt_len * t(i) | + | | | | | | + | | | | | Note: T(j) uses the T(i) value before T(i) is updated. | + | | | | | | + +---+------------+-----------------+-------------+----------------------------------------------------------+ Subport Traffic Class Oversubscription """""""""""""""""""""""""""""""""""""" @@ -969,40 +969,40 @@ Solution Space summarizes some of the possible approaches for handling this problem, with the third approach selected for implementation. -.. _pg_table_13: - -**Table 13. Subport Traffic Class Oversubscription** - -+-----+---------------------------+-------------------------------------------------------------------------+ -| No. | Approach | Description | -| | | | -+=====+===========================+=========================================================================+ -| 1 | Don't care | First come, first served. | -| | | | -| | | This approach is not fair among subport member pipes, as pipes that | -| | | are served first will use up as much bandwidth for TC X as they need, | -| | | while pipes that are served later will receive poor service due to | -| | | bandwidth for TC X at the subport level being scarce. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ -| 2 | Scale down all pipes | All pipes within the subport have their bandwidth limit for TC X scaled | -| | | down by the same factor. | -| | | | -| | | This approach is not fair among subport member pipes, as the low end | -| | | pipes (that is, pipes configured with low bandwidth) can potentially | -| | | experience severe service degradation that might render their service | -| | | unusable (if available bandwidth for these pipes drops below the | -| | | minimum requirements for a workable service), while the service | -| | | degradation for high end pipes might not be noticeable at all. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ -| 3 | Cap the high demand pipes | Each subport member pipe receives an equal share of the bandwidth | -| | | available at run-time for TC X at the subport level. Any bandwidth left | -| | | unused by the low-demand pipes is redistributed in equal portions to | -| | | the high-demand pipes. This way, the high-demand pipes are truncated | -| | | while the low-demand pipes are not impacted. | -| | | | -+-----+---------------------------+-------------------------------------------------------------------------+ +.. _table_qos_13: + +.. table:: Subport Traffic Class Oversubscription + + +-----+---------------------------+-------------------------------------------------------------------------+ + | No. | Approach | Description | + | | | | + +=====+===========================+=========================================================================+ + | 1 | Don't care | First come, first served. | + | | | | + | | | This approach is not fair amongst subport member pipes, as pipes that | + | | | are served first will use up as much bandwidth for TC X as they need, | + | | | while pipes that are served later will receive poor service due to | + | | | bandwidth for TC X at the subport level being scarce. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ + | 2 | Scale down all pipes | All pipes within the subport have their bandwidth limit for TC X scaled | + | | | down by the same factor. | + | | | | + | | | This approach is not fair among subport member pipes, as the low end | + | | | pipes (that is, pipes configured with low bandwidth) can potentially | + | | | experience severe service degradation that might render their service | + | | | unusable (if available bandwidth for these pipes drops below the | + | | | minimum requirements for a workable service), while the service | + | | | degradation for high end pipes might not be noticeable at all. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ + | 3 | Cap the high demand pipes | Each subport member pipe receives an equal share of the bandwidth | + | | | available at run-time for TC X at the subport level. Any bandwidth left | + | | | unused by the low-demand pipes is redistributed in equal portions to | + | | | the high-demand pipes. This way, the high-demand pipes are truncated | + | | | while the low-demand pipes are not impacted. | + | | | | + +-----+---------------------------+-------------------------------------------------------------------------+ Typically, the subport TC oversubscription feature is enabled only for the lowest priority traffic class (TC 3), which is typically used for best effort traffic, @@ -1035,100 +1035,100 @@ as a result of demand increase (when the watermark needs to be lowered) or deman When demand is low, the watermark is set high to prevent it from impeding the subport member pipes from consuming more bandwidth. The highest value for the watermark is picked as the highest rate configured for a subport member pipe. -Table 15 illustrates the watermark operation. - -.. _pg_table_14: - -**Table 14. Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period** - -+-----+---------------------------------+----------------------------------------------------+ -| No. | Subport Traffic Class Operation | Description | -| | | | -+=====+=================================+====================================================+ -| 1 | Initialization | **Subport level**: subport_period_id= 0 | -| | | | -| | | **Pipe level**: pipe_period_id = 0 | -| | | | -+-----+---------------------------------+----------------------------------------------------+ -| 2 | Credit update | **Subport Level**: | -| | | | -| | | if (time>=subport_tc_time) | -| | | | -| | | { | -| | | subport_wm = water_mark_update(); | -| | | | -| | | subport_tc_time = time + subport_tc_period; | -| | | | -| | | subport_period_id++; | -| | | | -| | | } | -| | | | -| | | **Pipelevel:** | -| | | | -| | | if(pipe_period_id != subport_period_id) | -| | | | -| | | { | -| | | | -| | | pipe_ov_credits = subport_wm \* pipe_weight; | -| | | | -| | | pipe_period_id = subport_period_id; | -| | | | -| | | } | -| | | | -+-----+---------------------------------+----------------------------------------------------+ -| 3 | Credit consumption | **Pipe level:** | -| | (on packet scheduling) | | -| | | pkt_credits = pk_len + frame_overhead; | -| | | | -| | | if(pipe_ov_credits >= pkt_credits{ | -| | | | -| | | pipe_ov_credits -= pkt_credits; | -| | | | -| | | } | -| | | | -+-----+---------------------------------+----------------------------------------------------+ - -.. _pg_table_15: - -**Table 15. Watermark Calculation** - -+-----+------------------+----------------------------------------------------------------------------------+ -| No. | Subport Traffic | Description | -| | Class Operation | | -+=====+==================+==================================================================================+ -| 1 | Initialization | **Subport level:** | -| | | | -| | | wm = WM_MAX | -| | | | -+-----+------------------+----------------------------------------------------------------------------------+ -| 2 | Credit update | **Subport level (water_mark_update):** | -| | | | -| | | tc0_cons = subport_tc0_credits_per_period - subport_tc0_credits; | -| | | | -| | | tc1_cons = subport_tc1_credits_per_period - subport_tc1_credits; | -| | | | -| | | tc2_cons = subport_tc2_credits_per_period - subport_tc2_credits; | -| | | | -| | | tc3_cons = subport_tc3_credits_per_period - subport_tc3_credits; | -| | | | -| | | tc3_cons_max = subport_tc3_credits_per_period - (tc0_cons + tc1_cons + | -| | | tc2_cons); | -| | | | -| | | if(tc3_consumption > (tc3_consumption_max - MTU)){ | -| | | | -| | | wm -= wm >> 7; | -| | | | -| | | if(wm < WM_MIN) wm = WM_MIN; | -| | | | -| | | } else { | -| | | | -| | | wm += (wm >> 7) + 1; | -| | | | -| | | if(wm > WM_MAX) wm = WM_MAX; | -| | | | -| | | } | -| | | | -+-----+------------------+----------------------------------------------------------------------------------+ +:numref:`table_qos_14` and :numref:`table_qos_15` illustrates the watermark operation. + +.. _table_qos_14: + +.. table:: Watermark Propagation from Subport Level to Member Pipes at the Beginning of Each Traffic Class Upper Limit Enforcement Period + + +-----+---------------------------------+----------------------------------------------------+ + | No. | Subport Traffic Class Operation | Description | + | | | | + +=====+=================================+====================================================+ + | 1 | Initialization | **Subport level**: subport_period_id= 0 | + | | | | + | | | **Pipe level**: pipe_period_id = 0 | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + | 2 | Credit update | **Subport Level**: | + | | | | + | | | if (time>=subport_tc_time) | + | | | | + | | | { | + | | | subport_wm = water_mark_update(); | + | | | | + | | | subport_tc_time = time + subport_tc_period; | + | | | | + | | | subport_period_id++; | + | | | | + | | | } | + | | | | + | | | **Pipelevel:** | + | | | | + | | | if(pipe_period_id != subport_period_id) | + | | | | + | | | { | + | | | | + | | | pipe_ov_credits = subport_wm \* pipe_weight; | + | | | | + | | | pipe_period_id = subport_period_id; | + | | | | + | | | } | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + | 3 | Credit consumption | **Pipe level:** | + | | (on packet scheduling) | | + | | | pkt_credits = pk_len + frame_overhead; | + | | | | + | | | if(pipe_ov_credits >= pkt_credits{ | + | | | | + | | | pipe_ov_credits -= pkt_credits; | + | | | | + | | | } | + | | | | + +-----+---------------------------------+----------------------------------------------------+ + +.. _table_qos_15: + +.. table:: Watermark Calculation + + +-----+------------------+----------------------------------------------------------------------------------+ + | No. | Subport Traffic | Description | + | | Class Operation | | + +=====+==================+==================================================================================+ + | 1 | Initialization | **Subport level:** | + | | | | + | | | wm = WM_MAX | + | | | | + +-----+------------------+----------------------------------------------------------------------------------+ + | 2 | Credit update | **Subport level (water_mark_update):** | + | | | | + | | | tc0_cons = subport_tc0_credits_per_period - subport_tc0_credits; | + | | | | + | | | tc1_cons = subport_tc1_credits_per_period - subport_tc1_credits; | + | | | | + | | | tc2_cons = subport_tc2_credits_per_period - subport_tc2_credits; | + | | | | + | | | tc3_cons = subport_tc3_credits_per_period - subport_tc3_credits; | + | | | | + | | | tc3_cons_max = subport_tc3_credits_per_period - (tc0_cons + tc1_cons + | + | | | tc2_cons); | + | | | | + | | | if(tc3_consumption > (tc3_consumption_max - MTU)){ | + | | | | + | | | wm -= wm >> 7; | + | | | | + | | | if(wm < WM_MIN) wm = WM_MIN; | + | | | | + | | | } else { | + | | | | + | | | wm += (wm >> 7) + 1; | + | | | | + | | | if(wm > WM_MAX) wm = WM_MAX; | + | | | | + | | | } | + | | | | + +-----+------------------+----------------------------------------------------------------------------------+ Worst Case Scenarios for Performance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1225,28 +1225,28 @@ and the mark empty operation is explained in :ref:`Section 2.23.3.3 <Queue_Empty Configuration ~~~~~~~~~~~~~ -A RED configuration contains the parameters given in Table 16. - -.. _pg_table_16: - -**Table 16. RED Configuration Parameters** - -+--------------------------+---------+---------+------------------+ -| Parameter | Minimum | Maximum | Typical | -| | | | | -+==========================+=========+=========+==================+ -| Minimum Threshold | 0 | 1022 | 1/4 x queue size | -| | | | | -+--------------------------+---------+---------+------------------+ -| Maximum Threshold | 1 | 1023 | 1/2 x queue size | -| | | | | -+--------------------------+---------+---------+------------------+ -| Inverse Mark Probability | 1 | 255 | 10 | -| | | | | -+--------------------------+---------+---------+------------------+ -| EWMA Filter Weight | 1 | 12 | 9 | -| | | | | -+--------------------------+---------+---------+------------------+ +A RED configuration contains the parameters given in :numref:`table_qos_16`. + +.. _table_qos_16: + +.. table:: RED Configuration Parameters + + +--------------------------+---------+---------+------------------+ + | Parameter | Minimum | Maximum | Typical | + | | | | | + +==========================+=========+=========+==================+ + | Minimum Threshold | 0 | 1022 | 1/4 x queue size | + | | | | | + +--------------------------+---------+---------+------------------+ + | Maximum Threshold | 1 | 1023 | 1/2 x queue size | + | | | | | + +--------------------------+---------+---------+------------------+ + | Inverse Mark Probability | 1 | 255 | 10 | + | | | | | + +--------------------------+---------+---------+------------------+ + | EWMA Filter Weight | 1 | 12 | 9 | + | | | | | + +--------------------------+---------+---------+------------------+ The meaning of these parameters is explained in more detail in the following sections. The format of these parameters as specified to the dropper module API @@ -1390,36 +1390,36 @@ These approaches include: The method that was finally selected (described above in Section 26.3.2.2.1) out performs all of these approaches in terms of run-time performance and memory requirements and also achieves accuracy comparable to floating-point evaluation. -Table 17 lists the performance of each of these alternative approaches relative to the method that is used in the dropper. +:numref:`table_qos_17` lists the performance of each of these alternative approaches relative to the method that is used in the dropper. As can be seen, the floating-point implementation achieved the worst performance. -.. _pg_table_17: - -**Table 17. Relative Performance of Alternative Approaches** - -+------------------------------------------------------------------------------------+----------------------+ -| Method | Relative Performance | -| | | -+====================================================================================+======================+ -| Current dropper method (see :ref:`Section 23.3.2.1.3 <Dropper>`) | 100% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Fixed-point method with small (512B) look-up table | 148% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| SSE method with small (512B) look-up table | 114% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Large (76KB) look-up table | 118% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| Floating-point | 595% | -| | | -+------------------------------------------------------------------------------------+----------------------+ -| **Note**: In this case, since performance is expressed as time spent executing the operation in a | -| specific condition, any relative performance value above 100% runs slower than the reference method. | -| | -+-----------------------------------------------------------------------------------------------------------+ +.. _table_qos_17: + +.. table:: Relative Performance of Alternative Approaches + + +------------------------------------------------------------------------------------+----------------------+ + | Method | Relative Performance | + | | | + +====================================================================================+======================+ + | Current dropper method (see :ref:`Section 23.3.2.1.3 <Dropper>`) | 100% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Fixed-point method with small (512B) look-up table | 148% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | SSE method with small (512B) look-up table | 114% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Large (76KB) look-up table | 118% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | Floating-point | 595% | + | | | + +------------------------------------------------------------------------------------+----------------------+ + | **Note**: In this case, since performance is expressed as time spent executing the operation in a | + | specific condition, any relative performance value above 100% runs slower than the reference method. | + | | + +-----------------------------------------------------------------------------------------------------------+ Drop Decision Block ^^^^^^^^^^^^^^^^^^^ @@ -1583,28 +1583,28 @@ A sample RED configuration is shown below. In this example, the queue size is 64 tc 3 wred weight = 9 9 9 With this configuration file, the RED configuration that applies to green, -yellow and red packets in traffic class 0 is shown in Table 18. - -.. _pg_table_18: - -**Table 18. RED Configuration Corresponding to RED Configuration File** - -+--------------------+--------------------+-------+--------+-----+ -| RED Parameter | Configuration Name | Green | Yellow | Red | -| | | | | | -+====================+====================+=======+========+=====+ -| Minimum Threshold | tc 0 wred min | 28 | 22 | 16 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| Maximum Threshold | tc 0 wred max | 32 | 32 | 32 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| Mark Probability | tc 0 wred inv prob | 10 | 10 | 10 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ -| EWMA Filter Weight | tc 0 wred weight | 9 | 9 | 9 | -| | | | | | -+--------------------+--------------------+-------+--------+-----+ +yellow and red packets in traffic class 0 is shown in :numref:`table_qos_18`. + +.. _table_qos_18: + +.. table:: RED Configuration Corresponding to RED Configuration File + + +--------------------+--------------------+-------+--------+-----+ + | RED Parameter | Configuration Name | Green | Yellow | Red | + | | | | | | + +====================+====================+=======+========+=====+ + | Minimum Threshold | tc 0 wred min | 28 | 22 | 16 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | Maximum Threshold | tc 0 wred max | 32 | 32 | 32 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | Mark Probability | tc 0 wred inv prob | 10 | 10 | 10 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ + | EWMA Filter Weight | tc 0 wred weight | 9 | 9 | 9 | + | | | | | | + +--------------------+--------------------+-------+--------+-----+ Application Programming Interface (API) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst index 745a7ac..e1a0c56 100644 --- a/doc/guides/sample_app_ug/index.rst +++ b/doc/guides/sample_app_ug/index.rst @@ -134,8 +134,8 @@ Sample Applications User Guide **Tables** -:ref:`Table 1.Output Traffic Marking <table_1>` +:numref:`table_qos_metering_1` :ref:`table_qos_metering_1` -:ref:`Table 2.Entity Types <table_2>` +:numref:`table_qos_scheduler_1` :ref:`table_qos_scheduler_1` -:ref:`Table 3.Table Types <table_3>` +:numref:`table_test_pipeline_1` :ref:`table_test_pipeline_1` diff --git a/doc/guides/sample_app_ug/qos_metering.rst b/doc/guides/sample_app_ug/qos_metering.rst index 344faf7..e1a6ac7 100644 --- a/doc/guides/sample_app_ug/qos_metering.rst +++ b/doc/guides/sample_app_ug/qos_metering.rst @@ -140,29 +140,29 @@ The traffic meter parameters are configured in the application source code with Assuming the input traffic is generated at line rate and all packets are 64 bytes Ethernet frames (IPv4 packet size of 46 bytes) and green, the expected output traffic should be marked as shown in the following table: -.. _table_1: - -**Table 1. Output Traffic Marking** - -+-------------+------------------+-------------------+----------------+ -| **Mode** | **Green (Mpps)** | **Yellow (Mpps)** | **Red (Mpps)** | -| | | | | -+=============+==================+===================+================+ -| srTCM blind | 1 | 1 | 12.88 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| srTCM color | 1 | 1 | 12.88 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| trTCM blind | 1 | 0.5 | 13.38 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| trTCM color | 1 | 0.5 | 13.38 | -| | | | | -+-------------+------------------+-------------------+----------------+ -| FWD | 14.88 | 0 | 0 | -| | | | | -+-------------+------------------+-------------------+----------------+ +.. _table_qos_metering_1: + +.. table:: Output Traffic Marking + + +-------------+------------------+-------------------+----------------+ + | **Mode** | **Green (Mpps)** | **Yellow (Mpps)** | **Red (Mpps)** | + | | | | | + +=============+==================+===================+================+ + | srTCM blind | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | srTCM color | 1 | 1 | 12.88 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM blind | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | trTCM color | 1 | 0.5 | 13.38 | + | | | | | + +-------------+------------------+-------------------+----------------+ + | FWD | 14.88 | 0 | 0 | + | | | | | + +-------------+------------------+-------------------+----------------+ To set up the policing scheme as desired, it is necessary to modify the main.h source file, where this policy is implemented as a static structure, as follows: diff --git a/doc/guides/sample_app_ug/qos_scheduler.rst b/doc/guides/sample_app_ug/qos_scheduler.rst index 66c261c..ffa8ee2 100644 --- a/doc/guides/sample_app_ug/qos_scheduler.rst +++ b/doc/guides/sample_app_ug/qos_scheduler.rst @@ -321,28 +321,28 @@ The Port/Subport/Pipe/Traffic Class/Queue are the hierarchical entities in a typ The traffic flows that need to be configured are application dependent. This application classifies based on the QinQ double VLAN tags and the IP destination address as indicated in the following table. -.. _table_2: - -**Table 2. Entity Types** - -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | -| | | | | -+================+=========================+==================================================+==================================+ -| Port | - | Ethernet port | Physical port | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ -| Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | -| | | | | -+----------------+-------------------------+--------------------------------------------------+----------------------------------+ +.. _table_qos_scheduler_1: + +.. table:: Entity Types + + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | **Level Name** | **Siblings per Parent** | **QoS Functional Description** | **Selected By** | + | | | | | + +================+=========================+==================================================+==================================+ + | Port | - | Ethernet port | Physical port | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Traffic Class | 4 | TCs of the same pipe services in strict priority | Destination IP address (0.0.X.0) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ + | Queue | 4 | Queue of the same TC serviced in WRR | Destination IP address (0.0.0.X) | + | | | | | + +----------------+-------------------------+--------------------------------------------------+----------------------------------+ Please refer to the "QoS Scheduler" chapter in the *DPDK Programmer's Guide* for more information about these parameters. diff --git a/doc/guides/sample_app_ug/test_pipeline.rst b/doc/guides/sample_app_ug/test_pipeline.rst index 3c2c25b..cd0cf9e 100644 --- a/doc/guides/sample_app_ug/test_pipeline.rst +++ b/doc/guides/sample_app_ug/test_pipeline.rst @@ -98,7 +98,7 @@ The PORTMASK parameter must contain 2 or 4 ports. Table Types and Behavior ~~~~~~~~~~~~~~~~~~~~~~~~ -Table 3 describes the table types used and how they are populated. +:numref:`table_test_pipeline_1` describes the table types used and how they are populated. The hash tables are pre-populated with 16 million keys. For hash tables, the following parameters can be selected: @@ -113,157 +113,157 @@ For hash tables, the following parameters can be selected: * **Table type (e.g. hash-spec-16-ext or hash-spec-16-lru).** The available options are ext (extendable bucket) or lru (least recently used). -.. _table_3: - -**Table 3. Table Types** - -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| **#** | **TABLE_TYPE** | **Description of Core B Table** | **Pre-added Table Entries** | -| | | | | -+=======+========================+==========================================================+=======================================================+ -| 1 | none | Core B is not implementing a DPDK pipeline. | N/A | -| | | Core B is implementing a pass-through from its input set | | -| | | of software queues to its output set of software queues. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 2 | stub | Stub table. Core B is implementing the same pass-through | N/A | -| | | functionality as described for the "none" option by | | -| | | using the DPDK Packet Framework by using one | | -| | | stub table for each input NIC port. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 3 | hash-[spec]-8-lru | LRU hash table with 8-byte key size and 16 million | 16 million entries are successfully added to the | -| | | entries. | hash table with the following key format: | -| | | | | -| | | | [4-byte index, 4 bytes of 0] | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Sendto output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for | -| | | | core B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 4 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 4 | hash-[spec]-8-ext | Extendable bucket hash table with 8-byte key size | Same as hash-[spec]-8-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 5 | hash-[spec]-16-lru | LRU hash table with 16-byte key size and 16 million | 16 million entries are successfully added to the hash | -| | | entries. | table with the following key format: | -| | | | | -| | | | [4-byte index, 12 bytes of 0] | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Send to output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for core | -| | | | B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 12 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 6 | hash-[spec]-16-ext | Extendable bucket hash table with 16-byte key size | Same as hash-[spec]-16-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 7 | hash-[spec]-32-lru | LRU hash table with 32-byte key size and 16 million | 16 million entries are successfully added to the hash | -| | | entries. | table with the following key format: | -| | | | | -| | | | [4-byte index, 28 bytes of 0]. | -| | | | | -| | | | The action configured for all table entries is | -| | | | "Send to output port", with the output port index | -| | | | uniformly distributed for the range of output ports. | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is creating the following lookup | -| | | | key and storing it into the packet meta data for | -| | | | Lpmcore B to use for table lookup: | -| | | | | -| | | | [destination IPv4 address, 28 bytes of 0] | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 8 | hash-[spec]-32-ext | Extendable bucket hash table with 32-byte key size | Same as hash-[spec]-32-lru table entries, above. | -| | | and 16 million entries. | | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 9 | lpm | Longest Prefix Match (LPM) IPv4 table. | In the case of two ports, two routes | -| | | | are added to the table: | -| | | | | -| | | | [0.0.0.0/9 => send to output port 0] | -| | | | | -| | | | [0.128.0.0/9 => send to output port 1] | -| | | | | -| | | | In case of four ports, four entries are added to the | -| | | | table: | -| | | | | -| | | | [0.0.0.0/10 => send to output port 0] | -| | | | | -| | | | [0.64.0.0/10 => send to output port 1] | -| | | | | -| | | | [0.128.0.0/10 => send to output port 2] | -| | | | | -| | | | [0.192.0.0/10 => send to output port 3] | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -| | | | At run time, core A is storing the IPv4 destination | -| | | | within the packet meta data to be later used by core | -| | | | B as the lookup key. | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ -| 10 | acl | Access Control List (ACL) table | In the case of two ports, two ACL rules are added to | -| | | | the table: | -| | | | | -| | | | [priority = 0 (highest), | -| | | | | -| | | | IPv4 source = ANY, | -| | | | | -| | | | IPv4 destination = 0.0.0.0/9, | -| | | | | -| | | | L4 protocol = ANY, | -| | | | | -| | | | TCP source port = ANY, | -| | | | | -| | | | TCP destination port = ANY | -| | | | | -| | | | => send to output port 0] | -| | | | | -| | | | | -| | | | [priority = 0 (highest), | -| | | | | -| | | | IPv4 source = ANY, | -| | | | | -| | | | IPv4 destination = 0.128.0.0/9, | -| | | | | -| | | | L4 protocol = ANY, | -| | | | | -| | | | TCP source port = ANY, | -| | | | | -| | | | TCP destination port = ANY | -| | | | | -| | | | => send to output port 0]. | -| | | | | -| | | | | -| | | | The default table rule (used in the case of a lookup | -| | | | miss) is to drop the packet. | -| | | | | -+-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ +.. _table_test_pipeline_1: + +.. table:: Table Types + + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | **#** | **TABLE_TYPE** | **Description of Core B Table** | **Pre-added Table Entries** | + | | | | | + +=======+========================+==========================================================+=======================================================+ + | 1 | none | Core B is not implementing a DPDK pipeline. | N/A | + | | | Core B is implementing a pass-through from its input set | | + | | | of software queues to its output set of software queues. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 2 | stub | Stub table. Core B is implementing the same pass-through | N/A | + | | | functionality as described for the "none" option by | | + | | | using the DPDK Packet Framework by using one | | + | | | stub table for each input NIC port. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 3 | hash-[spec]-8-lru | LRU hash table with 8-byte key size and 16 million | 16 million entries are successfully added to the | + | | | entries. | hash table with the following key format: | + | | | | | + | | | | [4-byte index, 4 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Sendto output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | core B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 4 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 4 | hash-[spec]-8-ext | Extendible bucket hash table with 8-byte key size | Same as hash-[spec]-8-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 5 | hash-[spec]-16-lru | LRU hash table with 16-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 12 bytes of 0] | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for core | + | | | | B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 12 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 6 | hash-[spec]-16-ext | Extendible bucket hash table with 16-byte key size | Same as hash-[spec]-16-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 7 | hash-[spec]-32-lru | LRU hash table with 32-byte key size and 16 million | 16 million entries are successfully added to the hash | + | | | entries. | table with the following key format: | + | | | | | + | | | | [4-byte index, 28 bytes of 0]. | + | | | | | + | | | | The action configured for all table entries is | + | | | | "Send to output port", with the output port index | + | | | | uniformly distributed for the range of output ports. | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is creating the following lookup | + | | | | key and storing it into the packet meta data for | + | | | | Lpmcore B to use for table lookup: | + | | | | | + | | | | [destination IPv4 address, 28 bytes of 0] | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 8 | hash-[spec]-32-ext | Extendible bucket hash table with 32-byte key size | Same as hash-[spec]-32-lru table entries, above. | + | | | and 16 million entries. | | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 9 | lpm | Longest Prefix Match (LPM) IPv4 table. | In the case of two ports, two routes | + | | | | are added to the table: | + | | | | | + | | | | [0.0.0.0/9 => send to output port 0] | + | | | | | + | | | | [0.128.0.0/9 => send to output port 1] | + | | | | | + | | | | In case of four ports, four entries are added to the | + | | | | table: | + | | | | | + | | | | [0.0.0.0/10 => send to output port 0] | + | | | | | + | | | | [0.64.0.0/10 => send to output port 1] | + | | | | | + | | | | [0.128.0.0/10 => send to output port 2] | + | | | | | + | | | | [0.192.0.0/10 => send to output port 3] | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + | | | | At run time, core A is storing the IPv4 destination | + | | | | within the packet meta data to be later used by core | + | | | | B as the lookup key. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ + | 10 | acl | Access Control List (ACL) table | In the case of two ports, two ACL rules are added to | + | | | | the table: | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.0.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0] | + | | | | | + | | | | | + | | | | [priority = 0 (highest), | + | | | | | + | | | | IPv4 source = ANY, | + | | | | | + | | | | IPv4 destination = 0.128.0.0/9, | + | | | | | + | | | | L4 protocol = ANY, | + | | | | | + | | | | TCP source port = ANY, | + | | | | | + | | | | TCP destination port = ANY | + | | | | | + | | | | => send to output port 0]. | + | | | | | + | | | | | + | | | | The default table rule (used in the case of a lookup | + | | | | miss) is to drop the packet. | + | | | | | + +-------+------------------------+----------------------------------------------------------+-------------------------------------------------------+ Input Traffic ~~~~~~~~~~~~~ -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [dpdk-dev] [PATCH v3 3/3] doc: add sphinx numref compatibility workaround 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 1/3] doc: refactored figure numbers " John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 2/3] doc: refactored table " John McNamara @ 2015-05-18 11:34 ` John McNamara 2015-05-25 14:05 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references Thomas Monjalon 3 siblings, 0 replies; 17+ messages in thread From: John McNamara @ 2015-05-18 11:34 UTC (permalink / raw) To: dev This change adds some simple handling for the :numref: directive for Sphinx versions prior to 1.3.1. This allows the Guides documentation to be built with older versions of Sphinx and still produce reasonable results. The patch replaces the :numref: reference with a link to the target (for all Sphinx doc types). It doesn't try to label figures/tables. Full numref support with automatic figure/table numbering and links can be obtained by upgrading to Sphinx 1.3.1 or later. Signed-off-by: John McNamara <john.mcnamara@intel.com> --- doc/guides/conf.py | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/doc/guides/conf.py b/doc/guides/conf.py index 1bc031f..cab97ac 100644 --- a/doc/guides/conf.py +++ b/doc/guides/conf.py @@ -31,6 +31,9 @@ import subprocess from sphinx.highlighting import PygmentsBridge from pygments.formatters.latex import LatexFormatter +from docutils import nodes +from distutils.version import LooseVersion +from sphinx import __version__ as sphinx_version project = 'DPDK' @@ -72,6 +75,7 @@ latex_elements = { 'preamble': latex_preamble } + # Override the default Latex formatter in order to modify the # code/verbatim blocks. class CustomLatexFormatter(LatexFormatter): @@ -82,3 +86,79 @@ class CustomLatexFormatter(LatexFormatter): # Replace the default latex formatter. PygmentsBridge.latex_formatter = CustomLatexFormatter + + +# The following hook functions add some simple handling for the :numref: +# directive for Sphinx versions prior to 1.3.1. The functions replace the +# :numref: reference with a link to the target (for all Sphinx doc types). It +# doesn't try to label figures/tables. + +def numref_role(reftype, rawtext, text, lineno, inliner): + """ + Add a Sphinx role to handle numref references. Note, we can't convert the + link here because the doctree isn't build and the target information isn't + available. + + """ + + # Add an identifier to distinguish numref from other references. + newnode = nodes.reference('', + '', + refuri='_local_numref_#%s' % text, + internal=True) + + return [newnode], [] + + +def process_numref(app, doctree, from_docname): + """ + Process the numref nodes once the doctree has been built and prior to + writing the files. The processing involves replacing the numref with a + link plus text to indicate if it is a Figure or Table link. + + """ + env = app.builder.env + + # Iterate over the reference nodes in the doctree. + for node in doctree.traverse(nodes.reference): + target = node.get('refuri', '') + + # Look for numref nodes. + if target.startswith('_local_numref_#'): + target = target.replace('_local_numref_#', '') + + # Get the target label and link information from the Sphinx env. + data = env.domains['std'].data + docname, label, _ = data['labels'].get(target, ('', '', '')) + relative_url = app.builder.get_relative_uri(from_docname, docname) + + # Add a text label to the link. + if target.startswith('figure'): + caption = 'Figure' + elif target.startswith('table'): + caption = 'Table' + else: + caption = 'Link' + + # Create a new reference node with the updated link information. + newnode = nodes.reference('', + caption, + refuri='%s#%s' % (relative_url, label), + internal=True) + + # Replace the node. + node.replace_self(newnode) + + +def setup(app): + + if LooseVersion(sphinx_version) < LooseVersion('1.3.1'): + + print('[dpdk docs] Upgrade sphinx to version >= 1.3.1 for ' + 'improved Figure/Table number handling.') + + # Add a role to handle :numref: references. + app.add_role('numref', numref_role) + + # Process the numref references once the doctree has been created. + app.connect('doctree-resolved', process_numref) -- 1.8.1.4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara ` (2 preceding siblings ...) 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 3/3] doc: add sphinx numref compatibility workaround John McNamara @ 2015-05-25 14:05 ` Thomas Monjalon 2015-05-25 16:37 ` Mcnamara, John 3 siblings, 1 reply; 17+ messages in thread From: Thomas Monjalon @ 2015-05-25 14:05 UTC (permalink / raw) To: John McNamara; +Cc: dev 2015-05-18 12:34, John McNamara: > This patchset adds automatic figure and table references to the docs. The > figure and table numbers in the generated Html and PDF docs can now be > automatically numbered. > > It replaces all hardcoded figure/table numbers and references. > > The numfig/numref feature requires Sphinx >= 1.3.1. For backward compatibility > with older versions workaround handling is added to the sphinx conf.py file in > patch 3/3. > > The workaround replaces the :numref: reference with a "Figure" or "Table" link > to the target (for all Sphinx doc types). It doesn't number the figures or > tables. This produces reasonable documentation links for users with older > versions of sphinx while allowing automatic numbering support for newer > versions. > > Tested with Sphinx 1.2.3 and 1.3.1. > > John McNamara (3): > doc: refactored figure numbers into references > doc: refactored table numbers into references > doc: add sphinx numref compatibility workaround Applied, thanks It will be easier to add/remove figures and tables now :) John, I did some light modifications in comments that I think you'd agree. This is a really nice feature for the doc and I think this fallback should be documented somewhere in the sphinx project. ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references 2015-05-25 14:05 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references Thomas Monjalon @ 2015-05-25 16:37 ` Mcnamara, John 0 siblings, 0 replies; 17+ messages in thread From: Mcnamara, John @ 2015-05-25 16:37 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Monday, May 25, 2015 3:06 PM > To: Mcnamara, John > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums > into references > > > Applied, thanks > It will be easier to add/remove figures and tables now :) Hi Thomas, Thanks for that. Anything that make the documentation process a little more automated is good. :-) Note, during the rebase I inadvertently reinstated a few typos within the tables that I had fixed in a previous patch. I'll submit another spellcheck patch to fix them and others prior to the 2.1 release. John ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-05-25 16:37 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-04-24 13:11 [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums into references John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 1/2] doc: refactored figure numbers " John McNamara 2015-04-24 13:11 ` [dpdk-dev] [PATCH 2/2] doc: refactored table " John McNamara 2015-04-29 16:12 ` [dpdk-dev] [PATCH 0/2] doc: refactored fig and table nums " Thomas Monjalon 2015-04-30 13:34 ` Mcnamara, John 2015-05-13 19:07 ` Thomas Monjalon 2015-05-14 10:41 ` Mcnamara, John 2015-05-18 10:48 ` Mcnamara, John 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 0/3] " John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH 1/3] doc: refactored figure numbers " John McNamara 2015-05-18 11:15 ` [dpdk-dev] [PATCH v2 3/3] doc: add sphinx numref compatibility workaround John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 1/3] doc: refactored figure numbers " John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 2/3] doc: refactored table " John McNamara 2015-05-18 11:34 ` [dpdk-dev] [PATCH v3 3/3] doc: add sphinx numref compatibility workaround John McNamara 2015-05-25 14:05 ` [dpdk-dev] [PATCH v3 0/3] doc: refactored fig and table nums into references Thomas Monjalon 2015-05-25 16:37 ` Mcnamara, John
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).