From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <amo@semihalf.com>
Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com
 [209.85.208.175]) by dpdk.org (Postfix) with ESMTP id 5C8CF532C
 for <dev@dpdk.org>; Tue, 25 Sep 2018 09:05:26 +0200 (CEST)
Received: by mail-lj1-f175.google.com with SMTP id r83-v6so2612505ljr.7
 for <dev@dpdk.org>; Tue, 25 Sep 2018 00:05:26 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=semihalf-com.20150623.gappssmtp.com; s=20150623;
 h=from:to:cc:subject:date:message-id:in-reply-to:references;
 bh=CVIBrqVMNQjzvYK6JvLgcHm1biUzmsbKPscez2Xd+Uc=;
 b=aENAmkK8nVXMpu+e5jDpFQ2+gsPVBMEslr2J3cpGwWXggEdJIyDxtzrnZTK9KDuEwG
 81UEsFzOtNdP/vqTfa1axpoygFUEVDRssDxJgSJCnmyQbNxFB8CJ5YquOEX2WnHHTGpR
 uVtv7NQXsF5dx74PG6z1Q+CkV1Eb48r06fshZwXDOBLwq3coF2Jd+BXITumgDHxnq27h
 8s+HRISt+xoQEyDt3xNg0MGt3VyctmibnJv58EW9p1I+89NndkY5AlB4EoNfPC9C46BG
 fQnaqNwuOMkr5UVDa38zKCTyJvQWYPahMsBqfnWzfghxxGJAKBC5qqIRy3dDq5v2mYpm
 PHDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references;
 bh=CVIBrqVMNQjzvYK6JvLgcHm1biUzmsbKPscez2Xd+Uc=;
 b=kKUiYCPTQZEYR8HJsOMjhpefoDGUnG6P+/jIWwWUyjyxOsEvl55WZv/itnWLO+iWcO
 OelhKD/8BB7CfRPnhMgJRUFXmZximelwLtfWE3Vt4PE2ervAm/EL+Cu8SfVW1jPQO9UY
 XtEZthHPc44rRzMH2r6FQP7XKlMnkfPcM/oY3f5Q5No3QbAygmrU6NZyffnu1QFY23Wl
 VrZCjYwrnrP1klgThn+GrsQe1ri881gmLAUN3biN8oPfzuQYl1iUiGV+dKdKsTw+PT9z
 MyRPcSkqWDhToSH7goFK+AZ5zcvFfOfnJcNj9NmxuPFrxi3vVnTE6YCbQg49vJ2CbPM/
 nxdg==
X-Gm-Message-State: ABuFfojBvVD53UTAh1op0zAJWM3+ubTYG350Gs0WqZxsM/auVfzJkV7c
 JddQ1xq15bq01x9yAs9PccuYwEPoGJBd1A==
X-Google-Smtp-Source: ACcGV61Jtvl9l5vLgmn082Oh2RghlHVQaxGg+RkmElRIfRp1z8dCG9lpo7qqa4NZaj+XBq4JW7LsIA==
X-Received: by 2002:a2e:94d5:: with SMTP id
 r21-v6mr1158944ljh.126.1537859125535; 
 Tue, 25 Sep 2018 00:05:25 -0700 (PDT)
Received: from amok.semihalf.local (31-172-191-173.noc.fibertech.net.pl.
 [31.172.191.173])
 by smtp.googlemail.com with ESMTPSA id 24-v6sm238306ljb.76.2018.09.25.00.05.24
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
 Tue, 25 Sep 2018 00:05:24 -0700 (PDT)
From: Andrzej Ostruszka <amo@semihalf.com>
To: dev@dpdk.org
Cc: mw@semihalf.com, nadavh@marvell.com,
 Natalie Samsonov <nsamsono@marvell.com>
Date: Tue, 25 Sep 2018 09:05:08 +0200
Message-Id: <1537859109-25659-13-git-send-email-amo@semihalf.com>
X-Mailer: git-send-email 2.7.4
In-Reply-To: <1537859109-25659-1-git-send-email-amo@semihalf.com>
References: <1536068953-9352-1-git-send-email-tdu@semihalf.com>
 <1537859109-25659-1-git-send-email-amo@semihalf.com>
Subject: [dpdk-dev] [PATCH v3 12/13] net/mvpp2: document MTR and TM usage
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 25 Sep 2018 07:05:27 -0000

From: Natalie Samsonov <nsamsono@marvell.com>

Document MTR (metering) and TM (traffic management) usage plus
do some small updates here and there.

Signed-off-by: Natalie Samsonov <nsamsono@marvell.com>
---
 doc/guides/nics/img/mvpp2_tm.svg |  71 +++++++
 doc/guides/nics/mvpp2.rst        | 386 +++++++++++++++++++++++++++++++++------
 2 files changed, 406 insertions(+), 51 deletions(-)
 create mode 100644 doc/guides/nics/img/mvpp2_tm.svg

diff --git a/doc/guides/nics/img/mvpp2_tm.svg b/doc/guides/nics/img/mvpp2_tm.svg
new file mode 100644
index 0000000..4aa9272
--- /dev/null
+++ b/doc/guides/nics/img/mvpp2_tm.svg
@@ -0,0 +1,71 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/PR-SVG-20010719/DTD/svg10.dtd">
+<svg width="16cm" height="4cm" viewBox="-1 -1 309 75" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+  <g>
+    <polyline style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" points="159.661,12.6759 141.655,12.6759 141.655,35.5606 88.1561,35.5606 88.1561,44.9245 "/>
+    <polygon style="fill: #000000" points="88.1561,49.4245 85.1561,43.4245 88.1561,44.9245 91.1561,43.4245 "/>
+    <polygon style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" points="88.1561,49.4245 85.1561,43.4245 88.1561,44.9245 91.1561,43.4245 "/>
+  </g>
+  <g>
+    <polyline style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" points="159.661,12.6759 176.28,12.6759 176.28,35.5606 281.681,35.5606 281.681,44.9245 "/>
+    <polygon style="fill: #000000" points="281.681,49.4245 278.681,43.4245 281.681,44.9245 284.681,43.4245 "/>
+    <polygon style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" points="281.681,49.4245 278.681,43.4245 281.681,44.9245 284.681,43.4245 "/>
+  </g>
+  <g>
+    <rect style="fill: #ffffff" x="126.066" y="0.98102" width="67.1901" height="23.3899"/>
+    <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="126.066" y="0.98102" width="67.1901" height="23.3899"/>
+  </g>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:middle;font-family:sans-serif;font-style:normal;font-weight:normal" x="159.661" y="17.1259">
+    <tspan x="159.661" y="17.1259">Port N</tspan>
+  </text>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="304.581" y="68.168">
+    <tspan x="304.581" y="68.168"></tspan>
+  </text>
+  <g>
+    <rect style="fill: #ffffff" x="62.5504" y="51.5478" width="51.2114" height="22.0925"/>
+    <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="62.5504" y="51.5478" width="51.2114" height="22.0925"/>
+  </g>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:middle;font-family:sans-serif;font-style:normal;font-weight:normal" x="88.1561" y="67.044">
+    <tspan x="88.1561" y="67.044">Txq 0</tspan>
+  </text>
+  <g>
+    <rect style="fill: #ffffff" x="134.1" y="51.355" width="51.1213" height="22.478"/>
+    <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="134.1" y="51.355" width="51.1213" height="22.478"/>
+  </g>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:middle;font-family:sans-serif;font-style:normal;font-weight:normal" x="159.661" y="67.044">
+    <tspan x="159.661" y="67.044">Txq 1</tspan>
+  </text>
+  <g>
+    <rect style="fill: #ffffff" x="256.416" y="51.5478" width="50.5306" height="22.0925"/>
+    <rect style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" x="256.416" y="51.5478" width="50.5306" height="22.0925"/>
+  </g>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:middle;font-family:sans-serif;font-style:normal;font-weight:normal" x="281.681" y="67.044">
+    <tspan x="281.681" y="67.044">Txq M</tspan>
+  </text>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="101.822" y="67.044">
+    <tspan x="101.822" y="67.044"></tspan>
+  </text>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="-0.537645" y="17.1259">
+    <tspan x="-0.537645" y="17.1259">Level 0:</tspan>
+  </text>
+  <text font-size="12.7998" style="fill: #000000;text-anchor:start;font-family:sans-serif;font-style:normal;font-weight:normal" x="-0.746688" y="67.044">
+    <tspan x="-0.746688" y="67.044">Level 1:</tspan>
+  </text>
+  <g>
+    <ellipse style="fill: #000000" cx="207.645" cy="62.594" rx="0.425344" ry="0.425344"/>
+    <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="207.645" cy="62.594" rx="0.425344" ry="0.425344"/>
+  </g>
+  <g>
+    <ellipse style="fill: #000000" cx="219.525" cy="62.594" rx="0.425344" ry="0.425344"/>
+    <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="219.525" cy="62.594" rx="0.425344" ry="0.425344"/>
+  </g>
+  <g>
+    <ellipse style="fill: #000000" cx="231.405" cy="62.594" rx="0.425345" ry="0.425345"/>
+    <ellipse style="fill: none; fill-opacity:0; stroke-width: 2; stroke: #000000" cx="231.405" cy="62.594" rx="0.425345" ry="0.425345"/>
+  </g>
+  <g>
+    <line style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" x1="159.661" y1="24.3709" x2="159.661" y2="45.737"/>
+    <polygon style="fill: #000000" points="159.661,50.237 156.661,44.237 159.661,45.737 162.661,44.237 "/>
+    <polygon style="fill: none; fill-opacity:0; stroke-width: 1; stroke: #000000" points="159.661,50.237 156.661,44.237 159.661,45.737 162.661,44.237 "/>
+  </g>
+</svg>
diff --git a/doc/guides/nics/mvpp2.rst b/doc/guides/nics/mvpp2.rst
index 3b3f8c6..59fa0e1 100644
--- a/doc/guides/nics/mvpp2.rst
+++ b/doc/guides/nics/mvpp2.rst
@@ -56,7 +56,7 @@ Features of the MVPP2 PMD are:
 
 - Speed capabilities
 - Link status
-- Queue start/stop
+- Tx Queue start/stop
 - MTU update
 - Jumbo frame
 - Promiscuous mode
@@ -70,12 +70,13 @@ Features of the MVPP2 PMD are:
 - L4 checksum offload
 - Packet type parsing
 - Basic stats
-- Extended stats
-- QoS
+- :ref:`Extended stats <extstats>`
 - RX flow control
-- TX queue start/stop
 - Scattered TX frames
-
+- :ref:`QoS <qossupport>`
+- :ref:`Flow API <flowapi>`
+- :ref:`Traffic metering and policing <mtrapi>`
+- :ref:`Traffic Management API <tmapi>`
 
 Limitations
 -----------
@@ -89,6 +90,20 @@ Limitations
   functionality. Current workaround is to reset board so that PPv2 has a
   chance to start in a sane state.
 
+- MUSDK architecture does not support changing configuration in run time.
+  All nessesary configurations should be done before first dev_start().
+
+- RX queue start/stop is not supported.
+
+- Current implementation does not support replacement of buffers in the HW buffer pool
+  at run time, so it is responsibility of the application to ensure that MTU does not exceed the configured buffer size.
+
+- Configuring TX flow control currently is not supported.
+
+- In current implementation, mechanism for acknowledging transmitted packets (``tx_done_cleanup``) is not supported.
+
+- Running more than one DPDK-MUSDK application simultaneously is not supported.
+
 
 Prerequisites
 -------------
@@ -139,6 +154,92 @@ The following options can be modified in the ``config`` file.
 
        When MVPP2 PMD is enabled ``CONFIG_RTE_LIBRTE_MVNETA_PMD`` must be disabled
 
+
+Building DPDK
+-------------
+
+Driver needs precompiled MUSDK library during compilation.
+
+.. code-block:: console
+
+   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
+   ./bootstrap
+   ./configure --host=aarch64-linux-gnu
+   make install
+
+MUSDK will be installed to `usr/local` under current directory.
+For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
+
+Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with
+the path to the MUSDK installation directory needs to be exported.
+
+For additional instructions regarding DPDK cross compilation please refer to :doc:`Cross compile DPDK for ARM64 <../linux_gsg/cross_build_dpdk_for_arm64>`.
+
+.. code-block:: console
+
+   export LIBMUSDK_PATH=<musdk>/usr/local
+   export CROSS=<toolchain>/bin/aarch64-linux-gnu-
+   export RTE_KERNELDIR=<kernel-dir>
+   export RTE_TARGET=arm64-armv8a-linuxapp-gcc
+
+   make config T=arm64-armv8a-linuxapp-gcc
+   sed -i "s/MVNETA_PMD=y/MVNETA_PMD=n/" build/.config
+   sed -i "s/MVPP2_PMD=n/MVPP2_PMD=y/" build/.config
+   make
+
+Usage Example
+-------------
+
+MVPP2 PMD requires extra out of tree kernel modules to function properly.
+`musdk_cma` sources are part of the MUSDK. Please consult
+``doc/musdk_get_started.txt`` for the detailed build instructions.
+For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
+detailed build instructions.
+
+.. code-block:: console
+
+   insmod musdk_cma.ko
+   insmod mvpp2x_sysfs.ko
+
+Additionally interfaces used by DPDK application need to be put up:
+
+.. code-block:: console
+
+   ip link set eth0 up
+   ip link set eth2 up
+
+In order to run testpmd example application following command can be used:
+
+.. code-block:: console
+
+   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
+     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
+     -i -a --rss-udp
+
+.. _extstats:
+
+Extended stats
+--------------
+
+MVPP2 PMD supports the following extended statistics:
+
+	- ``rx_bytes``:	number of RX bytes
+	- ``rx_packets``: number of RX packets
+	- ``rx_unicast_packets``: number of RX unicast packets
+	- ``rx_errors``: number of RX MAC errors
+	- ``rx_fullq_dropped``: number of RX packets dropped due to full RX queue
+	- ``rx_bm_dropped``: number of RX packets dropped due to no available buffers in the HW pool
+	- ``rx_early_dropped``: number of RX packets that were early dropped
+	- ``rx_fifo_dropped``: number of RX packets dropped due to RX fifo overrun
+	- ``rx_cls_dropped``: number of RX packets dropped by classifier
+	- ``tx_bytes``: number of TX bytes
+	- ``tx_packets``: number of TX packets
+	- ``tx_unicast_packets``: number of TX unicast packets
+	- ``tx_errors``: number of TX MAC errors
+
+
+.. _qossupport:
+
 QoS Configuration
 -----------------
 
@@ -302,39 +403,14 @@ Usage example
    ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2,cfg=/home/user/mrvl.conf \
      -c 7 -- -i -a --disable-hw-vlan-strip --rxq=3 --txq=3
 
-
-Building DPDK
--------------
-
-Driver needs precompiled MUSDK library during compilation.
-
-.. code-block:: console
-
-   export CROSS_COMPILE=<toolchain>/bin/aarch64-linux-gnu-
-   ./bootstrap
-   ./configure --host=aarch64-linux-gnu
-   make install
-
-MUSDK will be installed to `usr/local` under current directory.
-For the detailed build instructions please consult ``doc/musdk_get_started.txt``.
-
-Before the DPDK build process the environmental variable ``LIBMUSDK_PATH`` with
-the path to the MUSDK installation directory needs to be exported.
-
-.. code-block:: console
-
-   export LIBMUSDK_PATH=<musdk>/usr/local
-   export CROSS=aarch64-linux-gnu-
-   make config T=arm64-armv8a-linuxapp-gcc
-   sed -i "s/MVNETA_PMD=y/MVNETA_PMD=n/" build/.config
-   sed -i "s/MVPP2_PMD=n/MVPP2_PMD=y/" build/.config
-   make
+.. _flowapi:
 
 Flow API
 --------
 
 PPv2 offers packet classification capabilities via classifier engine which
 can be configured via generic flow API offered by DPDK.
+For an additional description please refer to DPDK :ref:`Generic flow API <Generic_flow_API>`.
 
 Supported flow actions
 ~~~~~~~~~~~~~~~~~~~~~~
@@ -495,31 +571,239 @@ Following limitations need to be taken into account while creating flow rules:
 For additional information about classifier please consult
 ``doc/musdk_cls_user_guide.txt``.
 
-Usage Example
--------------
+.. _mtrapi:
 
-MVPP2 PMD requires extra out of tree kernel modules to function properly.
-`musdk_cma` sources are part of the MUSDK. Please consult
-``doc/musdk_get_started.txt`` for the detailed build instructions.
-For `mvpp2x_sysfs` please consult ``Documentation/pp22_sysfs.txt`` for the
-detailed build instructions.
+Traffic metering and policing
+-----------------------------
 
-.. code-block:: console
+MVPP2 PMD supports DPDK traffic metering and policing that allows the following:
 
-   insmod musdk_cma.ko
-   insmod mvpp2x_sysfs.ko
+1. Meter ingress traffic.
+2. Do policing.
+3. Gather statistics.
 
-Additionally interfaces used by DPDK application need to be put up:
+For an additional description please refer to DPDK :doc:`Traffic Metering and Policing API <../prog_guide/traffic_metering_and_policing>`.
 
-.. code-block:: console
+The policer objects defined by this feature can work with the default policer defined via config file as discribed in :ref:`QoS Support <qossupport>`.
 
-   ip link set eth0 up
-   ip link set eth2 up
+Limitations
+~~~~~~~~~~~
 
-In order to run testpmd example application following command can be used:
+The following capabilities are not supported:
 
-.. code-block:: console
+- MTR object meter DSCP table update
+- MTR object policer action update
+- MTR object enabled statistics
 
-   ./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 7 -- \
-     --burst=128 --txd=2048 --rxd=1024 --rxq=2 --txq=2  --nb-cores=2 \
-     -i -a --rss-udp
+Usage example
+~~~~~~~~~~~~~
+
+1. Run testpmd user app:
+
+   .. code-block:: console
+
+		./testpmd --vdev=eth_mvpp2,iface=eth0,iface=eth2 -c 6 -- -i -p 3 -a --txd 1024 --rxd 1024
+
+2. Create meter profile:
+
+   .. code-block:: console
+
+		testpmd> add port meter profile 0 0 srtcm_rfc2697 2000 256 256
+
+3. Create meter:
+
+   .. code-block:: console
+
+		testpmd> create port meter 0 0 0 yes d d d 0 1 0
+
+4. Create flow rule witch meter attached:
+
+   .. code-block:: console
+
+		testpmd> flow create 0 ingress pattern ipv4 src is 10.10.10.1 / end actions meter mtr_id 0 / end
+
+For a detailed usage description please refer to "Traffic Metering and Policing" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
+
+
+
+.. _tmapi:
+
+Traffic Management API
+----------------------
+
+MVPP2 PMD supports generic DPDK Traffic Management API which allows to
+configure the following features:
+
+1. Hierarchical scheduling
+2. Traffic shaping
+3. Congestion management
+4. Packet marking
+
+Internally TM is represented by a hierarchy (tree) of nodes.
+Node which has a parent is called a leaf whereas node without
+parent is called a non-leaf (root).
+MVPP2 PMD supports two level hierarchy where level 0 represents ports and level 1 represents tx queues of a given port.
+
+.. figure:: img/mvpp2_tm.svg
+
+Nodes hold following types of settings:
+
+- for egress scheduler configuration: weight
+- for egress rate limiter: private shaper
+- bitmask indicating which statistics counters will be read
+
+Hierarchy is always constructed from the top, i.e first a root node is added
+then some number of leaf nodes. Number of leaf nodes cannot exceed number
+of configured tx queues.
+
+After hierarchy is complete it can be committed.
+
+
+For an additional description please refer to DPDK :doc:`Traffic Management API <../prog_guide/traffic_management>`.
+
+Limitations
+~~~~~~~~~~~
+
+The following capabilities are not supported:
+
+- Traffic manager WRED profile and WRED context
+- Traffic manager shared shaper update
+- Traffic manager packet marking
+- Maximum number of levels in hierarchy is 2
+- Currently dynamic change of a hierarchy is not supported
+
+Usage example
+~~~~~~~~~~~~~
+
+For a detailed usage description please refer to "Traffic Management" section in DPDK :doc:`Testpmd Runtime Functions <../testpmd_app_ug/testpmd_funcs>`.
+
+1. Run testpmd as follows:
+
+   .. code-block:: console
+
+		./testpmd --vdev=net_mrvl,iface=eth0,iface=eth2,cfg=./qos_config -c 7 -- \
+		-i -p 3 --disable-hw-vlan-strip --rxq 3 --txq 3 --txd 1024 --rxd 1024
+
+2. Stop all ports:
+
+   .. code-block:: console
+
+		testpmd> port stop all
+
+3. Add shaper profile:
+
+   .. code-block:: console
+
+		testpmd> add port tm node shaper profile 0 0 900000 70000 0
+
+   Parameters have following meaning::
+
+		0       - Id of a port.
+		0       - Id of a new shaper profile.
+		900000  - Shaper rate in bytes/s.
+		70000   - Bucket size in bytes.
+		0       - Packet length adjustment - ignored.
+
+4. Add non-leaf node for port 0:
+
+   .. code-block:: console
+
+		testpmd> add port tm nonleaf node 0 3 -1 0 0 0 0 0 1 3 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port
+		 3  - Id of a new node.
+		-1  - Indicate that root does not have a parent.
+		 0  - Priority of the node.
+		 0  - Weight of the node.
+		 0  - Id of a level. Since this is a root 0 is passed.
+		 0  - Id of the shaper profile.
+		 0  - Number of SP priorities.
+		 3  - Enable statistics for both number of transmitted packets and bytes.
+		 0  - Number of shared shapers.
+
+5. Add leaf node for tx queue 0:
+
+   .. code-block:: console
+
+		testpmd> add port tm leaf node 0 0 3 0 30 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 0  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 30 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+6. Add leaf node for tx queue 1:
+
+   .. code-block:: console
+
+	testpmd> add port tm leaf node 0 1 3 0 60 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 1  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 60 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+7. Add leaf node for tx queue 2:
+
+   .. code-block:: console
+
+		testpmd> add port tm leaf node 0 2 3 0 99 1 -1 0 0 1 0
+
+   Parameters have following meaning::
+
+		 0  - Id of a port.
+		 2  - Id of a new node.
+		 3  - Id of the parent node.
+		 0  - Priority of a node.
+		 99 - WRR weight.
+		 1  - Id of a level. Since this is a leaf node 1 is passed.
+		-1  - Id of a shaper. -1 indicates that shaper is not attached.
+		 0  - Congestion management is not supported.
+		 0  - Congestion management is not supported.
+		 1  - Enable statistics counter for number of transmitted packets.
+		 0  - Number of shared shapers.
+
+8. Commit hierarchy:
+
+   .. code-block:: console
+
+		testpmd> port tm hierarchy commit 0 no
+
+  Parameters have following meaning::
+
+		0  - Id of a port.
+		no - Do not flush TM hierarchy if commit fails.
+
+9. Start all ports
+
+   .. code-block:: console
+
+		testpmd> port start all
+
+
+
+10. Enable forwarding
+
+   .. code-block:: console
+
+		testpmd> start
-- 
2.7.4