[spp] [PATCH 0/5] Update documents for new features

[spp] [PATCH 0/5] Update documents for new features

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

* Add sub commands of `server`, `add` and `del` for managing the list of
  servers.

* Update pri command for referring lcores and `launch` sub command.

* Update nfv command for referring lcores.

* Add descriptions of helper tools.

Yasufumi Ogawa (5):
  docs: add multiple nodes section in usecases
  docs: update status command of primary
  docs: add launch command in primary
  docs: update status command of nfv
  docs: add helper tools

 docs/guides/commands/primary.rst           |  57 ++++++++--
 docs/guides/commands/secondary/spp_nfv.rst |   1 +
 docs/guides/tools/helpers/index.rst        |  15 +++
 docs/guides/tools/helpers/tools.rst        | 115 +++++++++++++++++++
 docs/guides/tools/index.rst                |   1 +
 docs/guides/use_cases/index.rst            |   1 +
 docs/guides/use_cases/multi_nodes.rst      | 174 +++++++++++++++++++++++++++++
 docs/guides/use_cases/spp_nfv.rst          | 170 ----------------------------
 8 files changed, 355 insertions(+), 179 deletions(-)
 create mode 100644 docs/guides/tools/helpers/index.rst
 create mode 100644 docs/guides/tools/helpers/tools.rst
 create mode 100644 docs/guides/use_cases/multi_nodes.rst

-- 
2.7.4

[spp] [PATCH 1/5] docs: add multiple nodes section in usecases

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

In `Use Case`, usecase of multiple nodes is described in `spp_nfv`
section, but it should be moved as a dependent one. This update is to
add `Multiple Nodes` in `Use Case` and move contents described in
`spp_nfv` to the section.

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 docs/guides/use_cases/index.rst       |   1 +
 docs/guides/use_cases/multi_nodes.rst | 174 ++++++++++++++++++++++++++++++++++
 docs/guides/use_cases/spp_nfv.rst     | 170 ---------------------------------
 3 files changed, 175 insertions(+), 170 deletions(-)
 create mode 100644 docs/guides/use_cases/multi_nodes.rst

diff --git a/docs/guides/use_cases/index.rst b/docs/guides/use_cases/index.rst
index c41dc2e..22c946b 100644
--- a/docs/guides/use_cases/index.rst
+++ b/docs/guides/use_cases/index.rst
@@ -9,3 +9,4 @@ Use Cases
    :numbered:
 
    spp_nfv
+   multi_nodes
diff --git a/docs/guides/use_cases/multi_nodes.rst b/docs/guides/use_cases/multi_nodes.rst
new file mode 100644
index 0000000..6be567e
--- /dev/null
+++ b/docs/guides/use_cases/multi_nodes.rst
@@ -0,0 +1,174 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+
+.. _usecase_multi_node:
+
+Multiple Nodes
+==============
+
+SPP provides multi-node support for configuring network across several nodes
+from SPP CLI. You can configure each of nodes step by step.
+
+In :numref:`figure_spp_multi_nodes_vhost`, there are four nodes on which
+SPP and service VMs are running. Host1 behaves as a patch panel for connecting
+between other nodes. A request is sent from a VM on host2 to a VM on host3 or
+host4. Host4 is a backup server for host3 and replaced with host3 by changing
+network configuration. Blue lines are paths for host3 and red lines are for
+host4, and changed alternatively.
+
+.. _figure_spp_multi_nodes_vhost:
+
+.. figure:: ../images/setup/use_cases/spp_multi_nodes_vhost.*
+   :width: 100%
+
+   Multiple nodes example
+
+Launch SPP on Multiple Nodes
+----------------------------
+
+Before SPP CLI, launch spp-ctl on each of nodes. You should give IP address
+with ``-b`` option to be accessed from outside of the node.
+This is an example for launching spp-ctl on host1.
+
+.. code-block:: console
+
+    # Launch on host1
+    $ python3 src/spp-ctl/spp-ctl -b 192.168.11.101
+
+You also need to launch it on host2, host3 and host4 in each of terminals.
+
+After all of spp-ctls are lauched, launch SPP CLI with four ``-b`` options
+for each of hosts. SPP CLI is able to be launched on any of nodes.
+
+.. code-block:: console
+
+    # Launch SPP CLI
+    $ python src/spp.py -b 192.168.11.101 \
+        -b 192.168.11.102 \
+        -b 192.168.11.103 \
+        -b 192.168.11.104 \
+
+If you succeeded to launch all of processes before, you can find them
+by running ``sever list`` command.
+
+.. code-block:: console
+
+    # Launch SPP CLI
+    spp > server list
+      1: 192.168.1.101:7777 *
+      2: 192.168.1.102:7777
+      3: 192.168.1.103:7777
+      4: 192.168.1.104:7777
+
+You might notice that first entry is marked with ``*``. It means that
+the current node under the management is the first node.
+
+Switch Nodes
+------------
+
+SPP CLI manages a node marked with ``*``. If you configure other nodes,
+change the managed node with ``server`` command.
+Here is an example to switch to third node.
+
+.. code-block:: console
+
+    # Launch SPP CLI
+    spp > server 3
+    Switch spp-ctl to "3: 192.168.1.103:7777".
+
+And the result after changed to host3.
+
+.. code-block:: console
+
+    spp > server list
+      1: 192.168.1.101:7777
+      2: 192.168.1.102:7777
+      3: 192.168.1.103:7777 *
+      4: 192.168.1.104:7777
+
+You can also confirm this change by checking IP address of spp-ctl from
+``status`` command.
+
+.. code-block:: console
+
+    spp > status
+    - spp-ctl:
+      - address: 192.168.1.103:7777
+    - primary:
+      - status: not running
+    ...
+
+Configure Patch Panel Node
+--------------------------
+
+First of all of the network configuration, setup blue lines on host1
+described in :numref:`figure_spp_multi_nodes_vhost`.
+You should confirm the managed server is host1.
+
+.. code-block:: console
+
+    spp > server list
+      1: 192.168.1.101:7777 *
+      2: 192.168.1.102:7777
+      ...
+
+Patch two sets of physical ports and start forwarding.
+
+.. code-block:: console
+
+    spp > nfv 1; patch phy:1 phy:2
+    Patch ports (phy:1 -> phy:2).
+    spp > nfv 1; patch phy:3 phy:0
+    Patch ports (phy:3 -> phy:0).
+    spp > nfv 1; forward
+    Start forwarding.
+
+Configure Service VM Nodes
+--------------------------
+
+It is almost similar as
+:ref:`Setup Network Configuration in spp_nfv<usecase_spp_nfv_l2fwd_vhost_nw>`
+to setup for host2, host3, and host4.
+
+For host2, swith server to host2 and run nfv commands.
+
+.. code-block:: console
+
+    # switch to server 2
+    spp > server 2
+    Switch spp-ctl to "2: 192.168.1.102:7777".
+
+    # configure
+    spp > nfv 1; patch phy:0 vhost:0
+    Patch ports (phy:0 -> vhost:0).
+    spp > nfv 1; patch vhost:0 phy:1
+    Patch ports (vhost:0 -> phy:1).
+    spp > nfv 1; forward
+    Start forwarding.
+
+Then, swith to host3 and host4 for doing the same configuration.
+
+Change Path to Backup Node
+--------------------------
+
+Finally, change path from blue lines to red lines.
+
+.. code-block:: console
+
+    # switch to server 1
+    spp > server 2
+    Switch spp-ctl to "2: 192.168.1.102:7777".
+
+    # remove blue path
+    spp > nfv 1; stop
+    Stop forwarding.
+    spp > nfv 1; patch reset
+
+    # configure red path
+    spp > nfv 2; patch phy:1 phy:4
+    Patch ports (phy:1 -> phy:4).
+    spp > nfv 2; patch phy:5 phy:0
+    Patch ports (phy:5 -> phy:0).
+    spp > nfv 2; forward
+    Start forwarding.
diff --git a/docs/guides/use_cases/spp_nfv.rst b/docs/guides/use_cases/spp_nfv.rst
index 8a8e509..39204e3 100644
--- a/docs/guides/use_cases/spp_nfv.rst
+++ b/docs/guides/use_cases/spp_nfv.rst
@@ -497,173 +497,3 @@ and watch received packets on pktgen.
     Start forwarding.
 
 After started forwarding, you can see that packet count is increased.
-
-
-Multiple Nodes
---------------
-
-SPP provides multi-node support for configuring network across several nodes
-from SPP CLI. You can configure each of nodes step by step.
-
-In :numref:`figure_spp_multi_nodes_vhost`, there are four nodes on which
-SPP and service VMs are running. Host1 behaves as a patch panel for connecting
-between other nodes. A request is sent from a VM on host2 to a VM on host3 or
-host4. Host4 is a backup server for host3 and replaced with host3 by changing
-network configuration. Blue lines are paths for host3 and red lines are for
-host4, and changed alternatively.
-
-.. _figure_spp_multi_nodes_vhost:
-
-.. figure:: ../images/setup/use_cases/spp_multi_nodes_vhost.*
-   :width: 100%
-
-   Multiple nodes example
-
-Launch SPP on Multiple Nodes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Before SPP CLI, launch spp-ctl on each of nodes. You should give IP address
-with ``-b`` option to be accessed from outside of the node.
-This is an example for launching spp-ctl on host1.
-
-.. code-block:: console
-
-    # Launch on host1
-    $ python3 src/spp-ctl/spp-ctl -b 192.168.11.101
-
-You also need to launch it on host2, host3 and host4 in each of terminals.
-
-After all of spp-ctls are lauched, launch SPP CLI with four ``-b`` options
-for each of hosts. SPP CLI is able to be launched on any of nodes.
-
-.. code-block:: console
-
-    # Launch SPP CLI
-    $ python src/spp.py -b 192.168.11.101 \
-        -b 192.168.11.102 \
-        -b 192.168.11.103 \
-        -b 192.168.11.104 \
-
-If you succeeded to launch all of processes before, you can find them
-by running ``sever list`` command.
-
-.. code-block:: console
-
-    # Launch SPP CLI
-    spp > server list
-      1: 192.168.1.101:7777 *
-      2: 192.168.1.102:7777
-      3: 192.168.1.103:7777
-      4: 192.168.1.104:7777
-
-You might notice that first entry is marked with ``*``. It means that
-the current node under the management is the first node.
-
-Switch Nodes
-~~~~~~~~~~~~
-
-SPP CLI manages a node marked with ``*``. If you configure other nodes,
-change the managed node with ``server`` command.
-Here is an example to switch to third node.
-
-.. code-block:: console
-
-    # Launch SPP CLI
-    spp > server 3
-    Switch spp-ctl to "3: 192.168.1.103:7777".
-
-And the result after changed to host3.
-
-.. code-block:: console
-
-    spp > server list
-      1: 192.168.1.101:7777
-      2: 192.168.1.102:7777
-      3: 192.168.1.103:7777 *
-      4: 192.168.1.104:7777
-
-You can also confirm this change by checking IP address of spp-ctl from
-``status`` command.
-
-.. code-block:: console
-
-    spp > status
-    - spp-ctl:
-      - address: 192.168.1.103:7777
-    - primary:
-      - status: not running
-    ...
-
-Configure Patch Panel Node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-First of all of the network configuration, setup blue lines on host1
-described in :numref:`figure_spp_multi_nodes_vhost`.
-You should confirm the managed server is host1.
-
-.. code-block:: console
-
-    spp > server list
-      1: 192.168.1.101:7777 *
-      2: 192.168.1.102:7777
-      ...
-
-Patch two sets of physical ports and start forwarding.
-
-.. code-block:: console
-
-    spp > nfv 1; patch phy:1 phy:2
-    Patch ports (phy:1 -> phy:2).
-    spp > nfv 1; patch phy:3 phy:0
-    Patch ports (phy:3 -> phy:0).
-    spp > nfv 1; forward
-    Start forwarding.
-
-Configure Service VM Nodes
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-It is almost similar as
-:ref:`Setup Network Configuration in spp_nfv<usecase_spp_nfv_l2fwd_vhost_nw>`
-to setup for host2, host3, and host4.
-
-For host2, swith server to host2 and run nfv commands.
-
-.. code-block:: console
-
-    # switch to server 2
-    spp > server 2
-    Switch spp-ctl to "2: 192.168.1.102:7777".
-
-    # configure
-    spp > nfv 1; patch phy:0 vhost:0
-    Patch ports (phy:0 -> vhost:0).
-    spp > nfv 1; patch vhost:0 phy:1
-    Patch ports (vhost:0 -> phy:1).
-    spp > nfv 1; forward
-    Start forwarding.
-
-Then, swith to host3 and host4 for doing the same configuration.
-
-Change Path to Backup Node
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Finally, change path from blue lines to red lines.
-
-.. code-block:: console
-
-    # switch to server 1
-    spp > server 2
-    Switch spp-ctl to "2: 192.168.1.102:7777".
-
-    # remove blue path
-    spp > nfv 1; stop
-    Stop forwarding.
-    spp > nfv 1; patch reset
-
-    # configure red path
-    spp > nfv 2; patch phy:1 phy:4
-    Patch ports (phy:1 -> phy:4).
-    spp > nfv 2; patch phy:5 phy:0
-    Patch ports (phy:5 -> phy:0).
-    spp > nfv 2; forward
-    Start forwarding.
-- 
2.7.4

[spp] [PATCH 2/5] docs: update status command of primary

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

This patch is to update status sub command of pri to include lcores.

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 docs/guides/commands/primary.rst | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)

diff --git a/docs/guides/commands/primary.rst b/docs/guides/commands/primary.rst
index 6742d28..11e378c 100644
--- a/docs/guides/commands/primary.rst
+++ b/docs/guides/commands/primary.rst
@@ -10,19 +10,22 @@ Primary process is managed with ``pri`` command.
 status
 ------
 
-Show forwarding statistics of each of ports.
+Show status fo spp_primary and forwarding statistics of each of ports.
 
 .. code-block:: console
 
     spp > pri; status
-    Physical Ports:
-      ID          rx          tx     tx_drop  mac_addr
-       0    78932932    78932931           1  56:48:4f:53:54:00
-    Ring Ports:
-      ID          rx          tx     rx_drop     tx_drop
-       0       89283       89283           0           0
-       1        9208        9203           0           5
-       ...
+    - lcores:
+      - [0]
+    - physical ports:
+        ID          rx          tx     tx_drop  mac_addr
+         0           0           0           0  56:48:4f:53:54:00
+         1           0           0           0  56:48:4f:53:54:01
+    - ring Ports:
+        ID          rx          tx     rx_drop     rx_drop
+         0           0           0           0           0
+         1           0           0           0           0
+
 
 clear
 -----
-- 
2.7.4

[spp] [PATCH 3/5] docs: add launch command in primary

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

This update is to add launch sub command of primary to `Commands`
section.

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 docs/guides/commands/primary.rst | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/docs/guides/commands/primary.rst b/docs/guides/commands/primary.rst
index 11e378c..5032b37 100644
--- a/docs/guides/commands/primary.rst
+++ b/docs/guides/commands/primary.rst
@@ -35,3 +35,39 @@ Clear statistics.
 .. code-block:: console
 
     spp > pri; clear
+
+
+launch
+------
+
+Launch secondary process.
+
+Spp_primary is able to launch a secondary process with given type, secondary
+ID and options of EAL and application itself. This is a list of supported type
+of secondary processes.
+
+  * nfv
+  * vf
+  * mirror
+  * pcap
+
+.. code-block:: console
+
+    # spp_nfv with sec ID 1
+    spp > pri; launch nfv 1 -l 1,2 -m 512 -- -n -s 192.168.1.100:6666
+
+    # spp_vf with sec ID 2
+    spp > pri; launch vf 2 -l 1,3-5 -m 512 -- --client-id -s 192.168.1.100:6666
+
+You notice that ``--proc-type secondary`` is not given for launching secondary
+processes. ``launch`` command adds this option before requesting to launch
+the process so that you do not need to input this option by yourself.
+
+``launch`` command supports TAB completion for type, secondary ID and the rest
+of options. Some of EAL and application options are just a template, so you
+should update them before launching.
+
+In terms of log, each of secondary processes are output its log messages to
+files under ``log`` directory of project root. The name of log file is defined
+with type of process and secondary ID. For instance, ``nfv 2``, the path of log
+file is ``log/spp_nfv-2.log``.
-- 
2.7.4

[spp] [PATCH 4/5] docs: update status command of nfv

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

This patch is to update status message of spp_nfv to add lcore info.

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 docs/guides/commands/secondary/spp_nfv.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/docs/guides/commands/secondary/spp_nfv.rst b/docs/guides/commands/secondary/spp_nfv.rst
index 5cb2d36..4057c25 100644
--- a/docs/guides/commands/secondary/spp_nfv.rst
+++ b/docs/guides/commands/secondary/spp_nfv.rst
@@ -49,6 +49,7 @@ source if it is not patched.
 
     spp > nfv 1; status
     - status: idling
+    - lcores: [1, 2]
     - ports:
       - phy:0 -> ring:0
       - phy:1
-- 
2.7.4

[spp] [PATCH 5/5] docs: add helper tools

[ogawa.yasufumi]

From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>

This update is to add description of helper tools in `Tools` chapter.
There are two helper tools currently.

* cpu_layout.py
* sec_launcher.py

Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
 docs/guides/tools/helpers/index.rst |  15 +++++
 docs/guides/tools/helpers/tools.rst | 115 ++++++++++++++++++++++++++++++++++++
 docs/guides/tools/index.rst         |   1 +
 3 files changed, 131 insertions(+)
 create mode 100644 docs/guides/tools/helpers/index.rst
 create mode 100644 docs/guides/tools/helpers/tools.rst

diff --git a/docs/guides/tools/helpers/index.rst b/docs/guides/tools/helpers/index.rst
new file mode 100644
index 0000000..c03b552
--- /dev/null
+++ b/docs/guides/tools/helpers/index.rst
@@ -0,0 +1,15 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+.. _spp_tools_helpers_index:
+
+Helper tools
+============
+
+Helper tools are intended to be used from other programs, such as ``spp-ctl``
+or SPP CLI.
+
+.. toctree::
+    :maxdepth: 1
+
+    tools
diff --git a/docs/guides/tools/helpers/tools.rst b/docs/guides/tools/helpers/tools.rst
new file mode 100644
index 0000000..b435d5f
--- /dev/null
+++ b/docs/guides/tools/helpers/tools.rst
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Nippon Telegraph and Telephone Corporation
+
+.. _spp_tools_helpers_tools:
+
+CPU Layout
+==========
+
+This tool is a customized script of DPDK's user tool ``cpu_layout.py``. It is
+used from ``spp-ctl`` to get CPU layout. The behaviour of this script is same
+as original one if you just run on terminal.
+
+.. code-block:: console
+
+    $ python3 tools/helpers/cpu_layout.py
+    ======================================================================
+    Core and Socket Information (as reported by '/sys/devices/system/cpu')
+    ======================================================================
+
+    cores =  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
+    sockets =  [0]
+
+            Socket 0
+            --------
+    Core 0  [0]
+    Core 1  [1]
+    ...
+
+Customized version of ``cpu_layout.py`` accepts an additional option
+``--json`` to output the result in JSON format.
+
+.. code-block:: console
+
+    # Output in JSON format
+    $ python3 tools/helpers/cpu_layout.py --json | jq
+    [
+      {
+        "socket_id": 0,
+        "cores": [
+          {
+            "core_id": 1,
+            "cpus": [
+              1
+            ]
+          },
+          {
+            "core_id": 0,
+            "cpus": [
+              0
+            ]
+          },
+          ...
+      }
+    ]
+
+You can almost the same result from ``spp-ctl``, but the order of params are
+just different.
+
+.. code-block:: console
+
+    # Retrieve CPU layout via REST API
+    $ curl -X GET http://192.168.1.100:7777/v1/cpus | jq
+      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                     Dload  Upload   Total   Spent    Left  Speed
+    100   505  100   505    0     0  18091      0 --:--:-- --:--:-- --:--:-- 18703
+    [
+      {
+        "cores": [
+          {
+            "cpus": [
+              1
+            ],
+            "core_id": 1
+          },
+          {
+            "cpus": [
+              0
+            ],
+            "core_id": 0
+          },
+          ...
+        ],
+        "socket_id": 0
+      }
+    ]
+
+
+Secondary Process Launcher
+==========================
+
+It is very simple python script used to lauch a secondary process from other
+program. It is intended to be used from spp_primary for launching. Here is
+whole lines of the script.
+
+.. code-block:: python
+
+    #!/usr/bin/env python
+    # coding: utf-8
+    """SPP secondary launcher."""
+
+    import sys
+    import subprocess
+
+    if len(sys.argv) > 1:
+        cmd = sys.argv[1:]
+        subprocess.call(cmd)
+
+As you may notice, it just runs given name or path of command with options,
+so you can any of command other than SPP secondary processes. However, it
+might be nouse for almost of users.
+
+The reason of why this script is required is to launch secondary process from
+``spp_primary`` indirectly to avoid launched secondaries to be zombies finally.
+In addtion, secondary processes other than ``spp_nfv`` do not work correctly
+after launched with execv() or other siblings directly from ``spp_primary``.
diff --git a/docs/guides/tools/index.rst b/docs/guides/tools/index.rst
index 445c486..c739d79 100644
--- a/docs/guides/tools/index.rst
+++ b/docs/guides/tools/index.rst
@@ -11,3 +11,4 @@ Tools
    :numbered:
 
    sppc/index
+   helpers/index
-- 
2.7.4