As forwarder thread is added to spp_primary, this series of patches is to add descriptions for commands for forwarder. Yasufumi Ogawa (5): docs: add pri APIs for forwarder docs: add desc for primary design docs: add intro in usecases docs: add desc for using worker thread of pri docs: add pri commands for forwarder docs/guides/api_ref/index.rst | 2 + docs/guides/api_ref/spp_mirror.rst | 4 +- docs/guides/api_ref/spp_nfv.rst | 4 +- docs/guides/api_ref/spp_pcap.rst | 4 +- docs/guides/api_ref/spp_primary.rst | 132 +++++++++++++++-- docs/guides/api_ref/spp_vf.rst | 4 +- docs/guides/commands/index.rst | 2 + docs/guides/commands/primary.rst | 160 +++++++++++++++++++-- docs/guides/commands/secondary/spp_nfv.rst | 8 +- docs/guides/design/impl/index.rst | 2 + docs/guides/design/index.rst | 2 + docs/guides/design/spp_primary.rst | 55 ++++++- docs/guides/gsg/howto_use.rst | 41 ++++-- docs/guides/usecases/index.rst | 18 +++ 14 files changed, 385 insertions(+), 53 deletions(-) -- 2.17.1
This update is to add REST APIs for `forward` and `patches` for spp_primary. Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com> --- docs/guides/api_ref/spp_mirror.rst | 4 +- docs/guides/api_ref/spp_nfv.rst | 4 +- docs/guides/api_ref/spp_pcap.rst | 4 +- docs/guides/api_ref/spp_primary.rst | 132 ++++++++++++++++++++++++++-- docs/guides/api_ref/spp_vf.rst | 4 +- 5 files changed, 132 insertions(+), 16 deletions(-) diff --git a/docs/guides/api_ref/spp_mirror.rst b/docs/guides/api_ref/spp_mirror.rst index 1d4efee..a3f7c02 100644 --- a/docs/guides/api_ref/spp_mirror.rst +++ b/docs/guides/api_ref/spp_mirror.rst @@ -4,8 +4,8 @@ .. _spp_ctl_rest_api_spp_mirror: -API for spp_mirror -================== +spp_mirror +========== GET /v1/mirrors/{client_id} --------------------------- diff --git a/docs/guides/api_ref/spp_nfv.rst b/docs/guides/api_ref/spp_nfv.rst index 949f772..7dbd43e 100644 --- a/docs/guides/api_ref/spp_nfv.rst +++ b/docs/guides/api_ref/spp_nfv.rst @@ -4,8 +4,8 @@ .. _spp_ctl_rest_api_spp_nfv: -API for spp_nfv -=============== +spp_nfv +======= GET /v1/nfvs/{client_id} ------------------------ diff --git a/docs/guides/api_ref/spp_pcap.rst b/docs/guides/api_ref/spp_pcap.rst index 56ed542..6f5bec5 100644 --- a/docs/guides/api_ref/spp_pcap.rst +++ b/docs/guides/api_ref/spp_pcap.rst @@ -4,8 +4,8 @@ .. _spp_ctl_rest_api_spp_pcap: -API for spp_pcap -================ +spp_pcap +======== GET /v1/pcaps/{client_id} ------------------------- diff --git a/docs/guides/api_ref/spp_primary.rst b/docs/guides/api_ref/spp_primary.rst index 1345a46..382915f 100644 --- a/docs/guides/api_ref/spp_primary.rst +++ b/docs/guides/api_ref/spp_primary.rst @@ -4,8 +4,8 @@ .. _spp_ctl_rest_api_spp_primary: -API for spp_primary -=================== +spp_primary +=========== GET /v1/primary/status ---------------------- @@ -136,6 +136,46 @@ Response example } +PUT /v1/primary/forward +----------------------- + +Start or stop forwarding. + +* Normal response codes: 204 +* Error response codes: 400, 404 + + +Request example +~~~~~~~~~~~~~~~ + +.. code-block:: console + + $ curl -X PUT -H 'application/json' -d '{"action": "start"}' \ + http://127.0.0.1:7777/v1/primary/forward + + +Response +~~~~~~~~ + +There is no body content for the response of a successful ``PUT`` request. + + +Equivalent CLI command +~~~~~~~~~~~~~~~~~~~~~~ + +Action is ``start``. + +.. code-block:: none + + spp > pri; forward + +Action is ``stop``. + +.. code-block:: none + + spp > pri; stop + + PUT /v1/primary/ports --------------------- @@ -207,6 +247,88 @@ Response There is no body content for the response of a successful ``DELETE`` request. +PUT /v1/primary/patches +----------------------- + +Add a patch. + +* Normal response codes: 204 +* Error response codes: 400, 404 + + +Request (body) +~~~~~~~~~~~~~~ + +.. _table_spp_ctl_spp_primary_ports_patches_body: + +.. table:: Request body params of patches of ``spp_primary``. + + +------+--------+------------------------------------+ + | Name | Type | Description | + | | | | + +======+========+====================================+ + | src | string | Source port id. | + +------+--------+------------------------------------+ + | dst | string | Destination port id. | + +------+--------+------------------------------------+ + + +Request example +~~~~~~~~~~~~~~~ + +.. code-block:: console + + $ curl -X PUT -H 'application/json' \ + -d '{"src": "ring:0", "dst": "ring:1"}' \ + http://127.0.0.1:7777/v1/primary/patches + + +Response +~~~~~~~~ + +There is no body content for the response of a successful ``PUT`` request. + + +Equivalent CLI command +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: none + + spp > pri; patch {src} {dst} + + +DELETE /v1/primary/patches +-------------------------- + +Reset patches. + +* Normal response codes: 204 +* Error response codes: 400, 404 + + +Request example +~~~~~~~~~~~~~~~ + +.. code-block:: console + + $ curl -X DELETE -H 'application/json' \ + http://127.0.0.1:7777/v1/primary/patches + + +Response +~~~~~~~~ + +There is no body content for the response of a successful ``DELETE`` request. + + +Equivalent CLI command +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: none + + spp > pri; patch reset + + DELETE /v1/primary ------------------ @@ -239,12 +361,6 @@ Launch a secondary process. * Error response codes: 400, 404 -Request (path) -~~~~~~~~~~~~~~ - -There is no params in this API. - - Request (body) ~~~~~~~~~~~~~~ diff --git a/docs/guides/api_ref/spp_vf.rst b/docs/guides/api_ref/spp_vf.rst index d9d3719..77a52c2 100644 --- a/docs/guides/api_ref/spp_vf.rst +++ b/docs/guides/api_ref/spp_vf.rst @@ -4,8 +4,8 @@ .. _spp_ctl_rest_api__spp_vf: -API for spp_vf -============== +spp_vf +====== GET /v1/vfs/{client_id} ----------------------- -- 2.17.1
As spp_primary is changed to support forwaring packets as similar to spp_nfv, and user can choose using this forwarder or monitor for statistics, add descriptions of worker threads and its usecases in design section. Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com> --- docs/guides/design/spp_primary.rst | 55 +++++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/docs/guides/design/spp_primary.rst b/docs/guides/design/spp_primary.rst index 79220e7..bfd1f65 100644 --- a/docs/guides/design/spp_primary.rst +++ b/docs/guides/design/spp_primary.rst @@ -15,12 +15,57 @@ of `Multi-process Sample Application <https://doc.dpdk.org/guides/sample_app_ug/multi_process.html>`_ in DPDK's sample applications. -``spp_primary`` is a server process for other secondary processes and -basically working as described in +``spp_primary`` is a server for other secondary processes and +basically working same as described in "How the Application Works" section of the sample application. -However, there are also differences between ``spp_primary`` and +However, there are some differences between ``spp_primary`` and the server process of the sample application. ``spp_primary`` has no limitation of the number of secondary processes. -It does not work for packet forwaring, but just provide rings and memory pools -for secondary processes. +It does not work for packet forwaring without in some usecases, but just +provide rings and memory pools for secondary processes. + + +Master and Worker Threads +------------------------- + +In SPP, Both of primary and secondary processes consist of master thread and +worker thread as slave. Master thread is for accepting commands from a user +for doing task, and running on a master lcore. On the other hand, +slave thread is for packet forwarding or other process specific jobs +as worker, and running on slave lcore. Only slave thread requires +dedicated core for running in pole mode, and launched from +``rte_eal_remote_launch()`` or ``rte_eal_mp_remote_launch()``. + +``spp_primary`` is able to run with or without worker thread selectively +, and requires at least one lcore for server process. +Using worker thread or not depends on your usecases. +``spp_primary`` provides two types of workers currently. + + +Worker Types +------------ + +There are two types of worker thread in ``spp_primary``. First one is +is forwarder thread, and second one is monitor thread. + +As default, ``spp_primary`` runs packet forwarder if two or more lcores +are given while launching the process. Behavior of this forwarder is +same as ``spp_nfv`` described in the next section. +This forwarder provides features for managing ports, patching them and +forwarding packets between ports. +It is useful for very simple usecase in which only few ports are patched +and no need to do forwarding packets in parallel with several processes. + +.. note:: + + In DPDK v18.11 or later, some of PMDs, such as vhost, do not work for + multi-process application. It means that packets cannot forwarded + to a VM or container via secondary process in SPP. + In this case, you use forwarder in ``spp_primary``. + +Another type is monitor for displaying status of ``spp_primary`` in which +statistics of RX and TX packets on each of physical ports and ring ports +are shown periodically in terminal ``spp_primary`` is launched. +Although statistics can be referred in SPP CLI by using ``pri; status`` +command, running monitor thread is useful if you always watch statistics. -- 2.17.1
To explain overview of usecases, and why spp_primary is not described in this chapter, add an introduction in the top of the chapter. Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com> --- docs/guides/api_ref/index.rst | 2 ++ docs/guides/commands/index.rst | 2 ++ docs/guides/design/impl/index.rst | 2 ++ docs/guides/design/index.rst | 2 ++ docs/guides/usecases/index.rst | 18 ++++++++++++++++++ 5 files changed, 26 insertions(+) diff --git a/docs/guides/api_ref/index.rst b/docs/guides/api_ref/index.rst index 490ec15..5619147 100644 --- a/docs/guides/api_ref/index.rst +++ b/docs/guides/api_ref/index.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright(c) 2018-2019 Nippon Telegraph and Telephone Corporation +.. _spp_api_ref_index: + API Reference ============= diff --git a/docs/guides/commands/index.rst b/docs/guides/commands/index.rst index ce67f4b..faaf50a 100644 --- a/docs/guides/commands/index.rst +++ b/docs/guides/commands/index.rst @@ -1,6 +1,8 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright(c) 2010-2014 Intel Corporation +.. _spp_commands_index: + SPP Commands ============ diff --git a/docs/guides/design/impl/index.rst b/docs/guides/design/impl/index.rst index 53f1c6c..5e6aa1b 100644 --- a/docs/guides/design/impl/index.rst +++ b/docs/guides/design/impl/index.rst @@ -2,6 +2,8 @@ Copyright(c) 2010-2014 Intel Corporation +.. _spp_design_impl_index: + Implementation ============== diff --git a/docs/guides/design/index.rst b/docs/guides/design/index.rst index f8d982d..3bf2c1d 100644 --- a/docs/guides/design/index.rst +++ b/docs/guides/design/index.rst @@ -2,6 +2,8 @@ Copyright(c) 2010-2014 Intel Corporation Copyright(c) 2018-2019 Nippon Telegraph and Telephone Corporation +.. _spp_design_index: + Design ====== diff --git a/docs/guides/usecases/index.rst b/docs/guides/usecases/index.rst index f67ea5d..538db87 100644 --- a/docs/guides/usecases/index.rst +++ b/docs/guides/usecases/index.rst @@ -1,9 +1,27 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright(c) 2019 Nippon Telegraph and Telephone Corporation +.. _spp_usecases_index: + Use Cases ========= +As described in :ref:`Design<spp_design_index>`, +SPP has several kinds of secondary process for +usecases such as simple forwarding to network entities, capturing or +mirroring packets for monitoring, or connecting VMs or containers for +Service Function Chaining in NFV. + +This chapter is focusing on explaining about each of secondary +processes with simple usecases. +Usecase of ``spp_primary`` is not covered here because it is almost +similar to ``spp_nfv`` and no need to explain both. + +Details of usages of each process is not covered in this chapter. +You can refer the details of SPP processes via CLI from +:ref:`SPP Commands<spp_commands_index>`, +or via REST API from :ref:`API Reference<spp_api_ref_index>`. + .. toctree:: :maxdepth: 1 :numbered: -- 2.17.1
Add description for using spp_primary with worker thread in `How to Use`. Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com> --- docs/guides/design/spp_primary.rst | 4 +-- docs/guides/gsg/howto_use.rst | 41 +++++++++++++++++++----------- 2 files changed, 28 insertions(+), 17 deletions(-) diff --git a/docs/guides/design/spp_primary.rst b/docs/guides/design/spp_primary.rst index bfd1f65..35e4bbb 100644 --- a/docs/guides/design/spp_primary.rst +++ b/docs/guides/design/spp_primary.rst @@ -37,8 +37,8 @@ as worker, and running on slave lcore. Only slave thread requires dedicated core for running in pole mode, and launched from ``rte_eal_remote_launch()`` or ``rte_eal_mp_remote_launch()``. -``spp_primary`` is able to run with or without worker thread selectively -, and requires at least one lcore for server process. +``spp_primary`` is able to run with or without worker thread selectively, +and requires at least one lcore for server process. Using worker thread or not depends on your usecases. ``spp_primary`` provides two types of workers currently. diff --git a/docs/guides/gsg/howto_use.rst b/docs/guides/gsg/howto_use.rst index 5e461eb..b0824d4 100644 --- a/docs/guides/gsg/howto_use.rst +++ b/docs/guides/gsg/howto_use.rst @@ -335,7 +335,7 @@ To launch SPP primary, run ``spp_primary`` with specific options. # terminal 3 $ sudo ./src/primary/x86_64-native-linuxapp-gcc/spp_primary \ - -l 1 -n 4 \ + -l 0 -n 4 \ --socket-mem 512,512 \ --huge-dir /dev/hugepages \ --proc-type primary \ @@ -349,9 +349,9 @@ SPP primary takes EAL options and application specific options. Core list option ``-l`` is for assigining cores and SPP primary requires just one core. You can use core mask option ``-c`` instead of ``-l``. -You can use ``-m 1024`` for memory reservation instead of -``--socket-mem 1024,0`` if you use single NUMA node. In this case, 512 MB is -reserved on each of nodes. +For memory, this example is for reserving 512 MB on each of two NUMA nodes +hardware, so you use ``-m 1024`` simply, or ``--socket-mem 1024,0`` +if you run the process on single NUMA node. .. note:: @@ -372,15 +372,25 @@ reserved on each of nodes. If you use DPDK v18.11 or later, ``--base-virtaddr 0x100000000`` is enabled in default. You need to use this option only for changing the default value. +If ``spp_primary`` is launched with two or more lcores, forwarder or monitor +is activated. The default is forwarder and monitor is optional in this case. +If you use monitor thread, additional option ``--disp-stat`` is required. +Here is an example for launching ``spp_primary`` with monitor thread. -In general, one lcore is enough for ``spp_primary``. If you give two or -more, it uses second lcore to display statistics periodically and does not -use others. - -.. note:: +.. code-block:: console - Anyway, you can get statistics in SPP CLI with ``pri; status`` command - actually even if you give only one core. + # terminal 3 + $ sudo ./src/primary/x86_64-native-linuxapp-gcc/spp_primary \ + -l 0-1 -n 4 \ # two lcores + --socket-mem 512,512 \ + --huge-dir /dev/hugepages \ + --proc-type primary \ + --base-virtaddr 0x100000000 + -- \ + -p 0x03 \ + -n 10 \ + -s 192.168.1.100:5555 + --disp-stats Primary process sets up physical ports of given port mask with ``-p`` option and ring ports of the number of ``-n`` option. Ports of ``-p`` option is for @@ -393,7 +403,7 @@ secondary processes. # terminal 3 $ sudo ./src/primary/x86_64-native-linuxapp-gcc/spp_primary \ - -l 1 -n 4 \ + -l 0 -n 4 \ --socket-mem 512,512 \ --huge-dir=/dev/hugepages \ --vdev eth_vhost1,iface=/tmp/sock1 # used as 1st phy port @@ -412,6 +422,7 @@ secondary processes. - ``--huge-dir``: Path of hugepage dir. - ``--proc-type``: Process type. - ``--base-virtaddr``: Specify base virtual address. + - ``--disp-stats``: Show statistics periodically. - Application options: @@ -476,7 +487,7 @@ spp_vf .. code-block:: console $ sudo ./src/vf/x86_64-native-linuxapp-gcc/spp_vf \ - -l 0,2-13 -n 4 \ + -l 2-13 -n 4 \ --proc-type secondary \ -- \ --client-id 1 \ @@ -500,7 +511,7 @@ and options are same as ``spp_vf``. .. code-block:: console $ sudo ./src/mirror/x86_64-native-linuxapp-gcc/spp_mirror \ - -l 1,2 -n 4 \ + -l 2,3 -n 4 \ --proc-type secondary \ -- \ --client-id 1 \ @@ -526,7 +537,7 @@ SPP provides ``spp_pcap`` for capturing comparatively heavy traffic. .. code-block:: console $ sudo ./src/pcap/x86_64-native-linuxapp-gcc/spp_pcap \ - -l 0-3 -n 4 \ + -l 2-5 -n 4 \ --proc-type secondary \ -- \ --client-id 1 \ -- 2.17.1
This update is to add `add`, `del`, `forward`, `stop` and `patch` commands for spp_primary in `SPP Commands`. Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com> --- docs/guides/commands/primary.rst | 160 +++++++++++++++++++-- docs/guides/commands/secondary/spp_nfv.rst | 8 +- 2 files changed, 151 insertions(+), 17 deletions(-) diff --git a/docs/guides/commands/primary.rst b/docs/guides/commands/primary.rst index da60947..54aead4 100644 --- a/docs/guides/commands/primary.rst +++ b/docs/guides/commands/primary.rst @@ -44,22 +44,49 @@ All of Sub commands are referred with ``help`` command. status ------ -Show status fo spp_primary and forwarding statistics of each of ports. +Show status fo ``spp_primary`` and forwarding statistics of each of ports. .. code-block:: console spp > pri; status - - lcores: - - [0] - - physical ports: - ID rx tx tx_drop mac_addr - 0 0 0 0 56:48:4f:53:54:00 - 1 0 0 0 56:48:4f:53:54:01 - - ring Ports: - ID rx tx rx_drop tx_drop - 0 0 0 0 0 - 1 0 0 0 0 - ... + - lcore_ids: + - master: 0 + - stats + - physical ports: + ID rx tx tx_drop mac_addr + 0 0 0 0 56:48:4f:53:54:00 + 1 0 0 0 56:48:4f:53:54:01 + - ring ports: + ID rx tx rx_drop tx_drop + 0 0 0 0 0 + 1 0 0 0 0 + 2 0 0 0 0 + ... + +If you run ``spp_primary`` with forwarder thread, status of the forwarder is +also displayed. + +.. code-block:: console + + spp > pri; status + - lcore_ids: + - master: 0 + - slave: 1 + - forwarder: + - status: idling + - ports: + - phy:0 + - phy:1 + - stats + - physical ports: + ID rx tx tx_drop mac_addr + 0 0 0 0 56:48:4f:53:54:00 + 1 0 0 0 56:48:4f:53:54:01 + - ring ports: + ID rx tx rx_drop tx_drop + 0 0 0 0 0 + 1 0 0 0 0 + ... .. _commands_primary_clear: @@ -75,12 +102,119 @@ Clear statistics. Clear port statistics. +.. _commands_primary_add: + +add +--- + +Add a port with resource ID. + +For example, adding ``ring:0`` by + +.. code-block:: console + + spp > pri; add ring:0 + Add ring:0. + +Or adding ``vhost:0`` by + +.. code-block:: console + + spp > pri; add vhost:0 + Add vhost:0. + + +.. _commands_primary_patch: + +patch +------ + +Create a path between two ports, source and destination ports. +This command just creates a path and does not start forwarding. + +.. code-block:: console + + spp > pri; patch phy:0 ring:0 + Patch ports (phy:0 -> ring:0). + + +.. _commands_primary_forward: + +forward +------- + +Start forwarding. + +.. code-block:: console + + spp > pri; forward + Start forwarding. + +Running status is changed from ``idling`` to ``running`` by +executing it. + +.. code-block:: console + + spp > pri; status + - lcore_ids: + - master: 0 + - slave: 1 + - forwarder: + - status: running + - ports: + - phy:0 + - phy:1 + ... + + +.. _commands_primary_stop: + +stop +---- + +Stop forwarding. + +.. code-block:: console + + spp > pri; stop + Stop forwarding. + +Running status is changed from ``running`` to ``idling`` by +executing it. + +.. code-block:: console + + spp > pri; status + - lcore_ids: + - master: 0 + - slave: 1 + - forwarder: + - status: idling + - ports: + - phy:0 + - phy:1 + ... + + +.. _commands_primary_del: + +del +--- + +Delete a port of given resource UID. + +.. code-block:: console + + spp > pri; del ring:0 + Delete ring:0. + + .. _commands_primary_launch: launch ------ -Launch secondary process. +Launch a secondary process. Spp_primary is able to launch a secondary process with given type, secondary ID and options of EAL and application itself. This is a list of supported type diff --git a/docs/guides/commands/secondary/spp_nfv.rst b/docs/guides/commands/secondary/spp_nfv.rst index d6db76e..6389b73 100644 --- a/docs/guides/commands/secondary/spp_nfv.rst +++ b/docs/guides/commands/secondary/spp_nfv.rst @@ -72,14 +72,14 @@ For example, adding ``ring:0`` by .. code-block:: console - spp> nfv 1; add ring:0 + spp > nfv 1; add ring:0 Add ring:0. Or adding ``vhost:0`` by .. code-block:: console - spp> nfv 1; add vhost:0 + spp > nfv 1; add vhost:0 Add vhost:0. @@ -154,7 +154,7 @@ Delete a port from the secondary. .. code-block:: console - spp> nfv 1; del ring:0 + spp > nfv 1; del ring:0 Delete ring:0. @@ -168,4 +168,4 @@ use ``bye sec`` command instead of it. .. code-block:: console - spp> nfv 1; exit + spp > nfv 1; exit -- 2.17.1