From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.valinux.co.jp (mail.valinux.co.jp [210.128.90.3]) by dpdk.org (Postfix) with ESMTP id DBB434CBB for ; Thu, 13 Sep 2018 01:25:49 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.valinux.co.jp (Postfix) with ESMTP id C001FB3B5E for ; Thu, 13 Sep 2018 08:25:47 +0900 (JST) X-Virus-Scanned: Debian amavisd-new at valinux.co.jp Received: from mail.valinux.co.jp ([127.0.0.1]) by localhost (mail.valinux.co.jp [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hEJ2svBRPvOj for ; Thu, 13 Sep 2018 08:25:47 +0900 (JST) Received: from [127.0.0.1] (vagw.valinux.co.jp [210.128.90.14]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.valinux.co.jp (Postfix) with ESMTPS id 969B2B3944 for ; Thu, 13 Sep 2018 08:25:47 +0900 (JST) Date: Thu, 13 Sep 2018 08:25:44 +0900 From: Itsuro ODA To: spp@dpdk.org Message-Id: <20180913082544.2D36.277DD91C@valinux.co.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.71.01 [ja] Subject: [spp] [PATCH] spp-ctl: SPP controller with Web API X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2018 23:25:50 -0000 From: Itsuro Oda spp-ctl is a SPP controller with a REST like web API. spp-ctl maintains the connections from the SPP processes and at the same time exposes the API for the user to request for the SPP processes. Background and motivation: Current CLI (spp.py/spp_vf.py) can be used by intaractive only. Therefore, spp-agent, a component of networking-spp which make SPP available on OpenStack environment, implements SPP controller in itself. (see. https://github.com/openstack/networking-spp ) Either CLI or spp-agent, there is a problem that other people can not request to SPP processes while using. spp-ctl is invented to solve this problem. Both CLI and spp-agent can be used spp-ctl to request SPP processes instead of owning contoroller itself. In that case, multiple people can request to SPP processes at the same time. Note that spp-agent has a plan to change to use spp-ctl. It is also available not using CLI but requesting spp-ctl directly. Signed-off-by: Itsuro Oda --- src/spp-ctl/README.rst | 102 +++++ src/spp-ctl/api-reference.rst | 790 ++++++++++++++++++++++++++++++++++ src/spp-ctl/requirements.txt | 3 + src/spp-ctl/spp-ctl | 11 + src/spp-ctl/spp_ctl.py | 158 +++++++ src/spp-ctl/spp_proc.py | 184 ++++++++ src/spp-ctl/spp_webapi.py | 440 +++++++++++++++++++ 7 files changed, 1688 insertions(+) create mode 100644 src/spp-ctl/README.rst create mode 100644 src/spp-ctl/api-reference.rst create mode 100644 src/spp-ctl/requirements.txt create mode 100755 src/spp-ctl/spp-ctl create mode 100644 src/spp-ctl/spp_ctl.py create mode 100644 src/spp-ctl/spp_proc.py create mode 100644 src/spp-ctl/spp_webapi.py diff --git a/src/spp-ctl/README.rst b/src/spp-ctl/README.rst new file mode 100644 index 0000000..847b9dc --- /dev/null +++ b/src/spp-ctl/README.rst @@ -0,0 +1,102 @@ +==================================== +spp-ctl: SPP controller with Web API +==================================== + +Overview +======== + +spp-ctl is a SPP controller with a REST like web API. + +spp-ctl maintains the connections from the SPP processes and at the same +time exposes the API for the user to request for the SPP processes. + +Background and motivation +------------------------- + +Current CLI (spp.py/spp_vf.py) can be used by intaractive only. +Therefore, spp-agent, a component of networking-spp which make SPP +available on OpenStack environment, implements SPP controller in +itself. (see. https://github.com/openstack/networking-spp ) + +Either CLI or spp-agent, there is a problem that other people can not +request to SPP processes while using. spp-ctl is invented to solve this +problem. + +Both CLI and spp-agent can be used spp-ctl to request SPP processes +instead of owning contoroller itself. In that case, multiple people +can request to SPP processes at the same time. +Note that spp-agent has a plan to change to use spp-ctl. +It is also available not using CLI but requesting spp-ctl directly. + +Architecture +------------ + +The design goal of spp-ctl is to be as simple as possible. +It is stateless. Basically, spp-ctl only converts API requests into +commands of SPP processes and throws request, thouth it does syntax and +lexical check for API requests. + +spp-ctl adopts bottle (it is simple and well known) as a web framework +and eventlet for parallel processing. spp-ctl can process multiple APIs +at the same time, however, requests for per SPP process are serialized +internally. + + +Setup +===== + +spp-ctl is a simple program written in python3. Installation of related +packages is as follows (assume ubuntu). + +:: + + $ sudo apt update + $ sudo apt install python3 + $ sudo apt install python3-pip + $ sudo pip3 install -r requirements.txt + +Usage +----- + +:: + + usage: spp-ctl [-p PRI_PORT] [-s SEC_PORT] [-a API_PORT] + + optional arguments: + -p PRI_PORT primary port. default is 5555. + -s SEC_PORT secondary port. default is 6666. + -a API_PORT web api port. default is 7777. + +Using systemd +------------- + +Although spp-ctl runs as a daemon process normaly, it assumes to the +use of systemd and does not daemonize itself. + +The service file for systemd is simple as shown below:: + + [Unit] + Description = SPP Controller + + [Service] + ExecStart = {SPP install path}/spp_ctl/spp-ctl -p 5555 -s 6666 -a 7777 + User = root + +API Usage +========= + +For API details, see API-reference_. + +.. _API-reference: ./api-reference.rst + +Since spp-ctl provides the web API, for example, you can use curl to execute +requests as follows:: + + $ curl http://localhost:7777/v1/processes + [{"type": "primary"}, {"client-id": 1, "type": "vf"}, {"client-id": 2, "type": "vf"}] + $ curl http://localhost:7777/v1/vfs/1 + ... snip + $ curl -X POST http://localhost:7777/v1/vfs/1/components \ + -d '{"core": 2, "name": "forward_0_tx", "type": "forward"}' + $ + diff --git a/src/spp-ctl/api-reference.rst b/src/spp-ctl/api-reference.rst new file mode 100644 index 0000000..b8bf488 --- /dev/null +++ b/src/spp-ctl/api-reference.rst @@ -0,0 +1,790 @@ +============= +API Reference +============= + +Overview +======== + +Spp-ctl provides simple REST like API. It supports http only, not https. + +Request and Response +-------------------- + +Request body is json format. It is accepted both "text/plain" and "application/json" for the content-type header. + +Response of GET is json format and the content-type is "application/json" if the request success. + +If a request fails, the response is a text which shows error reason and the content-type is "text/plain". + +Error code +---------- + +Spp-ctl does basic syntax and lexical check of a request. + ++-------+------------------------------------------------------------------------------+ +| Error | Description | ++=======+==============================================================================+ +| 400 | Syntax or lexical error of a request. | +| | Or an SPP process returns error for the command correspondings to a request. | ++-------+------------------------------------------------------------------------------+ +| 404 | URL is not supported for spp-ctl. | +| | Or there is no SPP process of client-id in a URL. | ++-------+------------------------------------------------------------------------------+ +| 500 | A system error occurs in the spp-ctl. | +| | ex. communication error between an SPP processes. | ++-------+------------------------------------------------------------------------------+ + + +API independent of the process type +=================================== + +GET /v1/processes +----------------- + +Show the SPP processes connected to the spp-ctl. + +Normarl response codes: 200 + +Response +^^^^^^^^ + +An array of process objects. + +process object: + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| type | string | process type. one of "primary", "vf" or "nfv". | ++-----------+---------+-------------------------------------------------------------+ +| client-id | integer | client id. if type is "primary" this member does not exist. | ++-----------+---------+-------------------------------------------------------------+ + +Response example +^^^^^^^^^^^^^^^^ + +:: + +[{"type": "primary"}, {"type": "vf", "client-id": 1}, {"type": "nfv", "client-id": 2}] + + +API for spp_vf +============== + +GET /v1/vfs/{client_id} +----------------------- + +Get the information of the spp_vf process. + +Normal response codes: 200 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Response +^^^^^^^^ + ++------------------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++==================+=========+=============================================================+ +| client-id | integer | client id. | ++------------------+---------+-------------------------------------------------------------+ +| ports | array | an array of port ids used by the process. | ++------------------+---------+-------------------------------------------------------------+ +| components | array | an array of component objects in the process. | ++------------------+---------+-------------------------------------------------------------+ +| classifier_table | array | an array of classifier tables in the process. | ++------------------+---------+-------------------------------------------------------------+ + +component object: + ++---------+---------+---------------------------------------------------------------------+ +| Name | Type | Description | ++=========+=========+=====================================================================+ +| core | integer | core id running on the component | ++---------+---------+---------------------------------------------------------------------+ +| name | string | an array of port ids used by the process. | ++---------+---------+---------------------------------------------------------------------+ +| type | string | an array of component objects in the process. | ++---------+---------+---------------------------------------------------------------------+ +| rx_port | array | an array of port objects connected to the rx side of the component. | ++---------+---------+---------------------------------------------------------------------+ +| tx_port | array | an array of port objects connected to the tx side of the component. | ++---------+---------+---------------------------------------------------------------------+ + +port object: + ++---------+---------+---------------------------------------------------------------------+ +| Name | Type | Description | ++=========+=========+=====================================================================+ +| port | string | port id. port id is the form {interface_type}:{interface_id}. | ++---------+---------+---------------------------------------------------------------------+ +| vlan | object | vlan operation which is applied to the port. | ++---------+---------+---------------------------------------------------------------------+ + +vlan object: + ++-----------+---------+-------------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+===================================================================+ +| operation | string | "add", "del" or "none". | ++-----------+---------+-------------------------------------------------------------------+ +| id | integer | vlan id. | ++-----------+---------+-------------------------------------------------------------------+ +| pcp | integer | vlan pcp. | ++-----------+---------+-------------------------------------------------------------------+ + +classifier table: + ++-----------+---------+----------------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+======================================================================+ +| type | string | "mac" or "vlan". | ++-----------+---------+----------------------------------------------------------------------+ +| value | string | "mac address" for "mac" type. "vlan id/mac address" for "vlan" type. | ++-----------+---------+----------------------------------------------------------------------+ +| port | string | port id applied to classify. | ++-----------+---------+----------------------------------------------------------------------+ + +Response example +^^^^^^^^^^^^^^^^ + +:: + +{ +"client-id": 1, +"ports": ["phy:0", "phy:1", "vhost:0", "vhost:1", "ring:0", "ring:1", "ring:2", "ring:3"], +"components": [ +{ +"core": 2, +"name": "forward_0_tx", +"type": "forward", +"rx_port": [ +{ +"port": "ring:0", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +], +"tx_port": [ +{ +"port": "vhost:0", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +] +}, +{ +"core": 3, +"type": "unuse" +}, +{ +"core": 4, +"type": "unuse" +}, +{ +"core": 5, +"name": "forward_1_rx", +"type": "forward", +"rx_port": [ +{ +"port": "vhost:1", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +], +"tx_port": [ +{ +"port": "ring:3", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +] +}, +{ +"core": 6, +"name": "classifier", +"type": "classifier_mac", +"rx_port": [ +{ +"port": "phy:0", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +], +"tx_port": [ +{ +"port": "ring:0", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +}, +{ +"port": "ring:2", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +] +}, +{ +"core": 7, +"name": "merger", +"type": "merge", +"rx_port": [ +{ +"port": "ring:1", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +}, +{ +"port": "ring:3", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +], +"tx_port": [ +{ +"port": "phy:0", +"vlan": { "operation": "none", "id": 0, "pcp": 0 } +} +] +} +}, +"classifier_table": [ +{ +"type": "mac", +"value": "FA:16:3E:7D:CC:35", +"port": "ring:0" +} +] +} + +Note: The component which type is "unused" is to indicate unused core. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};status + + +POST /v1/vfs/{client_id}/components +----------------------------------- + +Start the component. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + + +Request(body) +^^^^^^^^^^^^^ + ++-----------+---------+----------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+================================================================+ +| name | string | component name. must be unique in the process. | ++-----------+---------+----------------------------------------------------------------+ +| core | integer | core id. | ++-----------+---------+----------------------------------------------------------------+ +| type | string | component type. one of "forward", "merge" or "classifier_mac". | ++-----------+---------+----------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ +:: + +{"name": "forwarder1", "core": 12, "type": "forward"} + +Response +^^^^^^^^ + +There is no body content for the response of a successful POST request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};component start {name} {core} {type} + + +DELETE /v1/vfs/{sec id}/components/{name} +----------------------------------------- + +Stop the component. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ +| name | string | component name. | ++-----------+---------+-------------------------------------------------------------+ + +Response +^^^^^^^^ + +There is no body content for the response of a successful POST request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};component stop {name} + + +PUT /v1/vfs/{client_id}/components/{name}/ports +----------------------------------------------- + +Add or Delete port to the component. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ +| name | string | component name. | ++-----------+---------+-------------------------------------------------------------+ + +Request(body) +^^^^^^^^^^^^^ + ++---------+---------+---------------------------------------------------------------------+ +| Name | Type | Description | ++=========+=========+=====================================================================+ +| action | string | "attach" or "detach". | ++---------+---------+---------------------------------------------------------------------+ +| port | string | port id. port id is the form {interface_type}:{interface_id}. | ++---------+---------+---------------------------------------------------------------------+ +| dir | string | "rx" or "tx". | ++---------+---------+---------------------------------------------------------------------+ +| vlan | object | vlan operation which is applied to the port. it can be omitted. | ++---------+---------+---------------------------------------------------------------------+ + +vlan object: + ++-----------+---------+-------------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+===================================================================+ +| operation | string | "add", "del" or "none". | ++-----------+---------+-------------------------------------------------------------------+ +| id | integer | vlan id. ignored when operation is "del" or "none". | ++-----------+---------+-------------------------------------------------------------------+ +| pcp | integer | vlan pcp. ignored when operation is "del" or "none". | ++-----------+---------+-------------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ + +:: + +{"action": "attach", "port": "vhost:1", "dir": "rx", +"vlan": {"operation": "add", "id": 677, "pcp": 0} +} + +:: + +{"action": "detach", "port": "vhost:0", "dir": "tx"} + + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ +action is "attach" + +:: + +sec {client_id};port add {port} {dir} {name} [add_vlantag {id} {pcp} | del_vlantag] + +action is "detach" + +:: + +sec {client_id};port del {port} {dir} {name} + + +PUT /v1/vfs/{sec id}/classifier_table +------------------------------------- + +Set or Unset classifier table. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Request(body) +^^^^^^^^^^^^^ + ++-------------+-----------------+-------------------------------------------------------------------+ +| Name | Type | Description | ++=============+=================+===================================================================+ +| action | string | "add" or "del". | ++-------------+-----------------+-------------------------------------------------------------------+ +| type | string | "mac" or "vlan". | ++-------------+-----------------+-------------------------------------------------------------------+ +| vlan | integer or null | vlan id when type is "vlan. null or omitted when type is "mac". | ++-------------+-----------------+-------------------------------------------------------------------+ +| mac_address | string | mac address. | ++-------------+-----------------+-------------------------------------------------------------------+ +| port | string | port id. | ++-------------+-----------------+-------------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ + +:: + +{"action": "add", "type": "mac", +"mac_address": "FA:16:3E:7D:CC:35", "port": "ring:0" +} + +:: + +{"action": "del", "type": "vlan", "vlan": 475, +"mac_address": "FA:16:3E:7D:CC:35", "port": "ring:0" +} + + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +type is "mac" + +:: + +classifier_table {action} mac {mac_address} {port} + +type is "vlan" + +:: + +classifier_table {action} vlan {vlan} {mac_address} {port} + + +API for spp_nfv/spp_vm +====================== + +GET /v1/nfvs/{client_id} +------------------------ + +Get the information of the spp_nfv/spp_vm process. + +Normal response codes: 200 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Response +^^^^^^^^ + ++------------------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++==================+=========+=============================================================+ +| client-id | integer | client id. | ++------------------+---------+-------------------------------------------------------------+ +| status | string | "Running" or "Idle". | ++------------------+---------+-------------------------------------------------------------+ +| ports | array | an array of port ids used by the process. | ++------------------+---------+-------------------------------------------------------------+ +| patches | array | an array of patches. | ++------------------+---------+-------------------------------------------------------------+ + +patch objest + ++----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++==========+=========+=============================================================+ +| src | string | source port id. | ++----------+---------+-------------------------------------------------------------+ +| dst | string | destination port id. | ++----------+---------+-------------------------------------------------------------+ + +Response example +^^^^^^^^^^^^^^^^ + +:: + +{ +"client-id": 1, +"status": "Running", +"ports": ["phy:0", "phy:1", "vhost:0", "vhost:1", "ring:0", "ring:1", "ring:2", "ring:3"], +"patches": [ +{"src": "vhost:0", "dst": "ring:0"}, +{"src": "ring:1", "dst": "vhost:1"} +] +} + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};status + + +PUT /v1/nfvs/{client_id}/forward +-------------------------------- + +Start or Stop forwarding. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Request(body) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| action | string | "start" or "stop". | ++-----------+---------+-------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ + +:: + +{"action": "start"} + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +action is "start" + +:: + +sec {client_id};forward + +action is "stop" + +:: + +sec {client_id};stop + + +PUT /v1/nfvs/{client_id}/ports +------------------------------ + +Add or Delete port. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Request(body) +^^^^^^^^^^^^^ + ++-----------+---------+---------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+===============================================================+ +| action | string | "add" or "del". | ++-----------+---------+---------------------------------------------------------------+ +| port | string | port id. port id is the form {interface_type}:{interface_id}. | ++-----------+---------+---------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ + +:: + +{"action": "add", "port": "vhost:0"} + + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};{action} {interface_type} {interface_id} + + +PUT /v1/nfvs/{client_id}/patches +-------------------------------- + +Add a patch. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Request(body) +^^^^^^^^^^^^^ + ++----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++==========+=========+=============================================================+ +| src | string | source port id. | ++----------+---------+-------------------------------------------------------------+ +| dst | string | destination port id. | ++----------+---------+-------------------------------------------------------------+ + +Request example +^^^^^^^^^^^^^^^ + +:: + +{"src": "vhost:0", "dst": "ring:0"} + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};patch {src} {dst} + + +DELETE /v1/nfvs/{client_id}/patches +----------------------------------- + +Reset patches. + +Normal response codes: 204 + +Error response codes: 400, 404 + +Request(path) +^^^^^^^^^^^^^ + ++-----------+---------+-------------------------------------------------------------+ +| Name | Type | Description | ++===========+=========+=============================================================+ +| client_id | integer | client id. | ++-----------+---------+-------------------------------------------------------------+ + +Response +^^^^^^^^ + +There is no body content for the response of a successful DELETE request. + +Equivalent CLI command +^^^^^^^^^^^^^^^^^^^^^^ + +:: + +sec {client_id};patch reset + + +API for spp_primary +=================== + +GET /v1/primary/status +---------------------- + +Show statistical information. + +Normal response codes: 200 + +Response +^^^^^^^^ + +There is no data at the moment. The statistical information will be returned when spp_primary +implements it. + + +DELETE /v1/primary/status +------------------------- + +Clear statistical information. + +Normal response codes: 204 + +Response +^^^^^^^^ + +There is no body content for the response of a successful PUT request. + diff --git a/src/spp-ctl/requirements.txt b/src/spp-ctl/requirements.txt new file mode 100644 index 0000000..cc52d48 --- /dev/null +++ b/src/spp-ctl/requirements.txt @@ -0,0 +1,3 @@ +eventlet +bottle +netaddr diff --git a/src/spp-ctl/spp-ctl b/src/spp-ctl/spp-ctl new file mode 100755 index 0000000..645611b --- /dev/null +++ b/src/spp-ctl/spp-ctl @@ -0,0 +1,11 @@ +#!/usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Nippon Telegraph and Telephone Corporation + +import sys + +from spp_ctl import main + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/src/spp-ctl/spp_ctl.py b/src/spp-ctl/spp_ctl.py new file mode 100644 index 0000000..e168747 --- /dev/null +++ b/src/spp-ctl/spp_ctl.py @@ -0,0 +1,158 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Nippon Telegraph and Telephone Corporation + +import eventlet +eventlet.monkey_patch() + +import argparse +import errno +import json +import logging +import socket +import subprocess + +import spp_proc +import spp_webapi + + +LOG = logging.getLogger(__name__) + + +MSG_SIZE = 4096 + + +class Controller(object): + + def __init__(self, pri_port, sec_port, api_port): + self.web_server = spp_webapi.WebServer(self, api_port) + self.procs = {} + self.init_connection(pri_port, sec_port) + + def start(self): + self.web_server.start() + + def init_connection(self, pri_port, sec_port): + self.pri_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.pri_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + self.pri_sock.bind(('127.0.0.1', pri_port)) + self.pri_sock.listen(1) + self.primary_listen_thread = eventlet.greenthread.spawn( + self.accept_primary) + + self.sec_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.sec_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + self.sec_sock.bind(('127.0.0.1', sec_port)) + self.sec_sock.listen(1) + self.secondary_listen_thread = eventlet.greenthread.spawn( + self.accept_secondary) + + def accept_primary(self): + while True: + conn, _ = self.pri_sock.accept() + proc = self.procs.get(spp_proc.ID_PRIMARY) + if proc is not None: + LOG.warning("spp_primary reconnect !") + with proc.sem: + try: + proc.conn.close() + except Exception: + pass + proc.conn = conn + # NOTE: when spp_primary restart, all secondarys must be + # restarted. this is out of controle of spp-ctl. + else: + LOG.info("primary connected.") + self.procs[spp_proc.ID_PRIMARY] = spp_proc.PrimaryProc(conn) + + def accept_secondary(self): + while True: + conn, _ = self.sec_sock.accept() + LOG.debug("sec accepted: get process id") + proc = self._get_proc(conn) + if proc is None: + LOG.error("get process id failed") + conn.close() + continue + old_proc = self.procs.get(proc.id) + if old_proc: + LOG.warning("%s(%d) reconnect !", old_proc.type, old_proc.id) + if old_proc.type != proc.type: + LOG.warning("type changed ! new type: %s", proc.type) + with old_proc.sem: + try: + old_proc.conn.close() + except Exception: + pass + else: + LOG.info("%s(%d) connected.", proc.type, proc.id) + self.procs[proc.id] = proc + + @staticmethod + def _continue_recv(conn): + try: + # must set non-blocking to recieve remining data not to happen + # blocking here. + # NOTE: usually MSG_DONTWAIT flag is used for this purpose but + # this flag is not supported under eventlet. + conn.setblocking(False) + data = b"" + while True: + try: + rcv_data = conn.recv(MSG_SIZE) + data += rcv_data + if len(rcv_data) < MSG_SIZE: + break + except socket.error as e: + if e.args[0] == errno.EAGAIN: + # OK, no data remining. this happens when recieve data + # length is just multiple of MSG_SIZE. + break + raise e + return data + finally: + conn.setblocking(True) + + @staticmethod + def _send_command(conn, command): + conn.sendall(command.encode()) + data = conn.recv(MSG_SIZE) + if data and len(data) == MSG_SIZE: + # could not receive data at once. recieve remining data. + data += self._continue_recv(conn) + if data: + data = data.decode() + return data + + def _get_proc(self, conn): + # it is a bit ad hoc. send "_get_clinet_id" command and try to + # decode reply for each proc type. if success, that is the type. + data = self._send_command(conn, "_get_client_id") + for proc in [spp_proc.VfProc, spp_proc.NfvProc]: + sec_id = proc._decode_client_id(data) + if sec_id is not None: + return proc(sec_id, conn) + + def get_processes(self): + procs = [] + for proc in self.procs.values(): + p = {"type": proc.type} + if proc.id != spp_proc.ID_PRIMARY: + p["client-id"] = proc.id + procs.append(p) + return procs + + +def main(): + parser = argparse.ArgumentParser(description="SPP Controller") + parser.add_argument("-p", dest='pri_port', type=int, default=5555, + action='store', help="primary port") + parser.add_argument("-s", dest='sec_port', type=int, default=6666, + action='store', help="secondary port") + parser.add_argument("-a", dest='api_port', type=int, default=7777, + action='store', help="web api port") + args = parser.parse_args() + + logging.basicConfig(level=logging.DEBUG) + + controller = Controller(args.pri_port, args.sec_port, args.api_port) + controller.start() diff --git a/src/spp-ctl/spp_proc.py b/src/spp-ctl/spp_proc.py new file mode 100644 index 0000000..929942a --- /dev/null +++ b/src/spp-ctl/spp_proc.py @@ -0,0 +1,184 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Nippon Telegraph and Telephone Corporation + +import bottle +import eventlet +import json +import logging + +import spp_ctl + + +LOG = logging.getLogger(__name__) + +ID_PRIMARY = 0 +TYPE_PRIMARY = "primary" +TYPE_VF = "vf" +TYPE_NFV = "nfv" + + +def exec_command(func): + """Define the common function of sending command & receiving reply + as a decorator. + each method for executing command has only to return command string. + ex. + @exec_command + def some_command(self, ...): + return "command string of some_command" + """ + def wrapper(self, *args, **kwargs): + with self.sem: + command = func(self, *args, **kwargs) + LOG.info("%s(%d) command executed: %s", self.type, self.id, + command) + data = spp_ctl.Controller._send_command(self.conn, command) + if data is None: + raise RuntimeError("%s(%d): %s: no-data returned" % + (self.type, self.id, command)) + LOG.debug("reply: %s", data) + return self._decode_reply(data) + return wrapper + + +class SppProc(object): + def __init__(self, proc_type, id, conn): + self.id = id + self.type = proc_type + # NOTE: executing command is serialized by using a semaphore + # for each process. + self.sem = eventlet.semaphore.Semaphore(value=1) + self.conn = conn + + +class VfProc(SppProc): + + def __init__(self, id, conn): + super(VfProc, self).__init__(TYPE_VF, id, conn) + + @staticmethod + def _decode_reply(data): + data = json.loads(data) + result = data["results"][0] + if result["result"] == "error": + msg = result["error_details"]["message"] + raise bottle.HTTPError(400, "command error: %s" % msg) + return data + + @staticmethod + def _decode_client_id(data): + try: + data = VfProc._decode_reply(data) + return data["client_id"] + except: + return None + + @exec_command + def get_status(self): + return "status" + + @exec_command + def start_component(self, comp_name, core_id, comp_type): + return ("component start {comp_name} {core_id} {comp_type}" + .format(**locals())) + + @exec_command + def stop_component(self, comp_name): + return "component stop {comp_name}".format(**locals()) + + @exec_command + def port_add(self, port, direction, comp_name, op, vlan_id, pcp): + command = "port add {port} {direction} {comp_name}".format(**locals()) + if op != "none": + command += " %s" % op + if op == "add_vlantag": + command += " %d %d" % (vlan_id, pcp) + return command + + @exec_command + def port_del(self, port, direction, comp_name): + return "port del {port} {direction} {comp_name}".format(**locals()) + + @exec_command + def set_classifier_table(self, mac_address, port): + return ("classifier_table add mac {mac_address} {port}" + .format(**locals())) + + @exec_command + def clear_classifier_table(self, mac_address, port): + return ("classifier_table del mac {mac_address} {port}" + .format(**locals())) + + @exec_command + def set_classifier_table_with_vlan(self, mac_address, port, + vlan_id): + return ("classifier_table add vlan {vlan_id} {mac_address} {port}" + .format(**locals())) + + @exec_command + def clear_classifier_table_with_vlan(self, mac_address, port, + vlan_id): + return ("classifier_table del vlan {vlan_id} {mac_address} {port}" + .format(**locals())) + + +class NfvProc(SppProc): + + def __init__(self, id, conn): + super(NfvProc, self).__init__(TYPE_NFV, id, conn) + + @staticmethod + def _decode_reply(data): + return data.strip('\0') + + @staticmethod + def _decode_client_id(data): + try: + return int(NfvProc._decode_reply(data)) + except: + return None + + @exec_command + def get_status(self): + return "status" + + @exec_command + def port_add(self, if_type, if_num): + return "add {if_type} {if_num}".format(**locals()) + + @exec_command + def port_del(self, if_type, if_num): + return "del {if_type} {if_num}".format(**locals()) + + @exec_command + def patch_add(self, src_port, dst_port): + return "patch {src_port} {dst_port}".format(**locals()) + + @exec_command + def patch_reset(self): + return "patch reset" + + @exec_command + def forward(self): + return "forward" + + @exec_command + def stop(self): + return "stop" + + +class PrimaryProc(SppProc): + + def __init__(self, conn): + super(PrimaryProc, self).__init__(TYPE_PRIMARY, ID_PRIMARY, conn) + + @staticmethod + def _decode_reply(data): + return data.strip('\0') + + @exec_command + def status(self): + return "status" + + @exec_command + def clear(self): + return "clear" diff --git a/src/spp-ctl/spp_webapi.py b/src/spp-ctl/spp_webapi.py new file mode 100644 index 0000000..3ee7893 --- /dev/null +++ b/src/spp-ctl/spp_webapi.py @@ -0,0 +1,440 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2018 Nippon Telegraph and Telephone Corporation + +import bottle +import errno +import json +import logging +import netaddr +import re +import socket +import subprocess +import sys + +import spp_proc + + +LOG = logging.getLogger(__name__) + + +class KeyRequired(bottle.HTTPError): + + def __init__(self, key): + msg = "key(%s) required." % key + super(KeyRequired, self).__init__(400, msg) + + +class KeyInvalid(bottle.HTTPError): + + def __init__(self, key, value): + msg = "invalid key(%s): %s." % (key, value) + super(KeyRequired, self).__init__(400, msg) + + +class BaseHandler(bottle.Bottle): + """Define common methods for each handler.""" + + def __init__(self, controller): + super(BaseHandler, self).__init__() + self.ctrl = controller + + self.default_error_handler = self._error_handler + bottle.response.default_status = 404 + + def _error_handler(self, res): + # use "text/plain" as content_type rather than bottle's default + # "html". + res.content_type = "text/plain" + return res.body + + def _validate_port(self, port): + try: + if_type, if_num = port.split(":") + if if_type not in ["phy", "vhost", "ring"]: + raise + int(if_num) + except: + raise KeyInvalid('port', port) + + def log_url(self): + LOG.info("%s %s called", bottle.request.method, bottle.request.path) + + def log_response(self): + LOG.info("response: %s", bottle.response.status) + + # following three decorators do common works for each API. + # each handler 'install' appropriate decorators. + # + def get_body(self, func): + """Get body and set it to method argument. + content-type is OK whether application/json or plain text. + """ + def wrapper(*args, **kwargs): + req = bottle.request + if req.method in ["POST", "PUT"]: + if req.get_header('Content-Type') == "application/json": + body = req.json + else: + body = json.loads(req.body.read().decode()) + kwargs['body'] = body + LOG.info("body: %s", body) + return func(*args, **kwargs) + return wrapper + + def check_sec_id(self, func): + """Get and check proc and set it to method argument.""" + def wrapper(*args, **kwargs): + sec_id = kwargs.pop('sec_id', None) + if sec_id is not None: + proc = self.ctrl.procs.get(sec_id) + if proc is None or proc.type != self.type: + raise bottle.HTTPError(404, + "sec_id %d not found." % sec_id) + kwargs['proc'] = proc + return func(*args, **kwargs) + return wrapper + + def make_response(self, func): + """Convert plain response to bottle.HTTPResponse.""" + def wrapper(*args, **kwargs): + ret = func(*args, **kwargs) + if ret is None: + return bottle.HTTPResponse(status=204) + else: + r = bottle.HTTPResponse(status=200, body=json.dumps(ret)) + r.content_type = "application/json" + return r + return wrapper + + +class WebServer(BaseHandler): + """Top level handler. + handlers are hierarchized using 'mount' as follows: + / WebServer + /v1 V1Handler + /vfs V1VFHandler + /nfvs V1NFVHandler + /primary V1PrimaryHandler + """ + + def __init__(self, controller, api_port): + super(WebServer, self).__init__(controller) + self.api_port = api_port + + self.mount("/v1", V1Handler(controller)) + + # request and response logging. + self.add_hook("before_request", self.log_url) + self.add_hook("after_request", self.log_response) + + def start(self): + self.run(server='eventlet', host='localhost', port=self.api_port, + quiet=True) + + +class V1Handler(BaseHandler): + def __init__(self, controller): + super(V1Handler, self).__init__(controller) + + self.set_route() + + self.mount("/vfs", V1VFHandler(controller)) + self.mount("/nfvs", V1NFVHandler(controller)) + self.mount("/primary", V1PrimaryHandler(controller)) + + self.install(self.make_response) + + def set_route(self): + self.route('/processes', 'GET', callback=self.get_processes) + + def get_processes(self): + LOG.info("get processes called.") + return self.ctrl.get_processes() + + +class V1VFHandler(BaseHandler): + + def __init__(self, controller): + super(V1VFHandler, self).__init__(controller) + self.type = spp_proc.TYPE_VF + + self.set_route() + + self.install(self.check_sec_id) + self.install(self.get_body) + self.install(self.make_response) + + def set_route(self): + self.route('/', 'GET', callback=self.vf_get) + self.route('//components', 'POST', + callback=self.vf_comp_start) + self.route('//components/', 'DELETE', + callback=self.vf_comp_stop) + self.route('//components/', 'PUT', + callback=self.vf_comp_port) + self.route('//classifier_table', 'PUT', + callback=self.vf_classifier) + + def convert_vf_info(self, data): + info = data["info"] + vf = {} + vf["client-id"] = info["client-id"] + vf["ports"] = [] + for key in ["phy", "vhost", "ring"]: + for idx in info[key]: + vf["ports"].append(key + ":" + str(idx)) + vf["components"] = info["core"] + vf["classifier_table"] = info["classifier_table"] + + return vf + + def vf_get(self, proc): + return self.convert_vf_info(proc.get_status()) + + def _validate_vf_comp_start(self, body): + for key in ['name', 'core', 'type']: + if key not in body: + raise KeyRequired(key) + if not isinstance(body['name'], str): + raise KeyInvalid('name', body['name']) + if not isinstance(body['core'], int): + raise KeyInvalid('core', body['core']) + if body['type'] not in ["forward", "merge", "classifier_mac"]: + raise KeyInvalid('type', body['type']) + + def vf_comp_start(self, proc, body): + self._validate_vf_comp_start(body) + proc.start_component(body['name'], body['core'], body['type']) + + def vf_comp_stop(self, proc, name): + proc.stop_component(name) + + def _validate_vf_comp_port(self, body): + for key in ['action', 'port', 'dir']: + if key not in body: + raise KeyRequired(key) + if body['action'] not in ["attach", "detach"]: + raise KeyInvalid('action', body['action']) + if body['dir'] not in ["rx", "tx"]: + raise KeyInvalid('dir', body['dir']) + self._validate_port(body['port']) + + if body['action'] == "attach": + vlan = body.get('vlan') + if vlan: + try: + if vlan['operation'] not in ["none", "add", "del"]: + raise + if vlan['operation'] == "add": + int(vlan['id']) + int(vlan['pcp']) + except: + raise KeyInvalid('vlan', vlan) + + def vf_comp_port(self, proc, name, body): + self._validate_vf_comp_port(body) + + if body['action'] == "attach": + op = "none" + vlan_id = 0 + pcp = 0 + vlan = body.get('vlan') + if vlan: + if vlan['operation'] == "add": + op = "add_vlantag" + vlan_id = vlan['id'] + pcp = vlan['pcp'] + elif vlan['operation'] == "del": + op = "del_vlantag" + proc.port_add(body['port'], body['dir'], + name, op, vlan_id, pcp) + else: + proc.port_del(body['port'], body['dir'], name) + + def _validate_mac(self, mac_address): + try: + netaddr.EUI(mac_address) + except: + raise KeyInvalid('mac_address', mac_address) + + def _validate_vf_classifier(self, body): + for key in ['action', 'type', 'port', 'mac_address']: + if key not in body: + raise KeyRequired(key) + if body['action'] not in ["add", "del"]: + raise KeyInvalid('action', body['action']) + if body['type'] not in ["mac", "vlan"]: + raise KeyInvalid('type', body['type']) + self._validate_port(body['port']) + self._validate_mac(body['mac_address']) + + if body['type'] == "vlan": + try: + int(body['vlan']) + except: + raise KeyInvalid('vlan', body.get('vlan')) + + def vf_classifier(self, proc, body): + self._validate_vf_classifier(body) + + port = body['port'] + mac_address = body['mac_address'] + + if body['action'] == "add": + if body['type'] == "mac": + proc.set_classifier_table(mac_address, port) + else: + proc.set_classifier_table_with_vlan( + mac_address, port, body['vlan']) + else: + if body['type'] == "mac": + proc.clear_classifier_table(mac_address, port) + else: + proc.clear_classifier_table_with_vlan( + mac_address, port, body['vlan']) + + +class V1NFVHandler(BaseHandler): + + def __init__(self, controller): + super(V1NFVHandler, self).__init__(controller) + self.type = spp_proc.TYPE_NFV + + self.set_route() + + self.install(self.check_sec_id) + self.install(self.get_body) + self.install(self.make_response) + + def set_route(self): + self.route('/', 'GET', callback=self.nfv_get) + self.route('//forward', 'PUT', + callback=self.nfv_forward) + self.route('//ports', 'PUT', + callback=self.nfv_port) + self.route('//patches', 'PUT', + callback=self.nfv_patch_add) + self.route('//patches', 'DELETE', + callback=self.nfv_patch_del) + + def convert_nfv_info(self, data): + nfv = {} + lines = data.splitlines() + if len(lines) < 3: + return {} + p = re.compile("Client ID (\d+) (\w+)") + m = p.match(lines[0]) + if m: + nfv['client_id'] = int(m.group(1)) + nfv['status'] = m.group(2) + + ports = {} + outs = [] + for line in lines[2:]: + if not line.startswith("port_id"): + break + arg1, _, arg2, arg3 = line.split(",") + _, port_id = arg1.split(":") + if arg2 == "PHY": + port = "phy:" + port_id + else: + if_type, rest = arg2.split("(") + if_num = rest.rstrip(")") + if if_type == "RING": + port = "ring:" + if_num + elif if_type == "VHOST": + port = "vhost:" + if_num + else: + port = if_type + ":" + if_num + ports[port_id] = port + _, out_id = arg3.split(":") + if out_id != "none": + outs.append((port_id, out_id)) + nfv['ports'] = list(ports.values()) + patches = [] + if outs: + for src_id, dst_id in outs: + patches.append({"src": ports[src_id], "dst": ports[dst_id]}) + nfv['patches'] = patches + + return nfv + + def nfv_get(self, proc): + return self.convert_nfv_info(proc.get_status()) + + def _validate_nfv_forward(self, body): + if 'action' not in body: + raise KeyRequired('action') + if body['action'] not in ["start", "stop"]: + raise KeyInvalid('action', body['action']) + + def nfv_forward(self, proc, body): + if body['action'] == "start": + proc.forward() + else: + proc.stop() + + def _validate_nfv_port(self, body): + for key in ['action', 'port']: + if key not in body: + raise KeyRequired(key) + if body['action'] not in ["add", "del"]: + raise KeyInvalid('action', body['action']) + self._validate_port(body['port']) + + def nfv_port(self, proc, body): + self._validate_nfv_port(body) + + if_type, if_num = body['port'].split(":") + if body['action'] == "add": + proc.port_add(if_type, if_num) + else: + proc.port_del(if_type, if_num) + + def _validate_nfv_patch(self, body): + for key in ['src', 'dst']: + if key not in body: + raise KeyRequired(key) + self._validate_port(body['src']) + self._validate_port(body['dst']) + + def nfv_patch_add(self, proc, body): + self._validate_nfv_patch(body) + proc.patch_add(body['src'], body['dst']) + + def nfv_patch_del(self, proc): + proc.patch_reset() + + +class V1PrimaryHandler(BaseHandler): + + def __init__(self, controller): + super(V1PrimaryHandler, self).__init__(controller) + + self.set_route() + + self.install(self.make_response) + + def set_route(self): + self.route('/status', 'GET', callback=self.get_status) + self.route('/status', 'DELETE', callback=self.clear_status) + + def _get_proc(self): + proc = self.ctrl.procs.get(spp_proc.ID_PRIMARY) + if proc is None: + raise bottle.HTTPError(404, "primary not found.") + return proc + + def convert_status(self, data): + # no data returned at the moment. + # some data will be returned when the primary becomes to + # return statistical information. + return {} + + def get_status(self): + proc = self._get_proc() + return self.convert_status(proc.status()) + + def clear_status(self): + proc = self._get_proc() + proc.clear() -- 2.17.0 -- Itsuro ODA