* [spp] [PATCH 01/13] spp: move controller to sub directory
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 02/13] controller: move connection threads ogawa.yasufumi
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
SPP controller 'spp.py' is monolithic and has got to be large. It is
hard to be maintained because several classes are included in the file
and global variables are shared among its instances.
To improve maintainability, 'spp.py' is divided into main and class
files. Global variables are also moved to shared file 'spp_common.py'.
This update is halfway for refactoring and classes are going to be
more fine-grained.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
.gitignore | 1 +
src/controller/__init__.py | 0
src/controller/shell.py | 578 +++++++++++++++++++++++++
src/controller/spp.py | 363 ++++++++++++++++
src/controller/spp_common.py | 42 ++
src/spp.py | 984 +------------------------------------------
6 files changed, 988 insertions(+), 980 deletions(-)
create mode 100644 src/controller/__init__.py
create mode 100644 src/controller/shell.py
create mode 100644 src/controller/spp.py
create mode 100644 src/controller/spp_common.py
diff --git a/.gitignore b/.gitignore
index 08ac066..fbf8b76 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
.*.swp
+*.pyc
docs/guides/_build/*
diff --git a/src/controller/__init__.py b/src/controller/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/controller/shell.py b/src/controller/shell.py
new file mode 100644
index 0000000..145cbc3
--- /dev/null
+++ b/src/controller/shell.py
@@ -0,0 +1,578 @@
+import cmd
+import json
+import os
+from Queue import Empty
+import re
+import spp_common
+import subprocess
+
+# Turn true if activate logger to debug remote command.
+logger = None
+
+if logger is True:
+ import logging
+ logger = logging.getLogger(__name__)
+ handler = logging.StreamHandler()
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
+ handler.setFormatter(formatter)
+ logger.setLevel(logging.DEBUG)
+ logger.addHandler(handler)
+
+
+class Shell(cmd.Cmd, object):
+ """SPP command prompt"""
+
+ intro = 'Welcome to the spp. Type help or ? to list commands.\n'
+ prompt = 'spp > '
+ recorded_file = None
+
+ CMD_OK = "OK"
+ CMD_NG = "NG"
+ CMD_NOTREADY = "NOTREADY"
+ CMD_ERROR = "ERROR"
+
+ PORT_TYPES = ['phy', 'ring', 'vhost']
+
+ PRI_CMDS = ['status', 'exit', 'clear']
+ SEC_CMDS = ['status', 'exit', 'forward', 'stop', 'add', 'patch', 'del']
+ SEC_SUBCMDS = ['vhost', 'ring', 'pcap', 'nullpmd']
+ BYE_CMDS = ['sec', 'all']
+
+ def decorate_dir(self, curdir, filelist):
+ """Add '/' the end of dirname for path completion
+
+ 'filelist' is a list of files contained in a directory.
+ """
+
+ res = []
+ for f in filelist:
+ if os.path.isdir('%s/%s' % (curdir, f)):
+ res.append('%s/' % f)
+ else:
+ res.append(f)
+ return res
+
+ def compl_common(self, text, line, ftype=None):
+ """File path completion for 'complete_*' method
+
+ This method is called from 'complete_*' to complete 'do_*'.
+ 'text' and 'line' are arguments of 'complete_*'.
+
+ `complete_*` is a member method of builtin Cmd class and
+ called if tab key is pressed in a command defiend by 'do_*'.
+ 'text' and 'line' are contents of command line.
+ For example, if you type tab at 'command arg1 ar',
+ last token 'ar' is assigned to 'text' and whole line
+ 'command arg1 ar' is assigned to 'line'.
+
+ NOTE:
+ If tab is typed after '/', empty text '' is assigned to
+ 'text'. For example 'aaa b/', text is not 'b/' but ''.
+ """
+
+ if text == '': # tab is typed after command name or '/'
+ tokens = line.split(' ')
+ target_dir = tokens[-1] # get dirname for competion
+ if target_dir == '': # no dirname means current dir
+ res = self.decorate_dir(
+ '.', os.listdir(os.getcwd()))
+ else: # after '/'
+ res = self.decorate_dir(
+ target_dir, os.listdir(target_dir))
+ else: # tab is typed in the middle of a word
+ tokens = line.split(' ')
+ target = tokens[-1] # target dir for completion
+
+ if '/' in target: # word is a path such as 'path/to/file'
+ seg = target.split('/')[-1] # word to be completed
+ target_dir = '/'.join(target.split('/')[0:-1])
+ else:
+ seg = text
+ target_dir = os.getcwd()
+
+ matched = []
+ for t in os.listdir(target_dir):
+ if t.find(seg) == 0: # get words matched with 'seg'
+ matched.append(t)
+ res = self.decorate_dir(target_dir, matched)
+
+ if ftype is not None: # filtering by ftype
+ completions = []
+ if ftype == 'directory':
+ for fn in res:
+ if fn[-1] == '/':
+ completions.append(fn)
+ elif ftype == 'file':
+ for fn in res:
+ if fn[-1] != '/':
+ completions.append(fn)
+ else:
+ completions = res
+ else:
+ completions = res
+ return completions
+
+ def is_comment_line(self, line):
+ """Find commend line to not to interpret as a command
+
+ Return True if given line is a comment, or False.
+ Supported comment styles are
+ * python ('#')
+ * C ('//')
+ """
+
+ input_line = line.strip()
+ if len(input_line) > 0:
+ if (input_line[0] == '#') or (input_line[0:2] == '//'):
+ return True
+ else:
+ return False
+
+ def default(self, line):
+ """Define defualt behaviour
+
+ If user input is commend styled, controller simply echo
+ as a comment.
+ """
+
+ if self.is_comment_line(line):
+ print("%s" % line.strip())
+ else:
+ super(Shell, self).default(line)
+
+ def emptyline(self):
+ """Do nothin for empty input
+
+ It override Cmd.emptyline() which runs previous input as default
+ to do nothing.
+ """
+ pass
+
+ def close_all_secondary(self):
+ """Terminate all secondary processes"""
+
+ tmp_list = []
+ for i in spp_common.SECONDARY_LIST:
+ tmp_list.append(i)
+ for i in tmp_list:
+ self.command_secondary(i, 'exit')
+ spp_common.SECONDARY_COUNT = 0
+
+ def get_status(self):
+ """Return status of primary and secondary processes
+
+ It is called from do_status() method and return primary status
+ and a list of secondary processes as status.
+ """
+
+ secondary = []
+ for i in spp_common.SECONDARY_LIST:
+ secondary.append("%d" % i)
+ stat = {
+ # PRIMARY is 1 if it is running
+ "primary": "%d" % spp_common.PRIMARY,
+ "secondary": secondary
+ }
+ return stat
+
+ def print_status(self):
+ """Display information about connected clients"""
+
+ print ("Soft Patch Panel Status :")
+ print ("primary: %d" % spp_common.PRIMARY) # it is 1 if PRIMA == True
+ print ("secondary count: %d" % len(spp_common.SECONDARY_LIST))
+ for i in spp_common.SECONDARY_LIST:
+ print ("Connected secondary id: %d" % i)
+
+ def command_primary(self, command):
+ """Send command to primary process"""
+
+ if spp_common.PRIMARY:
+ spp_common.MAIN2PRIMARY.put(command)
+ recv = spp_common.PRIMARY2MAIN.get(True)
+ print (recv)
+ return self.CMD_OK, recv
+ else:
+ recv = "primary not started"
+ print (recv)
+ return self.CMD_NOTREADY, recv
+
+ def command_secondary(self, sec_id, command):
+ """Send command to secondary process with sec_id"""
+
+ if sec_id in spp_common.SECONDARY_LIST:
+ spp_common.MAIN2SEC[sec_id].put(command)
+ recv = spp_common.SEC2MAIN[sec_id].get(True)
+ print (recv)
+ return self.CMD_OK, recv
+ else:
+ message = "secondary id %d not exist" % sec_id
+ print(message)
+ return self.CMD_NOTREADY, message
+
+ def is_patched_ids_valid(self, id1, id2, delim=':'):
+ """Check if port IDs are valid
+
+ Supported format is port ID of integer or resource ID such as
+ 'phy:0' or 'ring:1'. Default delimiter ':' can be overwritten
+ by giving 'delim' option.
+ """
+
+ if str.isdigit(id1) and str.isdigit(id2):
+ return True
+ else:
+ ptn = r"\w+\%s\d+" % delim # Match "phy:0" or "ring:1" or so
+ if re.match(ptn, id1) and re.match(ptn, id2):
+ pt1 = id1.split(delim)[0]
+ pt2 = id2.split(delim)[0]
+ if (pt1 in self.PORT_TYPES) and (pt2 in self.PORT_TYPES):
+ return True
+ return False
+
+ def check_sec_cmds(self, cmds):
+ """Validate secondary commands before sending"""
+
+ level1 = ['status', 'exit', 'forward', 'stop']
+ level2 = ['add', 'patch', 'del']
+ patch_args = ['reset']
+ add_del_args = ['ring', 'vhost', 'pcap', 'nullpmd']
+ cmdlist = cmds.split(' ')
+ valid = 0
+
+ length = len(cmdlist)
+ if length == 1:
+ if cmdlist[0] in level1:
+ valid = 1
+ elif length == 2:
+ if cmdlist[0] == 'patch':
+ if cmdlist[1] in patch_args:
+ valid = 1
+ elif length == 3:
+ if cmdlist[0] in level2:
+ if cmdlist[0] == 'add' or cmdlist[0] == 'del':
+ if cmdlist[1] in add_del_args:
+ if str.isdigit(cmdlist[2]):
+ valid = 1
+ elif cmdlist[0] == 'patch':
+ if self.is_patched_ids_valid(cmdlist[1], cmdlist[2]):
+ valid = 1
+
+ return valid
+
+ def complete_pri(self, text, line, begidx, endidx):
+ """Completion for primary process commands"""
+
+ if not text:
+ completions = self.PRI_CMDS[:]
+ else:
+ completions = [p
+ for p in self.PRI_CMDS
+ if p.startswith(text)
+ ]
+ return completions
+
+ def clean_sec_cmd(self, cmdstr):
+ """remove unwanted spaces to avoid invalid command error"""
+
+ tmparg = re.sub(r'\s+', " ", cmdstr)
+ res = re.sub(r'\s?;\s?', ";", tmparg)
+ return res
+
+ def complete_sec(self, text, line, begidx, endidx):
+ """Completion for secondary process commands"""
+
+ try:
+ cleaned_line = self.clean_sec_cmd(line)
+ if len(cleaned_line.split()) == 1:
+ completions = [str(i)+";" for i in spp_common.SECONDARY_LIST]
+ elif len(cleaned_line.split()) == 2:
+ if not (";" in cleaned_line):
+ tmplist = [str(i) for i in spp_common.SECONDARY_LIST]
+ completions = [p+";"
+ for p in tmplist
+ if p.startswith(text)
+ ]
+ elif cleaned_line[-1] == ";":
+ completions = self.SEC_CMDS[:]
+ else:
+ seccmd = cleaned_line.split(";")[1]
+ if cleaned_line[-1] != " ":
+ completions = [p
+ for p in self.SEC_CMDS
+ if p.startswith(seccmd)
+ ]
+ elif ("add" in seccmd) or ("del" in seccmd):
+ completions = self.SEC_SUBCMDS[:]
+ else:
+ completions = []
+ elif len(cleaned_line.split()) == 3:
+ subcmd = cleaned_line.split()[-1]
+ if ("add" == subcmd) or ("del" == subcmd):
+ completions = self.SEC_SUBCMDS[:]
+ else:
+ if cleaned_line[-1] == " ":
+ completions = []
+ else:
+ completions = [p
+ for p in self.SEC_SUBCMDS
+ if p.startswith(subcmd)
+ ]
+ else:
+ completions = []
+ return completions
+ except Exception as e:
+ print(len(cleaned_line.split()))
+ print(e)
+
+ def complete_bye(self, text, line, begidx, endidx):
+ """Completion for bye commands"""
+
+ if not text:
+ completions = self.BYE_CMDS[:]
+ else:
+ completions = [p
+ for p in self.BYE_CMDS
+ if p.startswith(text)
+ ]
+ return completions
+
+ def response(self, result, message):
+ """Enqueue message from other than CLI"""
+
+ try:
+ rcmd = spp_common.RCMD_EXECUTE_QUEUE.get(False)
+ except Empty:
+ return
+
+ if (rcmd == spp_common.REMOTE_COMMAND):
+ param = result + '\n' + message
+ spp_common.RCMD_RESULT_QUEUE.put(param)
+ else:
+ if logger is not None:
+ logger.debug("unknown remote command = %s" % rcmd)
+
+ def do_status(self, _):
+ """Display status info of SPP processes
+
+ spp > status
+ """
+
+ self.print_status()
+ stat = self.get_status()
+ self.response(self.CMD_OK, json.dumps(stat))
+
+ def do_pri(self, command):
+ """Send command to primary process
+
+ Spp primary takes sub commands.
+
+ spp > pri;status
+ spp > pri;clear
+ """
+
+ if command and command in self.PRI_CMDS:
+ result, message = self.command_primary(command)
+ self.response(result, message)
+ else:
+ message = "primary invalid command"
+ print(message)
+ self.response(self.CMD_ERROR, message)
+
+ def do_sec(self, arg):
+ """Send command to secondary process
+
+ SPP secondary process is specified with secondary ID and takes
+ sub commands.
+
+ spp > sec 1;status
+ spp > sec 1;add ring 0
+ spp > sec 1;patch 0 2
+ """
+
+ # remove unwanted spaces to avoid invalid command error
+ tmparg = self.clean_sec_cmd(arg)
+ cmds = tmparg.split(';')
+ if len(cmds) < 2:
+ message = "error"
+ print(message)
+ self.response(self.CMD_ERROR, message)
+ elif str.isdigit(cmds[0]):
+ sec_id = int(cmds[0])
+ if self.check_sec_cmds(cmds[1]):
+ result, message = self.command_secondary(sec_id, cmds[1])
+ self.response(result, message)
+ else:
+ message = "invalid cmd"
+ print(message)
+ self.response(self.CMD_ERROR, message)
+ else:
+ print (cmds[0])
+ print ("first %s" % cmds[1])
+ self.response(self.CMD_ERROR, "invalid format")
+
+ def complete_record(self, text, line, begidx, endidx):
+ return self.compl_common(text, line)
+
+ def do_record(self, fname):
+ """Save commands to a log file
+
+ Save command history to a log file for loading from playback
+ command later as a config file.
+ Config is a series of SPP command and you can also create it
+ from scratch without playback command.
+
+ spp > record path/to/file
+ """
+
+ if fname == '':
+ print("Record file is required!")
+ else:
+ self.recorded_file = open(fname, 'w')
+ self.response(self.CMD_OK, "record")
+
+ def complete_playback(self, text, line, begidx, endidx):
+ return self.compl_common(text, line)
+
+ def do_playback(self, fname):
+ """Load a config file to reproduce network configuration
+
+ Config is a series of SPP command and you can also create it
+ from scratch without playback command.
+
+ spp > playback path/to/config
+ """
+
+ if fname == '':
+ print("Record file is required!")
+ else:
+ self.close()
+ try:
+ with open(fname) as recorded_file:
+ lines = []
+ for line in recorded_file:
+ if not self.is_comment_line(line):
+ lines.append("# %s" % line)
+ lines.append(line)
+ self.cmdqueue.extend(lines)
+ self.response(self.CMD_OK, "playback")
+ except IOError:
+ message = "Error: File does not exist."
+ print(message)
+ self.response(self.CMD_NG, message)
+
+ def precmd(self, line):
+ """Called before running a command
+
+ It is called for checking a contents of command line.
+ """
+
+ if self.recorded_file:
+ if not (
+ ('playback' in line) or
+ ('bye' in line) or
+ ('exit' in line)):
+ self.recorded_file.write("%s\n" % line)
+ return line
+
+ def close(self):
+ """Close record file"""
+
+ if self.recorded_file:
+ print("closing file")
+ self.recorded_file.close()
+ self.recorded_file = None
+
+ def do_pwd(self, args):
+ """Show corrent directory
+
+ It behaves as UNIX's pwd command.
+
+ spp > pwd
+ """
+
+ print(os.getcwd())
+
+ def complete_ls(self, text, line, begidx, endidx):
+ return self.compl_common(text, line)
+
+ def do_ls(self, args):
+ """Show a list of specified directory
+
+ It behaves as UNIX's ls command.
+
+ spp > ls path/to/dir
+ """
+
+ if args == '' or os.path.isdir(args):
+ c = 'ls -F %s' % args
+ subprocess.call(c, shell=True)
+ else:
+ print("No such a directory.")
+
+ def complete_cd(self, text, line, begidx, endidx):
+ return self.compl_common(text, line, 'directory')
+
+ def do_cd(self, args):
+ """Change current directory
+
+ spp > cd path/to/dir
+ """
+
+ if os.path.isdir(args):
+ os.chdir(args)
+ print(os.getcwd())
+ else:
+ print("No such a directory.")
+
+ def complete_mkdir(self, text, line, begidx, endidx):
+ return self.compl_common(text, line)
+
+ def do_mkdir(self, args):
+ """Create a new directory
+
+ It behaves as 'mkdir -p'.
+
+ spp > mkdir path/to/dir
+ """
+
+ c = 'mkdir -p %s' % args
+ subprocess.call(c, shell=True)
+
+ def do_bye(self, arg):
+ """Terminate SPP processes and controller
+
+ It also terminates logging if you activate recording.
+
+ (1) Terminate secondary processes
+ spp > bye sec
+
+ (2) Terminate primary and secondary processes
+ spp > bye all
+
+ (3) Terminate SPP controller (not for primary and secondary)
+ spp > bye
+ """
+
+ cmds = arg.split(' ')
+ if cmds[0] == 'sec':
+ self.close_all_secondary()
+ elif cmds[0] == 'all':
+ self.close_all_secondary()
+ self.command_primary('exit')
+ elif cmds[0] == '':
+ print('Thank you for using Soft Patch Panel')
+ self.close()
+ return True
+
+ def do_exit(self, args):
+ """Terminate SPP controller
+
+ It is an alias for bye command and same as bye command.
+
+ spp > exit
+ """
+ self.close()
+ print('Thank you for using Soft Patch Panel')
+ return True
diff --git a/src/controller/spp.py b/src/controller/spp.py
new file mode 100644
index 0000000..0515193
--- /dev/null
+++ b/src/controller/spp.py
@@ -0,0 +1,363 @@
+#!/usr/bin/python
+"""Soft Patch Panel"""
+
+from __future__ import print_function
+
+import argparse
+from Queue import Queue
+import select
+from shell import Shell
+import socket
+import SocketServer
+import spp_common
+import sys
+import threading
+import traceback
+
+# Turn true if activate logger to debug remote command.
+logger = None
+
+if logger is True:
+ import logging
+ logger = logging.getLogger(__name__)
+ handler = logging.StreamHandler()
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
+ handler.setFormatter(formatter)
+ logger.setLevel(logging.DEBUG)
+ logger.addHandler(handler)
+
+
+class CmdRequestHandler(SocketServer.BaseRequestHandler):
+ """Request handler for getting message from remote entities"""
+
+ CMD = None # contains a instance of Shell class
+
+ def handle(self):
+ self.data = self.request.recv(1024).strip()
+ cur_thread = threading.currentThread()
+ print(cur_thread.getName())
+ print(self.client_address[0])
+ print(self.data)
+ if CmdRequestHandler.CMD is not None:
+ spp_common.RCMD_EXECUTE_QUEUE.put(spp_common.REMOTE_COMMAND)
+ CmdRequestHandler.CMD.onecmd(self.data)
+ ret = spp_common.RCMD_RESULT_QUEUE.get()
+ if (ret is not None):
+ if logger is not None:
+ logger.debug("ret:%s" % ret)
+ self.request.send(ret)
+ else:
+ if logger is not None:
+ logger.debug("ret is none")
+ self.request.send("")
+ else:
+ if logger is not None:
+ logger.debug("CMD is None")
+ self.request.send("")
+
+
+class ConnectionThread(threading.Thread):
+ """Manage connection between controller and secondary"""
+
+ def __init__(self, client_id, conn):
+ super(ConnectionThread, self).__init__()
+ self.daemon = True
+
+ self.client_id = client_id
+ self.conn = conn
+ self.stop_event = threading.Event()
+ self.conn_opened = False
+
+ def stop(self):
+ self.stop_event.set()
+
+ def run(self):
+ cmd_str = 'hello'
+
+ # infinite loop so that function do not terminate and thread do not
+ # end.
+ while True:
+ try:
+ _, _, _ = select.select(
+ [self.conn, ], [self.conn, ], [], 5)
+ except select.error:
+ break
+
+ # Sending message to connected secondary
+ try:
+ cmd_str = spp_common.MAIN2SEC[self.client_id].get(True)
+ self.conn.send(cmd_str) # send only takes string
+ except KeyError:
+ break
+ except Exception as excep:
+ print(excep, ",Error while sending msg in connectionthread()!")
+ break
+
+ # Receiving from secondary
+ try:
+ # 1024 stands for bytes of data to be received
+ data = self.conn.recv(1024)
+ if data:
+ spp_common.SEC2MAIN[self.client_id].put(
+ "recv:%s:{%s}" % (str(self.conn.fileno()), data))
+ else:
+ spp_common.SEC2MAIN[self.client_id].put(
+ "closing:" + str(self.conn))
+ break
+ except Exception as excep:
+ print(
+ excep, ",Error while receiving msg in connectionthread()!")
+ break
+
+ spp_common.SECONDARY_LIST.remove(self.client_id)
+ self.conn.close()
+
+
+class AcceptThread(threading.Thread):
+ """Manage connection"""
+
+ def __init__(self, host, port):
+ super(AcceptThread, self).__init__()
+ self.daemon = True
+
+ # Creating secondary socket object
+ self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+ # Binding secondary socket to a address. bind() takes tuple of host
+ # and port.
+ self.sock.bind((host, port))
+
+ # Listening secondary at the address
+ self.sock.listen(spp_common.MAX_SECONDARY)
+
+ self.stop_event = threading.Event()
+ self.sock_opened = False
+
+ def getclientid(self, conn):
+ """Get client_id from client"""
+
+ try:
+ conn.send("_get_client_id")
+ except KeyError:
+ return -1
+
+ data = conn.recv(1024)
+ if data is None:
+ return -1
+
+ if logger is not None:
+ logger.debug("data: %s" % data)
+ client_id = int(data.strip('\0'))
+
+ if client_id < 0 or client_id > spp_common.MAX_SECONDARY:
+ logger.debug("Failed to get client_id: %d" % client_id)
+ return -1
+
+ found = 0
+ for i in spp_common.SECONDARY_LIST:
+ if client_id == i:
+ found = 1
+ break
+
+ if found == 0:
+ return client_id
+
+ # client_id in use, find a free one
+ free_client_id = -1
+ for i in range(spp_common.MAX_SECONDARY):
+ found = -1
+ for j in spp_common.SECONDARY_LIST:
+ if i == j:
+ found = i
+ break
+ if found == -1:
+ free_client_id = i
+ break
+
+ if logger is not None:
+ logger.debug("Found free_client_id: %d" % free_client_id)
+
+ if free_client_id < 0:
+ return -1
+
+ conn.send("_set_client_id %u" % free_client_id)
+ data = conn.recv(1024)
+
+ return free_client_id
+
+ def stop(self):
+ if self.sock_opened is True:
+ try:
+ self.sock.shutdown(socket.SHUT_RDWR)
+ except socket.error as excep:
+ print(excep, ", Error while closing sock in AcceptThread!")
+ traceback.print_exc()
+ self.sock.close()
+ self.stop_event.set()
+
+ def run(self):
+ try:
+ while True:
+ # Accepting incoming connections
+ conn, _ = self.sock.accept()
+
+ client_id = self.getclientid(conn)
+ if client_id < 0:
+ break
+
+ # Creating new thread.
+ # Calling secondarythread function for this function and
+ # passing conn as argument.
+ spp_common.SECONDARY_LIST.append(client_id)
+ spp_common.MAIN2SEC[client_id] = Queue()
+ spp_common.SEC2MAIN[client_id] = Queue()
+ connection_thread = ConnectionThread(client_id, conn)
+ connection_thread.daemon = True
+ connection_thread.start()
+
+ spp_common.SECONDARY_COUNT += 1
+ except Exception as excep:
+ print(excep, ", Error in AcceptThread!")
+ traceback.print_exc()
+ self.sock_opened = False
+ self.sock.close()
+
+
+class PrimaryThread(threading.Thread):
+
+ def __init__(self, host, port):
+ super(PrimaryThread, self).__init__()
+ self.daemon = True
+
+ self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ # Binding primary socket to a address. bind() takes tuple of host
+ # and port.
+ self.sock.bind((host, port))
+
+ # Listening primary at the address
+ self.sock.listen(1) # 5 denotes the number of clients can queue
+
+ self.stop_event = threading.Event()
+ self.sock_opened = False
+
+ def stop(self):
+ if self.sock_opened is True:
+ self.sock.shutdown(socket.SHUT_RDWR)
+ self.sock.close()
+ self.stop_event.set()
+
+ def run(self):
+ cmd_str = ''
+
+ while True:
+ # waiting for connection
+ spp_common.PRIMARY = False
+ conn, addr = self.sock.accept()
+ spp_common.PRIMARY = True
+
+ while conn:
+ try:
+ _, _, _ = select.select([conn, ], [conn, ], [], 5)
+ except select.error:
+ break
+
+ self.sock_opened = True
+ # Sending message to connected primary
+ try:
+ cmd_str = spp_common.MAIN2PRIMARY.get(True)
+ conn.send(cmd_str) # send only takes string
+ except KeyError:
+ break
+ except Exception as excep:
+ print(
+ excep,
+ ", Error while sending msg in primarythread()!")
+ break
+
+ # Receiving from primary
+ try:
+ # 1024 stands for bytes of data to be received
+ data = conn.recv(1024)
+ if data:
+ spp_common.PRIMARY2MAIN.put(
+ "recv:%s:{%s}" % (str(addr), data))
+ else:
+ spp_common.PRIMARY2MAIN.put("closing:" + str(addr))
+ conn.close()
+ self.sock_opened = False
+ break
+ except Exception as excep:
+ print(
+ excep,
+ ", Error while receiving msg in primarythread()!")
+ break
+
+
+def main(argv):
+ """main"""
+
+ parser = argparse.ArgumentParser(description="SPP Controller")
+
+ parser.add_argument(
+ "-p", "--pri-port",
+ type=int, default=5555,
+ help="primary port number")
+ parser.add_argument(
+ "-s", "--sec-port",
+ type=int, default=6666,
+ help="secondary port number")
+ parser.add_argument(
+ "-m", "--mng-port",
+ type=int, default=7777,
+ help="management port number")
+ parser.add_argument(
+ "-ip", "--ipaddr",
+ type=str, default='', # 'localhost' or '127.0.0.1' or '' are all same
+ help="IP address")
+ args = parser.parse_args()
+
+ host = args.ipaddr
+ primary_port = args.pri_port
+ secondary_port = args.sec_port
+ management_port = args.mng_port
+
+ print("primary port : %d" % primary_port)
+ print('secondary port : %d' % secondary_port)
+ print('management port : %d' % management_port)
+
+ primary_thread = PrimaryThread(host, primary_port)
+ primary_thread.start()
+
+ accept_thread = AcceptThread(host, secondary_port)
+ accept_thread.start()
+
+ shell = Shell()
+
+ # Run request handler as a TCP server thread
+ SocketServer.ThreadingTCPServer.allow_reuse_address = True
+ CmdRequestHandler.CMD = shell
+ command_server = SocketServer.ThreadingTCPServer(
+ (host, management_port), CmdRequestHandler)
+
+ t = threading.Thread(target=command_server.serve_forever)
+ t.setDaemon(True)
+ t.start()
+
+ shell.cmdloop()
+ shell = None
+
+ try:
+ primary_thread.stop()
+ accept_thread.stop()
+ except socket.error as excep:
+ print(excep, ", Error while terminating threads in main()!")
+ traceback.print_exc()
+
+
+if __name__ == "__main__":
+
+ main(sys.argv[1:])
diff --git a/src/controller/spp_common.py b/src/controller/spp_common.py
new file mode 100644
index 0000000..8c5af07
--- /dev/null
+++ b/src/controller/spp_common.py
@@ -0,0 +1,42 @@
+from Queue import Queue
+
+PRIMARY = ''
+SECONDARY_LIST = []
+
+# Initialize primary comm channel
+MAIN2PRIMARY = Queue()
+PRIMARY2MAIN = Queue()
+
+# Maximum num of sock queues for secondaries
+MAX_SECONDARY = 16
+
+PRIMARY = ''
+SECONDARY_COUNT = 0
+
+REMOTE_COMMAND = "RCMD"
+RCMD_EXECUTE_QUEUE = Queue()
+RCMD_RESULT_QUEUE = Queue()
+
+
+class GrowingList(list):
+ """Growing List
+
+ Custom list type for appending index over the range which is
+ similar to ruby's Array. Empty index is filled with 'None'.
+ It is used to contain queues for secondaries with any sec ID.
+
+ >>> gl = GrowingList()
+ >>> gl.[3] = 0
+ >>> gl
+ [None, None, None, 0]
+ """
+
+ def __setitem__(self, index, value):
+ if index >= len(self):
+ self.extend([None]*(index + 1 - len(self)))
+ list.__setitem__(self, index, value)
+
+
+# init secondary comm channel list
+MAIN2SEC = GrowingList()
+SEC2MAIN = GrowingList()
diff --git a/src/spp.py b/src/spp.py
index 73752bd..c494327 100755
--- a/src/spp.py
+++ b/src/spp.py
@@ -1,985 +1,9 @@
-#!/usr/bin/python
-"""Soft Patch Panel"""
+#!/usr/bin/env python
+# coding: utf-8
-from __future__ import print_function
-
-import argparse
-import cmd
-import json
-import os
-from Queue import Empty
-from Queue import Queue
-import re
-import select
-import socket
-import SocketServer
-import subprocess
+from controller import spp
import sys
-import threading
-import traceback
-
-# Turn true if activate logger to debug remote command.
-logger = None
-
-if logger is True:
- import logging
- logger = logging.getLogger(__name__)
- handler = logging.StreamHandler()
- handler.setLevel(logging.DEBUG)
- formatter = logging.Formatter(
- '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
- handler.setFormatter(formatter)
- logger.setLevel(logging.DEBUG)
- logger.addHandler(handler)
-
-# Maximum num of sock queues for secondaries
-MAX_SECONDARY = 16
-
-PRIMARY = ''
-SECONDARY_LIST = []
-SECONDARY_COUNT = 0
-
-# Initialize primary comm channel
-MAIN2PRIMARY = Queue()
-PRIMARY2MAIN = Queue()
-
-REMOTE_COMMAND = "RCMD"
-RCMD_EXECUTE_QUEUE = Queue()
-RCMD_RESULT_QUEUE = Queue()
-
-
-class GrowingList(list):
- """Growing List
-
- Custom list type for appending index over the range which is
- similar to ruby's Array. Empty index is filled with 'None'.
- It is used to contain queues for secondaries with any sec ID.
-
- >>> gl = GrowingList()
- >>> gl.[3] = 0
- >>> gl
- [None, None, None, 0]
- """
-
- def __setitem__(self, index, value):
- if index >= len(self):
- self.extend([None]*(index + 1 - len(self)))
- list.__setitem__(self, index, value)
-
-
-# init secondary comm channel list
-MAIN2SEC = GrowingList()
-SEC2MAIN = GrowingList()
-
-
-class CmdRequestHandler(SocketServer.BaseRequestHandler):
- """Request handler for getting message from remote entities"""
-
- CMD = None # contains a instance of Shell class
-
- def handle(self):
- self.data = self.request.recv(1024).strip()
- cur_thread = threading.currentThread()
- print(cur_thread.getName())
- print(self.client_address[0])
- print(self.data)
- if CmdRequestHandler.CMD is not None:
- RCMD_EXECUTE_QUEUE.put(REMOTE_COMMAND)
- CmdRequestHandler.CMD.onecmd(self.data)
- ret = RCMD_RESULT_QUEUE.get()
- if (ret is not None):
- if logger is not None:
- logger.debug("ret:%s" % ret)
- self.request.send(ret)
- else:
- if logger is not None:
- logger.debug("ret is none")
- self.request.send("")
- else:
- if logger is not None:
- logger.debug("CMD is None")
- self.request.send("")
-
-
-class ConnectionThread(threading.Thread):
- """Manage connection between controller and secondary"""
-
- def __init__(self, client_id, conn):
- super(ConnectionThread, self).__init__()
- self.daemon = True
-
- self.client_id = client_id
- self.conn = conn
- self.stop_event = threading.Event()
- self.conn_opened = False
-
- def stop(self):
- self.stop_event.set()
-
- def run(self):
- global SECONDARY_LIST
- global MAIN2SEC
- global SEC2MAIN
-
- cmd_str = 'hello'
-
- # infinite loop so that function do not terminate and thread do not
- # end.
- while True:
- try:
- _, _, _ = select.select(
- [self.conn, ], [self.conn, ], [], 5)
- except select.error:
- break
-
- # Sending message to connected secondary
- try:
- cmd_str = MAIN2SEC[self.client_id].get(True)
- self.conn.send(cmd_str) # send only takes string
- except KeyError:
- break
- except Exception as excep:
- print(excep, ",Error while sending msg in connectionthread()!")
- break
-
- # Receiving from secondary
- try:
- # 1024 stands for bytes of data to be received
- data = self.conn.recv(1024)
- if data:
- SEC2MAIN[self.client_id].put(
- "recv:%s:{%s}" % (str(self.conn.fileno()), data))
- else:
- SEC2MAIN[self.client_id].put("closing:" + str(self.conn))
- break
- except Exception as excep:
- print(
- excep, ",Error while receiving msg in connectionthread()!")
- break
-
- SECONDARY_LIST.remove(self.client_id)
- self.conn.close()
-
-
-class AcceptThread(threading.Thread):
- """Manage connection"""
-
- def __init__(self, host, port):
- super(AcceptThread, self).__init__()
- self.daemon = True
-
- # Creating secondary socket object
- self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
- self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
- # Binding secondary socket to a address. bind() takes tuple of host
- # and port.
- self.sock.bind((host, port))
-
- # Listening secondary at the address
- self.sock.listen(MAX_SECONDARY)
-
- self.stop_event = threading.Event()
- self.sock_opened = False
-
- def getclientid(self, conn):
- """Get client_id from client"""
-
- global SECONDARY_LIST
-
- try:
- conn.send("_get_client_id")
- except KeyError:
- return -1
-
- data = conn.recv(1024)
- if data is None:
- return -1
-
- if logger is not None:
- logger.debug("data: %s" % data)
- client_id = int(data.strip('\0'))
-
- if client_id < 0 or client_id > MAX_SECONDARY:
- logger.debug("Failed to get client_id: %d" % client_id)
- return -1
-
- found = 0
- for i in SECONDARY_LIST:
- if client_id == i:
- found = 1
- break
-
- if found == 0:
- return client_id
-
- # client_id in use, find a free one
- free_client_id = -1
- for i in range(MAX_SECONDARY):
- found = -1
- for j in SECONDARY_LIST:
- if i == j:
- found = i
- break
- if found == -1:
- free_client_id = i
- break
-
- if logger is not None:
- logger.debug("Found free_client_id: %d" % free_client_id)
-
- if free_client_id < 0:
- return -1
-
- conn.send("_set_client_id %u" % free_client_id)
- data = conn.recv(1024)
-
- return free_client_id
-
- def stop(self):
- if self.sock_opened is True:
- try:
- self.sock.shutdown(socket.SHUT_RDWR)
- except socket.error as excep:
- print(excep, ", Error while closing sock in AcceptThread!")
- traceback.print_exc()
- self.sock.close()
- self.stop_event.set()
-
- def run(self):
- global SECONDARY_COUNT
- global SECONDARY_LIST
- global MAIN2SEC
- global SEC2MAIN
-
- try:
- while True:
- # Accepting incoming connections
- conn, _ = self.sock.accept()
-
- client_id = self.getclientid(conn)
- if client_id < 0:
- break
-
- # Creating new thread.
- # Calling secondarythread function for this function and
- # passing conn as argument.
- SECONDARY_LIST.append(client_id)
- MAIN2SEC[client_id] = Queue()
- SEC2MAIN[client_id] = Queue()
- connection_thread = ConnectionThread(client_id, conn)
- connection_thread.daemon = True
- connection_thread.start()
-
- SECONDARY_COUNT += 1
- except Exception as excep:
- print(excep, ", Error in AcceptThread!")
- traceback.print_exc()
- self.sock_opened = False
- self.sock.close()
-
-
-class PrimaryThread(threading.Thread):
-
- def __init__(self, host, port):
- super(PrimaryThread, self).__init__()
- self.daemon = True
-
- self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
- self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
- # Binding primary socket to a address. bind() takes tuple of host
- # and port.
- self.sock.bind((host, port))
-
- # Listening primary at the address
- self.sock.listen(1) # 5 denotes the number of clients can queue
-
- self.stop_event = threading.Event()
- self.sock_opened = False
-
- def stop(self):
- if self.sock_opened is True:
- self.sock.shutdown(socket.SHUT_RDWR)
- self.sock.close()
- self.stop_event.set()
-
- def run(self):
- global PRIMARY
- cmd_str = ''
-
- while True:
- # waiting for connection
- PRIMARY = False
- conn, addr = self.sock.accept()
- PRIMARY = True
-
- while conn:
- try:
- _, _, _ = select.select([conn, ], [conn, ], [], 5)
- except select.error:
- break
-
- self.sock_opened = True
- # Sending message to connected primary
- try:
- cmd_str = MAIN2PRIMARY.get(True)
- conn.send(cmd_str) # send only takes string
- except KeyError:
- break
- except Exception as excep:
- print(
- excep,
- ", Error while sending msg in primarythread()!")
- break
-
- # Receiving from primary
- try:
- # 1024 stands for bytes of data to be received
- data = conn.recv(1024)
- if data:
- PRIMARY2MAIN.put(
- "recv:%s:{%s}" % (str(addr), data))
- else:
- PRIMARY2MAIN.put("closing:" + str(addr))
- conn.close()
- self.sock_opened = False
- break
- except Exception as excep:
- print(
- excep,
- ", Error while receiving msg in primarythread()!")
- break
-
-
-def clean_sec_cmd(cmdstr):
- """remove unwanted spaces to avoid invalid command error"""
-
- tmparg = re.sub(r'\s+', " ", cmdstr)
- res = re.sub(r'\s?;\s?', ";", tmparg)
- return res
-
-
-class Shell(cmd.Cmd, object):
- """SPP command prompt"""
-
- intro = 'Welcome to the spp. Type help or ? to list commands.\n'
- prompt = 'spp > '
- recorded_file = None
-
- CMD_OK = "OK"
- CMD_NG = "NG"
- CMD_NOTREADY = "NOTREADY"
- CMD_ERROR = "ERROR"
-
- PORT_TYPES = ['phy', 'ring', 'vhost']
-
- PRI_CMDS = ['status', 'exit', 'clear']
- SEC_CMDS = ['status', 'exit', 'forward', 'stop', 'add', 'patch', 'del']
- SEC_SUBCMDS = ['vhost', 'ring', 'pcap', 'nullpmd']
- BYE_CMDS = ['sec', 'all']
-
- def decorate_dir(self, curdir, filelist):
- """Add '/' the end of dirname for path completion
-
- 'filelist' is a list of files contained in a directory.
- """
-
- res = []
- for f in filelist:
- if os.path.isdir('%s/%s' % (curdir, f)):
- res.append('%s/' % f)
- else:
- res.append(f)
- return res
-
- def compl_common(self, text, line, ftype=None):
- """File path completion for 'complete_*' method
-
- This method is called from 'complete_*' to complete 'do_*'.
- 'text' and 'line' are arguments of 'complete_*'.
-
- `complete_*` is a member method of builtin Cmd class and
- called if tab key is pressed in a command defiend by 'do_*'.
- 'text' and 'line' are contents of command line.
- For example, if you type tab at 'command arg1 ar',
- last token 'ar' is assigned to 'text' and whole line
- 'command arg1 ar' is assigned to 'line'.
-
- NOTE:
- If tab is typed after '/', empty text '' is assigned to
- 'text'. For example 'aaa b/', text is not 'b/' but ''.
- """
-
- if text == '': # tab is typed after command name or '/'
- tokens = line.split(' ')
- target_dir = tokens[-1] # get dirname for competion
- if target_dir == '': # no dirname means current dir
- res = self.decorate_dir(
- '.', os.listdir(os.getcwd()))
- else: # after '/'
- res = self.decorate_dir(
- target_dir, os.listdir(target_dir))
- else: # tab is typed in the middle of a word
- tokens = line.split(' ')
- target = tokens[-1] # target dir for completion
-
- if '/' in target: # word is a path such as 'path/to/file'
- seg = target.split('/')[-1] # word to be completed
- target_dir = '/'.join(target.split('/')[0:-1])
- else:
- seg = text
- target_dir = os.getcwd()
-
- matched = []
- for t in os.listdir(target_dir):
- if t.find(seg) == 0: # get words matched with 'seg'
- matched.append(t)
- res = self.decorate_dir(target_dir, matched)
-
- if ftype is not None: # filtering by ftype
- completions = []
- if ftype == 'directory':
- for fn in res:
- if fn[-1] == '/':
- completions.append(fn)
- elif ftype == 'file':
- for fn in res:
- if fn[-1] != '/':
- completions.append(fn)
- else:
- completions = res
- else:
- completions = res
- return completions
-
- def is_comment_line(self, line):
- """Find commend line to not to interpret as a command
-
- Return True if given line is a comment, or False.
- Supported comment styles are
- * python ('#')
- * C ('//')
- """
-
- input_line = line.strip()
- if len(input_line) > 0:
- if (input_line[0] == '#') or (input_line[0:2] == '//'):
- return True
- else:
- return False
-
- def default(self, line):
- """Define defualt behaviour
-
- If user input is commend styled, controller simply echo
- as a comment.
- """
-
- if self.is_comment_line(line):
- print("%s" % line.strip())
- else:
- super(Shell, self).default(line)
-
- def emptyline(self):
- """Do nothin for empty input
-
- It override Cmd.emptyline() which runs previous input as default
- to do nothing.
- """
- pass
-
- def close_all_secondary(self):
- """Terminate all secondary processes"""
-
- global SECONDARY_COUNT
- global SECONDARY_LIST
-
- tmp_list = []
- for i in SECONDARY_LIST:
- tmp_list.append(i)
- for i in tmp_list:
- self.command_secondary(i, 'exit')
- SECONDARY_COUNT = 0
-
- def get_status(self):
- """Return status of primary and secondary processes
-
- It is called from do_status() method and return primary status
- and a list of secondary processes as status.
- """
-
- global SECONDARY_LIST
-
- secondary = []
- for i in SECONDARY_LIST:
- secondary.append("%d" % i)
- stat = {
- "primary": "%d" % PRIMARY, # PRIMARY is 1 if it is running
- "secondary": secondary
- }
- return stat
-
- def print_status(self):
- """Display information about connected clients"""
-
- global SECONDARY_LIST
-
- print ("Soft Patch Panel Status :")
- print ("primary: %d" % PRIMARY) # "primary: 1" if PRIMA == True
- print ("secondary count: %d" % len(SECONDARY_LIST))
- for i in SECONDARY_LIST:
- print ("Connected secondary id: %d" % i)
-
- def command_primary(self, command):
- """Send command to primary process"""
-
- if PRIMARY:
- MAIN2PRIMARY.put(command)
- recv = PRIMARY2MAIN.get(True)
- print (recv)
- return self.CMD_OK, recv
- else:
- recv = "primary not started"
- print (recv)
- return self.CMD_NOTREADY, recv
-
- def command_secondary(self, sec_id, command):
- """Send command to secondary process with sec_id"""
-
- global SECONDARY_LIST
-
- if sec_id in SECONDARY_LIST:
- MAIN2SEC[sec_id].put(command)
- recv = SEC2MAIN[sec_id].get(True)
- print (recv)
- return self.CMD_OK, recv
- else:
- message = "secondary id %d not exist" % sec_id
- print(message)
- return self.CMD_NOTREADY, message
-
- def is_patched_ids_valid(self, id1, id2, delim=':'):
- """Check if port IDs are valid
-
- Supported format is port ID of integer or resource ID such as
- 'phy:0' or 'ring:1'. Default delimiter ':' can be overwritten
- by giving 'delim' option.
- """
-
- if str.isdigit(id1) and str.isdigit(id2):
- return True
- else:
- ptn = r"\w+\%s\d+" % delim # Match "phy:0" or "ring:1" or so
- if re.match(ptn, id1) and re.match(ptn, id2):
- pt1 = id1.split(delim)[0]
- pt2 = id2.split(delim)[0]
- if (pt1 in self.PORT_TYPES) and (pt2 in self.PORT_TYPES):
- return True
- return False
-
- def check_sec_cmds(self, cmds):
- """Validate secondary commands before sending"""
-
- level1 = ['status', 'exit', 'forward', 'stop']
- level2 = ['add', 'patch', 'del']
- patch_args = ['reset']
- add_del_args = ['ring', 'vhost', 'pcap', 'nullpmd']
- cmdlist = cmds.split(' ')
- valid = 0
-
- length = len(cmdlist)
- if length == 1:
- if cmdlist[0] in level1:
- valid = 1
- elif length == 2:
- if cmdlist[0] == 'patch':
- if cmdlist[1] in patch_args:
- valid = 1
- elif length == 3:
- if cmdlist[0] in level2:
- if cmdlist[0] == 'add' or cmdlist[0] == 'del':
- if cmdlist[1] in add_del_args:
- if str.isdigit(cmdlist[2]):
- valid = 1
- elif cmdlist[0] == 'patch':
- if self.is_patched_ids_valid(cmdlist[1], cmdlist[2]):
- valid = 1
-
- return valid
-
- def complete_pri(self, text, line, begidx, endidx):
- """Completion for primary process commands"""
-
- if not text:
- completions = self.PRI_CMDS[:]
- else:
- completions = [p
- for p in self.PRI_CMDS
- if p.startswith(text)
- ]
- return completions
-
- def complete_sec(self, text, line, begidx, endidx):
- """Completion for secondary process commands"""
-
- global SECONDARY_LIST
-
- try:
- cleaned_line = clean_sec_cmd(line)
- if len(cleaned_line.split()) == 1:
- completions = [str(i)+";" for i in SECONDARY_LIST]
- elif len(cleaned_line.split()) == 2:
- if not (";" in cleaned_line):
- tmplist = [str(i) for i in SECONDARY_LIST]
- completions = [p+";"
- for p in tmplist
- if p.startswith(text)
- ]
- elif cleaned_line[-1] == ";":
- completions = self.SEC_CMDS[:]
- else:
- seccmd = cleaned_line.split(";")[1]
- if cleaned_line[-1] != " ":
- completions = [p
- for p in self.SEC_CMDS
- if p.startswith(seccmd)
- ]
- elif ("add" in seccmd) or ("del" in seccmd):
- completions = self.SEC_SUBCMDS[:]
- else:
- completions = []
- elif len(cleaned_line.split()) == 3:
- subcmd = cleaned_line.split()[-1]
- if ("add" == subcmd) or ("del" == subcmd):
- completions = self.SEC_SUBCMDS[:]
- else:
- if cleaned_line[-1] == " ":
- completions = []
- else:
- completions = [p
- for p in self.SEC_SUBCMDS
- if p.startswith(subcmd)
- ]
- else:
- completions = []
- return completions
- except Exception as e:
- print(len(cleaned_line.split()))
- print(e)
-
- def complete_bye(self, text, line, begidx, endidx):
- """Completion for bye commands"""
-
- if not text:
- completions = self.BYE_CMDS[:]
- else:
- completions = [p
- for p in self.BYE_CMDS
- if p.startswith(text)
- ]
- return completions
-
- def response(self, result, message):
- """Enqueue message from other than CLI"""
-
- try:
- rcmd = RCMD_EXECUTE_QUEUE.get(False)
- except Empty:
- return
-
- if (rcmd == REMOTE_COMMAND):
- param = result + '\n' + message
- RCMD_RESULT_QUEUE.put(param)
- else:
- if logger is not None:
- logger.debug("unknown remote command = %s" % rcmd)
-
- def do_status(self, _):
- """Display status info of SPP processes
-
- spp > status
- """
-
- self.print_status()
- stat = self.get_status()
- self.response(self.CMD_OK, json.dumps(stat))
-
- def do_pri(self, command):
- """Send command to primary process
-
- Spp primary takes sub commands.
-
- spp > pri;status
- spp > pri;clear
- """
-
- if command and command in self.PRI_CMDS:
- result, message = self.command_primary(command)
- self.response(result, message)
- else:
- message = "primary invalid command"
- print(message)
- self.response(self.CMD_ERROR, message)
-
- def do_sec(self, arg):
- """Send command to secondary process
-
- SPP secondary process is specified with secondary ID and takes
- sub commands.
-
- spp > sec 1;status
- spp > sec 1;add ring 0
- spp > sec 1;patch 0 2
- """
-
- # remove unwanted spaces to avoid invalid command error
- tmparg = clean_sec_cmd(arg)
- cmds = tmparg.split(';')
- if len(cmds) < 2:
- message = "error"
- print(message)
- self.response(self.CMD_ERROR, message)
- elif str.isdigit(cmds[0]):
- sec_id = int(cmds[0])
- if self.check_sec_cmds(cmds[1]):
- result, message = self.command_secondary(sec_id, cmds[1])
- self.response(result, message)
- else:
- message = "invalid cmd"
- print(message)
- self.response(self.CMD_ERROR, message)
- else:
- print (cmds[0])
- print ("first %s" % cmds[1])
- self.response(self.CMD_ERROR, "invalid format")
-
- def complete_record(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
-
- def do_record(self, fname):
- """Save commands to a log file
-
- Save command history to a log file for loading from playback
- command later as a config file.
- Config is a series of SPP command and you can also create it
- from scratch without playback command.
-
- spp > record path/to/file
- """
-
- if fname == '':
- print("Record file is required!")
- else:
- self.recorded_file = open(fname, 'w')
- self.response(self.CMD_OK, "record")
-
- def complete_playback(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
-
- def do_playback(self, fname):
- """Load a config file to reproduce network configuration
-
- Config is a series of SPP command and you can also create it
- from scratch without playback command.
-
- spp > playback path/to/config
- """
-
- if fname == '':
- print("Record file is required!")
- else:
- self.close()
- try:
- with open(fname) as recorded_file:
- lines = []
- for line in recorded_file:
- if not self.is_comment_line(line):
- lines.append("# %s" % line)
- lines.append(line)
- self.cmdqueue.extend(lines)
- self.response(self.CMD_OK, "playback")
- except IOError:
- message = "Error: File does not exist."
- print(message)
- self.response(self.CMD_NG, message)
-
- def precmd(self, line):
- """Called before running a command
-
- It is called for checking a contents of command line.
- """
-
- if self.recorded_file:
- if not (
- ('playback' in line) or
- ('bye' in line) or
- ('exit' in line)):
- print(line, file=self.recorded_file)
- return line
-
- def close(self):
- """Close record file"""
-
- if self.recorded_file:
- print("closing file")
- self.recorded_file.close()
- self.recorded_file = None
-
- def do_pwd(self, args):
- """Show corrent directory
-
- It behaves as UNIX's pwd command.
-
- spp > pwd
- """
-
- print(os.getcwd())
-
- def complete_ls(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
-
- def do_ls(self, args):
- """Show a list of specified directory
-
- It behaves as UNIX's ls command.
-
- spp > ls path/to/dir
- """
-
- if args == '' or os.path.isdir(args):
- c = 'ls -F %s' % args
- subprocess.call(c, shell=True)
- else:
- print("No such a directory.")
-
- def complete_cd(self, text, line, begidx, endidx):
- return self.compl_common(text, line, 'directory')
-
- def do_cd(self, args):
- """Change current directory
-
- spp > cd path/to/dir
- """
-
- if os.path.isdir(args):
- os.chdir(args)
- print(os.getcwd())
- else:
- print("No such a directory.")
-
- def complete_mkdir(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
-
- def do_mkdir(self, args):
- """Create a new directory
-
- It behaves as 'mkdir -p'.
-
- spp > mkdir path/to/dir
- """
-
- c = 'mkdir -p %s' % args
- subprocess.call(c, shell=True)
-
- def do_bye(self, arg):
- """Terminate SPP processes and controller
-
- It also terminates logging if you activate recording.
-
- (1) Terminate secondary processes
- spp > bye sec
-
- (2) Terminate primary and secondary processes
- spp > bye all
-
- (3) Terminate SPP controller (not for primary and secondary)
- spp > bye
- """
-
- cmds = arg.split(' ')
- if cmds[0] == 'sec':
- self.close_all_secondary()
- elif cmds[0] == 'all':
- self.close_all_secondary()
- self.command_primary('exit')
- elif cmds[0] == '':
- print('Thank you for using Soft Patch Panel')
- self.close()
- return True
-
- def do_exit(self, args):
- """Terminate SPP controller
-
- It is an alias for bye command and same as bye command.
-
- spp > exit
- """
- self.close()
- print('Thank you for using Soft Patch Panel')
- return True
-
-
-def main(argv):
- """main"""
-
- parser = argparse.ArgumentParser(description="SPP Controller")
-
- parser.add_argument(
- "-p", "--pri-port",
- type=int, default=5555,
- help="primary port number")
- parser.add_argument(
- "-s", "--sec-port",
- type=int, default=6666,
- help="secondary port number")
- parser.add_argument(
- "-m", "--mng-port",
- type=int, default=7777,
- help="management port number")
- parser.add_argument(
- "-ip", "--ipaddr",
- type=str, default='', # 'localhost' or '127.0.0.1' or '' are all same
- help="IP address")
- args = parser.parse_args()
-
- host = args.ipaddr
- primary_port = args.pri_port
- secondary_port = args.sec_port
- management_port = args.mng_port
-
- print("primary port : %d" % primary_port)
- print('secondary port : %d' % secondary_port)
- print('management port : %d' % management_port)
-
- primary_thread = PrimaryThread(host, primary_port)
- primary_thread.start()
-
- accept_thread = AcceptThread(host, secondary_port)
- accept_thread.start()
-
- shell = Shell()
-
- # Run request handler as a TCP server thread
- SocketServer.ThreadingTCPServer.allow_reuse_address = True
- CmdRequestHandler.CMD = shell
- command_server = SocketServer.ThreadingTCPServer(
- (host, management_port), CmdRequestHandler)
-
- t = threading.Thread(target=command_server.serve_forever)
- t.setDaemon(True)
- t.start()
-
- shell.cmdloop()
- shell = None
-
- try:
- primary_thread.stop()
- accept_thread.stop()
- except socket.error as excep:
- print(excep, ", Error while terminating threads in main()!")
- traceback.print_exc()
-
if __name__ == "__main__":
- main(sys.argv[1:])
+ spp.main(sys.argv[1:])
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 02/13] controller: move connection threads
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 01/13] spp: move controller to sub directory ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 03/13] controller: aggregate logger to spp_common.py ogawa.yasufumi
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
There are three classes in 'spp.py' for managing connection as threads.
This update is for separating them from main file to improve
maintainability.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/conn_thread.py | 259 ++++++++++++++++++++++++++++++++++++++++++
src/controller/spp.py | 243 +--------------------------------------
2 files changed, 261 insertions(+), 241 deletions(-)
create mode 100644 src/controller/conn_thread.py
diff --git a/src/controller/conn_thread.py b/src/controller/conn_thread.py
new file mode 100644
index 0000000..7ba3b00
--- /dev/null
+++ b/src/controller/conn_thread.py
@@ -0,0 +1,259 @@
+from Queue import Queue
+import select
+import socket
+import spp_common
+import threading
+import traceback
+
+# Turn true if activate logger to debug remote command.
+logger = None
+
+if logger is True:
+ import logging
+ logger = logging.getLogger(__name__)
+ handler = logging.StreamHandler()
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
+ handler.setFormatter(formatter)
+ logger.setLevel(logging.DEBUG)
+ logger.addHandler(handler)
+
+
+class ConnectionThread(threading.Thread):
+ """Manage connection between controller and secondary"""
+
+ def __init__(self, client_id, conn):
+ super(ConnectionThread, self).__init__()
+ self.daemon = True
+
+ self.client_id = client_id
+ self.conn = conn
+ self.stop_event = threading.Event()
+ self.conn_opened = False
+
+ def stop(self):
+ self.stop_event.set()
+
+ def run(self):
+ cmd_str = 'hello'
+
+ # infinite loop so that function do not terminate and thread do not
+ # end.
+ while True:
+ try:
+ _, _, _ = select.select(
+ [self.conn, ], [self.conn, ], [], 5)
+ except select.error:
+ break
+
+ # Sending message to connected secondary
+ try:
+ cmd_str = spp_common.MAIN2SEC[self.client_id].get(True)
+ self.conn.send(cmd_str) # send only takes string
+ except KeyError:
+ break
+ except Exception as excep:
+ print(excep, ",Error while sending msg in connectionthread()!")
+ break
+
+ # Receiving from secondary
+ try:
+ # 1024 stands for bytes of data to be received
+ data = self.conn.recv(1024)
+ if data:
+ spp_common.SEC2MAIN[self.client_id].put(
+ "recv:%s:{%s}" % (str(self.conn.fileno()), data))
+ else:
+ spp_common.SEC2MAIN[self.client_id].put(
+ "closing:" + str(self.conn))
+ break
+ except Exception as excep:
+ print(
+ excep, ",Error while receiving msg in connectionthread()!")
+ break
+
+ spp_common.SECONDARY_LIST.remove(self.client_id)
+ self.conn.close()
+
+
+class AcceptThread(threading.Thread):
+ """Manage connection"""
+
+ def __init__(self, host, port):
+ super(AcceptThread, self).__init__()
+ self.daemon = True
+
+ # Creating secondary socket object
+ self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+ # Binding secondary socket to a address. bind() takes tuple of host
+ # and port.
+ self.sock.bind((host, port))
+
+ # Listening secondary at the address
+ self.sock.listen(spp_common.MAX_SECONDARY)
+
+ self.stop_event = threading.Event()
+ self.sock_opened = False
+
+ def getclientid(self, conn):
+ """Get client_id from client"""
+
+ try:
+ conn.send("_get_client_id")
+ except KeyError:
+ return -1
+
+ data = conn.recv(1024)
+ if data is None:
+ return -1
+
+ if logger is not None:
+ logger.debug("data: %s" % data)
+ client_id = int(data.strip('\0'))
+
+ if client_id < 0 or client_id > spp_common.MAX_SECONDARY:
+ logger.debug("Failed to get client_id: %d" % client_id)
+ return -1
+
+ found = 0
+ for i in spp_common.SECONDARY_LIST:
+ if client_id == i:
+ found = 1
+ break
+
+ if found == 0:
+ return client_id
+
+ # client_id in use, find a free one
+ free_client_id = -1
+ for i in range(spp_common.MAX_SECONDARY):
+ found = -1
+ for j in spp_common.SECONDARY_LIST:
+ if i == j:
+ found = i
+ break
+ if found == -1:
+ free_client_id = i
+ break
+
+ if logger is not None:
+ logger.debug("Found free_client_id: %d" % free_client_id)
+
+ if free_client_id < 0:
+ return -1
+
+ conn.send("_set_client_id %u" % free_client_id)
+ data = conn.recv(1024)
+
+ return free_client_id
+
+ def stop(self):
+ if self.sock_opened is True:
+ try:
+ self.sock.shutdown(socket.SHUT_RDWR)
+ except socket.error as excep:
+ print(excep, ", Error while closing sock in AcceptThread!")
+ traceback.print_exc()
+ self.sock.close()
+ self.stop_event.set()
+
+ def run(self):
+ try:
+ while True:
+ # Accepting incoming connections
+ conn, _ = self.sock.accept()
+
+ client_id = self.getclientid(conn)
+ if client_id < 0:
+ break
+
+ # Creating new thread.
+ # Calling secondarythread function for this function and
+ # passing conn as argument.
+ spp_common.SECONDARY_LIST.append(client_id)
+ spp_common.MAIN2SEC[client_id] = Queue()
+ spp_common.SEC2MAIN[client_id] = Queue()
+ connection_thread = ConnectionThread(client_id, conn)
+ connection_thread.daemon = True
+ connection_thread.start()
+
+ spp_common.SECONDARY_COUNT += 1
+ except Exception as excep:
+ print(excep, ", Error in AcceptThread!")
+ traceback.print_exc()
+ self.sock_opened = False
+ self.sock.close()
+
+
+class PrimaryThread(threading.Thread):
+
+ def __init__(self, host, port):
+ super(PrimaryThread, self).__init__()
+ self.daemon = True
+
+ self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ # Binding primary socket to a address. bind() takes tuple of host
+ # and port.
+ self.sock.bind((host, port))
+
+ # Listening primary at the address
+ self.sock.listen(1) # 5 denotes the number of clients can queue
+
+ self.stop_event = threading.Event()
+ self.sock_opened = False
+
+ def stop(self):
+ if self.sock_opened is True:
+ self.sock.shutdown(socket.SHUT_RDWR)
+ self.sock.close()
+ self.stop_event.set()
+
+ def run(self):
+ cmd_str = ''
+
+ while True:
+ # waiting for connection
+ spp_common.PRIMARY = False
+ conn, addr = self.sock.accept()
+ spp_common.PRIMARY = True
+
+ while conn:
+ try:
+ _, _, _ = select.select([conn, ], [conn, ], [], 5)
+ except select.error:
+ break
+
+ self.sock_opened = True
+ # Sending message to connected primary
+ try:
+ cmd_str = spp_common.MAIN2PRIMARY.get(True)
+ conn.send(cmd_str) # send only takes string
+ except KeyError:
+ break
+ except Exception as excep:
+ print(
+ excep,
+ ", Error while sending msg in primarythread()!")
+ break
+
+ # Receiving from primary
+ try:
+ # 1024 stands for bytes of data to be received
+ data = conn.recv(1024)
+ if data:
+ spp_common.PRIMARY2MAIN.put(
+ "recv:%s:{%s}" % (str(addr), data))
+ else:
+ spp_common.PRIMARY2MAIN.put("closing:" + str(addr))
+ conn.close()
+ self.sock_opened = False
+ break
+ except Exception as excep:
+ print(
+ excep,
+ ", Error while receiving msg in primarythread()!")
+ break
diff --git a/src/controller/spp.py b/src/controller/spp.py
index 0515193..d0d7bc9 100644
--- a/src/controller/spp.py
+++ b/src/controller/spp.py
@@ -4,8 +4,8 @@
from __future__ import print_function
import argparse
-from Queue import Queue
-import select
+from conn_thread import AcceptThread
+from conn_thread import PrimaryThread
from shell import Shell
import socket
import SocketServer
@@ -58,245 +58,6 @@ class CmdRequestHandler(SocketServer.BaseRequestHandler):
self.request.send("")
-class ConnectionThread(threading.Thread):
- """Manage connection between controller and secondary"""
-
- def __init__(self, client_id, conn):
- super(ConnectionThread, self).__init__()
- self.daemon = True
-
- self.client_id = client_id
- self.conn = conn
- self.stop_event = threading.Event()
- self.conn_opened = False
-
- def stop(self):
- self.stop_event.set()
-
- def run(self):
- cmd_str = 'hello'
-
- # infinite loop so that function do not terminate and thread do not
- # end.
- while True:
- try:
- _, _, _ = select.select(
- [self.conn, ], [self.conn, ], [], 5)
- except select.error:
- break
-
- # Sending message to connected secondary
- try:
- cmd_str = spp_common.MAIN2SEC[self.client_id].get(True)
- self.conn.send(cmd_str) # send only takes string
- except KeyError:
- break
- except Exception as excep:
- print(excep, ",Error while sending msg in connectionthread()!")
- break
-
- # Receiving from secondary
- try:
- # 1024 stands for bytes of data to be received
- data = self.conn.recv(1024)
- if data:
- spp_common.SEC2MAIN[self.client_id].put(
- "recv:%s:{%s}" % (str(self.conn.fileno()), data))
- else:
- spp_common.SEC2MAIN[self.client_id].put(
- "closing:" + str(self.conn))
- break
- except Exception as excep:
- print(
- excep, ",Error while receiving msg in connectionthread()!")
- break
-
- spp_common.SECONDARY_LIST.remove(self.client_id)
- self.conn.close()
-
-
-class AcceptThread(threading.Thread):
- """Manage connection"""
-
- def __init__(self, host, port):
- super(AcceptThread, self).__init__()
- self.daemon = True
-
- # Creating secondary socket object
- self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
- self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
- # Binding secondary socket to a address. bind() takes tuple of host
- # and port.
- self.sock.bind((host, port))
-
- # Listening secondary at the address
- self.sock.listen(spp_common.MAX_SECONDARY)
-
- self.stop_event = threading.Event()
- self.sock_opened = False
-
- def getclientid(self, conn):
- """Get client_id from client"""
-
- try:
- conn.send("_get_client_id")
- except KeyError:
- return -1
-
- data = conn.recv(1024)
- if data is None:
- return -1
-
- if logger is not None:
- logger.debug("data: %s" % data)
- client_id = int(data.strip('\0'))
-
- if client_id < 0 or client_id > spp_common.MAX_SECONDARY:
- logger.debug("Failed to get client_id: %d" % client_id)
- return -1
-
- found = 0
- for i in spp_common.SECONDARY_LIST:
- if client_id == i:
- found = 1
- break
-
- if found == 0:
- return client_id
-
- # client_id in use, find a free one
- free_client_id = -1
- for i in range(spp_common.MAX_SECONDARY):
- found = -1
- for j in spp_common.SECONDARY_LIST:
- if i == j:
- found = i
- break
- if found == -1:
- free_client_id = i
- break
-
- if logger is not None:
- logger.debug("Found free_client_id: %d" % free_client_id)
-
- if free_client_id < 0:
- return -1
-
- conn.send("_set_client_id %u" % free_client_id)
- data = conn.recv(1024)
-
- return free_client_id
-
- def stop(self):
- if self.sock_opened is True:
- try:
- self.sock.shutdown(socket.SHUT_RDWR)
- except socket.error as excep:
- print(excep, ", Error while closing sock in AcceptThread!")
- traceback.print_exc()
- self.sock.close()
- self.stop_event.set()
-
- def run(self):
- try:
- while True:
- # Accepting incoming connections
- conn, _ = self.sock.accept()
-
- client_id = self.getclientid(conn)
- if client_id < 0:
- break
-
- # Creating new thread.
- # Calling secondarythread function for this function and
- # passing conn as argument.
- spp_common.SECONDARY_LIST.append(client_id)
- spp_common.MAIN2SEC[client_id] = Queue()
- spp_common.SEC2MAIN[client_id] = Queue()
- connection_thread = ConnectionThread(client_id, conn)
- connection_thread.daemon = True
- connection_thread.start()
-
- spp_common.SECONDARY_COUNT += 1
- except Exception as excep:
- print(excep, ", Error in AcceptThread!")
- traceback.print_exc()
- self.sock_opened = False
- self.sock.close()
-
-
-class PrimaryThread(threading.Thread):
-
- def __init__(self, host, port):
- super(PrimaryThread, self).__init__()
- self.daemon = True
-
- self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
- self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
- # Binding primary socket to a address. bind() takes tuple of host
- # and port.
- self.sock.bind((host, port))
-
- # Listening primary at the address
- self.sock.listen(1) # 5 denotes the number of clients can queue
-
- self.stop_event = threading.Event()
- self.sock_opened = False
-
- def stop(self):
- if self.sock_opened is True:
- self.sock.shutdown(socket.SHUT_RDWR)
- self.sock.close()
- self.stop_event.set()
-
- def run(self):
- cmd_str = ''
-
- while True:
- # waiting for connection
- spp_common.PRIMARY = False
- conn, addr = self.sock.accept()
- spp_common.PRIMARY = True
-
- while conn:
- try:
- _, _, _ = select.select([conn, ], [conn, ], [], 5)
- except select.error:
- break
-
- self.sock_opened = True
- # Sending message to connected primary
- try:
- cmd_str = spp_common.MAIN2PRIMARY.get(True)
- conn.send(cmd_str) # send only takes string
- except KeyError:
- break
- except Exception as excep:
- print(
- excep,
- ", Error while sending msg in primarythread()!")
- break
-
- # Receiving from primary
- try:
- # 1024 stands for bytes of data to be received
- data = conn.recv(1024)
- if data:
- spp_common.PRIMARY2MAIN.put(
- "recv:%s:{%s}" % (str(addr), data))
- else:
- spp_common.PRIMARY2MAIN.put("closing:" + str(addr))
- conn.close()
- self.sock_opened = False
- break
- except Exception as excep:
- print(
- excep,
- ", Error while receiving msg in primarythread()!")
- break
-
-
def main(argv):
"""main"""
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 03/13] controller: aggregate logger to spp_common.py
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 01/13] spp: move controller to sub directory ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 02/13] controller: move connection threads ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 04/13] controller: add load command ogawa.yasufumi
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/conn_thread.py | 15 +--------------
src/controller/shell.py | 15 +--------------
src/controller/spp.py | 15 +--------------
src/controller/spp_common.py | 14 ++++++++++++++
4 files changed, 17 insertions(+), 42 deletions(-)
diff --git a/src/controller/conn_thread.py b/src/controller/conn_thread.py
index 7ba3b00..1620046 100644
--- a/src/controller/conn_thread.py
+++ b/src/controller/conn_thread.py
@@ -2,23 +2,10 @@ from Queue import Queue
import select
import socket
import spp_common
+from spp_common import logger
import threading
import traceback
-# Turn true if activate logger to debug remote command.
-logger = None
-
-if logger is True:
- import logging
- logger = logging.getLogger(__name__)
- handler = logging.StreamHandler()
- handler.setLevel(logging.DEBUG)
- formatter = logging.Formatter(
- '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
- handler.setFormatter(formatter)
- logger.setLevel(logging.DEBUG)
- logger.addHandler(handler)
-
class ConnectionThread(threading.Thread):
"""Manage connection between controller and secondary"""
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 145cbc3..06b0012 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -4,22 +4,9 @@ import os
from Queue import Empty
import re
import spp_common
+from spp_common import logger
import subprocess
-# Turn true if activate logger to debug remote command.
-logger = None
-
-if logger is True:
- import logging
- logger = logging.getLogger(__name__)
- handler = logging.StreamHandler()
- handler.setLevel(logging.DEBUG)
- formatter = logging.Formatter(
- '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
- handler.setFormatter(formatter)
- logger.setLevel(logging.DEBUG)
- logger.addHandler(handler)
-
class Shell(cmd.Cmd, object):
"""SPP command prompt"""
diff --git a/src/controller/spp.py b/src/controller/spp.py
index d0d7bc9..7672cc9 100644
--- a/src/controller/spp.py
+++ b/src/controller/spp.py
@@ -10,24 +10,11 @@ from shell import Shell
import socket
import SocketServer
import spp_common
+from spp_common import logger
import sys
import threading
import traceback
-# Turn true if activate logger to debug remote command.
-logger = None
-
-if logger is True:
- import logging
- logger = logging.getLogger(__name__)
- handler = logging.StreamHandler()
- handler.setLevel(logging.DEBUG)
- formatter = logging.Formatter(
- '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
- handler.setFormatter(formatter)
- logger.setLevel(logging.DEBUG)
- logger.addHandler(handler)
-
class CmdRequestHandler(SocketServer.BaseRequestHandler):
"""Request handler for getting message from remote entities"""
diff --git a/src/controller/spp_common.py b/src/controller/spp_common.py
index 8c5af07..4cbfffd 100644
--- a/src/controller/spp_common.py
+++ b/src/controller/spp_common.py
@@ -1,5 +1,19 @@
from Queue import Queue
+# Turn true if activate logger to debug remote command.
+logger = None
+
+if logger is True:
+ import logging
+ logger = logging.getLogger(__name__)
+ handler = logging.StreamHandler()
+ handler.setLevel(logging.DEBUG)
+ formatter = logging.Formatter(
+ '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
+ handler.setFormatter(formatter)
+ logger.setLevel(logging.DEBUG)
+ logger.addHandler(handler)
+
PRIMARY = ''
SECONDARY_LIST = []
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 04/13] controller: add load command
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (2 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 03/13] controller: aggregate logger to spp_common.py ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 05/13] controller: move common methods to shell_lib ogawa.yasufumi
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Load command is for loading a command plugin. It enables users to
activate a command after SPP controller is launched. Plugin files
are included in 'src/controller/command/'.
This update also includes a sample command 'hello'. It says hello
message with given name.
spp > load hello
spp > hello alice
Hello, alice!
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/command/__init__.py | 0
src/controller/command/hello.py | 28 ++++++++++++++++++++++++++++
src/controller/shell.py | 21 +++++++++++++++++++++
3 files changed, 49 insertions(+)
create mode 100644 src/controller/command/__init__.py
create mode 100644 src/controller/command/hello.py
diff --git a/src/controller/command/__init__.py b/src/controller/command/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/controller/command/hello.py b/src/controller/command/hello.py
new file mode 100644
index 0000000..f898234
--- /dev/null
+++ b/src/controller/command/hello.py
@@ -0,0 +1,28 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+
+class Hello(object):
+ def __init__(self, name):
+ self.name = name
+
+ def say(self):
+ print("Hello, %s!" % self.name)
+
+
+def do_hello(name):
+ """Say hello to given user
+
+ spp > hello alice
+ Hello, alice!
+ """
+
+ if name == '':
+ print('name is required!')
+ else:
+ hl = Hello(name)
+ hl.say()
+
+if __name__ == "__main__":
+ hello = Hello()
+ print(hello.say())
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 06b0012..7c7d94a 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -1,4 +1,5 @@
import cmd
+# import importlib
import json
import os
from Queue import Empty
@@ -563,3 +564,23 @@ class Shell(cmd.Cmd, object):
self.close()
print('Thank you for using Soft Patch Panel')
return True
+
+ def do_load(self, args):
+ """Load command plugin
+
+ Path of plugin file is 'spp/src/controller/command'.
+
+ spp > load hello
+ """
+
+ args = re.sub(',', ' ', args)
+ args = re.sub(r'\s+', ' ', args)
+ list_args = args.split(' ')
+
+ libdir = 'command'
+ loaded = '%s.%s' % (libdir, list_args[0])
+ # importlib.import_module(loaded)
+ exec('import %s' % loaded)
+ do_cmd = '%s.do_%s' % (loaded, list_args[0])
+ setattr(self, 'do_%s' % list_args[0], eval(do_cmd))
+ print("Module '%s' loaded." % loaded)
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 05/13] controller: move common methods to shell_lib
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (3 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 04/13] controller: add load command ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 06/13] controller: add filter for py to compl_common ogawa.yasufumi
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
For refactoring, move common methods in 'shell.py' to shell_lib/.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/shell.py | 106 +++--------------------------------
src/controller/shell_lib/__init__.py | 0
src/controller/shell_lib/common.py | 94 +++++++++++++++++++++++++++++++
3 files changed, 102 insertions(+), 98 deletions(-)
create mode 100644 src/controller/shell_lib/__init__.py
create mode 100644 src/controller/shell_lib/common.py
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 7c7d94a..3cb5f96 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -1,9 +1,9 @@
import cmd
-# import importlib
import json
import os
from Queue import Empty
import re
+from shell_lib import common
import spp_common
from spp_common import logger
import subprocess
@@ -28,96 +28,6 @@ class Shell(cmd.Cmd, object):
SEC_SUBCMDS = ['vhost', 'ring', 'pcap', 'nullpmd']
BYE_CMDS = ['sec', 'all']
- def decorate_dir(self, curdir, filelist):
- """Add '/' the end of dirname for path completion
-
- 'filelist' is a list of files contained in a directory.
- """
-
- res = []
- for f in filelist:
- if os.path.isdir('%s/%s' % (curdir, f)):
- res.append('%s/' % f)
- else:
- res.append(f)
- return res
-
- def compl_common(self, text, line, ftype=None):
- """File path completion for 'complete_*' method
-
- This method is called from 'complete_*' to complete 'do_*'.
- 'text' and 'line' are arguments of 'complete_*'.
-
- `complete_*` is a member method of builtin Cmd class and
- called if tab key is pressed in a command defiend by 'do_*'.
- 'text' and 'line' are contents of command line.
- For example, if you type tab at 'command arg1 ar',
- last token 'ar' is assigned to 'text' and whole line
- 'command arg1 ar' is assigned to 'line'.
-
- NOTE:
- If tab is typed after '/', empty text '' is assigned to
- 'text'. For example 'aaa b/', text is not 'b/' but ''.
- """
-
- if text == '': # tab is typed after command name or '/'
- tokens = line.split(' ')
- target_dir = tokens[-1] # get dirname for competion
- if target_dir == '': # no dirname means current dir
- res = self.decorate_dir(
- '.', os.listdir(os.getcwd()))
- else: # after '/'
- res = self.decorate_dir(
- target_dir, os.listdir(target_dir))
- else: # tab is typed in the middle of a word
- tokens = line.split(' ')
- target = tokens[-1] # target dir for completion
-
- if '/' in target: # word is a path such as 'path/to/file'
- seg = target.split('/')[-1] # word to be completed
- target_dir = '/'.join(target.split('/')[0:-1])
- else:
- seg = text
- target_dir = os.getcwd()
-
- matched = []
- for t in os.listdir(target_dir):
- if t.find(seg) == 0: # get words matched with 'seg'
- matched.append(t)
- res = self.decorate_dir(target_dir, matched)
-
- if ftype is not None: # filtering by ftype
- completions = []
- if ftype == 'directory':
- for fn in res:
- if fn[-1] == '/':
- completions.append(fn)
- elif ftype == 'file':
- for fn in res:
- if fn[-1] != '/':
- completions.append(fn)
- else:
- completions = res
- else:
- completions = res
- return completions
-
- def is_comment_line(self, line):
- """Find commend line to not to interpret as a command
-
- Return True if given line is a comment, or False.
- Supported comment styles are
- * python ('#')
- * C ('//')
- """
-
- input_line = line.strip()
- if len(input_line) > 0:
- if (input_line[0] == '#') or (input_line[0:2] == '//'):
- return True
- else:
- return False
-
def default(self, line):
"""Define defualt behaviour
@@ -125,7 +35,7 @@ class Shell(cmd.Cmd, object):
as a comment.
"""
- if self.is_comment_line(line):
+ if common.is_comment_line(line):
print("%s" % line.strip())
else:
super(Shell, self).default(line)
@@ -401,7 +311,7 @@ class Shell(cmd.Cmd, object):
self.response(self.CMD_ERROR, "invalid format")
def complete_record(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
+ return common.compl_common(text, line)
def do_record(self, fname):
"""Save commands to a log file
@@ -421,7 +331,7 @@ class Shell(cmd.Cmd, object):
self.response(self.CMD_OK, "record")
def complete_playback(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
+ return common.compl_common(text, line)
def do_playback(self, fname):
"""Load a config file to reproduce network configuration
@@ -440,7 +350,7 @@ class Shell(cmd.Cmd, object):
with open(fname) as recorded_file:
lines = []
for line in recorded_file:
- if not self.is_comment_line(line):
+ if not common.is_comment_line(line):
lines.append("# %s" % line)
lines.append(line)
self.cmdqueue.extend(lines)
@@ -483,7 +393,7 @@ class Shell(cmd.Cmd, object):
print(os.getcwd())
def complete_ls(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
+ return common.compl_common(text, line)
def do_ls(self, args):
"""Show a list of specified directory
@@ -500,7 +410,7 @@ class Shell(cmd.Cmd, object):
print("No such a directory.")
def complete_cd(self, text, line, begidx, endidx):
- return self.compl_common(text, line, 'directory')
+ return common.compl_common(text, line, 'directory')
def do_cd(self, args):
"""Change current directory
@@ -515,7 +425,7 @@ class Shell(cmd.Cmd, object):
print("No such a directory.")
def complete_mkdir(self, text, line, begidx, endidx):
- return self.compl_common(text, line)
+ return common.compl_common(text, line)
def do_mkdir(self, args):
"""Create a new directory
diff --git a/src/controller/shell_lib/__init__.py b/src/controller/shell_lib/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/controller/shell_lib/common.py b/src/controller/shell_lib/common.py
new file mode 100644
index 0000000..d011684
--- /dev/null
+++ b/src/controller/shell_lib/common.py
@@ -0,0 +1,94 @@
+import os
+
+
+def decorate_dir(curdir, filelist):
+ """Add '/' the end of dirname for path completion
+
+ 'filelist' is a list of files contained in a directory.
+ """
+
+ res = []
+ for f in filelist:
+ if os.path.isdir('%s/%s' % (curdir, f)):
+ res.append('%s/' % f)
+ else:
+ res.append(f)
+ return res
+
+
+def compl_common(text, line, ftype=None):
+ """File path completion for 'complete_*' method
+
+ This method is called from 'complete_*' to complete 'do_*'.
+ 'text' and 'line' are arguments of 'complete_*'.
+
+ `complete_*` is a member method of builtin Cmd class and
+ called if tab key is pressed in a command defiend by 'do_*'.
+ 'text' and 'line' are contents of command line.
+ For example, if you type tab at 'command arg1 ar',
+ last token 'ar' is assigned to 'text' and whole line
+ 'command arg1 ar' is assigned to 'line'.
+
+ NOTE:
+ If tab is typed after '/', empty text '' is assigned to
+ 'text'. For example 'aaa b/', text is not 'b/' but ''.
+ """
+
+ if text == '': # tab is typed after command name or '/'
+ tokens = line.split(' ')
+ target_dir = tokens[-1] # get dirname for competion
+ if target_dir == '': # no dirname means current dir
+ res = decorate_dir(
+ '.', os.listdir(os.getcwd()))
+ else: # after '/'
+ res = decorate_dir(
+ target_dir, os.listdir(target_dir))
+ else: # tab is typed in the middle of a word
+ tokens = line.split(' ')
+ target = tokens[-1] # target dir for completion
+
+ if '/' in target: # word is a path such as 'path/to/file'
+ seg = target.split('/')[-1] # word to be completed
+ target_dir = '/'.join(target.split('/')[0:-1])
+ else:
+ seg = text
+ target_dir = os.getcwd()
+
+ matched = []
+ for t in os.listdir(target_dir):
+ if t.find(seg) == 0: # get words matched with 'seg'
+ matched.append(t)
+ res = decorate_dir(target_dir, matched)
+
+ if ftype is not None: # filtering by ftype
+ completions = []
+ if ftype == 'directory':
+ for fn in res:
+ if fn[-1] == '/':
+ completions.append(fn)
+ elif ftype == 'file':
+ for fn in res:
+ if fn[-1] != '/':
+ completions.append(fn)
+ else:
+ completions = res
+ else:
+ completions = res
+ return completions
+
+
+def is_comment_line(line):
+ """Find commend line to not to interpret as a command
+
+ Return True if given line is a comment, or False.
+ Supported comment styles are
+ * python ('#')
+ * C ('//')
+ """
+
+ input_line = line.strip()
+ if len(input_line) > 0:
+ if (input_line[0] == '#') or (input_line[0:2] == '//'):
+ return True
+ else:
+ return False
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 06/13] controller: add filter for py to compl_common
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (4 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 05/13] controller: move common methods to shell_lib ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 07/13] controller: refactor shell.py ogawa.yasufumi
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
To filter python code with comple_common() method, add a case for
matching '.py' source files.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/shell_lib/common.py | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/controller/shell_lib/common.py b/src/controller/shell_lib/common.py
index d011684..b4e8fb9 100644
--- a/src/controller/shell_lib/common.py
+++ b/src/controller/shell_lib/common.py
@@ -66,6 +66,10 @@ def compl_common(text, line, ftype=None):
for fn in res:
if fn[-1] == '/':
completions.append(fn)
+ elif ftype == 'py' or ftype == 'python':
+ for fn in res:
+ if fn[-3:] == '.py':
+ completions.append(fn)
elif ftype == 'file':
for fn in res:
if fn[-1] != '/':
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 07/13] controller: refactor shell.py
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (5 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 06/13] controller: add filter for py to compl_common ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 08/13] controller: change logger output to logfile ogawa.yasufumi
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Change loading command plugin from usign setattr() to directly assigning
it to Shell class. It is because Shell object provides completion and
showing help only for class methods and does not for methods added with
setattr() mehtod.
This update also includes misc refactoring of whole of shell.py.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/command/hello.py | 2 +-
src/controller/shell.py | 246 +++++++++++++++++++++++-----------------
2 files changed, 141 insertions(+), 107 deletions(-)
diff --git a/src/controller/command/hello.py b/src/controller/command/hello.py
index f898234..e3d974b 100644
--- a/src/controller/command/hello.py
+++ b/src/controller/command/hello.py
@@ -10,7 +10,7 @@ class Hello(object):
print("Hello, %s!" % self.name)
-def do_hello(name):
+def do_hello(self, name):
"""Say hello to given user
spp > hello alice
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 3cb5f96..c5ef82d 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -28,6 +28,8 @@ class Shell(cmd.Cmd, object):
SEC_SUBCMDS = ['vhost', 'ring', 'pcap', 'nullpmd']
BYE_CMDS = ['sec', 'all']
+ PLUGIN_DIR = 'command'
+
def default(self, line):
"""Define defualt behaviour
@@ -159,18 +161,6 @@ class Shell(cmd.Cmd, object):
return valid
- def complete_pri(self, text, line, begidx, endidx):
- """Completion for primary process commands"""
-
- if not text:
- completions = self.PRI_CMDS[:]
- else:
- completions = [p
- for p in self.PRI_CMDS
- if p.startswith(text)
- ]
- return completions
-
def clean_sec_cmd(self, cmdstr):
"""remove unwanted spaces to avoid invalid command error"""
@@ -178,64 +168,6 @@ class Shell(cmd.Cmd, object):
res = re.sub(r'\s?;\s?', ";", tmparg)
return res
- def complete_sec(self, text, line, begidx, endidx):
- """Completion for secondary process commands"""
-
- try:
- cleaned_line = self.clean_sec_cmd(line)
- if len(cleaned_line.split()) == 1:
- completions = [str(i)+";" for i in spp_common.SECONDARY_LIST]
- elif len(cleaned_line.split()) == 2:
- if not (";" in cleaned_line):
- tmplist = [str(i) for i in spp_common.SECONDARY_LIST]
- completions = [p+";"
- for p in tmplist
- if p.startswith(text)
- ]
- elif cleaned_line[-1] == ";":
- completions = self.SEC_CMDS[:]
- else:
- seccmd = cleaned_line.split(";")[1]
- if cleaned_line[-1] != " ":
- completions = [p
- for p in self.SEC_CMDS
- if p.startswith(seccmd)
- ]
- elif ("add" in seccmd) or ("del" in seccmd):
- completions = self.SEC_SUBCMDS[:]
- else:
- completions = []
- elif len(cleaned_line.split()) == 3:
- subcmd = cleaned_line.split()[-1]
- if ("add" == subcmd) or ("del" == subcmd):
- completions = self.SEC_SUBCMDS[:]
- else:
- if cleaned_line[-1] == " ":
- completions = []
- else:
- completions = [p
- for p in self.SEC_SUBCMDS
- if p.startswith(subcmd)
- ]
- else:
- completions = []
- return completions
- except Exception as e:
- print(len(cleaned_line.split()))
- print(e)
-
- def complete_bye(self, text, line, begidx, endidx):
- """Completion for bye commands"""
-
- if not text:
- completions = self.BYE_CMDS[:]
- else:
- completions = [p
- for p in self.BYE_CMDS
- if p.startswith(text)
- ]
- return completions
-
def response(self, result, message):
"""Enqueue message from other than CLI"""
@@ -251,6 +183,28 @@ class Shell(cmd.Cmd, object):
if logger is not None:
logger.debug("unknown remote command = %s" % rcmd)
+ def precmd(self, line):
+ """Called before running a command
+
+ It is called for checking a contents of command line.
+ """
+
+ if self.recorded_file:
+ if not (
+ ('playback' in line) or
+ ('bye' in line) or
+ ('exit' in line)):
+ self.recorded_file.write("%s\n" % line)
+ return line
+
+ def close(self):
+ """Close record file"""
+
+ if self.recorded_file:
+ print("closing file")
+ self.recorded_file.close()
+ self.recorded_file = None
+
def do_status(self, _):
"""Display status info of SPP processes
@@ -278,6 +232,18 @@ class Shell(cmd.Cmd, object):
print(message)
self.response(self.CMD_ERROR, message)
+ def complete_pri(self, text, line, begidx, endidx):
+ """Completion for primary process commands"""
+
+ if not text:
+ completions = self.PRI_CMDS[:]
+ else:
+ completions = [p
+ for p in self.PRI_CMDS
+ if p.startswith(text)
+ ]
+ return completions
+
def do_sec(self, arg):
"""Send command to secondary process
@@ -310,8 +276,51 @@ class Shell(cmd.Cmd, object):
print ("first %s" % cmds[1])
self.response(self.CMD_ERROR, "invalid format")
- def complete_record(self, text, line, begidx, endidx):
- return common.compl_common(text, line)
+ def complete_sec(self, text, line, begidx, endidx):
+ """Completion for secondary process commands"""
+
+ try:
+ cleaned_line = self.clean_sec_cmd(line)
+ if len(cleaned_line.split()) == 1:
+ completions = [str(i)+";" for i in spp_common.SECONDARY_LIST]
+ elif len(cleaned_line.split()) == 2:
+ if not (";" in cleaned_line):
+ tmplist = [str(i) for i in spp_common.SECONDARY_LIST]
+ completions = [p+";"
+ for p in tmplist
+ if p.startswith(text)
+ ]
+ elif cleaned_line[-1] == ";":
+ completions = self.SEC_CMDS[:]
+ else:
+ seccmd = cleaned_line.split(";")[1]
+ if cleaned_line[-1] != " ":
+ completions = [p
+ for p in self.SEC_CMDS
+ if p.startswith(seccmd)
+ ]
+ elif ("add" in seccmd) or ("del" in seccmd):
+ completions = self.SEC_SUBCMDS[:]
+ else:
+ completions = []
+ elif len(cleaned_line.split()) == 3:
+ subcmd = cleaned_line.split()[-1]
+ if ("add" == subcmd) or ("del" == subcmd):
+ completions = self.SEC_SUBCMDS[:]
+ else:
+ if cleaned_line[-1] == " ":
+ completions = []
+ else:
+ completions = [p
+ for p in self.SEC_SUBCMDS
+ if p.startswith(subcmd)
+ ]
+ else:
+ completions = []
+ return completions
+ except Exception as e:
+ print(len(cleaned_line.split()))
+ print(e)
def do_record(self, fname):
"""Save commands to a log file
@@ -330,7 +339,7 @@ class Shell(cmd.Cmd, object):
self.recorded_file = open(fname, 'w')
self.response(self.CMD_OK, "record")
- def complete_playback(self, text, line, begidx, endidx):
+ def complete_record(self, text, line, begidx, endidx):
return common.compl_common(text, line)
def do_playback(self, fname):
@@ -360,27 +369,8 @@ class Shell(cmd.Cmd, object):
print(message)
self.response(self.CMD_NG, message)
- def precmd(self, line):
- """Called before running a command
-
- It is called for checking a contents of command line.
- """
-
- if self.recorded_file:
- if not (
- ('playback' in line) or
- ('bye' in line) or
- ('exit' in line)):
- self.recorded_file.write("%s\n" % line)
- return line
-
- def close(self):
- """Close record file"""
-
- if self.recorded_file:
- print("closing file")
- self.recorded_file.close()
- self.recorded_file = None
+ def complete_playback(self, text, line, begidx, endidx):
+ return common.compl_common(text, line)
def do_pwd(self, args):
"""Show corrent directory
@@ -392,9 +382,6 @@ class Shell(cmd.Cmd, object):
print(os.getcwd())
- def complete_ls(self, text, line, begidx, endidx):
- return common.compl_common(text, line)
-
def do_ls(self, args):
"""Show a list of specified directory
@@ -409,8 +396,8 @@ class Shell(cmd.Cmd, object):
else:
print("No such a directory.")
- def complete_cd(self, text, line, begidx, endidx):
- return common.compl_common(text, line, 'directory')
+ def complete_ls(self, text, line, begidx, endidx):
+ return common.compl_common(text, line)
def do_cd(self, args):
"""Change current directory
@@ -424,8 +411,8 @@ class Shell(cmd.Cmd, object):
else:
print("No such a directory.")
- def complete_mkdir(self, text, line, begidx, endidx):
- return common.compl_common(text, line)
+ def complete_cd(self, text, line, begidx, endidx):
+ return common.compl_common(text, line, 'directory')
def do_mkdir(self, args):
"""Create a new directory
@@ -438,6 +425,9 @@ class Shell(cmd.Cmd, object):
c = 'mkdir -p %s' % args
subprocess.call(c, shell=True)
+ def complete_mkdir(self, text, line, begidx, endidx):
+ return common.compl_common(text, line)
+
def do_bye(self, arg):
"""Terminate SPP processes and controller
@@ -464,6 +454,18 @@ class Shell(cmd.Cmd, object):
self.close()
return True
+ def complete_bye(self, text, line, begidx, endidx):
+ """Completion for bye commands"""
+
+ if not text:
+ completions = self.BYE_CMDS[:]
+ else:
+ completions = [p
+ for p in self.BYE_CMDS
+ if p.startswith(text)
+ ]
+ return completions
+
def do_exit(self, args):
"""Terminate SPP controller
@@ -475,7 +477,13 @@ class Shell(cmd.Cmd, object):
print('Thank you for using Soft Patch Panel')
return True
- def do_load(self, args):
+ def do_inspect(self, args):
+ from pprint import pprint
+ if args == '':
+ pprint(vars(self))
+ pprint(self.__class__.__name__)
+
+ def do_load_cmd(self, args):
"""Load command plugin
Path of plugin file is 'spp/src/controller/command'.
@@ -488,9 +496,35 @@ class Shell(cmd.Cmd, object):
list_args = args.split(' ')
libdir = 'command'
- loaded = '%s.%s' % (libdir, list_args[0])
- # importlib.import_module(loaded)
+ mod_name = list_args[0]
+ method_name = 'do_%s' % mod_name
+ loaded = '%s.%s' % (libdir, mod_name)
exec('import %s' % loaded)
- do_cmd = '%s.do_%s' % (loaded, list_args[0])
- setattr(self, 'do_%s' % list_args[0], eval(do_cmd))
+ do_cmd = '%s.%s' % (loaded, method_name)
+ exec('Shell.%s = %s' % (method_name, do_cmd))
+
print("Module '%s' loaded." % loaded)
+
+ def complete_load_cmd(self, text, line, begidx, endidx):
+ """Complete command plugins
+
+ Search under PLUGIN_DIR with compl_common() method.
+ This method is intended to be used for searching current
+ directory, but not in this case. If text is not '',
+ compl_common() does not work correctly and do filtering
+ for the result by self.
+ """
+
+ curdir = os.path.dirname(__file__)
+ res = common.compl_common(
+ '', '%s/%s' % (curdir, self.PLUGIN_DIR), 'py')
+
+ completions = []
+ for t in res:
+ if text == '':
+ if t[:2] != '__':
+ completions.append(t[:-3])
+ else:
+ if t[:len(text)] == text:
+ completions.append(t[:-3])
+ return completions
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 08/13] controller: change logger output to logfile
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (6 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 07/13] controller: refactor shell.py ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 09/13] controller: add do_topo to shell.py ogawa.yasufumi
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Update .gitignore to logfile is ignored.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
.gitignore | 4 +++-
src/controller/spp_common.py | 26 +++++++++++++-------------
2 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/.gitignore b/.gitignore
index fbf8b76..e2c6c1e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,5 @@
-.*.swp
+*.swp
*.pyc
+*.log
docs/guides/_build/*
+src/controller/3rd_party/*
diff --git a/src/controller/spp_common.py b/src/controller/spp_common.py
index 4cbfffd..3a92d75 100644
--- a/src/controller/spp_common.py
+++ b/src/controller/spp_common.py
@@ -1,18 +1,18 @@
+import logging
+import os
from Queue import Queue
-# Turn true if activate logger to debug remote command.
-logger = None
-
-if logger is True:
- import logging
- logger = logging.getLogger(__name__)
- handler = logging.StreamHandler()
- handler.setLevel(logging.DEBUG)
- formatter = logging.Formatter(
- '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
- handler.setFormatter(formatter)
- logger.setLevel(logging.DEBUG)
- logger.addHandler(handler)
+# Setup logger object
+logger = logging.getLogger(__name__)
+# handler = logging.StreamHandler()
+logfile = '%s/log/%s' % (os.path.dirname(__file__), 'spp.log')
+handler = logging.FileHandler(logfile)
+handler.setLevel(logging.DEBUG)
+formatter = logging.Formatter(
+ '%(asctime)s,[%(filename)s][%(name)s][%(levelname)s]%(message)s')
+handler.setFormatter(formatter)
+logger.setLevel(logging.DEBUG)
+logger.addHandler(handler)
PRIMARY = ''
SECONDARY_LIST = []
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 09/13] controller: add do_topo to shell.py
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (7 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 08/13] controller: change logger output to logfile ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 10/13] controller: add topo.py ogawa.yasufumi
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
'topo' is an experimental command for SPP controller to outptu topology
of network configuration. It enables to display a graphical image of
network configuration on terminal, or browser for which websocket
server is running with.
'topo' command supports four types of output. For multiple output
support, it uses graphviz, imagemagick and other external tools.
* terminal (but very few terminals supporting to display images)
* browser (websocket server is required)
* image file (jpg, png, bmp)
* text (dot, json, yaml)
Here is a list of required tools for 'topo'. Notice that SPP controller
and 'topo' command support MacOS on a remote host.
* graphviz
* imagemagick
* libsixel-bin (for Ubuntu) and terminal app supporting img2sixel
* iTerm2 and imgcat (for MacOS)
For terminal, only few terminal applications are supported. You can use
mlterm or xterm for which 'img2sixel' supports on linux
or iTerm on MacOS. If you use iTerm2, you have to get a shell script
'imgcat' from [1] and save it as
'spp/src/controller/3rd_party/imgcat.sh'.
[1] https://iterm2.com/documentation-images.html
For browser support, you are required to install websocket libraries and
use 'spp_ws' tool. 'spp_ws' is not released yet, but soon.
There are some example usecases.
(1) Display on terminal or browser.
spp > topo term
spp > topo http
(2) Output to a file.
spp > topo myconfig.jpg
spp > topo myconfig dot
This update is only for 'shell.py' and requires a module file 'topo.py'
in which methods for 'topo' command are implemented.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/shell.py | 85 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 84 insertions(+), 1 deletion(-)
diff --git a/src/controller/shell.py b/src/controller/shell.py
index c5ef82d..8a0580a 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -7,6 +7,7 @@ from shell_lib import common
import spp_common
from spp_common import logger
import subprocess
+import topo
class Shell(cmd.Cmd, object):
@@ -481,7 +482,89 @@ class Shell(cmd.Cmd, object):
from pprint import pprint
if args == '':
pprint(vars(self))
- pprint(self.__class__.__name__)
+
+ def do_topo(self, args):
+ """Output network topology
+
+ Support four types of output.
+ * terminal (but very few terminals supporting to display images)
+ * browser (websocket server is required)
+ * image file (jpg, png, bmp)
+ * text (dot, json, yaml)
+
+ spp > topo term # terminal
+ spp > topo http # browser
+ spp > topo network_conf.jpg # image
+ spp > topo network_conf.dot # text
+ """
+
+ if len(spp_common.SECONDARY_LIST) == 0:
+ message = "secondary not exist"
+ print(message)
+ self.response(self.CMD_NOTREADY, message)
+ else:
+ tp = topo.Topo(
+ spp_common.SECONDARY_LIST,
+ spp_common.MAIN2SEC,
+ spp_common.SEC2MAIN)
+ args_ary = args.split()
+ if len(args_ary) == 0:
+ print("Usage: topo dst [ftype]")
+ return False
+ elif (args_ary[0] == "term") or (args_ary[0] == "http"):
+ res_ary = tp.show(args_ary[0])
+ elif len(args_ary) == 1:
+ ftype = args_ary[0].split(".")[-1]
+ res_ary = tp.output(args_ary[0], ftype)
+ elif len(args_ary) == 2:
+ res_ary = tp.output(args_ary[0], args_ary[1])
+ else:
+ print("Usage: topo dst [ftype]")
+ return False
+ self.response(self.CMD_OK, json.dumps(res_ary))
+
+ def complete_topo(self, text, line, begidx, endidx):
+ """Complete topo command
+
+ If no token given, return 'term' and 'http'.
+ On the other hand, complete 'term' or 'http' if token starts
+ from it, or complete file name if is one of supported formats.
+ """
+
+ terms = ['term', 'http']
+ # Supported formats
+ img_exts = ['jpg', 'png', 'bmp']
+ txt_exts = ['dot', 'yml', 'js']
+
+ # Number of given tokens is expected as two. First one is
+ # 'topo'. If it is three or more, this methods returns nothing.
+ tokens = re.sub(r"\s+", " ", line).split(' ')
+ if (len(tokens) == 2):
+ if (text == ''):
+ completions = terms
+ else:
+ completions = []
+ # Check if 2nd token is a part of terms.
+ for t in terms:
+ if t.startswith(tokens[1]):
+ completions.append(t)
+ # Not a part of terms, so check for completion for
+ # output file name.
+ if len(completions) == 0:
+ if tokens[1].endswith('.'):
+ completions = img_exts + txt_exts
+ elif ('.' in tokens[1]):
+ fname = tokens[1].split('.')[0]
+ token = tokens[1].split('.')[-1]
+ for ext in img_exts:
+ if ext.startswith(token):
+ completions.append(fname + '.' + ext)
+ for ext in txt_exts:
+ if ext.startswith(token):
+ completions.append(fname + '.' + ext)
+ return completions
+ else: # do nothing for three or more tokens
+ pass
def do_load_cmd(self, args):
"""Load command plugin
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 10/13] controller: add topo.py
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (8 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 09/13] controller: add do_topo to shell.py ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 11/13] controller: add topo_subgraph command ogawa.yasufumi
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
This update adds 'topo.py' in which methods for 'topo' command are
implemented.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/topo.py | 312 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 312 insertions(+)
create mode 100644 src/controller/topo.py
diff --git a/src/controller/topo.py b/src/controller/topo.py
new file mode 100644
index 0000000..30a9c1a
--- /dev/null
+++ b/src/controller/topo.py
@@ -0,0 +1,312 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+import os
+import re
+from spp_common import logger
+import subprocess
+import traceback
+import uuid
+import websocket
+
+
+class Topo(object):
+ """Setup and output network topology for topo command
+
+ Topo command supports four types of output.
+ * terminal (but very few terminals supporting to display images)
+ * browser (websocket server is required)
+ * image file (jpg, png, bmp)
+ * text (dot, json, yaml)
+ """
+
+ def __init__(self, sec_ids, m2s_queues, s2m_queues):
+ logger.info("Topo initialized with sec IDs %s" % sec_ids)
+ self.sec_ids = sec_ids
+ self.m2s_queues = m2s_queues
+ self.s2m_queues = s2m_queues
+
+ def show(self, dtype):
+ res_ary = []
+ for sec_id in self.sec_ids:
+ self.m2s_queues[sec_id].put("status")
+ res = self.format_sec_status(self.s2m_queues[sec_id].get(True))
+ res_ary.append(res)
+ if dtype == "http":
+ self.to_http(res_ary)
+ elif dtype == "term":
+ self.to_term(res_ary)
+ else:
+ print("Invalid file type")
+ return res_ary
+
+ def output(self, fname, ftype="dot"):
+ res_ary = []
+ for sec_id in self.sec_ids:
+ self.m2s_queues[sec_id].put("status")
+ res = self.format_sec_status(self.s2m_queues[sec_id].get(True))
+ res_ary.append(res)
+
+ if ftype == "dot":
+ self.to_dot(res_ary, fname)
+ elif ftype == "json" or ftype == "js":
+ self.to_json(res_ary, fname)
+ elif ftype == "yaml" or ftype == "yml":
+ self.to_yaml(res_ary, fname)
+ elif ftype == "jpg" or ftype == "png" or ftype == "bmp":
+ self.to_img(res_ary, fname)
+ else:
+ print("Invalid file type")
+ return res_ary
+ print("Create topology: '%s'" % fname)
+ return res_ary
+
+ def to_dot(self, sec_list, output_fname):
+ # Label given if outport is "none"
+ NO_PORT = "none"
+
+ # Graphviz params
+ SEC_COLORS = [
+ "blue", "green", "orange", "chocolate", "black",
+ "cyan3", "green3", "indianred", "lawngreen", "limegreen"]
+ PORT_COLORS = {
+ "PHY": "white",
+ "RING": "yellow",
+ "VHOST": "limegreen"}
+ LINE_STYLE = {
+ "RUNNING": "solid",
+ "IDLING": "dashed"}
+ GRAPH_TYPE = "digraph"
+ LINK_TYPE = "->"
+
+ node_attrs = 'node[shape="rectangle", style="filled"];'
+
+ phys = []
+ rings = []
+ vhosts = []
+ links = []
+
+ for sec in sec_list:
+ for port in sec["ports"]:
+ if port["iface"]["type"] == "PHY":
+ phys.append(port)
+ elif port["iface"]["type"] == "RING":
+ rings.append(port)
+ elif port["iface"]["type"] == "VHOST":
+ vhosts.append(port)
+ else:
+ raise ValueError(
+ "Invaid interface type: %s" % port["iface"]["type"])
+
+ if port["out"] != NO_PORT:
+ out_id = int(port["out"])
+ if sec["forward"] is True:
+ l_style = LINE_STYLE["RUNNING"]
+ else:
+ l_style = LINE_STYLE["IDLING"]
+ attrs = '[label="%s", color="%s", style="%s"]' % (
+ "sec" + sec["sec_id"],
+ SEC_COLORS[int(sec["sec_id"])],
+ l_style
+ )
+ tmp = "%s%s %s %s%s%s;" % (
+ port["iface"]["type"],
+ port["iface"]["id"],
+ LINK_TYPE,
+ sec["ports"][out_id]["iface"]["type"],
+ sec["ports"][out_id]["iface"]["id"],
+ attrs
+ )
+ links.append(tmp)
+
+ output = ["%s spp{" % GRAPH_TYPE]
+ output.append("newrank=true;")
+ output.append(node_attrs)
+
+ phy_labels = []
+ for p in phys:
+ phy_labels.append(p["iface"]["type"] + p["iface"]["id"])
+ phy_labels = list(set(phy_labels))
+ for l in phy_labels:
+ output.append(
+ '%s[label="%s", fillcolor="%s"];' % (
+ l, l, PORT_COLORS["PHY"]))
+
+ ring_labels = []
+ for p in rings:
+ ring_labels.append(p["iface"]["type"] + p["iface"]["id"])
+ ring_labels = list(set(ring_labels))
+ for l in ring_labels:
+ output.append(
+ '%s[label="%s", fillcolor="%s"];' % (
+ l, l, PORT_COLORS["RING"]))
+
+ vhost_labels = []
+ for p in vhosts:
+ vhost_labels.append(p["iface"]["type"] + p["iface"]["id"])
+ vhost_labels = list(set(vhost_labels))
+ for l in vhost_labels:
+ output.append(
+ '%s[label="%s", fillcolor="%s"];' % (
+ l, l, PORT_COLORS["VHOST"]))
+
+ # rank
+ output.append(
+ '{rank=same; %s}' % ("; ".join(ring_labels)))
+ if len(phys) > 0:
+ output.append(
+ '{rank=max; %s}' % (
+ phys[0]["iface"]["type"] + phys[0]["iface"]["id"]))
+ output.append(
+ '{rank=same; %s}' % ("; ".join(phy_labels)))
+
+ # subgraph
+ cluster_id = "cluster0"
+ sg_label = "Host"
+ sg_ports = "; ".join(phy_labels + ring_labels)
+ output.append(
+ 'subgraph %s {label="%s" %s}' % (cluster_id, sg_label, sg_ports))
+
+ for link in links:
+ output.append(link)
+
+ output.append("}")
+
+ # remove duplicated entries
+ f = open(output_fname, "w+")
+ f.write("\n".join(output))
+ f.close()
+
+ def to_json(self, sec_list, output_fname):
+ import json
+ f = open(output_fname, "w+")
+ f.write(json.dumps(sec_list))
+ f.close()
+
+ def to_yaml(self, sec_list, output_fname):
+ import yaml
+ f = open(output_fname, "w+")
+ f.write(yaml.dump(sec_list))
+ f.close()
+
+ def to_img(self, sec_list, output_fname):
+ tmpfile = "%s.dot" % uuid.uuid4().hex
+ self.to_dot(sec_list, tmpfile)
+ fmt = output_fname.split(".")[-1]
+ cmd = "dot -T%s %s -o %s" % (fmt, tmpfile, output_fname)
+ subprocess.call(cmd, shell=True)
+ subprocess.call("rm -f %s" % tmpfile, shell=True)
+
+ def to_http(self, sec_list):
+ tmpfile = "%s.dot" % uuid.uuid4().hex
+ self.to_dot(sec_list, tmpfile)
+ msg = open(tmpfile).read()
+ subprocess.call("rm -f %s" % tmpfile, shell=True)
+ ws_url = "ws://localhost:8989/spp_ws"
+ ws = websocket.create_connection(ws_url)
+ ws.send(msg)
+ ws.close()
+
+ def to_term(self, sec_list):
+ tmpfile = "%s.jpg" % uuid.uuid4().hex
+ self.to_img(sec_list, tmpfile)
+ from distutils import spawn
+
+ # TODO(yasufum) Add check for using only supported terminal
+
+ if spawn.find_executable("img2sixel") is not None:
+ img_cmd = "img2sixel"
+ else:
+ img_cmd = "%s/%s/imgcat.sh" % (
+ os.path.dirname(__file__), '3rd_party')
+ # Resize image to fit the terminal
+ img_size = "60%"
+ cmd = "convert -resize %s %s %s" % (img_size, tmpfile, tmpfile)
+ subprocess.call(cmd, shell=True)
+ subprocess.call("%s %s" % (img_cmd, tmpfile), shell=True)
+ subprocess.call(["rm", "-f", tmpfile])
+
+ def format_sec_status(self, stat):
+ """Return formatted secondary status as a hash
+
+ By running status command on controller, status is sent from
+ secondary process and receiving message is displayed.
+
+ This is an example of receiving status message.
+ recv:8:{Client ID 1 Idling
+ client_id:1
+ port_id:0,on,PHY,outport:2
+ port_id:1,on,PHY,outport:none
+ port_id:2,on,RING(0),outport:3
+ port_id:3,on,VHOST(1),outport:none
+ }
+
+ This method returns as following.
+ {
+ 'forward': False,
+ 'ports': [
+ {
+ 'out': 'none',
+ 'id': '0',
+ 'iface': {'type': 'PHY', 'id': '0'}
+ },
+ {
+ 'out': 'none',
+ 'id': '1',
+ 'iface': {'type': 'PHY', 'id': '1'}
+ }
+ ],
+ 'sec_id': '2'
+ }
+ """
+
+ stat_ary = stat.split("\n")
+ res = {}
+
+ try:
+ # Check running status
+ if "Idling" in stat_ary[0]:
+ res["forward"] = False
+ elif "Running" in stat_ary[0]:
+ res["forward"] = True
+ else:
+ print("Invalid forwarding status:", stat_ary[0])
+
+ ptn = re.compile(r"clinet_id:(\d+)")
+ m = ptn.match(stat_ary[1])
+ if m is not None:
+ res["sec_id"] = m.group(1)
+ else:
+ raise Exception("No client ID matched!")
+
+ ports = []
+ # match PHY, for exp. 'port_id:0,on,PHY,outport:none'
+ ptn_p = re.compile(r"port_id:(\d+),on,(\w+),outport:(\w+)")
+
+ # match RING for exp. 'port_id:2,on,RING(0),outport:3'
+ # or VHOST for exp. 'port_id:3,on,VHOST(1),outport:none'
+ ptn_v = re.compile(
+ r"port_id:(\d+),on,(\w+)\((\d+)\),outport:(\w+)")
+
+ for i in range(2, len(stat_ary)-1):
+ m = ptn_p.match(stat_ary[i])
+ if m is not None:
+ ports.append({
+ "id": m.group(1),
+ "iface": {"type": m.group(2), "id": m.group(1)},
+ "out": m.group(3)})
+ continue
+
+ m = ptn_v.match(stat_ary[i])
+ if m is not None:
+ ports.append({
+ "id": m.group(1),
+ "iface": {"type": m.group(2), "id": m.group(3)},
+ "out": m.group(4)})
+
+ res["ports"] = ports
+ return res
+
+ except Exception:
+ traceback.print_exc()
+ return None
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 11/13] controller: add topo_subgraph command
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (9 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 10/13] controller: add topo.py ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 12/13] controller: add cat and less command ogawa.yasufumi
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
'topo_subgraph' is for defining subgraph of resources in a network
topology generated by 'topo'. Typically it is used for grouping vhost
interfaces of each of VMs or containers to be more understandable the
generated network topology.
Usage: topo_subgraph [VERB] [LABEL] [SUBGRAPH]
VERB: 'add' or 'del'
If you assign vhost1 and vhost2 to VM1, define subgraph as following.
spp > topo_subgraph add VM1 vhost1,vhost2
Or delete defined subgraph.
spp > topo_subgraph del VM1
To show all of defined subgraphs, simply run without arguments.
spp > topo_subgraph
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/shell.py | 85 ++++++++++++++++++++++++++++++++++++++++++++++++-
src/controller/topo.py | 43 +++++++++++++++++++++----
2 files changed, 120 insertions(+), 8 deletions(-)
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 8a0580a..8aae86d 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -30,6 +30,7 @@ class Shell(cmd.Cmd, object):
BYE_CMDS = ['sec', 'all']
PLUGIN_DIR = 'command'
+ subgraphs = {}
def default(self, line):
"""Define defualt behaviour
@@ -483,6 +484,87 @@ class Shell(cmd.Cmd, object):
if args == '':
pprint(vars(self))
+ def terms_topo_subgraph(self):
+ """Define terms of topo_subgraph command"""
+
+ return ['add', 'del']
+
+ def do_topo_subgraph(self, args):
+ """Subgarph manager for topo
+
+ Subgraph is a group of object defined in dot language.
+ For topo command, it is used for grouping resources of each
+ of VM or container to topology be more understandable.
+
+ Add subgraph labeled 'vm1'. Resource name is capitalized and
+ both of them is OK.
+ spp > topo_subgraph add vm1 VHOST1;VHOST2 # upper case
+ spp > topo_subgraph add vm1 vhost1;vhost2 # lower case
+
+ Delete subgraph 'vm1'.
+ spp > topo_subgraph del vm1
+
+ To show subgraphs, run topo_subgraph without args.
+ spp > topo_subgraph
+ {'vm1', 'VHOST1;VHOST2'}
+ """
+
+ args_cleaned = re.sub(r"\s+", ' ', args).strip()
+ # Show subgraphs if given no argments
+ if (args_cleaned == ''):
+ if len(self.subgraphs) == 0:
+ print("No subgraph.")
+ else:
+ for label, subg in self.subgraphs.items():
+ print('label: %s\tsubgraph: "%s"' % (label, subg))
+ else: # add or del
+ tokens = args_cleaned.split(' ')
+ # Add subgraph
+ if tokens[0] == 'add':
+ if len(tokens) == 3:
+ label = tokens[1]
+ subg = tokens[2].upper()
+ if ',' in subg:
+ subg = re.sub(r",", ";", subg)
+
+ # TODO(yasufum) add validation for subgraph
+ self.subgraphs[label] = subg
+ print("Add subgraph '%s'" % label)
+ else:
+ print("Invalid syntax '%s'!" % args_cleaned)
+ # Delete subgraph
+ elif ((tokens[0] == 'del') or
+ (tokens[0] == 'delete') or
+ (tokens[0] == 'remove')):
+ del(self.subgraphs[tokens[1]])
+ print("Delete subgraph '%s'" % tokens[1])
+
+ else:
+ print("Ivalid subcommand '%s'!" % tokens[0])
+
+ def complete_topo_subgraph(self, text, line, begidx, endidx):
+ terms = self.terms_topo_subgraph()
+
+ tokens = re.sub(r"\s+", ' ', line).strip().split(' ')
+ if text == '':
+ if len(tokens) == 1:
+ return terms
+ elif len(tokens) == 2 and tokens[1] == 'del':
+ return self.subgraphs.keys()
+ elif text != '':
+ completions = []
+ if len(tokens) == 3 and tokens[1] == 'del':
+ for t in self.subgraphs.keys():
+ if t.startswith(tokens[2]):
+ completions.append(t)
+ elif len(tokens) == 2:
+ for t in terms:
+ if t.startswith(text):
+ completions.append(t)
+ return completions
+ else:
+ pass
+
def do_topo(self, args):
"""Output network topology
@@ -506,7 +588,8 @@ class Shell(cmd.Cmd, object):
tp = topo.Topo(
spp_common.SECONDARY_LIST,
spp_common.MAIN2SEC,
- spp_common.SEC2MAIN)
+ spp_common.SEC2MAIN,
+ self.subgraphs)
args_ary = args.split()
if len(args_ary) == 0:
print("Usage: topo dst [ftype]")
diff --git a/src/controller/topo.py b/src/controller/topo.py
index 30a9c1a..92d1634 100644
--- a/src/controller/topo.py
+++ b/src/controller/topo.py
@@ -20,11 +20,12 @@ class Topo(object):
* text (dot, json, yaml)
"""
- def __init__(self, sec_ids, m2s_queues, s2m_queues):
+ def __init__(self, sec_ids, m2s_queues, s2m_queues, sub_graphs):
logger.info("Topo initialized with sec IDs %s" % sec_ids)
self.sec_ids = sec_ids
self.m2s_queues = m2s_queues
self.s2m_queues = s2m_queues
+ self.sub_graphs = sub_graphs
def show(self, dtype):
res_ary = []
@@ -153,26 +154,54 @@ class Topo(object):
# rank
output.append(
'{rank=same; %s}' % ("; ".join(ring_labels)))
+ output.append(
+ '{rank=same; %s}' % ("; ".join(vhost_labels)))
+
if len(phys) > 0:
output.append(
'{rank=max; %s}' % (
phys[0]["iface"]["type"] + phys[0]["iface"]["id"]))
- output.append(
- '{rank=same; %s}' % ("; ".join(phy_labels)))
+ elif len(vhosts) > 0:
+ output.append(
+ '{rank=max; %s}' % (
+ vhosts[0]["iface"]["type"] + vhosts[0]["iface"]["id"]))
+
+ if len(phy_labels) > 0:
+ output.append(
+ '{rank=same; %s}' % ("; ".join(phy_labels)))
+
+ # Add subgraph
+ ssgs = []
+ if len(self.sub_graphs) > 0:
+ cnt = 1
+ for label, val in self.sub_graphs.items():
+ cluster_id = "cluster%d" % cnt
+ ssg_label = label
+ ssg_ports = val
+ ssg = 'subgraph %s {label="%s" %s}' % (
+ cluster_id, ssg_label, ssg_ports)
+ ssgs.append(ssg)
+ cnt += 1
- # subgraph
cluster_id = "cluster0"
sg_label = "Host"
sg_ports = "; ".join(phy_labels + ring_labels)
- output.append(
- 'subgraph %s {label="%s" %s}' % (cluster_id, sg_label, sg_ports))
+ if len(ssgs) == 0:
+ output.append(
+ 'subgraph %s {label="%s" %s}' % (
+ cluster_id, sg_label, sg_ports))
+ else:
+ tmp = 'label="%s" %s' % (sg_label, sg_ports)
+ contents = [tmp] + ssgs
+ output.append(
+ 'subgraph %s {%s}' % (cluster_id, '; '.join(contents)))
+ # Add links
for link in links:
output.append(link)
output.append("}")
- # remove duplicated entries
f = open(output_fname, "w+")
f.write("\n".join(output))
f.close()
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 12/13] controller: add cat and less command
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (10 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 11/13] controller: add topo_subgraph command ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-06 10:50 ` [spp] [PATCH 13/13] controller: create log directory ogawa.yasufumi
2018-03-27 23:41 ` [spp] [PATCH 00/13] Change structure of SPP controller Ferruh Yigit
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/shell.py | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/src/controller/shell.py b/src/controller/shell.py
index 8aae86d..86598d3 100644
--- a/src/controller/shell.py
+++ b/src/controller/shell.py
@@ -468,6 +468,34 @@ class Shell(cmd.Cmd, object):
]
return completions
+ def do_cat(self, arg):
+ """View contents of a file
+
+ spp > cat file
+ """
+ if os.path.isfile(arg):
+ c = 'cat %s' % arg
+ subprocess.call(c, shell=True)
+ else:
+ print("No such a directory.")
+
+ def complete_cat(self, text, line, begidx, endidx):
+ return common.compl_common(text, line)
+
+ def do_less(self, arg):
+ """View contents of a file
+
+ spp > less file
+ """
+ if os.path.isfile(arg):
+ c = 'less %s' % arg
+ subprocess.call(c, shell=True)
+ else:
+ print("No such a directory.")
+
+ def complete_less(self, text, line, begidx, endidx):
+ return common.compl_common(text, line)
+
def do_exit(self, args):
"""Terminate SPP controller
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [spp] [PATCH 13/13] controller: create log directory
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (11 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 12/13] controller: add cat and less command ogawa.yasufumi
@ 2018-03-06 10:50 ` ogawa.yasufumi
2018-03-27 23:41 ` [spp] [PATCH 00/13] Change structure of SPP controller Ferruh Yigit
13 siblings, 0 replies; 15+ messages in thread
From: ogawa.yasufumi @ 2018-03-06 10:50 UTC (permalink / raw)
To: ferruh.yigit, spp; +Cc: Yasufumi Ogawa
From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
To avoid error for no existing log directory, add creating the
directory while launching controller if it is not exist.
Signed-off-by: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
---
src/controller/spp_common.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/controller/spp_common.py b/src/controller/spp_common.py
index 3a92d75..59ba504 100644
--- a/src/controller/spp_common.py
+++ b/src/controller/spp_common.py
@@ -5,6 +5,8 @@ from Queue import Queue
# Setup logger object
logger = logging.getLogger(__name__)
# handler = logging.StreamHandler()
+os.system("mkdir -p %s/log" % (os.path.dirname(__file__)))
+
logfile = '%s/log/%s' % (os.path.dirname(__file__), 'spp.log')
handler = logging.FileHandler(logfile)
handler.setLevel(logging.DEBUG)
--
2.13.1
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [spp] [PATCH 00/13] Change structure of SPP controller
2018-03-06 10:50 [spp] [PATCH 00/13] Change structure of SPP controller ogawa.yasufumi
` (12 preceding siblings ...)
2018-03-06 10:50 ` [spp] [PATCH 13/13] controller: create log directory ogawa.yasufumi
@ 2018-03-27 23:41 ` Ferruh Yigit
13 siblings, 0 replies; 15+ messages in thread
From: Ferruh Yigit @ 2018-03-27 23:41 UTC (permalink / raw)
To: ogawa.yasufumi, spp
On 3/6/2018 10:50 AM, ogawa.yasufumi@lab.ntt.co.jp wrote:
> From: Yasufumi Ogawa <ogawa.yasufumi@lab.ntt.co.jp>
>
> SPP controller 'spp.py' is monolithic and has got to be large. It is
> hard to be maintained because several classes are included in the file
> and global variables are shared among its instances.
>
> This seriese of update is to move classes and methods in spp.py to
> 'controller/' as separated files for maintainability. It also includes
> additional commands.
>
> Yasufumi Ogawa (13):
> spp: move controller to sub directory
> controller: move connection threads
> controller: aggregate logger to spp_common.py
> controller: add load command
> controller: move common methods to shell_lib
> controller: add filter for py to compl_common
> controller: refactor shell.py
> controller: change logger output to logfile
> controller: add do_topo to shell.py
> controller: add topo.py
> controller: add topo_subgraph command
> controller: add cat and less command
> controller: create log directory
Series applied thanks.
^ permalink raw reply [flat|nested] 15+ messages in thread