test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [dts 0/9] Support virtual scenarios
@ 2015-06-18  3:06 Yong Liu
  2015-06-18  3:06 ` [dts] [dts 1/9] Optimize config load module with proper execption raised Yong Liu
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

This patch set will support virtualization scenario creation and destroy.
All scenarios will be based on scenerio configuration file. And there're three
typical virtual scenarios configurations supplied in this patch set.

Marvin Liu (9):
  Optimize config load module with proper execption raised
  Implement virtual scenario module
  Support virtual scenario in dts main process
  Optimize network topology discovery routine in virtual scenario
  Optimize set_target function for virtual scenario
  Support auto_map and set_target flag in vm initialization
  Optimize VF devices creation and network devices pass-through function
  Optimize ssh_pexpect and virt_resource module
  Add three typical virtual scenario configuration files

 conf/scene/pf_passthrough.cfg   |  25 ++
 conf/scene/vf_passthrough.cfg   |  28 +++
 conf/scene/vf_passthrough_1.cfg |  27 +++
 framework/config.py             |  17 +-
 framework/dts.py                |  45 +++-
 framework/dut.py                |  10 +-
 framework/exception.py          |   6 +
 framework/project_dpdk.py       |   5 +-
 framework/qemu_kvm.py           |  12 +-
 framework/settings.py           |  11 +
 framework/ssh_pexpect.py        |   7 +-
 framework/test_case.py          |   5 +-
 framework/virt_base.py          |  51 +++--
 framework/virt_dut.py           | 122 ++++++++--
 framework/virt_resource.py      |   4 +
 framework/virt_scene.py         | 495 ++++++++++++++++++++++++++++++++++++++++
 16 files changed, 812 insertions(+), 58 deletions(-)
 create mode 100644 conf/scene/pf_passthrough.cfg
 create mode 100644 conf/scene/vf_passthrough.cfg
 create mode 100644 conf/scene/vf_passthrough_1.cfg
 create mode 100644 framework/virt_scene.py

-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 1/9] Optimize config load module with proper execption raised
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 2/9] Implement virtual scenario module Yong Liu
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/config.py b/framework/config.py
index 7e2436a..5948edb 100644
--- a/framework/config.py
+++ b/framework/config.py
@@ -37,6 +37,8 @@ import re
 import ConfigParser  # config parse module
 import argparse      # prase arguments module
 
+from exception import ConfigParseException, VirtConfigParseException
+
 PORTCONF = "conf/ports.cfg"
 CRBCONF = "conf/crbs.cfg"
 VIRTCONF = "conf/virt_global.cfg"
@@ -50,7 +52,7 @@ class UserConf():
         if load_files == []:
             print "FAILED LOADING %s!!!" % config
             self.conf = None
-            raise
+            raise ConfigParseException(config)
 
     def get_sections(self):
         if self.conf is None:
@@ -91,9 +93,9 @@ class VirtConf(UserConf):
         self.virt_cfg = {}
         try:
             self.virt_conf = UserConf(self.config_file)
-        except Exception as e:
-            print "FAILED LOADING VIRT CONFIG!!!"
+        except ConfigParseException:
             self.virt_conf = None
+            raise VirtConfigParseException
 
     def load_virt_config(self, name):
         self.virt_cfgs = []
@@ -130,9 +132,9 @@ class PortConf(UserConf):
         self.pci_regex = "([\da-f]{2}:[\da-f]{2}.\d{1})$"
         try:
             self.port_conf = UserConf(self.config_file)
-        except Exception as e:
-            print "FAILED LOADING PORT CONFIG!!!"
+        except ConfigParseException:
             self.port_conf = None
+            raise PortConfigParseException
 
     def load_ports_config(self, crbIP):
         self.ports_cfg = {}
@@ -185,7 +187,10 @@ if __name__ == '__main__':
     args = parser.parse_args()
 
     # not existed configuration file
-    VirtConf('/tmp/not-existed.cfg')
+    try:
+        VirtConf('/tmp/not-existed.cfg')
+    except VirtConfigParseException:
+        print "Capture config parse failure"
 
     # example for basic use configuration file
     conf = UserConf(PORTCONF)
diff --git a/framework/exception.py b/framework/exception.py
index 6e112c8..facbeed 100644
--- a/framework/exception.py
+++ b/framework/exception.py
@@ -131,3 +131,9 @@ class VirtDutConnectException(Exception):
 class VirtDutInitException(Exception):
     def __init__(self, vm_dut):
         self.vm_dut = vm_dut
+
+class VirtDeviceCreateException(Exception):
+    pass
+
+class VirtVmOperationException(Exception):
+    pass
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 2/9] Implement virtual scenario module
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
  2015-06-18  3:06 ` [dts] [dts 1/9] Optimize config load module with proper execption raised Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 3/9] Support virtual scenario in dts main process Yong Liu
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Virtual scenario module will handle the scenario creation and destroy.
This module will be created based on scenerio configuration file.
All abstracted configurations will be handled and converted to configurations
for virtual machine.
This module also support previous vm start command list and post vm start
command list.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/virt_scene.py b/framework/virt_scene.py
new file mode 100644
index 0000000..3725ca2
--- /dev/null
+++ b/framework/virt_scene.py
@@ -0,0 +1,495 @@
+#!/usr/bin/python
+# BSD LICENSE
+#
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import dts
+
+from settings import CONFIG_ROOT_PATH, get_netdev
+from config import VirtConf
+from config import VIRTCONF
+from exception import *
+from qemu_kvm import QEMUKvm
+
+# scenario module for handling scenario
+# 1. load configurations
+# config saved in conf/scenarios/name.cfg
+# load configurations will saved in vm list
+# 2. handle special config
+#   pf_idx=0,vf_num=2,driver=default;
+#   PF0 igb_uio, create 2VFs by default driver
+# 3. create scenario
+#   allocate hareware resource for this vm
+#   cpu, memory, pf devices, vf devices
+#   configuration vm
+#   run pre_vm commands
+#   create vm
+#   run post_vm commands
+
+
+class VirtScene(object):
+
+    def __init__(self, dut, tester, scene_name):
+        self.name = scene_name
+        self.host_dut = dut
+        self.tester_dut = tester
+        self.pre_cmds = []
+        self.post_cmds = []
+
+        self.vm_dut_enable = False
+        self.auto_portmap = True
+        self.vm_type = 'kvm'
+
+    def load_config(self):
+        try:
+            self.vm_confs = {}
+            conf = VirtConf(CONFIG_ROOT_PATH + 'scene/' + self.name + '.cfg')
+            self.sections = conf.virt_conf.get_sections()
+            for vm in self.sections:
+                conf.load_virt_config(vm)
+                vm_conf = conf.get_virt_config()
+                self.vm_confs[vm] = vm_conf
+        except:
+            raise VirtConfigParseException
+
+    def prepare_vm(self):
+        for conf in self.vm_confs.keys():
+            if conf == 'scene':
+                suite_cfg = self.vm_confs['scene'][0]
+                self.prepare_suite(suite_cfg)
+                self.vm_confs.pop('scene')
+            else:
+                vm_name = conf
+                vm_conf = self.vm_confs[vm_name]
+                self.prepare_cpu(vm_name, vm_conf)
+                self.prepare_devices(vm_conf)
+                self.prepare_vmdevice(vm_conf)
+
+    def cleanup_vm(self):
+        # reload config for has been changed when handle config
+        self.load_config()
+        for conf in self.vm_confs.keys():
+            if conf != 'scene':
+                vm_name = conf
+                vm_conf = self.vm_confs[vm_name]
+                self.cleanup_devices(vm_conf)
+
+    def prepare_suite(self, conf):
+        if 'suite' in conf.keys():
+            for param in conf['suite']:
+                if 'dut' in param.keys():
+                    if param['dut'] == 'vm_dut':
+                        self.vm_dut_enable = True
+                if 'type' in param.keys():
+                    if param['type'] == 'xen':
+                        self.vm_type = 'xen'
+                    # not implement yet
+                    if param['type'] == 'vmware':
+                        self.vm_type = 'vmware'
+                    # not implement yet
+                    if param['type'] == 'container':
+                        self.vm_type = 'container'
+                if 'portmap' in param.keys():
+                    if param['portmap'] == 'cfg':
+                        self.auto_portmap = False
+
+    def prepare_cpu(self, vm_name, conf):
+        cpu_param = {}
+        for params in conf:
+            if 'cpu' in params.keys():
+                cpu_conf = params['cpu'][0]
+                break
+
+        if 'skipcores' in cpu_conf.keys():
+            cpus = cpu_conf['skipcores'].split()
+            # remove invalid configured core
+            for cpu in cpus:
+                if int(cpu) not in self.host_dut.virt_pool.cores:
+                    cpus.remove(cpu)
+            # create core mask for reserver cores
+            core_mask = dts.create_mask(cpus)
+            # reserve those skipped cores
+            self.host_dut.virt_pool.reserve_cpu(core_mask)
+
+        if 'numa' in cpu_conf.keys():
+            if cpu_conf['numa'] == 'auto':
+                numa = self.host_dut.ports_info[0]['port'].socket
+            else:
+                numa = int(cpu_conf['numa'])
+        else:
+            numa = 0
+
+        if 'number' in cpu_conf.keys():
+            num = int(cpu_conf['number'])
+        else:
+            num = 2
+
+        if 'model' in cpu_conf.keys():
+            model = cpu_conf['model']
+        else:
+            model = 'host'
+
+        cores = self.host_dut.virt_pool.alloc_cpu(vm=vm_name, number=num,
+                                                  socket=numa)
+        core_cfg = ''
+        for core in cores:
+            core_cfg += '%s ' % core
+        core_cfg = core_cfg[:-1]
+
+        cpu_param['number'] = num
+        cpu_param['model'] = model
+        cpu_param['cpupin'] = core_cfg
+
+        # replace with allocated cpus
+        params['cpu'] = [cpu_param]
+
+    def prepare_devices(self, conf):
+        for params in conf:
+            if 'dev_gen' in params.keys():
+                index = conf.index(params)
+                for param in params['dev_gen']:
+                    self.handle_dev_gen(**param)
+                # remove handled 'dev_gen' configuration
+                conf.remove(conf[index])
+
+    def cleanup_devices(self, conf):
+        for params in conf:
+            if 'dev_gen' in params.keys():
+                for param in params['dev_gen']:
+                    self.handle_dev_destroy(**param)
+
+    def prepare_vmdevice(self, conf):
+        for params in conf:
+            if 'device' in params.keys():
+                for param in params['device']:
+                    if 'vf_idx' in param.keys():
+                        new_param = self.prepare_vf_conf(param)
+                        index = params['device'].index(param)
+                        params['device'][index] = new_param
+                    elif 'pf_idx' in param.keys():
+                        new_param = self.prepare_pf_conf(param)
+                        index = params['device'].index(param)
+                        params['device'][index] = new_param
+
+                for param in params['device']:
+                    netdev = get_netdev(self.host_dut, param['opt_host'])
+                    if netdev is not None:
+                        netdev.bind_driver('pci-stub')
+
+    def prepare_pf_conf(self, param):
+        pf_param = {}
+        # strip pf pci id
+        pf = int(param['pf_idx'])
+        if pf >= len(self.host_dut.ports_info):
+            raise VirtDeviceCreateException
+        pf_pci = self.host_dut.ports_info[pf]['pci']
+        pf_param['driver'] = 'pci-assign'
+        pf_param['opt_host'] = pf_pci
+        if param['guestpci'] != 'auto':
+            pf_param['opt_addr'] = param['guestpci']
+
+        return pf_param
+
+    def prepare_vf_conf(self, param):
+        vf_param = {}
+        # strip vf pci id
+        if 'pf_dev' in param.keys():
+            pf = int(param['pf_dev'])
+            vfs = self.host_dut.ports_info[pf]['vfs_port']
+            vf_idx = int(param['vf_idx'])
+            if vf_idx >= len(vfs):
+                raise VirtDeviceCreateException
+            vf_pci = vfs[vf_idx].pci
+            vf_param['driver'] = 'pci-assign'
+            vf_param['opt_host'] = vf_pci
+            if param['guestpci'] != 'auto':
+                vf_param['opt_addr'] = param['guestpci']
+        else:
+            print dts.RED("Invalid vf device config, request pf_dev")
+
+        return vf_param
+
+    def reset_pf_cmds(self, port):
+        command = {}
+        command['type'] = 'host'
+        intf = self.host_dut.ports_info[port]['intf']
+        command['command'] = 'ifconfig %s up' % intf
+        self.reg_postvm_cmds(command)
+
+    def handle_dev_gen(self, **opts):
+        if 'pf_idx' in opts.keys():
+            port = int(opts['pf_idx'])
+            if 'vf_num' in opts.keys():
+                vf_num = int(opts['vf_num'])
+            else:
+                print dts.RED("No vf_num for port %d, assum one VF" % port)
+                vf_num = 1
+            if 'driver' in opts.keys():
+                driver = opts['driver']
+
+            try:
+                print dts.GREEN("create vf %d %d %s" % (port, vf_num, driver))
+                self.host_dut.generate_sriov_vfs_by_port(port, vf_num, driver)
+                self.reset_pf_cmds(port)
+            except:
+                print dts.RED("Failed to create vf as requested!!!")
+                raise VirtDeviceCreateException
+
+    def handle_dev_destroy(self, **opts):
+        if 'pf_idx' in opts.keys():
+            port = int(opts['pf_idx'])
+
+            try:
+                print dts.GREEN("destroy vfs on port %d" % port)
+                self.host_dut.destroy_sriov_vfs_by_port(port)
+            except:
+                print dts.RED("Failed to destroy vf as requested!!!")
+
+    def reg_prevm_cmds(self, command):
+        """
+        command: {'type':'host/tester/vm',
+                    define which crb command progress
+                  'command':'XXX',
+                    command send to crb
+                  'expect':'XXX',
+                    expected output for command
+                  'timeout': 60,
+                  'verify': True or False
+                    check whether command sucessfully
+                 }
+        """
+        self.pre_cmds.append(command)
+
+    def run_pre_cmds(self):
+        for cmd in self.pre_cmds:
+            if cmd['type'] == 'vm':
+                print dts.RED("Can't run vm command when vm not ready")
+            elif cmd['type'] == 'host':
+                crb = self.host_dut
+            elif cmd['type'] == 'tester':
+                crb = self.tester
+            else:
+                crb = self.host_dut
+
+            if 'expect' not in cmd.keys():
+                expect = "# "
+            else:
+                expect = cmd['expect']
+
+            if 'verify' not in cmd.keys():
+                verify = False
+            else:
+                verify = cmd['verify']
+
+            if 'timeout' not in cmd.keys():
+                timeout = 5
+            else:
+                timeout = cmd['timeout']
+
+            ret = crb.send_expect(cmd['command'], expect, timeout=timeout,
+                                  verify=verify)
+
+            if type(ret) is int and ret != 0:
+                print dts.RED("Failed to run command %s" % cmd['command'])
+                raise VirtVmOperationException
+
+    def reg_postvm_cmds(self, command):
+        """
+        command: {'type':'host/tester/vm',
+                    define which crb command progress
+                  'command':'XXX',
+                    command send to crb
+                  'expect':'XXX',
+                    expected output for command
+                  'verify':'yes or no'
+                    check whether command sucessfully
+        """
+        self.post_cmds.append(command)
+        pass
+
+    def run_post_cmds(self):
+        for cmd in self.post_cmds:
+            if cmd['type'] == 'vm':
+                crb = self.vm_dut
+            elif cmd['type'] == 'host':
+                crb = self.host_dut
+            elif cmd['type'] == 'tester':
+                crb = self.tester
+            else:
+                crb = self.host_dut
+
+            if 'expect' not in cmd.keys():
+                expect = "# "
+            else:
+                expect = cmd['expect']
+
+            if 'verify' not in cmd.keys():
+                verify = False
+            else:
+                verify = cmd['verify']
+
+            if 'timeout' not in cmd.keys():
+                timeout = 5
+            else:
+                timeout = cmd['timeout']
+
+            ret = crb.send_expect(cmd['command'], expect, timeout=timeout,
+                                  verify=verify)
+
+            if type(ret) is int and ret != 0:
+                print dts.RED("Failed to run command %s" % cmd['command'])
+                raise VirtVmOperationException
+
+    def merge_params(self, vm, params):
+        for param in params:
+            index = vm.find_option_index(param.keys()[0])
+            if index is not None:
+                vm.params[index] = param
+            else:
+                vm.params.append(param)
+        index = vm.find_option_index('name')
+        # update vm name
+        vm.params[index]['name'][0]['name'] = vm.vm_name
+
+    def start_vms(self):
+        self.vms = []
+        if self.vm_type == 'kvm':
+            for vm_name in self.vm_confs.keys():
+                # tricky here, QEMUKvm based on suite and vm name
+                # suite is virt_global, vm_name just the type
+                vm = QEMUKvm(self.host_dut, self.vm_type.upper(),
+                             'virt_global')
+                vm.load_config()
+                vm.vm_name = vm_name
+                # merge default config and scene config
+                scene_params = self.vm_confs[vm_name]
+                # reload merged configurations
+                self.merge_params(vm, scene_params)
+                try:
+                    vm_dut = vm.start(load_config=False, set_target=False,
+                                      auto_portmap=self.auto_portmap)
+                    if vm_dut is None:
+                        raise Exception("Set up VM ENV failed!")
+
+                    vm_info = {}
+                    vm_info[vm_name] = vm
+                    vm_info[vm_name + '_session'] = vm_dut
+                    self.vms.append(vm_info)
+
+                except Exception as e:
+                    print dts.RED("Failure for %s" % str(e))
+
+    def get_vm_duts(self):
+        duts = []
+        for vm_info in self.vms:
+            for vm_obj in vm_info.keys():
+                if 'session' in vm_obj:
+                    duts.append(vm_info[vm_obj])
+
+        return duts
+
+    def create_scene(self):
+        self.prepare_vm()
+        self.run_pre_cmds()
+        self.start_vms()
+        self.run_post_cmds()
+        pass
+
+    def set_target(self, target):
+        for vm_info in self.vms:
+            for vm_obj in vm_info.keys():
+                if 'session' in vm_obj:
+                    vm_info[vm_obj].set_target(target)
+
+    def destroy_scene(self):
+        for vm_info in self.vms:
+            for vm_obj in vm_info.keys():
+                if 'session' in vm_obj:
+                    vm_info[vm_obj].kill_all()
+                    vm_info[vm_obj].close()
+                    vm_info[vm_obj].logger.logger_exit()
+            for vm_obj in vm_info.keys():
+                if 'session' not in vm_obj:
+                    vm_info[vm_obj].stop()
+                    vm_info[vm_obj] = None
+        self.cleanup_vm()
+
+
+if __name__ == "__main__":
+
+    class QEMUKvm():
+
+        def __init__(self, dut, vm_name, suite_name):
+            print vm_name
+            print suite_name
+
+        def start(self):
+            print self.params
+            return True
+
+    class simple_dev(object):
+
+        def __init__(self, pci):
+            self.pci = pci
+            self.socket = 1
+
+    emu_dev1 = simple_dev("00:00.1")
+    emu_dev2 = simple_dev("00:00.2")
+    emu_dev3 = simple_dev("00:00.3")
+    emu_dev4 = simple_dev("00:00.4")
+
+    class simple_dut(object):
+
+        def __init__(self):
+            self.ports_info = [
+                {'vfs_port': [emu_dev1, emu_dev2]},
+                {'vfs_port': [emu_dev3, emu_dev4]},
+            ]
+            self.virt_pool = simple_resource()
+
+        def send_expect(self, cmds, expected, timeout=5,
+                        alt_session=False, verify=False):
+            print cmds + "---" + expected
+
+    class simple_resource(object):
+
+        def __init__(self):
+            pass
+
+        def reserve_cpu(self, coremask):
+            print "reserve " + coremask
+
+        def alloc_cpu(self, vm='', number=-1, socket=-1, corelist=None):
+            print "alloc %s num %d on socket %d" % (vm, number, socket)
+
+    dut = simple_dut()
+    scene = VirtScene(dut, None, "vf_passthrough")
+    scene.load_config()
+    scene.create_scene()
+    scene.destroy_scene()
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 3/9] Support virtual scenario in dts main process
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
  2015-06-18  3:06 ` [dts] [dts 1/9] Optimize config load module with proper execption raised Yong Liu
  2015-06-18  3:06 ` [dts] [dts 2/9] Implement virtual scenario module Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 4/9] Optimize network topology discovery routine in virtual scenario Yong Liu
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

When section "scenario" contained in execution.cfg, virtual scenario module
will be loaded and create the test scenario.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/dts.py b/framework/dts.py
index cf07d51..cf944da 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -55,6 +55,7 @@ from exception import TimeoutException
 from logger import getLogger
 import logger
 import debugger
+from virt_scene import VirtScene
 
 import sys
 reload(sys)
@@ -144,6 +145,11 @@ def dts_parse_config(section):
     """
     Parse execution file configuration.
     """
+    try:
+        scenario = config.get(section, 'scenario')
+    except:
+        scenario = ''
+
     duts = [dut_.strip() for dut_ in config.get(section,
                                                 'crbs').split(',')]
     targets = [target.strip()
@@ -157,7 +163,7 @@ def dts_parse_config(section):
 
     nic = [_.strip() for _ in paramDict['nic_type'].split(',')][0]
 
-    return duts[0], targets, test_suites, nic
+    return duts[0], targets, test_suites, nic, scenario
 
 
 def get_project_obj(project_name, super_class, crbInst, serializer):
@@ -261,16 +267,29 @@ def dts_run_prerequisties(pkgName, patch):
         return False
 
 
-def dts_run_target(crbInst, targets, test_suites, nic):
+def dts_run_target(crbInst, targets, test_suites, nic, scenario):
     """
     Run each target in execution targets.
     """
+    if scenario != '':
+        scene = VirtScene(dut, tester, scenario)
+    else:
+        scene = None
+
+    if scene:
+       scene.load_config()
+       scene.create_scene()
+
     for target in targets:
         log_handler.info("\nTARGET " + target)
         result.target = target
 
         try:
-            dut.set_target(target)
+            if scene:
+                scene.set_target(target)
+                dut.set_target(target, build_only=True)
+            else:
+                dut.set_target(target)
         except AssertionError as ex:
             log_handler.error(" TARGET ERROR: " + str(ex))
             result.add_failed_target(result.dut, target, str(ex))
@@ -284,7 +303,11 @@ def dts_run_target(crbInst, targets, test_suites, nic):
             paramDict['nic_type'] = 'any'
             nic = 'any'
 
-        dts_run_suite(crbInst, test_suites, target, nic)
+        dts_run_suite(crbInst, test_suites, target, nic, scene)
+
+    if scene:
+        scene.destroy_scene()
+        scene = None
 
     dut.restore_interfaces()
     dut.close()
@@ -292,7 +315,7 @@ def dts_run_target(crbInst, targets, test_suites, nic):
     tester.close()
 
 
-def dts_run_suite(crbInst, test_suites, target, nic):
+def dts_run_suite(crbInst, test_suites, target, nic, scene):
     """
     Run each suite in test suite list.
     """
@@ -306,7 +329,12 @@ def dts_run_suite(crbInst, test_suites, target, nic):
             module = test_module
             for test_classname, test_class in get_subclasses(test_module, TestCase):
 
-                test_suite = test_class(dut, tester, target, test_suite)
+                if scene.vm_dut_enable:
+                    duts = scene.get_vm_duts()
+                    tester.dut = duts[0]
+                    test_suite = test_class(duts[0], tester, target, test_suite)
+                else:
+                    test_suite = test_class(dut, tester, target, test_suite)
                 result.nic = test_suite.nic
 
                 dts_log_testsuite(test_suite, log_handler, test_classname)
@@ -352,6 +380,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
     global debug_case
     global Package
     global Patches
+    global scenario
 
     # save global variable
     Package = pkgName
@@ -405,7 +434,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
         dts_parse_param(section)
 
         # verify if the delimiter is good if the lists are vertical
-        dutIP, targets, test_suites, nics = dts_parse_config(section)
+        dutIP, targets, test_suites, nics, scenario = dts_parse_config(section)
 
         log_handler.info("\nDUT " + dutIP)
 
@@ -431,7 +460,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
             dts_crbs_exit()
             continue
 
-        dts_run_target(crbInst, targets, test_suites, nics)
+        dts_run_target(crbInst, targets, test_suites, nics, scenario)
 
         dts_crbs_exit()
 
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 4/9] Optimize network topology discovery routine in virtual scenario
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (2 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 3/9] Support virtual scenario in dts main process Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 5/9] Optimize set_target function for " Yong Liu
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/dut.py b/framework/dut.py
index a9b4ed0..96c362b 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -781,10 +781,12 @@ class Dut(Crb):
                     continue
 
                 if getattr(self, 'send_ping6', None):
-                    self.send_ping6(
-                        dutPort, self.tester.ports_info[remotePort]['ipv6'])
-                out = self.tester.send_ping6(
-                    remotePort, ipv6, self.get_mac_address(dutPort))
+                    out = self.send_ping6(
+                        dutPort, self.tester.ports_info[remotePort]['ipv6'],
+                        self.get_mac_address(dutPort))
+                else:
+                    out = self.tester.send_ping6(
+                        remotePort, ipv6, self.get_mac_address(dutPort))
 
                 if ('64 bytes from' in out):
                     self.logger.info("PORT MAP: [dut %d: tester %d]" % (dutPort, remotePort))
diff --git a/framework/virt_dut.py b/framework/virt_dut.py
index b52109d..79ecc2a 100644
--- a/framework/virt_dut.py
+++ b/framework/virt_dut.py
@@ -35,7 +35,7 @@ import time
 import dts
 import settings
 from config import PortConf
-from settings import NICS, LOG_NAME_SEP
+from settings import NICS, LOG_NAME_SEP, get_netdev
 from ssh_connection import SSHConnection
 from project_dpdk import DPDKdut
 from dut import Dut
@@ -54,7 +54,9 @@ class VirtDut(DPDKdut):
     or CRBBareMetal.
     """
 
-    def __init__(self, crb, serializer, virttype, vm_name, suite):
+    def __init__(self, hyper, crb, serializer, virttype, vm_name, suite):
+        self.vm_name = vm_name
+        self.hyper = hyper
         super(Dut, self).__init__(crb, serializer)
         self.vm_ip = self.get_ip_address()
         self.NAME = 'virtdut' + LOG_NAME_SEP + '%s' % self.vm_ip
@@ -96,8 +98,27 @@ class VirtDut(DPDKdut):
         """
         Load port config for this virtual machine
         """
+        self.conf = PortConf()
+        self.conf.load_ports_config(self.vm_name)
+        self.ports_cfg = self.conf.get_ports_config()
+
         return
 
+    def create_portmap(self):
+        port_num = len(self.ports_info)
+        self.ports_map = [-1] * port_num
+        for key in self.ports_cfg.keys():
+            index = int(key)
+            if index >= port_num:
+                print dts.RED("Can not found [%d ]port info" % index)
+                continue
+
+            if 'peer' in self.ports_cfg[key].keys():
+                tester_pci = self.ports_cfg[key]['peer']
+                # find tester_pci index
+                pci_idx = self.tester.get_local_index(tester_pci)
+                self.ports_map[index] = pci_idx
+
     def set_target(self, target):
         """
         Set env variable, these have to be setup all the time. Some tests
@@ -121,7 +142,7 @@ class VirtDut(DPDKdut):
 
         self.bind_interfaces_linux('igb_uio')
 
-    def prerequisites(self, pkgName, patch):
+    def prerequisites(self, pkgName, patch, auto_portmap):
         """
         Prerequest function should be called before execute any test case.
         Will call function to scan all lcore's information which on DUT.
@@ -153,13 +174,19 @@ class VirtDut(DPDKdut):
 
         # no need to rescan ports for guest os just bootup
         # load port infor from config file
-        self.load_portconf()
+        if auto_portmap is False:
+            self.load_portconf()
 
         # enable tester port ipv6
         self.host_dut.enable_tester_ipv6()
         self.mount_procfs()
-        # auto detect network topology
-        self.map_available_ports()
+
+        if auto_portmap:
+            # auto detect network topology
+            self.map_available_ports()
+        else:
+            self.create_portmap()
+
         # disable tester port ipv6
         self.host_dut.disable_tester_ipv6()
 
@@ -167,6 +194,22 @@ class VirtDut(DPDKdut):
         for port_info in self.ports_info:
             self.logger.info(port_info)
 
+    def init_core_list(self):
+        self.cores = []
+        cpuinfo = self.send_expect("grep --color=never \"processor\""
+                                   " /proc/cpuinfo", "#", alt_session=False)
+        cpuinfo = cpuinfo.split('\r\n')
+        for line in cpuinfo:
+            m = re.search("processor\t: (\d+)", line)
+            if m:
+                thread = m.group(1)
+                socket = 0
+                core = thread
+            self.cores.append(
+                {'thread': thread, 'socket': socket, 'core': core})
+
+        self.number_of_cores = len(self.cores)
+
     def restore_interfaces_domu(self):
         """
         Restore Linux interfaces.
@@ -227,13 +270,66 @@ class VirtDut(DPDKdut):
         Load or generate network connection mapping list.
         """
         self.map_available_ports_uncached()
-        self.logger.warning("DUT PORT MAP: " + str(self.ports_map))
+        self.logger.warning("VM DUT PORT MAP: " + str(self.ports_map))
 
-    def send_ping6(self, localPort, ipv6, mac=''):
+    def map_available_ports_uncached(self):
         """
-        Send ping6 packet from local port with destination ipv6 address.
+        Generate network connection mapping list.
         """
-        if self.ports_info[localPort]['type'] == 'ixia':
-            pass
-        else:
-            return self.send_expect("ping6 -w 1 -c 1 -A -I %s %s" % (self.ports_info[localPort]['intf'], ipv6), "# ", 10)
+        nrPorts = len(self.ports_info)
+        if nrPorts == 0:
+            return
+
+        remove = []
+        self.ports_map = [-1] * nrPorts
+
+        hits = [False] * len(self.tester.ports_info)
+
+        for dutPort in range(nrPorts):
+            peer = self.get_peer_pci(dutPort)
+            # if peer pci configured
+            if peer is not None:
+                for remotePort in range(len(self.tester.ports_info)):
+                    if self.tester.ports_info[remotePort]['pci'] == peer:
+                        hits[remotePort] = True
+                        self.ports_map[dutPort] = remotePort
+                        break
+                if self.ports_map[dutPort] == -1:
+                    self.logger.error("CONFIGURED TESTER PORT CANNOT FOUND!!!")
+                else:
+                    continue  # skip ping6 map
+
+            if hasattr(self.hyper, 'pt_devices'):
+                hostpci = self.hyper.pt_devices[dutPort]
+                netdev = get_netdev(self.host_dut, hostpci)
+
+            # auto ping port map
+            for remotePort in range(len(self.tester.ports_info)):
+                # for two vfs connected to same tester port
+                # can't skip those teste ports even have hits
+                # need skip ping self vf port
+                remotepci = self.tester.ports_info[remotePort]['pci']
+                remoteport =  self.tester.ports_info[remotePort]['port']
+                vfs = []
+                # vm_dut and tester in same dut
+                host_ip = self.crb['IP'].split(':')[0]
+                if self.crb['tester IP'] == host_ip:
+                    vfs = remoteport.get_sriov_vfs_pci()
+                # if hostpci is vf of tester port
+                if netdev.pci in vfs:
+                    print dts.RED("Skip ping from PF device")
+                    continue
+
+                ipv6 = self.get_ipv6_address(dutPort)
+                if ipv6 == "Not connected":
+                    continue
+
+                out = self.tester.send_ping6(
+                    remotePort, ipv6, self.get_mac_address(dutPort))
+
+                if ('64 bytes from' in out):
+                    self.logger.info(
+                        "PORT MAP: [dut %d: tester %d]" % (dutPort, remotePort))
+                    self.ports_map[dutPort] = remotePort
+                    hits[remotePort] = True
+                    continue
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 5/9] Optimize set_target function for virtual scenario
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (3 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 4/9] Optimize network topology discovery routine in virtual scenario Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 6/9] Support auto_map and set_target flag in vm initialization Yong Liu
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Add build_only parameter for set_target function, when enable build only,
ports on dut will not be bound to igb_uio driver.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
index 0b94b99..bc1a9bd 100644
--- a/framework/project_dpdk.py
+++ b/framework/project_dpdk.py
@@ -53,7 +53,7 @@ class DPDKdut(Dut):
         super(DPDKdut, self).__init__(crb, serializer)
         self.testpmd = None
 
-    def set_target(self, target):
+    def set_target(self, target, build_only=False):
         """
         Set env variable, these have to be setup all the time. Some tests
         need to compile example apps by themselves and will fail otherwise.
@@ -76,7 +76,7 @@ class DPDKdut(Dut):
         self.setup_memory()
         self.setup_modules(target)
 
-        if self.get_os_type() == 'linux':
+        if build_only is False and self.get_os_type() == 'linux':
             self.bind_interfaces_linux(dts.drivername)
 
     def setup_modules(self, target):
@@ -101,6 +101,7 @@ class DPDKdut(Dut):
             if "igb_uio" in out:
                 self.send_expect("rmmod -f igb_uio", "#", 70)
             self.send_expect("insmod ./" + target + "/kmod/igb_uio.ko", "#", 60)
+
             out = self.send_expect("lsmod | grep igb_uio", "#")
             assert ("igb_uio" in out), "Failed to insmod igb_uio"
 
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 6/9] Support auto_map and set_target flag in vm initialization
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (4 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 5/9] Optimize set_target function for " Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 7/9] Optimize VF devices creation and network devices pass-through function Yong Liu
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

When auto_map flag is True, virt dut will use ping6 function detect network
topology automatically. When set_target flag is False, will not build and use
dpdk in vm dut.

Optimize configuration file load routine and vm start, stop routine.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/virt_base.py b/framework/virt_base.py
index 3daf42f..9a0035d 100644
--- a/framework/virt_base.py
+++ b/framework/virt_base.py
@@ -45,11 +45,10 @@ from utils import remove_old_rsa_key
 
 
 class VirtBase(object):
-
     """
-    Basic module for customer special virtual type. This module implement functions
-    configurated and composed the VM boot command. With these function, we can get
-    and set the VM boot command, and instantiate the VM.
+    Basic module for customer special virtual type. This module implement
+    functions configurated and composed the VM boot command. With these
+    function, we can get and set the VM boot command, and instantiate the VM.
     """
 
     def __init__(self, dut, vm_name, suite_name):
@@ -202,22 +201,29 @@ class VirtBase(object):
         Check whether VM existed.
         """
         vm_status = self.host_session.send_expect(
-            "ps aux | grep qemu | grep 'name %s '| grep -v grep" % self.vm_name, "# ")
+            "ps aux | grep qemu | grep 'name %s '| grep -v grep"
+            % self.vm_name, "# ")
 
         if self.vm_name in vm_status:
             return True
         else:
             return False
 
-    def start(self):
+    def load_config(self):
+        """
+        Load configurations for VM
+        """
+        # load global and suite configuration file
+        self.load_global_config()
+        self.load_local_config(self.suite)
+
+    def start(self, load_config=True, set_target=True, auto_portmap=True):
         """
         Start VM and instantiate the VM with VirtDut.
         """
         try:
-            # load global and suite configuration file
-            self.load_global_config()
-            self.load_local_config(self.suite)
-
+            if load_config is True:
+                self.load_config()
             # compose boot command for different hypervisors
             self.compose_boot_param()
 
@@ -225,12 +231,12 @@ class VirtBase(object):
             self._start_vm()
 
             # connect vm dut and init running environment
-            vm_dut = self.instantiate_vm_dut()
+            vm_dut = self.instantiate_vm_dut(set_target, auto_portmap)
         except Exception as vm_except:
             if self.handle_exception(vm_except):
                 print dts.RED("Handled expection " + str(type(vm_except)))
             else:
-                print dts.RED("Unhandled expection " + str(type(vm_except)) + " !!!")
+                print dts.RED("Unhandled expection " + str(type(vm_except)))
 
             if callable(self.callback):
                 self.callback()
@@ -268,13 +274,19 @@ class VirtBase(object):
         else:
             return False
 
-    def __start_vm(self):
+    def _start_vm(self):
         """
         Start VM.
         """
         NotImplemented
 
-    def instantiate_vm_dut(self):
+    def _stop_vm(self):
+        """
+        Stop VM.
+        """
+        NotImplemented
+
+    def instantiate_vm_dut(self, set_target=True, auto_portmap=True):
         """
         Instantiate the Dut class for VM.
         """
@@ -290,6 +302,7 @@ class VirtBase(object):
 
         try:
             vm_dut = VirtDut(
+                self,
                 crb,
                 serializer,
                 self.virt_type,
@@ -316,9 +329,10 @@ class VirtBase(object):
 
         try:
             # setting up dpdk in vm, must call at last
-            vm_dut.prerequisites(dts.Package, dts.Patches)
-            target = self.host_dut.target
-            vm_dut.set_target(target)
+            vm_dut.prerequisites(dts.Package, dts.Patches, auto_portmap)
+            if set_target:
+                target = self.host_dut.target
+                vm_dut.set_target(target)
         except:
             raise exception.VirtDutInitException(vm_dut)
             return None
@@ -329,7 +343,8 @@ class VirtBase(object):
         """
         Stop the VM.
         """
-        NotImplemented
+        self._stop_vm()
+        self.virt_pool.free_all_resource(self.vm_name)
 
     def register_exit_callback(self, callback):
         """
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 7/9] Optimize VF devices creation and network devices pass-through function
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (5 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 6/9] Support auto_map and set_target flag in vm initialization Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 8/9] Optimize ssh_pexpect and virt_resource module Yong Liu
  2015-06-18  3:06 ` [dts] [dts 9/9] Add three typical virtual scenario configuration files Yong Liu
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Add pt devices list in qemu_kvm module which can support devices free in
virtual scenario module.
Support netdevie object get by pci address.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/qemu_kvm.py b/framework/qemu_kvm.py
index 8e0582a..6320a43 100644
--- a/framework/qemu_kvm.py
+++ b/framework/qemu_kvm.py
@@ -88,6 +88,9 @@ class QEMUKvm(VirtBase):
         self.char_idx = 0
         self.netdev_idx = 0
 
+        # devices pass-through into vm
+        self.pt_devices = []
+
     def set_vm_default(self):
         self.set_vm_name(self.vm_name)
         self.set_vm_enable_kvm()
@@ -534,6 +537,7 @@ class QEMUKvm(VirtBase):
         if 'opt_host' in options.keys() and \
                 options['opt_host']:
             dev_boot_line += separator + 'host=%s' % options['opt_host']
+            self.pt_devices.append(options['opt_host'])
         if 'opt_addr' in options.keys() and \
                 options['opt_addr']:
             dev_boot_line += separator + 'addr=%s' % options['opt_addr']
@@ -773,7 +777,7 @@ class QEMUKvm(VirtBase):
         cpus = self.virt_pool.alloc_cpu(vm=self.vm_name, corelist=req_cpus)
 
         if len(req_cpus) != len(cpus):
-            self.host_logger.warn("VCPUs not enough, required [ %s ], just [ %s ]" %
+            self.host_logger.warning("VCPUs not enough, required [ %s ], just [ %s ]" %
                                   (req_cpus, cpus))
             raise Exception("No enough required vcpus!!!")
 
@@ -904,6 +908,9 @@ class QEMUKvm(VirtBase):
                     return ip
         return ''
 
+    def get_vm_pt_devices(self):
+        return self.pt_devices
+
     def __control_session(self, command, *args):
         """
         Use the qemu guest agent service to control VM.
@@ -935,10 +942,9 @@ class QEMUKvm(VirtBase):
 
         return out
 
-    def stop(self):
+    def _stop_vm(self):
         """
         Stop VM.
         """
         self.__control_session('powerdown')
         time.sleep(5)
-        self.virt_pool.free_all_resource(self.vm_name)
diff --git a/framework/settings.py b/framework/settings.py
index 19ebe6b..d3cdf80 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -189,3 +189,14 @@ def accepted_nic(pci_id):
             return True
 
     return False
+
+def get_netdev(crb, pci):
+    for port in crb.ports_info:
+        if pci == port['pci']:
+            return port['port']
+        if 'vfs_port' in port.keys():
+            for vf in port['vfs_port']:
+                if pci == vf.pci:
+                    return vf
+
+    return None
diff --git a/framework/test_case.py b/framework/test_case.py
index a519691..5013123 100644
--- a/framework/test_case.py
+++ b/framework/test_case.py
@@ -49,7 +49,10 @@ class TestCase(object):
         for portid in range(len(self.dut.ports_info)):
             nic_type = self.dut.ports_info[portid]['type']
             self.nics.append(nic_name_from_type(nic_type))
-        self.nic = self.nics[0]
+        if len(self.nics):
+            self.nic = self.nics[0]
+        else:
+            self.nic = ''
 
     def set_up_all(self):
         pass
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 8/9] Optimize ssh_pexpect and virt_resource module
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (6 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 7/9] Optimize VF devices creation and network devices pass-through function Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  2015-06-18  3:06 ` [dts] [dts 9/9] Add three typical virtual scenario configuration files Yong Liu
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
index a474500..879b9de 100644
--- a/framework/ssh_pexpect.py
+++ b/framework/ssh_pexpect.py
@@ -18,8 +18,8 @@ class SSHPexpect(object):
         self.magic_prompt = "MAGIC PROMPT"
         try:
             self.session = pxssh.pxssh()
-            self.username = username
             self.host = host
+            self.username = username
             self.password = password
             if ':' in host:
                 self.ip = host.split(':')[0]
@@ -63,12 +63,13 @@ class SSHPexpect(object):
             if not int(ret_status):
                 return ret
             else:
+                self.logger.error("Command: %s failure!" % command)
                 self.logger.error(ret)
-                return ret_status
+                return int(ret_status)
         else:
             return ret
 
-    def get_session_before(self, timeout):
+    def get_session_before(self, timeout=15):
         """
         Get all output before timeout
         """
diff --git a/framework/virt_resource.py b/framework/virt_resource.py
index ce60751..b830f4e 100644
--- a/framework/virt_resource.py
+++ b/framework/virt_resource.py
@@ -161,6 +161,10 @@ class VirtResource(object):
             print "Alloc cpu request vitual machine name!!!"
             return cores
 
+        # if vm has been alloacted cores, just return them
+        if self.__vm_has_resource(vm, 'cores'):
+            return self.allocated_info[vm]['cores']
+
         if number != -1:
             for core in self.unused_cores:
                 if core != -1 and number != 0:
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [dts] [dts 9/9] Add three typical virtual scenario configuration files
  2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
                   ` (7 preceding siblings ...)
  2015-06-18  3:06 ` [dts] [dts 8/9] Optimize ssh_pexpect and virt_resource module Yong Liu
@ 2015-06-18  3:06 ` Yong Liu
  8 siblings, 0 replies; 10+ messages in thread
From: Yong Liu @ 2015-06-18  3:06 UTC (permalink / raw)
  To: dts

From: Marvin Liu <yong.liu@intel.com>

pf_passthrough.cfg for two pf devices pass-through into vm scene.
vf_passthrough for vf devices pass-through into vm. This two vfs created on
two pfs.
vf_passthrough_1 for vf devices pass-through into vm. This two vfs created on
same pf devices.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/conf/scene/pf_passthrough.cfg b/conf/scene/pf_passthrough.cfg
new file mode 100644
index 0000000..91b5eaf
--- /dev/null
+++ b/conf/scene/pf_passthrough.cfg
@@ -0,0 +1,25 @@
+# vm configuration for pf passthrough cases
+# numa 0,1,yes yes mean cpu numa match the first port
+# skipcores list mean those core will not used by vm
+# dut=vm_dut; mean vm_dut act as dut
+# dut=dut; mean host dut act as dut
+# portmap=cfg; mean vm_dut port map will be load from cfg
+# portmap=auto; mean vm_dut will create portmap automatically
+# devices = dev_gen/host/dev_gen+host not useful now
+[scene]
+suite =
+    dut=vm_dut,portmap=auto;
+    tester=tester;
+    type=kvm;
+[vm]
+cpu =
+    model=host,number=4,numa_aware=yes;
+mem = 
+    size=2048,hugepage=no;
+disk =
+    file=/storage/vm-image/vm0.img;
+device =
+    pf_idx=0,guestpci=auto;
+    pf_idx=1,guestpci=auto;
+vnc = 
+    displayNum=1;
diff --git a/conf/scene/vf_passthrough.cfg b/conf/scene/vf_passthrough.cfg
new file mode 100644
index 0000000..aa5493d
--- /dev/null
+++ b/conf/scene/vf_passthrough.cfg
@@ -0,0 +1,28 @@
+# vm configuration for vf passthrough cases
+# numa 0,1,yes yes mean cpu numa match the first port
+# skipcores list mean those core will not used by vm
+# dut=vm_dut; mean vm_dut act as dut
+# dut=dut; mean host dut act as dut
+# portmap=cfg; mean vm_dut port map will be load from cfg
+# portmap=auto; mean vm_dut will create portmap automatically
+# devices = dev_gen/host/dev_gen+host not useful now
+[scene]
+suite =
+    dut=vm_dut,portmap=auto;
+    tester=tester;
+    type=kvm;
+[vm0]
+cpu =
+    model=host,number=4,numa=auto,skipcores=0 1 2 3;
+mem = 
+    size=2048,hugepage=no;
+disk =
+    file=/storage/vm-image/vm0.img;
+dev_gen =
+    pf_idx=0,vf_num=2,driver=default;
+    pf_idx=1,vf_num=2,driver=default;
+device =
+    vf_idx=0,pf_dev=0,guestpci=auto;
+    vf_idx=0,pf_dev=1,guestpci=auto;
+vnc = 
+    displayNum=1;
diff --git a/conf/scene/vf_passthrough_1.cfg b/conf/scene/vf_passthrough_1.cfg
new file mode 100644
index 0000000..afd760a
--- /dev/null
+++ b/conf/scene/vf_passthrough_1.cfg
@@ -0,0 +1,27 @@
+# vm configuration for vf passthrough cases
+# numa 0,1,yes yes mean cpu numa match the first port
+# skipcores list mean those core will not used by vm
+# dut=vm_dut; mean vm_dut act as dut
+# dut=dut; mean host dut act as dut
+# portmap=cfg; mean vm_dut port map will be load from cfg
+# portmap=auto; mean vm_dut will create portmap automatically
+# devices = dev_gen/host/dev_gen+host not useful now
+[scene]
+suite =
+    dut=vm_dut,portmap=auto;
+    tester=tester;
+    type=kvm;
+[vm0]
+cpu =
+    model=host,number=4,numa=auto,skipcores=0 1 2 3;
+mem = 
+    size=2048,hugepage=no;
+disk =
+    file=/storage/vm-image/vm0.img;
+dev_gen =
+    pf_idx=0,vf_num=2,driver=default;
+device =
+    vf_idx=0,pf_dev=0,guestpci=auto;
+    vf_idx=1,pf_dev=0,guestpci=auto;
+vnc = 
+    displayNum=1;
-- 
1.9.3

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2015-06-18  3:07 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-18  3:06 [dts] [dts 0/9] Support virtual scenarios Yong Liu
2015-06-18  3:06 ` [dts] [dts 1/9] Optimize config load module with proper execption raised Yong Liu
2015-06-18  3:06 ` [dts] [dts 2/9] Implement virtual scenario module Yong Liu
2015-06-18  3:06 ` [dts] [dts 3/9] Support virtual scenario in dts main process Yong Liu
2015-06-18  3:06 ` [dts] [dts 4/9] Optimize network topology discovery routine in virtual scenario Yong Liu
2015-06-18  3:06 ` [dts] [dts 5/9] Optimize set_target function for " Yong Liu
2015-06-18  3:06 ` [dts] [dts 6/9] Support auto_map and set_target flag in vm initialization Yong Liu
2015-06-18  3:06 ` [dts] [dts 7/9] Optimize VF devices creation and network devices pass-through function Yong Liu
2015-06-18  3:06 ` [dts] [dts 8/9] Optimize ssh_pexpect and virt_resource module Yong Liu
2015-06-18  3:06 ` [dts] [dts 9/9] Add three typical virtual scenario configuration files Yong Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).