test suite reviews and discussions
 help / color / mirror / Atom feed
From: "Qiu, Michael" <michael.qiu@intel.com>
To: "Liu, Yong" <yong.liu@intel.com>, "dts@dpdk.org" <dts@dpdk.org>
Subject: Re: [dts] [PATCH v2 3/4] framework: reorganize DUT and Tester port	initialize sequence
Date: Wed, 4 Feb 2015 06:59:34 +0000	[thread overview]
Message-ID: <533710CFB86FA344BFBF2D6802E60286CD4647@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <1423032214-19856-4-git-send-email-yong.liu@intel.com>

On 2/4/2015 2:44 PM, Yong Liu wrote:
> DUT port initialization sequence: scan,restore,rescan,load config
> Test port initialization sequence: modprobe, interface up
> Separate restore interface function from DUT and Tester.
>
> Signed-off-by: Marvinliu <yong.liu@intel.com>
> ---
>  framework/crb.py            |  66 ++++-------------
>  framework/dut.py            | 172 ++++++++++++++++++++++++++++++--------------
>  framework/project_dpdk.py   |  22 +++---
>  framework/ssh_connection.py |   2 +-
>  framework/tester.py         |  32 ++++++++-
>  5 files changed, 175 insertions(+), 119 deletions(-)
>
> diff --git a/framework/crb.py b/framework/crb.py
> index d41f51b..fa03757 100644
> --- a/framework/crb.py
> +++ b/framework/crb.py
> @@ -133,56 +133,6 @@ class Crb(object):
>          """
>          self.send_expect("ip link set  %s %s" % (eth, status), "# ")
>  
> -    def restore_interfaces(self):
> -        """
> -        Restore Linux interfaces.
> -        """
> -        if dts.drivername == "vfio-pci":
> -            self.send_expect("rmmod vfio_iommu_type1", "# ", 10)
> -            self.send_expect("rmmod vfio_pci", "# ", 10)
> -            self.send_expect("rmmod vfio", "# ", 10)
> -        else:
> -            out = self.send_expect("lsmod | grep igb_uio", "#")
> -            if "igb_uio" in out:
> -                self.send_expect("rmmod -f igb_uio", "# ", 10)
> -        self.send_expect("modprobe igb", "# ", 20)
> -        self.send_expect("modprobe ixgbe", "# ", 20)
> -        self.send_expect("modprobe e1000e", "# ", 20)
> -        self.send_expect("modprobe e1000", "# ", 20)
> -        self.send_expect("modprobe virtio_net", "# ", 20)
> -
> -        try:
> -            for (pci_bus, pci_id) in self.pci_devices_info:
> -                """
> -                Check if driver is already bound before binding it.
> -                """
> -                if pci_id in ('8086:10fb', '8086:151c', '8086:1528', '8086:1512', '8086:154a'):
> -                    if not os.path.exists("/sys/bus/pci/drivers/ixgbe/"+"0000:"+pci_bus):
> -                        self.send_expect("echo -n 0000:%s > /sys/bus/pci/drivers/ixgbe/bind" % pci_bus, "# ")
> -                elif pci_id in ('8086:10e8', '8086:150e', '8086:1521', '8086:10c9', '8086:1526', '8086:1533'):
> -                    if not os.path.exists("/sys/bus/pci/drivers/igb/"+"0000:"+pci_bus):
> -                        self.send_expect("echo -n 0000:%s > /sys/bus/pci/drivers/igb/bind" % pci_bus, "# ")
> -                elif pci_id in('8086:10d3', '8086:10b9'):
> -                    if not os.path.exists("/sys/bus/pci/drivers/e1000e/"+"0000:"+pci_bus):
> -                        self.send_expect("echo -n 0000:%s > /sys/bus/pci/drivers/e1000e/bind" % pci_bus, "# ")
> -                elif pci_id in ('8086:153a', '8086:153b', '8086:155a', '8086:1559'):
> -                    if not os.path.exists("/sys/bus/pci/drivers/e1000e/"+"0000:"+pci_bus):
> -                        self.send_expect("echo -n 0000:%s > /sys/bus/pci/drivers/e1000e/bind" % pci_bus, "# ")
> -                elif pci_id in ('8086:100f', '8086:100e'):
> -                    if not os.path.exists("/sys/bus/pci/drivers/e1000/"+"0000:"+pci_bus):
> -                        self.send_expect("echo -n 0000:%s > /sys/bus/pci/drivers/e1000/bind" % pci_bus, "# ")
> -                elif pci_id in ('1af4:1000'):
> -                    self.send_expect("echo 0000%s > /sys/bus/pci/drivers/virtio-pci/bind" % pci_bus, "# ")
> -                else:
> -                    continue
> -
> -                addr_array = pci_bus.split(':')
> -                itf = self.get_interface_name(addr_array[0], addr_array[1])
> -                self.send_expect("ifconfig %s up" % itf, "# ")
> -
> -        except Exception as e:
> -            self.logger.error("   !!! Restore ITF: " + e.message)
> -
>      def pci_devices_information(self):
>          """
>          Scan CRB pci device information and save it into cache file.
> @@ -206,7 +156,7 @@ class Crb(object):
>          Look for the NIC's information (PCI Id and card type).
>          """
>          out = self.send_expect("lspci -nn | grep -i eth", "# ")
> -        rexp = r"([\da-f]{2}:[\da-f]{2}.\d{1}) Ethernet .* Intel Corporation .*?([\da-f]{4}:[\da-f]{4})"
> +        rexp = r"([\da-f]{2}:[\da-f]{2}.\d{1}) Ethernet .*?([\da-f]{4}:[\da-f]{4})"
>          pattern = re.compile(rexp)
>          match = pattern.findall(out)
>          self.pci_devices_info = []
> @@ -276,6 +226,20 @@ class Crb(object):
>          match = pattern.findall(out)
>          return match[0]
>  
> +    def get_device_numa(self, bus_id, devfun_id):
> +        """
> +        Get numa id of specified pci device
> +        """
> +        numa = self.send_expect("cat /sys/bus/pci/devices/0000\:%s\:%s/numa_node" %
> +                                (bus_id, devfun_id), "# ")
> +
> +        try:
> +            numa = int(numa)
> +        except ValueError:
> +            numa = -1
> +            self.logger.warning("NUMA not available")
> +        return numa
> +
>      def get_ipv6_addr(self, intf):
>          """
>          Get ipv6 address of specified pci device.
> diff --git a/framework/dut.py b/framework/dut.py
> index 9879335..cddc248 100644
> --- a/framework/dut.py
> +++ b/framework/dut.py
> @@ -33,6 +33,7 @@ import os
>  import re
>  import time
>  import dts
> +from config import UserConf
>  from settings import NICS
>  from ssh_connection import SSHConnection
>  from crb import Crb
> @@ -70,6 +71,7 @@ class Dut(Crb):
>          self.cores = []
>          self.architecture = None
>          self.ports_info = None
> +        self.conf = UserConf()
>  
>      def change_config_option(self, target, parameter, value):
>          """
> @@ -78,6 +80,14 @@ class Dut(Crb):
>          self.send_expect("sed -i 's/%s=.*$/%s=%s/'  config/defconfig_%s" %
>                           (parameter, parameter, value, target), "# ")
>  
> +    def set_nic_types(self, nics):
> +        """
> +        Set CRB NICS ready to validated.
> +        """
> +        self.nics = nics
> +        if 'cfg' in nics:
> +            self.conf.load_ports_config(self.get_ip_address())
> +
>      def set_toolchain(self, target):
>          """
>          This looks at the current target and instantiates an attribute to
> @@ -144,8 +154,14 @@ class Dut(Crb):
>  
>          self.init_core_list()
>          self.pci_devices_information()
> -        self.restore_interfaces()
> +        # scan ports before restore interface
>          self.scan_ports()
> +        # restore dut ports to kernel
> +        self.restore_interfaces()
> +        # rescan ports after interface up
> +        self.rescan_ports()
> +        # load port infor from config file
> +        self.load_portconf()
>          self.mount_procfs()
>  
>      def restore_interfaces(self):
> @@ -167,11 +183,27 @@ class Dut(Crb):
>          """
>          Restore Linux interfaces.
>          """
> -        Crb.restore_interfaces(self)
> -
> -        if self.ports_info is not None:
> -            for port in self.ports_info:
> -                self.admin_ports_linux(port['intf'], 'up')
> +        for port in self.ports_info:
> +            pci_bus = port['pci']
> +            pci_id = port['type']
> +            # get device driver
> +            driver = dts.get_nic_driver(pci_id)
> +            if driver is not None:
> +                # unbind device driver
> +                addr_array = pci_bus.split(':')
> +                bus_id = addr_array[0]
> +                devfun_id = addr_array[1]
> +
> +                self.send_expect('echo 0000:%s > /sys/bus/pci/devices/0000\:%s\:%s/driver/unbind'
> +                                 % (pci_bus, bus_id, devfun_id), '# ')
> +                # bind to linux kernel driver
> +                self.send_expect('modprobe %s' % driver, '# ')
> +                self.send_expect('echo 0000:%s > /sys/bus/pci/drivers/%s/bind'
> +                                 % (pci_bus, driver), '# ')
> +                itf = self.get_interface_name(addr_array[0], addr_array[1])
> +                self.send_expect("ifconfig %s up" % itf, "# ")
> +            else:
> +                self.logger.info("NOT FOUND DRIVER FOR PORT (%s|%s)!!!" % (pci_bus, pci_id))
>  
>      def setup_memory(self, hugepages=-1):
>          """
> @@ -259,7 +291,7 @@ class Dut(Crb):
>  
>          self.send_expect('tools/dpdk_nic_bind.py %s' % binding_list, '# ', 30)
>  
> -    def get_ports(self, nic_type, perf=None, socket=None):
> +    def get_ports(self, nic_type='any', perf=None, socket=None):
>          """
>          Return DUT port list with the filter of NIC type, whether run IXIA
>          performance test, whether request specified socket.
> @@ -301,7 +333,7 @@ class Dut(Crb):
>          else:
>              return ports
>  
> -    def get_ports_performance(self, nic_type, perf=None, socket=None,
> +    def get_ports_performance(self, nic_type='any', perf=None, socket=None,
>                                force_same_socket=True,
>                                force_different_nic=True):
>          """
> @@ -339,6 +371,15 @@ class Dut(Crb):
>  
>          return biggest_set
>  
> +    def get_peer_pci(self, port_num):
> +        """
> +        return the peer pci address of dut port
> +        """
> +        if 'peer' not in self.ports_info[port_num]:
> +            return None
> +        else:
> +            return self.ports_info[port_num]['peer']
> +
>      def get_mac_address(self, port_num):
>          """
>          return the port mac on dut
> @@ -381,6 +422,55 @@ class Dut(Crb):
>          else:
>              return 1
>  
> +    def check_ports_available(self, pci_bus, pci_id):
> +        """
> +        Check that whether auto scanned ports ready to use
> +        """
> +        pci_addr = "%s:%s" % (pci_bus, pci_id)
> +        codenames = []
> +        for nic in self.nics:
> +            if nic == 'any':
> +                return True
> +            elif nic == 'cfg':
> +                if self.conf.check_port_available(pci_bus) is True:
> +                    return True
> +            elif nic not in NICS.keys():
> +                self.logger.warning("NOT SUPPORTED NIC TYPE: %s" % nic)
> +            else:
> +                codenames.append(NICS[nic])
> +
> +        if pci_id in codenames:
> +            return True
> +
> +        return False
> +
> +    def rescan_ports(self):
> +        unknow_interface = dts.RED('Skipped: unknow_interface')
> +
> +        for port_info in self.ports_info:
> +            pci_bus = port_info['pci']
> +            addr_array = pci_bus.split(':')
> +            bus_id = addr_array[0]
> +            devfun_id = addr_array[1]
> +            intf = self.get_interface_name(bus_id, devfun_id)
> +            if "No such file" in intf:
> +                self.logger.info("DUT: [0000:%s] %s" % (pci_bus, unknow_interface))
> +            out = self.send_expect("ip link show %s" % intf, "# ")
> +            if "DOWN" in out:
> +                self.send_expect("ip link set %s up" % intf, "# ")
> +                time.sleep(5)
> +            macaddr = self.get_mac_addr(intf, bus_id, devfun_id)
> +            out = self.send_expect("ip -family inet6 address show dev %s | awk '/inet6/ { print $2 }'"
> +                                   % intf, "# ")
> +            ipv6 = out.split('/')[0]
> +            # Unconnected ports don't have IPv6
> +            if ":" not in ipv6:
> +                ipv6 = "Not connected"
> +
> +            port_info['mac'] = macaddr
> +            port_info['intf'] = intf
> +            port_info['ipv6'] = ipv6
> +
>      def scan_ports(self):
>          """
>          Scan ports information or just read it from cache file.
> @@ -411,8 +501,7 @@ class Dut(Crb):
>          unknow_interface = dts.RED('Skipped: unknow_interface')
>  
>          for (pci_bus, pci_id) in self.pci_devices_info:
> -
> -            if not dts.accepted_nic(pci_id):
> +            if self.check_ports_available(pci_bus, pci_id) is False:
>                  self.logger.info("DUT: [000:%s %s] %s" % (pci_bus, pci_id,
>                                                            skipped))
>                  continue
> @@ -420,50 +509,10 @@ class Dut(Crb):
>              addr_array = pci_bus.split(':')
>              bus_id = addr_array[0]
>              devfun_id = addr_array[1]
> -            self.send_expect("echo 0000:%s > /sys/bus/pci/drivers/igb_uio/unbind" % pci_bus, "# ")
> -            if pci_id == '8086:10fb':
> -                self.send_expect("echo 0000:%s > /sys/bus/pci/drivers/ixgbe/bind" % pci_bus, "# ")
> -            intf = self.get_interface_name(bus_id, devfun_id)
> -
> -            out = self.send_expect("ip link show %s" % intf, "# ")
> -            if "DOWN" in out:
> -                self.send_expect("ip link set %s up" % intf, "# ")
> -                """
> -                I217 and I218 requires more time to acquire V6 address.
> -                """
> -                time.sleep(25)
> -
> -            self.logger.info("DUT: [000:%s %s] %s" % (pci_bus,
> -                                                      pci_id,
> -                                                      intf))
> -
> -            if "No such file" in intf:
> -                self.logger.info("DUT: [000:%s %s] %s" % (pci_bus, pci_id,
> -                                                          unknow_interface))
> -                continue
> -
> -            macaddr = self.get_mac_addr(intf, bus_id, devfun_id)
> -
> -            out = self.send_expect("ip -family inet6 address show dev %s | awk '/inet6/ { print $2 }'"
> -                                   % intf, "# ")
> -            ipv6 = out.split('/')[0]
> -
> -            # Unconnected ports don't have IPv6
> -            if ":" not in ipv6:
> -                ipv6 = "Not connected"
> -
> -            numa = self.send_expect("cat /sys/bus/pci/devices/0000\:%s\:%s/numa_node" %
> -                                    (bus_id, devfun_id), "# ")
> -
> -            try:
> -                numa = int(numa)
> -            except ValueError:
> -                numa = -1
> -                self.logger.warning("NUMA not available")
>  
> +            numa = self.get_device_numa(bus_id, devfun_id)
>              # store the port info to port mapping
> -            self.ports_info.append({'pci': pci_bus, 'type': pci_id, 'intf':
> -                                    intf, 'mac': macaddr, 'ipv6': ipv6, 'numa': numa})
> +            self.ports_info.append({'pci': pci_bus, 'type': pci_id, 'numa': numa})
>  
>      def scan_ports_uncached_freebsd(self):
>          """
> @@ -505,3 +554,22 @@ class Dut(Crb):
>              # store the port info to port mapping
>              self.ports_info.append({'pci': pci_str, 'type': pci_id, 'intf':
>                                      intf, 'mac': macaddr, 'ipv6': ipv6, 'numa': -1})
> +
> +    def load_portconf(self):
> +        """
> +        Load port configurations for ports_info. If manually configured infor
> +        not same as auto scanned, still use infor in configuration file.
> +        """
> +        for port in self.ports_info:
> +            pci_bus = port['pci']
> +            if pci_bus in self.conf.ports_cfg:
> +                port_cfg = self.conf.ports_cfg[pci_bus]
> +                port_cfg['source'] = 'cfg'
> +            else:
> +                port_cfg = {}
> +
> +            for key in ['intf', 'mac', 'numa', 'peer']:
> +                if key in port_cfg:
> +                    if key in port and port_cfg[key] != port[key]:
> +                        self.logger.warning("CONGGURED %s NOT SAME AS SCANNED!!!" % (key.upper()))
> +                    port[key] = port_cfg[key]
> diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
> index 3577eb7..8cb144f 100644
> --- a/framework/project_dpdk.py
> +++ b/framework/project_dpdk.py
> @@ -236,13 +236,10 @@ class DPDKdut(Dut):
>          binding_list = '--bind=%s ' % driver
>  
>          current_nic = 0
> -        for (pci_bus, pci_id) in self.pci_devices_info:
> -            if dts.accepted_nic(pci_id):
> -
> -                if nics_to_bind is None or current_nic in nics_to_bind:
> -                    binding_list += '%s ' % (pci_bus)
> -
> -                current_nic += 1
> +        for port_info in self.ports_info:
> +            if nics_to_bind is None or current_nic in nics_to_bind:
> +                binding_list += '%s ' % (port_info['pci'])
> +            current_nic += 1
>  
>          self.send_expect('tools/dpdk_nic_bind.py %s' % binding_list, '# ')
>  
> @@ -254,13 +251,10 @@ class DPDKdut(Dut):
>          binding_list = '-u '
>  
>          current_nic = 0
> -        for (pci_bus, pci_id) in self.pci_devices_info:
> -            if dts.accepted_nic(pci_id):
> -
> -                if nics_to_bind is None or current_nic in nics_to_bind:
> -                    binding_list += '%s ' % (pci_bus)
> -
> -                current_nic += 1
> +        for port_info in self.ports_info:
> +            if nics_to_bind is None or current_nic in nics_to_bind:
> +                binding_list += '%s ' % (port_info['pci'])
> +            current_nic += 1
>  
>          self.send_expect('tools/dpdk_nic_bind.py %s' % binding_list, '# ', 30)
>  
> diff --git a/framework/ssh_connection.py b/framework/ssh_connection.py
> index 4306162..25f249a 100644
> --- a/framework/ssh_connection.py
> +++ b/framework/ssh_connection.py
> @@ -40,7 +40,7 @@ class SSHConnection(object):
>      Implement send_expect/copy function upper SSHPexpet module.
>      """
>  
> -    def __init__(self, host, session_name, password = ''):
> +    def __init__(self, host, session_name, password=''):
>          self.session = SSHPexpect(host, USERNAME, password)
>          self.name = session_name
>  
> diff --git a/framework/tester.py b/framework/tester.py
> index 0ebe29a..437d8c4 100644
> --- a/framework/tester.py
> +++ b/framework/tester.py
> @@ -133,7 +133,7 @@ class Tester(Crb):
>          self.restore_interfaces()
>          self.scan_ports()
>          self.map_available_ports()
> -        if self.ports_map == None or len(self.ports_map) == 0:
> +        if self.ports_map is None or len(self.ports_map) == 0:
>              raise ValueError("ports_map should not be empty, please check all links")
>  
>      def get_local_port(self, remotePort):
> @@ -179,6 +179,24 @@ class Tester(Crb):
>          else:
>              return 'down'
>  
> +    def restore_interfaces(self):
> +        """
> +        Restore Linux interfaces.
> +        """
> +        self.send_expect("modprobe igb", "# ", 20)
> +        self.send_expect("modprobe ixgbe", "# ", 20)
> +        self.send_expect("modprobe e1000e", "# ", 20)
> +        self.send_expect("modprobe e1000", "# ", 20)
> +
> +        try:
> +            for (pci_bus, pci_id) in self.pci_devices_info:
> +                addr_array = pci_bus.split(':')
> +                itf = self.get_interface_name(addr_array[0], addr_array[1])
> +                self.send_expect("ifconfig %s up" % itf, "# ")
> +
> +        except Exception as e:
> +            self.logger.error("   !!! Restore ITF: " + e.message)
> +
>      def scan_ports(self):
>          """
>          Scan all ports on tester and save port's pci/mac/interface.
> @@ -262,6 +280,18 @@ class Tester(Crb):
>          hits = [False] * len(self.ports_info)
>  
>          for dutPort in range(nrPorts):
> +            peer = self.dut.get_peer_pci(dutPort)
> +            if peer is not None:
> +                for localPort in range(len(self.ports_info)):
> +                    if self.ports_info[localPort]['pci'] == peer:
> +                        hits[localPort] = True
> +                        self.ports_map[dutPort] = localPort
> +                        break
> +                if self.ports_map[dutPort] == -1:
> +                    self.logger.error("CONFIGURED TESTER PORT CANNOT FOUND!!!")
> +                else:
> +                    continue  # skip ping6 map
> +
>              for localPort in range(len(self.ports_info)):
>                  if hits[localPort]:
>                      continue
Acked-by: Michael Qiu <michael.qiu@intel.com>

  reply	other threads:[~2015-02-04  6:59 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-02-04  6:43 [dts] [PATCH v2 0/4] Support additional port configuration file Yong Liu
2015-02-04  6:43 ` [dts] [PATCH v2 1/4] framework: add new module for load " Yong Liu
2015-02-04  6:47   ` Qiu, Michael
2015-02-04  6:43 ` [dts] [PATCH v2 2/4] framework: execuction file support port config nic_type Yong Liu
2015-02-04  7:07   ` Qiu, Michael
2015-02-04  6:43 ` [dts] [PATCH v2 3/4] framework: reorganize DUT and Tester port initialize sequence Yong Liu
2015-02-04  6:59   ` Qiu, Michael [this message]
2015-02-04  6:43 ` [dts] [PATCH v2 4/4] suites: remove nic type check from testsuites Yong Liu
2015-02-04  6:50   ` Qiu, Michael
2015-02-05  1:32 ` [dts] [PATCH v2 0/4] Support additional port configuration file Liu, Yong
2015-02-05  2:18 ` [dts] [PATCH] framework: add port config pci bus id check suites: seperated checksum_offload function and performance port requirements Yong Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=533710CFB86FA344BFBF2D6802E60286CD4647@SHSMSX101.ccr.corp.intel.com \
    --to=michael.qiu@intel.com \
    --cc=dts@dpdk.org \
    --cc=yong.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).