[dts] framework v2: hugepages, cpuinfo, connect x3 support and bug fixes

[dts] framework v2: hugepages, cpuinfo, connect x3 support and bug fixes

[Gowrishankar]

Following are set of patches addressing below:

1. hugepages setting and cpu info parsing for powerpc environment in a platform
   independent way.
2. enable connect x3 support to run DTS.
3. some bug fixes in framework as well as tests handling numa, cpu coremask
   and expected output.
   
Changes from version 1:
 - interface name for second port sharing same PCI id is stored in variable 'intf2'
   instead 'peer' as earlier, also relavant function names.
 - check interface name and mac retrieved from DUT scan.
 

Thanks,
Gowrishankar

[dts] [PATCH v2 1/9] framework: check hugepage size and add pages

[Gowrishankar]

Add number of huge pages according to enabled huge page size. Also, added
expected number of pages in case ppc_64 is DUT platform.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 framework/crb.py |   10 ++++++----
 framework/dut.py |    2 ++
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index f2009b9..c6fd9fb 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -161,18 +161,20 @@ class Crb(object):
         """
         Set numbers of huge pages
         """
+        page_size = self.send_expect("awk '/Hugepagesize/ {print $2}' /proc/meminfo", "# ")
+
         if numa == -1:
-            self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages' % huge_pages, '# ', 5)
+            self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages, page_size), '# ', 5)
         else:
             #sometimes we set hugepage on kernel cmdline, so we need clear default hugepage
-            self.send_expect('echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages', '# ', 5)
+            self.send_expect('echo 0 > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (page_size), '# ', 5)
             
             #some platform not support numa, example vm dut
             try:
-                self.send_expect('echo %d > /sys/devices/system/node/node%d/hugepages/hugepages-2048kB/nr_hugepages' % (huge_pages, numa), '# ', 5)
+                self.send_expect('echo %d > /sys/devices/system/node/node%d/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages, numa, page_size), '# ', 5)
             except:
                 self.logger.warning("set %d hugepage on socket %d error" % (huge_pages, numa))
-                self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages' % huge_pages, '# ', 5)
+                self.send_expect('echo %d > /sys/kernel/mm/hugepages/hugepages-%skB/nr_hugepages' % (huge_pages. page_size), '# ', 5)
 
     def set_speedup_options(self, read_cache, skip_setup):
         """
diff --git a/framework/dut.py b/framework/dut.py
index bd437cb..c57aa79 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -301,6 +301,8 @@ class Dut(Crb):
             elif self.architecture == "x86_x32":
                 arch_huge_pages = hugepages if hugepages > 0 else 256
                 force_socket = True
+            elif self.architecture == "ppc_64":
+                arch_huge_pages = hugepages if hugepages > 0 else 512
 
             if total_huge_pages != arch_huge_pages:
                  # before all hugepage average distribution  by all socket,
-- 
1.7.10.4

[dts] [PATCH v2 2/9] framework: platform independent cpu info parsing

[Gowrishankar]

To collect thread/core/socket, /proc/cpuinfo would not help in case of powerpc.
Instead, lscpu seems to be a better alternative and platform neutral approach.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 framework/crb.py |   40 ++++++++++++----------------------------
 1 file changed, 12 insertions(+), 28 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index c6fd9fb..1711f37 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -495,41 +495,25 @@ class Crb(object):
 
         cpuinfo = \
             self.send_expect(
-                "grep --color=never \"processor\\|physical id\\|core id\\|^$\" /proc/cpuinfo",
+                "lscpu -p|grep -v \#",
                 "#", alt_session=True)
 
-        if "processor" not in cpuinfo:              
-            # yocto not support --color=never, but ubuntu must need --color=never, 
-            # so check cpuinfo, before parsing cpuinfo, if cpuifo get error, delete --color=never
-            # and get cpuinfo again
-            cpuinfo = \
-                self.send_expect(
-                    r'grep "processor\|physical id\|core id\|^$" /proc/cpuinfo',
-                    "#", alt_session=True)
-
-        cpuinfo = cpuinfo.split('\r\n\r\n')
+        cpuinfo = cpuinfo.split()
         # haswell cpu on cottonwood core id not correct
         # need addtional coremap for haswell cpu
         core_id = 0
         coremap = {}
         for line in cpuinfo:
-            m = re.search("processor\t: (\d+)\r\n" +
-                          "physical id\t: (\d+)\r\n" +
-                          "core id\t\t: (\d+)", line)
-
-            if m:
-                thread = m.group(1)
-                socket = m.group(2)
-                core = m.group(3)
-
-                if core not in coremap.keys():
-                    coremap[core] = core_id
-                    core_id += 1
-
-                if self.crb['bypass core0'] and core == '0' and socket == '0':
-                    self.logger.info("Core0 bypassed")
-                    continue
-                self.cores.append(
+            (thread, core, socket) = line.split(',')[0:3]
+
+            if core not in coremap.keys():
+                coremap[core] = core_id
+                core_id += 1
+
+            if self.crb['bypass core0'] and core == '0' and socket == '0':
+                self.logger.info("Core0 bypassed")
+                continue
+            self.cores.append(
                     {'thread': thread, 'socket': socket, 'core': coremap[core]})
 
         self.number_of_cores = len(self.cores)
-- 
1.7.10.4

[dts] [PATCH v2 3/9] framework: include domain id in pci tuple

[Gowrishankar]

Sometimes, PCI domain id can not always 0x0000 as it is hardcoded in framework.
This patch adds changes in API to also include pci domain id while handling a
network device.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 framework/config.py       |    2 +-
 framework/crb.py          |   34 ++++-----
 framework/dut.py          |   42 ++++++-----
 framework/project_dpdk.py |    2 +-
 framework/tester.py       |   19 ++---
 framework/virt_dut.py     |    7 +-
 nics/net_device.py        |  183 +++++++++++++++++++++++----------------------
 7 files changed, 150 insertions(+), 139 deletions(-)

diff --git a/framework/config.py b/framework/config.py
index 10624f0..6fd3630 100644
--- a/framework/config.py
+++ b/framework/config.py
@@ -130,7 +130,7 @@ class PortConf(UserConf):
     def __init__(self, port_conf=PORTCONF):
         self.config_file = port_conf
         self.ports_cfg = {}
-        self.pci_regex = "([\da-f]{2}:[\da-f]{2}.\d{1})$"
+        self.pci_regex = "([\da-f]{4}:[\da-f]{2}:[\da-f]{2}.\d{1})$"
         try:
             self.port_conf = UserConf(self.config_file)
         except ConfigParseException:
diff --git a/framework/crb.py b/framework/crb.py
index 1711f37..453796f 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -237,8 +237,8 @@ class Crb(object):
         Look for the NIC's information (PCI Id and card type).
         """
         out = self.send_expect(
-            "lspci -nn | grep -i eth", "# ", alt_session=True)
-        rexp = r"([\da-f]{2}:[\da-f]{2}.\d{1}) .*Eth.*?ernet .*?([\da-f]{4}:[\da-f]{4})"
+            "lspci -Dnn | grep -i eth", "# ", alt_session=True)
+        rexp = r"([\da-f]{4}:[\da-f]{2}:[\da-f]{2}.\d{1}) .*Eth.*?ernet .*?([\da-f]{4}:[\da-f]{4})"
         pattern = re.compile(rexp)
         match = pattern.findall(out)
         self.pci_devices_info = []
@@ -259,20 +259,20 @@ class Crb(object):
             card_type = "8086:%s" % match[i][1]
             self.pci_devices_info.append((match[i][0], card_type))
 
-    def get_pci_dev_driver(self, bus_id, devfun_id):
+    def get_pci_dev_driver(self, domain_id, bus_id, devfun_id):
         """
         Get the driver of specified pci device.
         """
         get_pci_dev_driver = getattr(
             self, 'get_pci_dev_driver_%s' % self.get_os_type())
-        return get_pci_dev_driver(bus_id, devfun_id)
+        return get_pci_dev_driver(domain_id, bus_id, devfun_id)
 
-    def get_pci_dev_driver_linux(self, bus_id, devfun_id):
+    def get_pci_dev_driver_linux(self, domain_id, bus_id, devfun_id):
         """
         Get the driver of specified pci device on linux.
         """
-        out = self.send_expect("cat /sys/bus/pci/devices/0000\:%s\:%s/uevent" %
-                               (bus_id, devfun_id), "# ", alt_session=True)
+        out = self.send_expect("cat /sys/bus/pci/devices/%s\:%s\:%s/uevent" %
+                               (domain_id, bus_id, devfun_id), "# ", alt_session=True)
         rexp = r"DRIVER=(.+?)\r"
         pattern = re.compile(rexp)
         match = pattern.search(out)
@@ -286,20 +286,20 @@ class Crb(object):
         """
         return True
 
-    def get_pci_dev_id(self, bus_id, devfun_id):
+    def get_pci_dev_id(self, domain_id, bus_id, devfun_id):
         """
         Get the pci id of specified pci device.
         """
         get_pci_dev_id = getattr(
             self, 'get_pci_dev_id_%s' % self.get_os_type())
-        return get_pci_dev_id(bus_id, devfun_id)
+        return get_pci_dev_id(domain_id, bus_id, devfun_id)
 
-    def get_pci_dev_id_linux(self, bus_id, devfun_id):
+    def get_pci_dev_id_linux(self, domain_id, bus_id, devfun_id):
         """
         Get the pci id of specified pci device on linux.
         """
-        out = self.send_expect("cat /sys/bus/pci/devices/0000\:%s\:%s/uevent" %
-                               (bus_id, devfun_id), "# ", alt_session=True)
+        out = self.send_expect("cat /sys/bus/pci/devices/%s\:%s\:%s/uevent" %
+                               (domain_id, bus_id, devfun_id), "# ", alt_session=True)
         rexp = r"PCI_ID=(.+)"
         pattern = re.compile(rexp)
         match = re.search(out)
@@ -307,21 +307,21 @@ class Crb(object):
             return None
         return match.group(1)
 
-    def get_device_numa(self, bus_id, devfun_id):
+    def get_device_numa(self, domain_id, bus_id, devfun_id):
         """
         Get numa number of specified pci device.
         """
         get_device_numa = getattr(
             self, "get_device_numa_%s" % self.get_os_type())
-        return get_device_numa(bus_id, devfun_id)
+        return get_device_numa(domain_id, bus_id, devfun_id)
 
-    def get_device_numa_linux(self, bus_id, devfun_id):
+    def get_device_numa_linux(self, domain_id, bus_id, devfun_id):
         """
         Get numa number of specified pci device on Linux.
         """
         numa = self.send_expect(
-            "cat /sys/bus/pci/devices/0000\:%s\:%s/numa_node" %
-            (bus_id, devfun_id), "# ", alt_session=True)
+            "cat /sys/bus/pci/devices/%s\:%s\:%s/numa_node" %
+            (domain_id, bus_id, devfun_id), "# ", alt_session=True)
 
         try:
             numa = int(numa)
diff --git a/framework/dut.py b/framework/dut.py
index c57aa79..ff40fac 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -237,9 +237,10 @@ class Dut(Crb):
             if driver is not None:
                 # unbind device driver
                 addr_array = pci_bus.split(':')
-                bus_id = addr_array[0]
-                devfun_id = addr_array[1]
-                port = GetNicObj(self, bus_id, devfun_id)
+                domain_id = addr_array[0]
+                bus_id = addr_array[1]
+                devfun_id = addr_array[2]
+                port = GetNicObj(self, domain_id, bus_id, devfun_id)
                 port.stop()
 
     def restore_interfaces_linux(self):
@@ -254,16 +255,17 @@ class Dut(Crb):
             if driver is not None:
                 # unbind device driver
                 addr_array = pci_bus.split(':')
-                bus_id = addr_array[0]
-                devfun_id = addr_array[1]
+                domain_id = addr_array[0]
+                bus_id = addr_array[1]
+                devfun_id = addr_array[2]
 
-                port = GetNicObj(self, bus_id, devfun_id)
+                port = GetNicObj(self, domain_id, bus_id, devfun_id)
 
-                self.send_expect('echo 0000:%s > /sys/bus/pci/devices/0000\:%s\:%s/driver/unbind'
-                                 % (pci_bus, bus_id, devfun_id), '# ')
+                self.send_expect('echo %s > /sys/bus/pci/devices/%s\:%s\:%s/driver/unbind'
+                                 % (pci_bus, domain_id, bus_id, devfun_id), '# ')
                 # bind to linux kernel driver
                 self.send_expect('modprobe %s' % driver, '# ')
-                self.send_expect('echo 0000:%s > /sys/bus/pci/drivers/%s/bind'
+                self.send_expect('echo %s > /sys/bus/pci/drivers/%s/bind'
                                  % (pci_bus, driver), '# ')
                 itf = port.get_interface_name()
                 self.send_expect("ifconfig %s up" % itf, "# ")
@@ -569,7 +571,7 @@ class Dut(Crb):
             port = port_info['port']
             intf = port.get_interface_name()
             if "No such file" in intf:
-                self.logger.info("DUT: [0000:%s] %s" % (pci_bus, unknow_interface))
+                self.logger.info("DUT: [%s] %s" % (pci_bus, unknow_interface))
                 continue
             out = self.send_expect("ip link show %s" % intf, "# ")
             if "DOWN" in out:
@@ -594,7 +596,7 @@ class Dut(Crb):
             port = port_info['port']
             intf = port.get_interface_name()
             if "No such file" in intf:
-                self.logger.info("DUT: [0000:%s] %s" % (pci_bus, unknow_interface))
+                self.logger.info("DUT: [%s] %s" % (pci_bus, unknow_interface))
                 continue
             self.send_expect("ifconfig %s up" % intf, "# ")
             time.sleep(5)
@@ -654,7 +656,7 @@ class Dut(Crb):
             port = GetNicObj(self, port_info['pci'], port_info['type'])
             intf = port.get_interface_name()
 
-            self.logger.info("DUT cached: [000:%s %s] %s" % (port_info['pci'],
+            self.logger.info("DUT cached: [%s %s] %s" % (port_info['pci'],
                              port_info['type'], intf))
 
             port_info['port'] = port
@@ -677,15 +679,16 @@ class Dut(Crb):
 
         for (pci_bus, pci_id) in self.pci_devices_info:
             if self.check_ports_available(pci_bus, pci_id) is False:
-                self.logger.info("DUT: [000:%s %s] %s" % (pci_bus, pci_id,
+                self.logger.info("DUT: [%s %s] %s" % (pci_bus, pci_id,
                                                           skipped))
                 continue
 
             addr_array = pci_bus.split(':')
-            bus_id = addr_array[0]
-            devfun_id = addr_array[1]
+            domain_id = addr_array[0]
+            bus_id = addr_array[1]
+            devfun_id = addr_array[2]
 
-            port = GetNicObj(self, bus_id, devfun_id)
+            port = GetNicObj(self, domain_id, bus_id, devfun_id)
             numa = port.socket
             # store the port info to port mapping
             self.ports_info.append(
@@ -758,9 +761,10 @@ class Dut(Crb):
         vfs_port = []
         for vf_pci in sriov_vfs_pci:
             addr_array = vf_pci.split(':')
-            bus_id = addr_array[0]
-            devfun_id = addr_array[1]
-            vf_port = GetNicObj(self, bus_id, devfun_id)
+            domain_id = addr_array[0]
+            bus_id = addr_array[1]
+            devfun_id = addr_array[2]
+            vf_port = GetNicObj(self, domain_id, bus_id, devfun_id)
             vfs_port.append(vf_port)
         self.ports_info[port_id]['vfs_port'] = vfs_port
 
diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
index a96aba8..383aea3 100644
--- a/framework/project_dpdk.py
+++ b/framework/project_dpdk.py
@@ -340,7 +340,7 @@ class DPDKdut(Dut):
         self.bind_interfaces_linux()
         for port in range(0, len(self.ports_info)):
             if(port not in dutPorts):
-                blacklist += '-b 0000:%s ' % self.ports_info[port]['pci']
+                blacklist += '-b %s ' % self.ports_info[port]['pci']
         return blacklist
 
     def get_blacklist_string_freebsd(self, target, nic):
diff --git a/framework/tester.py b/framework/tester.py
index 4944da3..1d69355 100644
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -203,7 +203,7 @@ class Tester(Crb):
         try:
             for (pci_bus, pci_id) in self.pci_devices_info:
                 addr_array = pci_bus.split(':')
-                port = GetNicObj(self, addr_array[0], addr_array[1])
+                port = GetNicObj(self, addr_array[0], addr_array[1], addr_array[2])
                 itf = port.get_interface_name()
                 self.enable_ipv6(itf)
                 self.send_expect("ifconfig %s up" % itf, "# ")
@@ -217,7 +217,7 @@ class Tester(Crb):
         try:
             for (pci_bus, pci_id) in self.pci_devices_info:
                 addr_array = pci_bus.split(':')
-                port = GetNicObj(self, addr_array[0], addr_array[1])
+                port = GetNicObj(self, addr_array[0], addr_array[1], addr_array[2])
                 itf = port.get_interface_name()
                 self.enable_promisc(itf)
         except Exception as e:
@@ -284,23 +284,24 @@ class Tester(Crb):
         for (pci_bus, pci_id) in self.pci_devices_info:
             # ignore unknown card types
             if pci_id not in NICS.values():
-                self.logger.info("Tester: [000:%s %s] %s" % (pci_bus, pci_id,
+                self.logger.info("Tester: [%s %s] %s" % (pci_bus, pci_id,
                                                              "unknow_nic"))
                 continue
 
             addr_array = pci_bus.split(':')
-            bus_id = addr_array[0]
-            devfun_id = addr_array[1]
+            domain_id = addr_array[0]
+            bus_id = addr_array[1]
+            devfun_id = addr_array[2]
 
-            port = GetNicObj(self, bus_id, devfun_id)
+            port = GetNicObj(self, domain_id, bus_id, devfun_id)
             intf = port.get_interface_name()
 
             if "No such file" in intf:
-                self.logger.info("Tester: [000:%s %s] %s" % (pci_bus, pci_id,
+                self.logger.info("Tester: [%s %s] %s" % (pci_bus, pci_id,
                                                              "unknow_interface"))
                 continue
 
-            self.logger.info("Tester: [000:%s %s] %s" % (pci_bus, pci_id, intf))
+            self.logger.info("Tester: [%s %s] %s" % (pci_bus, pci_id, intf))
             macaddr = port.get_mac_addr()
 
             ipv6 = port.get_ipv6_addr()
@@ -327,7 +328,7 @@ class Tester(Crb):
         Return tester local port numa.
         """
         pci = self.ports_info[port]['pci']
-        out = self.send_expect("cat /sys/bus/pci/devices/0000:%s/numa_node" % pci, "#")
+        out = self.send_expect("cat /sys/bus/pci/devices/%s/numa_node" % pci, "#")
         return int(out)
 
     def check_port_list(self, portList, ftype='normal'):
diff --git a/framework/virt_dut.py b/framework/virt_dut.py
index 8b366c9..43d9b57 100644
--- a/framework/virt_dut.py
+++ b/framework/virt_dut.py
@@ -248,9 +248,10 @@ class VirtDut(DPDKdut):
             driver = settings.get_nic_driver(pci_id)
             if driver is not None:
                 addr_array = pci_bus.split(':')
-                bus_id = addr_array[0]
-                devfun_id = addr_array[1]
-                port = GetNicObj(self, bus_id, devfun_id)
+                domain_id = addr_array[0]
+                bus_id = addr_array[1]
+                devfun_id = addr_array[2]
+                port = GetNicObj(self, domain_id, bus_id, devfun_id)
                 itf = port.get_interface_name()
                 self.send_expect("ifconfig %s up" % itf, "# ")
                 time.sleep(30)
diff --git a/nics/net_device.py b/nics/net_device.py
index 73750f5..a9127cb 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -52,14 +52,15 @@ class NetDevice(object):
     Abstract the device which is PF or VF.
     """
 
-    def __init__(self, crb, bus_id, devfun_id):
+    def __init__(self, crb, domain_id, bus_id, devfun_id):
         if not isinstance(crb, Crb):
             raise Exception("  Please input the instance of Crb!!!")
         self.crb = crb
+        self.domain_id = domain_id
         self.bus_id = bus_id
         self.devfun_id = devfun_id
-        self.pci = bus_id + ':' + devfun_id
-        self.pci_id = get_pci_id(crb, bus_id, devfun_id)
+        self.pci = domain_id + ':' + bus_id + ':' + devfun_id
+        self.pci_id = get_pci_id(crb, domain_id, bus_id, devfun_id)
         self.default_driver = settings.get_nic_driver(self.pci_id)
 
         if self.nic_is_pf():
@@ -111,7 +112,7 @@ class NetDevice(object):
         """
         Get the NIC driver.
         """
-        return self.crb.get_pci_dev_driver(self.bus_id, self.devfun_id)
+        return self.crb.get_pci_dev_driver(self.domain_id, self.bus_id, self.devfun_id)
 
     def get_nic_socket(self):
         """
@@ -120,11 +121,11 @@ class NetDevice(object):
         get_nic_socket = getattr(
             self, 'get_nic_socket_%s' %
             self.__get_os_type())
-        return get_nic_socket(self.bus_id, self.devfun_id)
+        return get_nic_socket(self.domain_id, self.bus_id, self.devfun_id)
 
-    def get_nic_socket_linux(self, bus_id, devfun_id):
-        command = ('cat /sys/bus/pci/devices/0000\:%s\:%s/numa_node' %
-                   (bus_id, devfun_id))
+    def get_nic_socket_linux(self, domain_id, bus_id, devfun_id):
+        command = ('cat /sys/bus/pci/devices/%s\:%s\:%s/numa_node' %
+                   (domain_id, bus_id, devfun_id))
         try:
             out = self.__send_expect(command, '# ')
             socket = int(out)
@@ -144,7 +145,7 @@ class NetDevice(object):
         get_interface_name = getattr(
             self, 'get_interface_name_%s' %
             self.__get_os_type())
-        out = get_interface_name(self.bus_id, self.devfun_id, self.current_driver)
+        out = get_interface_name(self.domain_id, self.bus_id, self.devfun_id, self.current_driver)
         if "No such file or directory" in out:
             self.intf_name = 'N/A'
         else:
@@ -152,7 +153,7 @@ class NetDevice(object):
 
         return self.intf_name
 
-    def get_interface_name_linux(self, bus_id, devfun_id, driver):
+    def get_interface_name_linux(self, domain_id, bus_id, devfun_id, driver):
         """
         Get interface name of specified pci device on linux.
         """
@@ -167,22 +168,22 @@ class NetDevice(object):
             get_interface_name_linux = getattr(self,
                                                'get_interface_name_linux_%s' % generic_driver)
 
-        return get_interface_name_linux(bus_id, devfun_id)
+        return get_interface_name_linux(domain_id, bus_id, devfun_id)
 
-    def get_interface_name_linux_virtio_pci(self, bus_id, devfun_id):
+    def get_interface_name_linux_virtio_pci(self, domain_id, bus_id, devfun_id):
         """
         Get virtio device interface name by the default way on linux.
         """
-        command = 'ls --color=never /sys/bus/pci/devices/0000\:%s\:%s/virtio*/net' % (
-            bus_id, devfun_id)
+        command = 'ls --color=never /sys/bus/pci/devices/%s\:%s\:%s/virtio*/net' % (
+            domain_id, bus_id, devfun_id)
         return self.__send_expect(command, '# ')
 
-    def get_interface_name_linux_generic(self, bus_id, devfun_id):
+    def get_interface_name_linux_generic(self, domain_id, bus_id, devfun_id):
         """
         Get the interface name by the default way on linux.
         """
-        command = 'ls --color=never /sys/bus/pci/devices/0000\:%s\:%s/net' % (
-            bus_id, devfun_id)
+        command = 'ls --color=never /sys/bus/pci/devices/%s\:%s\:%s/net' % (
+            domain_id, bus_id, devfun_id)
         return self.__send_expect(command, '# ')
 
     def get_interface_name_freebsd(self, bus_id, devfun_id, driver):
@@ -235,13 +236,13 @@ class NetDevice(object):
         Get mac address of specified pci device.
         """
         get_mac_addr = getattr(self, 'get_mac_addr_%s' % self.__get_os_type())
-        out = get_mac_addr(self.intf_name, self.bus_id, self.devfun_id, self.current_driver)
+        out = get_mac_addr(self.intf_name, self.domain_id, self.bus_id, self.devfun_id, self.current_driver)
         if "No such file or directory" in out:
             return 'N/A'
         else:
             return out
 
-    def get_mac_addr_linux(self, intf, bus_id, devfun_id, driver):
+    def get_mac_addr_linux(self, intf, domain_id, bus_id, devfun_id, driver):
         """
         Get mac address of specified pci device on linux.
         """
@@ -258,26 +259,26 @@ class NetDevice(object):
                 'get_mac_addr_linux_%s' %
                 generic_driver)
 
-        return get_mac_addr_linux(intf, bus_id, devfun_id, driver)
+        return get_mac_addr_linux(intf, domain_id, bus_id, devfun_id, driver)
 
-    def get_mac_addr_linux_generic(self, intf, bus_id, devfun_id, driver):
+    def get_mac_addr_linux_generic(self, intf, domain_id, bus_id, devfun_id, driver):
         """
         Get MAC by the default way on linux.
         """
-        command = ('cat /sys/bus/pci/devices/0000\:%s\:%s/net/%s/address' %
-                   (bus_id, devfun_id, intf))
+        command = ('cat /sys/bus/pci/devices/%s\:%s\:%s/net/%s/address' %
+                   (domain_id, bus_id, devfun_id, intf))
         return self.__send_expect(command, '# ')
 
-    def get_mac_addr_linux_virtio_pci(self, intf, bus_id, devfun_id, driver):
+    def get_mac_addr_linux_virtio_pci(self, intf, domain_id, bus_id, devfun_id, driver):
         """
         Get MAC by the default way on linux.
         """
-        virtio_cmd = ('ls /sys/bus/pci/devices/0000\:%s\:%s/ | grep --color=never virtio' %
-                      (bus_id, devfun_id))
+        virtio_cmd = ('ls /sys/bus/pci/devices/%s\:%s\:%s/ | grep --color=never virtio' %
+                      (domain_id, bus_id, devfun_id))
         virtio = self.__send_expect(virtio_cmd, '# ')
 
-        command = ('cat /sys/bus/pci/devices/0000\:%s\:%s/%s/net/%s/address' %
-                   (bus_id, devfun_id, virtio, intf))
+        command = ('cat /sys/bus/pci/devices/%s\:%s\:%s/%s/net/%s/address' %
+                   (domain_id, bus_id, devfun_id, virtio, intf))
         return self.__send_expect(command, '# ')
 
     def get_mac_addr_freebsd(self, intf, bus_id, devfun_id, driver):
@@ -329,7 +330,7 @@ class NetDevice(object):
                 self, 'get_ipv4_linux_%s' %
                 generic_driver)
 
-        return get_ipv4_addr_linux(intf, bus_id, devfun_id, driver)
+        return get_ipv4_addr_linux(intf, domain_id, bus_id, devfun_id, driver)
 
     def get_ipv4_addr_linux_generic(self, intf):
         """
@@ -496,13 +497,13 @@ class NetDevice(object):
         """
         Get numa number of specified pci device.
         """
-        self.crb.get_nic_numa(self.bus_id, self.devfun_id)
+        self.crb.get_nic_numa(self.domain_id, self.bus_id, self.devfun_id)
 
     def get_card_type(self):
         """
         Get card type of specified pci device.
         """
-        return self.crb.get_pci_dev_id(self.bus_id, self.devfun_id)
+        return self.crb.get_pci_dev_id(self.domain_id, self.bus_id, self.devfun_id)
 
     @nic_has_driver
     def get_sriov_vfs_pci(self):
@@ -511,9 +512,9 @@ class NetDevice(object):
         """
         get_sriov_vfs_pci = getattr(
             self, 'get_sriov_vfs_pci_%s' % self.__get_os_type())
-        return get_sriov_vfs_pci(self.bus_id, self.devfun_id, self.current_driver)
+        return get_sriov_vfs_pci(self.domain_id, self.bus_id, self.devfun_id, self.current_driver)
 
-    def get_sriov_vfs_pci_linux(self, bus_id, devfun_id, driver):
+    def get_sriov_vfs_pci_linux(self, domain_id, bus_id, devfun_id, driver):
         """
         Get all SRIOV VF pci bus of specified pci device on linux.
         """
@@ -529,15 +530,15 @@ class NetDevice(object):
                 'get_sriov_vfs_pci_linux_%s' %
                 generic_driver)
 
-        return get_sriov_vfs_pci_linux(bus_id, devfun_id)
+        return get_sriov_vfs_pci_linux(domain_id, bus_id, devfun_id)
 
-    def get_sriov_vfs_pci_linux_generic(self, bus_id, devfun_id):
+    def get_sriov_vfs_pci_linux_generic(self, domain_id, bus_id, devfun_id):
         """
         Get all the VF PCIs of specified PF by the default way on linux.
         """
         sriov_numvfs = self.__send_expect(
-            "cat /sys/bus/pci/devices/0000\:%s\:%s/sriov_numvfs" %
-            (bus_id, devfun_id), "# ")
+            "cat /sys/bus/pci/devices/%s\:%s\:%s/sriov_numvfs" %
+            (domain_id, bus_id, devfun_id), "# ")
         sriov_vfs_pci = []
 
         if "No such file" in sriov_numvfs:
@@ -548,18 +549,18 @@ class NetDevice(object):
         else:
             try:
                 virtfns = self.__send_expect(
-                    "ls -d /sys/bus/pci/devices/0000\:%s\:%s/virtfn*" %
-                    (bus_id, devfun_id), "# ")
+                    "ls -d /sys/bus/pci/devices/%s\:%s\:%s/virtfn*" %
+                    (domain_id, bus_id, devfun_id), "# ")
                 for virtfn in virtfns.split():
                     vf_uevent = self.__send_expect(
                         "cat %s" %
                         os.path.join(virtfn, "uevent"), "# ")
                     vf_pci = re.search(
-                        r"PCI_SLOT_NAME=0000:([0-9a-f]+:[0-9a-f]+\.[0-9a-f]+)",
+                        r"PCI_SLOT_NAME=%s:([0-9a-f]+:[0-9a-f]+\.[0-9a-f]+)" %domain_id,
                         vf_uevent).group(1)
                     sriov_vfs_pci.append(vf_pci)
             except Exception as e:
-                print "Scan linux port [0000:%s.%s] sriov vf failed: %s" % (bus_id, devfun_id, e)
+                print "Scan linux port [%s:%s.%s] sriov vf failed: %s" % (domain_id, bus_id, devfun_id, e)
 
         return sriov_vfs_pci
 
@@ -574,6 +575,7 @@ class NetDevice(object):
             self, 'generate_sriov_vfs_%s' %
             self.__get_os_type())
         generate_sriov_vfs(
+            self.domain_id,
             self.bus_id,
             self.devfun_id,
             vf_num,
@@ -583,16 +585,17 @@ class NetDevice(object):
 
             vf_pci = self.sriov_vfs_pci[0]
             addr_array = vf_pci.split(':')
-            bus_id = addr_array[0]
-            devfun_id = addr_array[1]
+            domain_id = addr_array[0]
+            bus_id = addr_array[1]
+            devfun_id = addr_array[2]
 
             self.default_vf_driver = self.crb.get_pci_dev_driver(
-                bus_id, devfun_id)
+                domain_id, bus_id, devfun_id)
         else:
             self.sriov_vfs_pci = []
         time.sleep(1)
 
-    def generate_sriov_vfs_linux(self, bus_id, devfun_id, vf_num, driver):
+    def generate_sriov_vfs_linux(self, domain_id, bus_id, devfun_id, vf_num, driver):
         """
         Generate some numbers of SRIOV VF.
         """
@@ -608,9 +611,9 @@ class NetDevice(object):
                 'generate_sriov_vfs_linux_%s' %
                 generic_driver)
 
-        return generate_sriov_vfs_linux(bus_id, devfun_id, vf_num)
+        return generate_sriov_vfs_linux(domain_id, bus_id, devfun_id, vf_num)
 
-    def generate_sriov_vfs_linux_generic(self, bus_id, devfun_id, vf_num):
+    def generate_sriov_vfs_linux_generic(self, domain_id, bus_id, devfun_id, vf_num):
         """
         Generate SRIOV VFs by the default way on linux.
         """
@@ -620,12 +623,12 @@ class NetDevice(object):
             return None
 
         vf_reg_file = "sriov_numvfs"
-        vf_reg_path = os.path.join("/sys/bus/pci/devices/0000:%s:%s" %
-                                   (bus_id, devfun_id), vf_reg_file)
+        vf_reg_path = os.path.join("/sys/bus/pci/devices/%s:%s:%s" %
+                                   (domain_id, bus_id, devfun_id), vf_reg_file)
         self.__send_expect("echo %d > %s" %
                            (int(vf_num), vf_reg_path), "# ")
 
-    def generate_sriov_vfs_linux_igb_uio(self, bus_id, devfun_id, vf_num):
+    def generate_sriov_vfs_linux_igb_uio(self, domain_id, bus_id, devfun_id, vf_num):
         """
         Generate SRIOV VFs by the special way of igb_uio driver on linux.
         """
@@ -636,11 +639,11 @@ class NetDevice(object):
 
         vf_reg_file = "max_vfs"
         if self.default_driver == 'i40e':
-            regx_reg_path = "find /sys -name %s | grep %s:%s" % (vf_reg_file, bus_id, devfun_id)
+            regx_reg_path = "find /sys -name %s | grep %s:%s:%s" % (vf_reg_file, domain_id, bus_id, devfun_id)
             vf_reg_path = self.__send_expect(regx_reg_path, "# ")
         else:
-            vf_reg_path = os.path.join("/sys/bus/pci/devices/0000:%s:%s" %
-                                       (bus_id, devfun_id), vf_reg_file)
+            vf_reg_path = os.path.join("/sys/bus/pci/devices/%s:%s:%s" %
+                                       (domain_id, bus_id, devfun_id), vf_reg_file)
         self.__send_expect("echo %d > %s" %
                            (int(vf_num), vf_reg_path), "# ")
 
@@ -667,16 +670,18 @@ class NetDevice(object):
                 return
             for vf_pci in self.sriov_vfs_pci:
                 addr_array = vf_pci.split(':')
-                bus_id = addr_array[0]
-                devfun_id = addr_array[1]
+                domain_id = addr_array[0]
+                bus_id = addr_array[1]
+                devfun_id = addr_array[2]
 
-                bind_vf_driver(bus_id, devfun_id, driver)
+                bind_vf_driver(domain_id, bus_id, devfun_id, driver)
         else:
             addr_array = pci.split(':')
-            bus_id = addr_array[0]
-            devfun_id = addr_array[1]
+            domain_id = addr_array[0]
+            bus_id = addr_array[1]
+            devfun_id = addr_array[2]
 
-            bind_vf_driver(bus_id, devfun_id, driver)
+            bind_vf_driver(domain_id, bus_id, devfun_id, driver)
 
     def bind_driver(self, driver=''):
         """
@@ -688,11 +693,11 @@ class NetDevice(object):
                 print "Must specify a driver because default driver is NULL!"
                 return
             driver = self.default_driver
-        ret = bind_driver(self.bus_id, self.devfun_id, driver)
+        ret = bind_driver(self.domain_id, self.bus_id, self.devfun_id, driver)
         time.sleep(1)
         return ret
 
-    def bind_driver_linux(self, bus_id, devfun_id, driver):
+    def bind_driver_linux(self, domain_id, bus_id, devfun_id, driver):
         """
         Bind NIC port to specified driver on linux.
         """
@@ -702,26 +707,26 @@ class NetDevice(object):
                 self,
                 'bind_driver_linux_%s' %
                 driver_alias)
-            return bind_driver_linux(bus_id, devfun_id)
+            return bind_driver_linux(domain_id, bus_id, devfun_id)
         except Exception as e:
             driver_alias = 'generic'
             bind_driver_linux = getattr(
                 self,
                 'bind_driver_linux_%s' %
                 driver_alias)
-            return bind_driver_linux(bus_id, devfun_id, driver)
+            return bind_driver_linux(domain_id, bus_id, devfun_id, driver)
 
-    def bind_driver_linux_generic(self, bus_id, devfun_id, driver):
+    def bind_driver_linux_generic(self, domain_id, bus_id, devfun_id, driver):
         """
         Bind NIC port to specified driver by the default way on linux.
         """
         new_id = self.pci_id.replace(':', ' ')
-        nic_pci_num = ':'.join(['0000', bus_id, devfun_id])
+        nic_pci_num = ':'.join([domain_id, bus_id, devfun_id])
         self.__send_expect(
             "echo %s > /sys/bus/pci/drivers/%s/new_id" % (new_id, driver), "# ")
         self.__send_expect(
-            "echo %s > /sys/bus/pci/devices/0000\:%s\:%s/driver/unbind" %
-            (nic_pci_num, bus_id, devfun_id), "# ")
+            "echo %s > /sys/bus/pci/devices/%s\:%s\:%s/driver/unbind" %
+            (nic_pci_num, domain_id, bus_id, devfun_id), "# ")
         self.__send_expect(
             "echo %s > /sys/bus/pci/drivers/%s/bind" %
             (nic_pci_num, driver), "# ")
@@ -729,7 +734,7 @@ class NetDevice(object):
             itf = self.get_interface_name()
             self.__send_expect("ifconfig %s up" % itf, "# ")
 
-    def bind_driver_linux_pci_stub(self, bus_id, devfun_id):
+    def bind_driver_linux_pci_stub(self, domain_id, bus_id, devfun_id):
         """
         Bind NIC port to the pci-stub driver on linux.
         """
@@ -737,8 +742,8 @@ class NetDevice(object):
         self.__send_expect(
             "echo %s > /sys/bus/pci/drivers/pci-stub/new_id" % new_id, "# ")
         self.__send_expect(
-            "echo %s > /sys/bus/pci/devices/0000\:%s\:%s/driver/unbind" %
-            (nic_pci_num, bus_id, devfun_id), "# ")
+            "echo %s > /sys/bus/pci/devices/%s\:%s\:%s/driver/unbind" %
+            (nic_pci_num, domain_id, bus_id, devfun_id), "# ")
         self.__send_expect(
             "echo %s > /sys/bus/pci/drivers/pci-stub/bind" %
             nic_pci_num, "# ")
@@ -753,11 +758,11 @@ class NetDevice(object):
             self.__get_os_type())
         if not driver:
             driver = 'generic'
-        ret = unbind_driver(self.bus_id, self.devfun_id, driver)
+        ret = unbind_driver(self.domain_id, self.bus_id, self.devfun_id, driver)
         time.sleep(1)
         return ret
 
-    def unbind_driver_linux(self, bus_id, devfun_id, driver):
+    def unbind_driver_linux(self, domain_id, bus_id, devfun_id, driver):
         """
         Unbind driver on linux.
         """
@@ -765,15 +770,15 @@ class NetDevice(object):
 
         unbind_driver_linux = getattr(
             self, 'unbind_driver_linux_%s' % driver_alias)
-        return unbind_driver_linux(bus_id, devfun_id)
+        return unbind_driver_linux(domain_id, bus_id, devfun_id)
 
-    def unbind_driver_linux_generic(self, bus_id, devfun_id):
+    def unbind_driver_linux_generic(self, domain_id, bus_id, devfun_id):
         """
         Unbind driver by the default way on linux.
         """
-        nic_pci_num = ':'.join(['0000', bus_id, devfun_id])
-        cmd = "echo %s > /sys/bus/pci/devices/0000\:%s\:%s/driver/unbind"
-        self.__send_expect(cmd % (nic_pci_num, bus_id, devfun_id), "# ")
+        nic_pci_num = ':'.join([domain_id, bus_id, devfun_id])
+        cmd = "echo %s > /sys/bus/pci/devices/%s\:%s\:%s/driver/unbind"
+        self.__send_expect(cmd % (nic_pci_num, domain_id, bus_id, devfun_id), "# ")
 
     def _cal_mtu(self, framesize):
         return framesize - HEADER_SIZE['eth']
@@ -791,16 +796,16 @@ class NetDevice(object):
         self.__send_expect(cmd % (self.intf_name, mtu), "# ")
 
 
-def get_pci_id(crb, bus_id, devfun_id):
+def get_pci_id(crb, domain_id, bus_id, devfun_id):
     """
     Return pci device type
     """
-    command = ('cat /sys/bus/pci/devices/0000\:%s\:%s/vendor' %
-               (bus_id, devfun_id))
+    command = ('cat /sys/bus/pci/devices/%s\:%s\:%s/vendor' %
+               (domain_id, bus_id, devfun_id))
     out = crb.send_expect(command, "# ")
     vender = out[2:]
-    command = ('cat /sys/bus/pci/devices/0000\:%s\:%s/device' %
-               (bus_id, devfun_id))
+    command = ('cat /sys/bus/pci/devices/%s\:%s\:%s/device' %
+               (domain_id, bus_id, devfun_id))
     out = crb.send_expect(command, '# ')
     device = out[2:]
     return "%s:%s" % (vender, device)
@@ -818,31 +823,31 @@ def add_to_list(host, obj):
     NICS_LIST.append(nic)
 
 
-def get_from_list(host, bus_id, devfun_id):
+def get_from_list(host, domain_id, bus_id, devfun_id):
     """
     Get network device object from global structure
-    Parameter will by host ip, pci bus id, pci function id
+    Parameter will by host ip, pci domain id, pci bus id, pci function id
     """
     for nic in NICS_LIST:
         if host == nic['host']:
-            pci = ':'.join((bus_id, devfun_id))
+            pci = ':'.join((domain_id, bus_id, devfun_id))
             if pci == nic['pci']:
                 return nic['port']
     return None
 
 
-def GetNicObj(crb, bus_id, devfun_id):
+def GetNicObj(crb, domain_id, bus_id, devfun_id):
     """
     Get network device object. If network device has been initialized, just
     return object. Based on nic type, some special nics like RRC will return
     object different from default.
     """
     # find existed NetDevice object
-    obj = get_from_list(crb.crb['My IP'], bus_id, devfun_id)
+    obj = get_from_list(crb.crb['My IP'], domain_id, bus_id, devfun_id)
     if obj:
         return obj
 
-    pci_id = get_pci_id(crb, bus_id, devfun_id)
+    pci_id = get_pci_id(crb, domain_id, bus_id, devfun_id)
     nic = settings.get_nic_name(pci_id)
 
     if nic == 'redrockcanyou':
@@ -858,7 +863,7 @@ def GetNicObj(crb, bus_id, devfun_id):
         from br import BoulderRapid
         obj = BoulderRapid(crb, bus_id, devfun_id)
     else:
-        obj = NetDevice(crb, bus_id, devfun_id)
+        obj = NetDevice(crb, domain_id, bus_id, devfun_id)
 
     add_to_list(crb.crb['My IP'], obj)
     return obj
-- 
1.7.10.4

[dts] [PATCH v2 4/9] framework: enable connect X3 support

[Gowrishankar]

Connect X3 dual port adapter shares single pci device ID. This limits dts frame-
work to use both the ports for the tests. With this patch, a new attribute also
to refer second interface in such single pci port lookup is introduced. Added
supporting APIs to derive this second interface and its MAC addresses whereever
needed.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 framework/dut.py      |   33 +++++++++++++++++++++++++--------
 framework/settings.py |    2 ++
 framework/tester.py   |   26 ++++++++++++++++++++++++++
 nics/net_device.py    |   27 +++++++++++++++++++++++++++
 4 files changed, 80 insertions(+), 8 deletions(-)

diff --git a/framework/dut.py b/framework/dut.py
index ff40fac..bcfedb6 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -569,15 +569,12 @@ class Dut(Crb):
 
         for port_info in self.ports_info:
             port = port_info['port']
-            intf = port.get_interface_name()
-            if "No such file" in intf:
-                self.logger.info("DUT: [%s] %s" % (pci_bus, unknow_interface))
-                continue
+            intf = port_info['intf']
             out = self.send_expect("ip link show %s" % intf, "# ")
             if "DOWN" in out:
                 self.send_expect("ip link set %s up" % intf, "# ")
                 time.sleep(5)
-            macaddr = port.get_mac_addr()
+            macaddr = port_info['mac']
             out = self.send_expect("ip -family inet6 address show dev %s | awk '/inet6/ { print $2 }'"
                                    % intf, "# ")
             ipv6 = out.split('/')[0]
@@ -585,8 +582,6 @@ class Dut(Crb):
             if ":" not in ipv6:
                 ipv6 = "Not connected"
 
-            port_info['mac'] = macaddr
-            port_info['intf'] = intf
             port_info['ipv6'] = ipv6
 
     def rescan_ports_uncached_freebsd(self):
@@ -689,10 +684,32 @@ class Dut(Crb):
             devfun_id = addr_array[2]
 
             port = GetNicObj(self, domain_id, bus_id, devfun_id)
+            intf = port.get_interface_name()
+            if "No such file" in intf:
+                self.logger.info("DUT: [%s] %s" % (pci_bus, unknow_interface))
+                continue
+
+            macaddr = port.get_mac_addr()
+            if "No such file" in intf:
+                self.logger.info("DUT: [%s] %s" % (pci_bus, unknow_interface))
+                continue
+
+            numa = port.socket
+            # store the port info to port mapping
+            self.ports_info.append(
+                {'port': port, 'pci': pci_bus, 'type': pci_id, 'numa': numa,
+                 'intf': intf, 'mac': macaddr})
+
+            if not port.get_interface2_name():
+                continue
+
+            intf = port.get_interface2_name()
+            macaddr = port.get_intf2_mac_addr()
             numa = port.socket
             # store the port info to port mapping
             self.ports_info.append(
-                {'port': port, 'pci': pci_bus, 'type': pci_id, 'numa': numa})
+                {'port': port, 'pci': pci_bus, 'type': pci_id, 'numa': numa,
+                 'intf': intf, 'mac': macaddr})
 
     def scan_ports_uncached_freebsd(self):
         """
diff --git a/framework/settings.py b/framework/settings.py
index 068afce..f3e3b9a 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -78,6 +78,7 @@ NICS = {
     'fortpark':'8086:374c',
     'fvl10g_vf':'8086:154c',
     'atwood': '8086:15d5',
+    'ConnectX3':'15b3:1003',
     'ConnectX4':'15b3:1013',
     'boulderrapid': '8086:15d0',
 }
@@ -114,6 +115,7 @@ DRIVERS = {
     'fortpark':'i40e',
     'fvl10g_vf':'i40evf',
     'atwood': 'fm10k',
+    'ConnectX3':'mlx4_core',
     'ConnectX4':'mlx5_core',
     'boulderrapid': 'fm10k',
 }
diff --git a/framework/tester.py b/framework/tester.py
index 1d69355..76ceeb4 100644
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -207,6 +207,10 @@ class Tester(Crb):
                 itf = port.get_interface_name()
                 self.enable_ipv6(itf)
                 self.send_expect("ifconfig %s up" % itf, "# ")
+                if port.get_interface2_name():
+                    itf = port.get_interface2_name()
+                    self.enable_ipv6(itf)
+                    self.send_expect("ifconfig %s up" % itf, "# ")
 
         except Exception as e:
             self.logger.error("   !!! Restore ITF: " + e.message)
@@ -220,6 +224,9 @@ class Tester(Crb):
                 port = GetNicObj(self, addr_array[0], addr_array[1], addr_array[2])
                 itf = port.get_interface_name()
                 self.enable_promisc(itf)
+                if port.get_interface2_name():
+                    itf = port.get_interface2_name()
+                    self.enable_promisc(itf)
         except Exception as e:
             pass
 
@@ -314,6 +321,25 @@ class Tester(Crb):
                                     'mac': macaddr,
                                     'ipv6': ipv6})
 
+            # return if port is not connect x3
+            if not port.get_interface2_name():
+                continue
+
+            intf = port.get_interface2_name()
+
+            self.logger.info("Tester: [%s %s] %s" % (pci_bus, pci_id, intf))
+            macaddr = port.get_intf2_mac_addr()
+
+            ipv6 = port.get_ipv6_addr()
+
+            # store the port info to port mapping
+            self.ports_info.append({'port': port,
+                                    'pci': pci_bus,
+                                    'type': pci_id,
+                                    'intf': intf,
+                                    'mac': macaddr,
+                                    'ipv6': ipv6})
+
     def send_ping6(self, localPort, ipv6, mac):
         """
         Send ping6 packet from local port with destination ipv6 address.
diff --git a/nics/net_device.py b/nics/net_device.py
index a9127cb..941be99 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -150,9 +150,21 @@ class NetDevice(object):
             self.intf_name = 'N/A'
         else:
             self.intf_name = out
+            self.intf2_name = None
+
+        # not a complete fix for CX3.
+        if len(out.split()) > 1 and self.default_driver == 'mlx4_core':
+            self.intf_name = out.split()[0]
+            self.intf2_name = out.split()[1]
 
         return self.intf_name
 
+    def get_interface2_name(self):
+        """
+        Get interface name of second port of this pci device.
+        """
+        return self.intf2_name
+
     def get_interface_name_linux(self, domain_id, bus_id, devfun_id, driver):
         """
         Get interface name of specified pci device on linux.
@@ -242,6 +254,18 @@ class NetDevice(object):
         else:
             return out
 
+    @nic_has_driver
+    def get_intf2_mac_addr(self):
+        """
+        Get mac address of 2nd port of specified pci device.
+        """
+        get_mac_addr = getattr(self, 'get_mac_addr_%s' % self.__get_os_type())
+        out = get_mac_addr(self.get_interface2_name(), self.domain_id, self.bus_id, self.devfun_id, self.current_driver)
+        if "No such file or directory" in out:
+            return 'N/A'
+        else:
+            return out
+
     def get_mac_addr_linux(self, intf, domain_id, bus_id, devfun_id, driver):
         """
         Get mac address of specified pci device on linux.
@@ -733,6 +757,9 @@ class NetDevice(object):
         if driver == self.default_driver:
             itf = self.get_interface_name()
             self.__send_expect("ifconfig %s up" % itf, "# ")
+            if self.get_interface2_name():
+                itf = self.get_interface2_name()
+                self.__send_expect("ifconfig %s up" % itf, "# ")
 
     def bind_driver_linux_pci_stub(self, domain_id, bus_id, devfun_id):
         """
-- 
1.7.10.4

[dts] [PATCH v2 5/9] framework: fix numa number lookup for a dev

[Gowrishankar]

Fix get_nic_numa() to return numa associated with a given device correctly.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 nics/net_device.py |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/nics/net_device.py b/nics/net_device.py
index 941be99..99e44ab 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -521,7 +521,7 @@ class NetDevice(object):
         """
         Get numa number of specified pci device.
         """
-        self.crb.get_nic_numa(self.domain_id, self.bus_id, self.devfun_id)
+        self.crb.get_device_numa(self.domain_id, self.bus_id, self.devfun_id)
 
     def get_card_type(self):
         """
-- 
1.7.10.4

[dts] [PATCH v2 6/9] framework: fix get_core_list to return all lcores

[Gowrishankar]

From: Gowri Shankar <gowrishankar.m@linux.vnet.ibm.com>

Irrespective of whether threading is enabled or not, "all" option to get_core_
list should return all available lcores. Current "th" option breaks the test
on loading PMD on an unavailable lcore because cpu core ranges is returned.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 framework/crb.py |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index 453796f..33ceb8c 100644
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -574,17 +574,13 @@ class Crb(object):
         # return thread list
         return map(str, thread_list)
 
-    def get_core_list(self, config, th=False, socket=-1):
+    def get_core_list(self, config, socket=-1):
         """
         Get lcore array according to the core config like "all", "1S/1C/1T".
         We can specify the physical CPU socket by paramter "socket".
         """
         if config == 'all':
-
-            if th:
-                return [n['thread'] for n in self.cores]
-            else:
-                return [n for n in range(0, self.number_of_cores - 1)]
+            return [n['thread'] for n in self.cores]
 
         m = re.match("([1234])S/([1-9]+)C/([12])T", config)
 
-- 
1.7.10.4

[dts] [PATCH v2 7/9] tests: fix multiprocess test to set coremask through library

[Gowrishankar]

This patch removes fixed core mask values which break PMD to load on unavailable
cpu with in this mask. Instead, use get_core_list and create_mask based on need.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 tests/TestSuite_multiprocess.py |   42 ++++++++++++++++++++++++++++-----------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/tests/TestSuite_multiprocess.py b/tests/TestSuite_multiprocess.py
index 197bdb4..97d3092 100644
--- a/tests/TestSuite_multiprocess.py
+++ b/tests/TestSuite_multiprocess.py
@@ -83,9 +83,12 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         Basic operation.
         """
         # Send message from secondary to primary
-        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 --proc-type=primary" % self.target, "Finished Process Init", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=primary" % (self.target, coremask), "Finished Process Init", 100)
         time.sleep(20)
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C --proc-type=secondary" % self.target, "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=secondary" % (self.target, coremask), "Finished Process Init", 100)
 
         self.session_secondary.send_expect("send hello_primary", ">")
         out = self.dut.get_session_output()
@@ -93,9 +96,12 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         self.dut.send_expect("quit","# ")
         self.verify("Received 'hello_primary'" in out, "Message not received on primary process")
         # Send message from primary to secondary
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 --proc-type=primary " % self.target, "Finished Process Init", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=primary " % (self.target, coremask), "Finished Process Init", 100)
         time.sleep(20)
-        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C --proc-type=secondary" % self.target, "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=secondary" % (self.target, coremask), "Finished Process Init", 100)
         self.session_secondary.send_expect("send hello_secondary", ">")
         out = self.dut.get_session_output()
         self.session_secondary.send_expect("quit", "# ")
@@ -109,9 +115,12 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         Load test of Simple MP application.
         """
 
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 --proc-type=primary" % self.target, "Finished Process Init", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=primary" % (self.target, coremask), "Finished Process Init", 100)
         time.sleep(20)
-        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C --proc-type=secondary" % self.target, "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=secondary" % (self.target, coremask), "Finished Process Init", 100)
         stringsSent = 0
         for line in open('/usr/share/dict/words', 'r').readlines():
             line = line.split('\n')[0]
@@ -130,10 +139,13 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         """
 
         # Send message from secondary to primary (auto process type)
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 --proc-type=auto " % self.target, "Finished Process Init", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=auto " % (self.target, coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: PRIMARY" in out, "The type of process (PRIMARY) was not detected properly")
         time.sleep(20)
-        out = self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C --proc-type=auto" % self.target, "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        out = self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=auto" % (self.target, coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: SECONDARY" in out,
                     "The type of process (SECONDARY) was not detected properly")
 
@@ -144,10 +156,13 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         self.verify("Received 'hello_primary'" in out, "Message not received on primary process")
 
         # Send message from primary to secondary (auto process type)
-        out = self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 --proc-type=auto" % self.target, "Finished Process Init", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        out = self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=auto" % (self.target, coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: PRIMARY" in out, "The type of process (PRIMARY) was not detected properly")
         time.sleep(20)
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C --proc-type=auto" % self.target, "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s --proc-type=auto" % (self.target, coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: SECONDARY" in out, "The type of process (SECONDARY) was not detected properly")
         self.session_secondary.send_expect("send hello_secondary", ">",100)
         out = self.dut.get_session_output()
@@ -162,8 +177,11 @@ class TestMultiprocess(TestCase, IxiaPacketGenerator):
         Multiple processes without "--proc-type" flag.
         """
 
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c 3 -m 64" % self.target, "Finished Process Init", 100)
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c C" % self.target, "# ", 100)
+        cores = self.dut.get_core_list('1S/2C/1T')
+        coremask = dts.create_mask(cores)
+        self.session_secondary.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s -m 64" % (self.target, coremask), "Finished Process Init", 100)
+        coremask = hex(int(coremask, 16) * 0x10000).rstrip("L")
+        out = self.dut.send_expect("./examples/multi_process/simple_mp/simple_mp/%s/simple_mp -n 1 -c %s" % (self.target, coremask), "# ", 100)
 
         self.verify("Is another primary process running" in out,
                     "No other primary process detected")
-- 
1.7.10.4

[dts] [PATCH v2 8/9] tests: fix coremask test to check expected EAL output

[Gowrishankar]

From: Gowri Shankar <gowrishankar.m@linux.vnet.ibm.com>

In expected output of EAL, there is type mismatch due to which test breaks.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 tests/TestSuite_coremask.py |   20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/TestSuite_coremask.py b/tests/TestSuite_coremask.py
index d1cc6e3..4f6f4bd 100644
--- a/tests/TestSuite_coremask.py
+++ b/tests/TestSuite_coremask.py
@@ -60,11 +60,11 @@ class TestCoremask(TestCase):
 
             out = self.dut.send_expect(command, "RTE>>", 10)
 
-            self.verify("EAL: Detected lcore %d as core" % core in out,
-                        "Core %d not detected" % core)
+            self.verify("EAL: Detected lcore %s as core" % core in out,
+                        "Core %s not detected" % core)
 
-            self.verify("EAL: Master lcore %d is ready" % core in out,
-                        "Core %d not ready" % core)
+            self.verify("EAL: Master lcore %s is ready" % core in out,
+                        "Core %s not ready" % core)
 
             self.dut.send_expect("quit", "# ", 10)
 
@@ -86,11 +86,11 @@ class TestCoremask(TestCase):
                     "Core 0 not detected")
 
         for core in self.all_cores[1:]:
-            self.verify("EAL: lcore %d is ready" % core in out,
-                        "Core %d not ready" % core)
+            self.verify("EAL: lcore %s is ready" % core in out,
+                        "Core %s not ready" % core)
 
-            self.verify("EAL: Detected lcore %d as core" % core in out,
-                        "Core %d not detected" % core)
+            self.verify("EAL: Detected lcore %s as core" % core in out,
+                        "Core %s not detected" % core)
 
         self.dut.send_expect("quit", "# ", 10)
 
@@ -135,8 +135,8 @@ class TestCoremask(TestCase):
 
         for core in self.all_cores[1:]:
 
-            self.verify("EAL: Detected lcore %d as core" % core in out,
-                        "Core %d not detected" % core)
+            self.verify("EAL: Detected lcore %s as core" % core in out,
+                        "Core %s not detected" % core)
 
         self.dut.send_expect("quit", "# ", 10)
 
-- 
1.7.10.4

[dts] [PATCH v2 9/9] tests: fix blacklist test to discard extra pci domain id in verification string

[Gowrishankar]

Discard 0x0000 from expected output as it is now added in pci id tuple.

Signed-off-by: Gowrishankar <gowrishankar.m@linux.vnet.ibm.com>
---
 tests/TestSuite_blacklist.py |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/TestSuite_blacklist.py b/tests/TestSuite_blacklist.py
index ba39b36..06dbf87 100644
--- a/tests/TestSuite_blacklist.py
+++ b/tests/TestSuite_blacklist.py
@@ -50,9 +50,9 @@ class TestBlackList(TestCase):
         [arch, machine, self.env, toolchain] = self.target.split('-')
 
         if self.env == 'bsdapp':
-            self.regexp_blacklisted_port = "EAL: PCI device 0000:%02x:%s on NUMA socket [-0-9]+[^\n]*\nEAL:   probe driver[^\n]*\nEAL:   Device is blacklisted, not initializing"
+            self.regexp_blacklisted_port = "EAL: PCI device %02x:%s on NUMA socket [-0-9]+[^\n]*\nEAL:   probe driver[^\n]*\nEAL:   Device is blacklisted, not initializing"
         else:
-            self.regexp_blacklisted_port = "EAL: PCI device 0000:%s on NUMA socket [-0-9]+[^\n]*\nEAL:   probe driver[^\n]*\nEAL:   Device is blacklisted, not initializing"
+            self.regexp_blacklisted_port = "EAL: PCI device %s on NUMA socket [-0-9]+[^\n]*\nEAL:   probe driver[^\n]*\nEAL:   Device is blacklisted, not initializing"
         self.pmdout = PmdOutput(self.dut)
 
     def set_up(self):
@@ -101,7 +101,7 @@ class TestBlackList(TestCase):
         Run testpmd with one port blacklisted.
         """
         self.dut.kill_all()
-        out = self.pmdout.start_testpmd("Default", eal_param="-b 0000:%s" % self.dut.ports_info[0]['pci'])
+        out = self.pmdout.start_testpmd("Default", eal_param="-b %s" % self.dut.ports_info[0]['pci'])
         self.check_blacklisted_ports(out, self.ports[1:])
 
     def test_bl_allbutoneportblacklisted(self):
@@ -112,7 +112,7 @@ class TestBlackList(TestCase):
         ports_to_blacklist = self.ports[:-1]
         cmdline = ""
         for port in ports_to_blacklist:
-            cmdline += " -b 0000:%s" % self.dut.ports_info[port]['pci']
+            cmdline += " -b %s" % self.dut.ports_info[port]['pci']
         out = self.pmdout.start_testpmd("Default", eal_param=cmdline)
         blacklisted_ports = self.check_blacklisted_ports(out,
                                               ports_to_blacklist, True)
-- 
1.7.10.4

Re: [dts] framework v2: hugepages, cpuinfo, connect x3 support and bug fixes

[Liu, Yong]

Thanks, Gowrishankar. Your patches have been merged into next branch. There'll be some basic validations on it before merged back to master.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Gowrishankar
> Sent: Tuesday, March 08, 2016 6:07 PM
> To: dts
> Subject: [dts] framework v2: hugepages, cpuinfo, connect x3 support and
> bug fixes
> 
> Following are set of patches addressing below:
> 
> 1. hugepages setting and cpu info parsing for powerpc environment in a
> platform
>    independent way.
> 2. enable connect x3 support to run DTS.
> 3. some bug fixes in framework as well as tests handling numa, cpu
> coremask
>    and expected output.
> 
> Changes from version 1:
>  - interface name for second port sharing same PCI id is stored in
> variable 'intf2'
>    instead 'peer' as earlier, also relavant function names.
>  - check interface name and mac retrieved from DUT scan.
> 
> 
> Thanks,
> Gowrishankar

Re: [dts] framework v2: hugepages, cpuinfo, connect x3 support and bug fixes

[gowrishankar]

On Wednesday 09 March 2016 10:04 AM, Liu, Yong wrote:
> Thanks, Gowrishankar. Your patches have been merged into next branch. There'll be some basic validations on it before merged back to master.
Thank you Yong.

As far as yocto specific need on lscpu, either their os image itself 
need to include concerned util package
or we need to fallback to /proc/cpuinfo parsing only for yocto in run 
time. If latter option is considered,
Is huilongx planning to add patch on top of next ?.

Regards,
Gowrishankar

>
>> -----Original Message-----
>> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Gowrishankar
>> Sent: Tuesday, March 08, 2016 6:07 PM
>> To: dts
>> Subject: [dts] framework v2: hugepages, cpuinfo, connect x3 support and
>> bug fixes
>>
>> Following are set of patches addressing below:
>>
>> 1. hugepages setting and cpu info parsing for powerpc environment in a
>> platform
>>     independent way.
>> 2. enable connect x3 support to run DTS.
>> 3. some bug fixes in framework as well as tests handling numa, cpu
>> coremask
>>     and expected output.
>>
>> Changes from version 1:
>>   - interface name for second port sharing same PCI id is stored in
>> variable 'intf2'
>>     instead 'peer' as earlier, also relavant function names.
>>   - check interface name and mac retrieved from DUT scan.
>>
>>
>> Thanks,
>> Gowrishankar
>

Re: [dts] framework v2: hugepages, cpuinfo, connect x3 support and bug fixes

[Liu, Yong]

Thanks, Gowrishankar. My suggestion is to install util-linux in yocto.
We'd better use those tools which can make our life easier.

> -----Original Message-----
> From: gowrishankar [mailto:gowrishankar.m@linux.vnet.ibm.com]
> Sent: Thursday, March 10, 2016 5:29 PM
> To: Liu, Yong
> Cc: dts; Xu, HuilongX
> Subject: Re: [dts] framework v2: hugepages, cpuinfo, connect x3 support
> and bug fixes
> 
> On Wednesday 09 March 2016 10:04 AM, Liu, Yong wrote:
> > Thanks, Gowrishankar. Your patches have been merged into next branch.
> There'll be some basic validations on it before merged back to master.
> Thank you Yong.
> 
> As far as yocto specific need on lscpu, either their os image itself
> need to include concerned util package
> or we need to fallback to /proc/cpuinfo parsing only for yocto in run
> time. If latter option is considered,
> Is huilongx planning to add patch on top of next ?.
> 
> Regards,
> Gowrishankar
> 
> >
> >> -----Original Message-----
> >> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Gowrishankar
> >> Sent: Tuesday, March 08, 2016 6:07 PM
> >> To: dts
> >> Subject: [dts] framework v2: hugepages, cpuinfo, connect x3 support
> and
> >> bug fixes
> >>
> >> Following are set of patches addressing below:
> >>
> >> 1. hugepages setting and cpu info parsing for powerpc environment in a
> >> platform
> >>     independent way.
> >> 2. enable connect x3 support to run DTS.
> >> 3. some bug fixes in framework as well as tests handling numa, cpu
> >> coremask
> >>     and expected output.
> >>
> >> Changes from version 1:
> >>   - interface name for second port sharing same PCI id is stored in
> >> variable 'intf2'
> >>     instead 'peer' as earlier, also relavant function names.
> >>   - check interface name and mac retrieved from DUT scan.
> >>
> >>
> >> Thanks,
> >> Gowrishankar
> >
>