[PATCH v1 0/2] fix pylama framework errors in crb and virt

[PATCH v1 0/2] fix pylama framework errors in crb and virt

[Juraj Linkeš]

The patches must be submitted together because there would be conflicts
otherwise. The commit messages of the respective commits describe the
Pylama errors being fixed.

Juraj Linkeš (2):
  framework/crb: fix pylama errors
  framework/virt_*: fix pylama errors

 framework/crb.py           | 31 +++++++++++++++++++++++++++----
 framework/dut.py           | 11 +++++------
 framework/project_dpdk.py  |  5 +++--
 framework/tester.py        |  2 +-
 framework/virt_base.py     | 28 ++++++++++++++++++++++++++--
 framework/virt_dut.py      | 12 +++++-------
 framework/virt_resource.py |  7 +++++--
 framework/virt_scene.py    | 12 +++++++-----
 8 files changed, 79 insertions(+), 29 deletions(-)

-- 
2.20.1

[PATCH v1 1/2] framework/crb: fix pylama errors

[Juraj Linkeš]

Pylama found the following errors:
framework/crb.py:68: [E] E1101 Instance of 'Crb' has no 'get_ip_address' member [pylint]
framework/crb.py:69: [E] E1101 Instance of 'Crb' has no 'get_username' member [pylint]
framework/crb.py:70: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:74: [E] E1101 Instance of 'Crb' has no 'get_ip_address' member [pylint]
framework/crb.py:76: [E] E1101 Instance of 'Crb' has no 'get_username' member [pylint]
framework/crb.py:77: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:109: [E] E1101 Instance of 'Crb' has no 'get_ip_address' member [pylint]
framework/crb.py:111: [E] E1101 Instance of 'Crb' has no 'get_username' member [pylint]
framework/crb.py:112: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:140: [E] E1101 Instance of 'Crb' has no 'loggger' member; maybe 'logger'? [pylint]
framework/crb.py:144: [E] E1101 Instance of 'Crb' has no 'get_ip_address' member [pylint]
framework/crb.py:146: [E] E1101 Instance of 'Crb' has no 'get_username' member [pylint]
framework/crb.py:147: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:150: [E] E1101 Instance of 'Crb' has no 'get_ip_address' member [pylint]
framework/crb.py:151: [E] E1101 Instance of 'Crb' has no 'get_username' member [pylint]
framework/crb.py:151: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:254: [E] E1136 Value 'self.ports_info' is unsubscriptable [pylint]
framework/crb.py:270: [E] E1101 Instance of 'Crb' has no 'PCI_DEV_CACHE_KEY' member [pylint]
framework/crb.py:274: [E] E1101 Instance of 'Crb' has no 'PCI_DEV_CACHE_KEY' member [pylint]
framework/crb.py:391: [E] E1120 No value for argument 'string' in function call [pylint]
framework/crb.py:478: [E] E1101 Instance of 'Crb' has no 'get_password' member [pylint]
framework/crb.py:560: [E] E0203 Access to member 'prefix_list' before its definition line 564 [pylint]
framework/crb.py:562: [E] E0203 Access to member 'prefix_list' before its definition line 564 [pylint]
framework/crb.py:606: [E] E1101 Instance of 'Crb' has no 'NUMBER_CORES_CACHE_KEY' member [pylint]
framework/crb.py:607: [E] E1101 Instance of 'Crb' has no 'CORE_LIST_CACHE_KEY' member [pylint]
framework/crb.py:611: [E] E1101 Instance of 'Crb' has no 'NUMBER_CORES_CACHE_KEY' member [pylint]
framework/crb.py:612: [E] E1101 Instance of 'Crb' has no 'CORE_LIST_CACHE_KEY' member [pylint]
framework/crb.py:880: [E] E1133 Non-iterable value self.ports_info is used in an iterating context [pylint]
framework/crb.py:888: [E] E1136 Value 'self.ports_info' is unsubscriptable [pylint]

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 framework/crb.py      | 29 ++++++++++++++++++++++++++---
 framework/dut.py      |  2 --
 framework/virt_dut.py |  2 --
 3 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index bd4f565d..0f22c5d3 100755
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -51,18 +51,23 @@ class Crb(object):
     CPU/PCI/NIC on the board and setup running environment for DPDK.
     """
 
+    PCI_DEV_CACHE_KEY = None
+    NUMBER_CORES_CACHE_KEY = None
+    CORE_LIST_CACHE_KEY = None
+
     def __init__(self, crb, serializer, name, alt_session=True, dut_id=0):
         self.dut_id = dut_id
         self.crb = crb
         self.read_cache = False
         self.skip_setup = False
         self.serializer = serializer
-        self.ports_info = None
+        self.ports_info = []
         self.sessions = []
         self.stage = 'pre-init'
         self.name = name
         self.trex_prefix = None
         self.default_hugepages_cleared = False
+        self.prefix_list = []
 
         self.logger = getLogger(name)
         self.session = SSHConnection(self.get_ip_address(), name,
@@ -80,6 +85,24 @@ class Crb(object):
         else:
             self.alt_session = None
 
+    def get_ip_address(self):
+        """
+        Get CRB's ip address.
+        """
+        raise NotImplementedError
+
+    def get_password(self):
+        """
+        Get CRB's login password.
+        """
+        raise NotImplementedError
+
+    def get_username(self):
+        """
+        Get CRB's login username.
+        """
+        raise NotImplementedError
+
     def send_expect(self, cmds, expected, timeout=TIMEOUT,
                     alt_session=False, verify=False, trim_whitespace=True):
         """
@@ -137,7 +160,7 @@ class Crb(object):
             else:
                 self.session.close(force=True)
         except Exception as e:
-            self.loggger.error("Session close failed for [%s]" % e)
+            self.logger.error("Session close failed for [%s]" % e)
 
         if alt_session:
             session = SSHConnection(
@@ -388,7 +411,7 @@ class Crb(object):
                                (domain_id, bus_id, devfun_id), "# ", alt_session=True)
         rexp = r"PCI_ID=(.+)"
         pattern = re.compile(rexp)
-        match = re.search(out)
+        match = re.search(pattern, out)
         if not match:
             return None
         return match.group(1)
diff --git a/framework/dut.py b/framework/dut.py
index dc3fc874..45db4616 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -71,14 +71,12 @@ class Dut(Crb):
         self.tester = None
         self.cores = []
         self.architecture = None
-        self.ports_info = []
         self.conf = PortConf()
         self.ports_map = []
         self.virt_pool = None
         # hypervisor pid list, used for cleanup
         self.virt_pids = []
         self.prefix_subfix = str(os.getpid()) + '_' + time.strftime("%Y%m%d%H%M%S", time.localtime())
-        self.prefix_list = []
         self.hugepage_path = None
         self.apps_name_conf = {}
         self.apps_name = {}
diff --git a/framework/virt_dut.py b/framework/virt_dut.py
index bde65667..ca8c8c6b 100644
--- a/framework/virt_dut.py
+++ b/framework/virt_dut.py
@@ -78,11 +78,9 @@ class VirtDut(DPDKdut):
         self.tester = None
         self.cores = []
         self.architecture = None
-        self.ports_info = None
         self.ports_map = []
         self.virttype = virttype
         self.prefix_subfix = str(os.getpid()) + '_' + time.strftime("%Y%m%d%H%M%S", time.localtime())
-        self.prefix_list = []
         self.apps_name_conf = {}
         self.apps_name = {}
 
-- 
2.20.1

[PATCH v1 2/2] framework/virt_*: fix pylama errors

[Juraj Linkeš]

Pylama found the following errors:
framework/virt_base.py:271: [E] E1101 Instance of 'VirtBase' has no 'add_vm_login' member [pylint]
framework/virt_base.py:274: [E] E1101 Instance of 'VirtBase' has no '_attach_vm' member [pylint]
framework/virt_base.py:319: [E] E1101 Instance of 'VirtBase' has no '_quick_start_vm' member [pylint]
framework/virt_base.py:413: [E] E1111 Assigning result of a function call, where the function has no return [pylint]
framework/virt_base.py:416: [E] E1101 Instance of 'VirtBase' has no 'get_vm_login' member [pylint]
framework/virt_base.py:88: [E] E1111 Assigning result of a function call, where the function has no return [pylint]
framework/virt_dut.py:73: [E] E1003 Bad first argument 'Dut' given to super() [pylint]
- unify the init function of VirtDut's parents
framework/virt_resource.py:242: [E] E1101 Instance of 'VirtResource' has no '__port_isused' member [pylint]
framework/virt_resource.py:246: [E] E0602 Undefined variable 'core' [pylint]
framework/virt_resource.py:247: [E] E0602 Undefined variable 'core' [pylint]
framework/virt_scene.py:350: [E] E1101 Instance of 'VirtScene' has no 'tester' member [pylint]
framework/virt_scene.py:393: [E] E1101 Instance of 'VirtScene' has no 'vm_dut' member [pylint]
framework/virt_scene.py:397: [E] E1101 Instance of 'VirtScene' has no 'tester' member [pylint]
framework/virt_scene.py:458: [E] E1123 Unexpected keyword argument 'auto_portmap' in method call [pylint]
framework/virt_scene.py:510: [E] E0102 class already defined line 38 [pylint]
framework/virt_scene.py:517: [E] E1101 Instance of 'QEMUKvm' has no 'params' member [pylint]

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 framework/crb.py           |  2 +-
 framework/dut.py           |  9 +++++----
 framework/project_dpdk.py  |  5 +++--
 framework/tester.py        |  2 +-
 framework/virt_base.py     | 28 ++++++++++++++++++++++++++--
 framework/virt_dut.py      | 10 +++++-----
 framework/virt_resource.py |  7 +++++--
 framework/virt_scene.py    | 12 +++++++-----
 8 files changed, 53 insertions(+), 22 deletions(-)

diff --git a/framework/crb.py b/framework/crb.py
index 0f22c5d3..57883f00 100755
--- a/framework/crb.py
+++ b/framework/crb.py
@@ -55,7 +55,7 @@ class Crb(object):
     NUMBER_CORES_CACHE_KEY = None
     CORE_LIST_CACHE_KEY = None
 
-    def __init__(self, crb, serializer, name, alt_session=True, dut_id=0):
+    def __init__(self, crb, serializer, dut_id=0, name=None, alt_session=True):
         self.dut_id = dut_id
         self.crb = crb
         self.read_cache = False
diff --git a/framework/dut.py b/framework/dut.py
index 45db4616..a2dc82db 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -62,10 +62,11 @@ class Dut(Crb):
     CORE_LIST_CACHE_KEY = 'dut_core_list'
     PCI_DEV_CACHE_KEY = 'dut_pci_dev_info'
 
-    def __init__(self, crb, serializer, dut_id):
-        self.NAME = 'dut' + LOG_NAME_SEP + '%s' % crb['My IP']
-        super(Dut, self).__init__(crb, serializer, self.NAME, alt_session=True, dut_id=dut_id)
-
+    def __init__(self, crb, serializer, dut_id=0, name=None, alt_session=True):
+        if not name:
+            name = 'dut' + LOG_NAME_SEP + '%s' % crb['My IP']
+            self.NAME = name
+        super(Dut, self).__init__(crb, serializer, dut_id, name, alt_session)
         self.host_init_flag = False
         self.number_of_cores = 0
         self.tester = None
diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
index 9927bcc1..37fbc4e6 100644
--- a/framework/project_dpdk.py
+++ b/framework/project_dpdk.py
@@ -60,8 +60,9 @@ class DPDKdut(Dut):
     build, memory and kernel module.
     """
 
-    def __init__(self, crb, serializer, dut_id):
-        super(DPDKdut, self).__init__(crb, serializer, dut_id)
+    def __init__(self, crb, serializer, dut_id=0, name=None, alt_session=True):
+        super(DPDKdut, self).__init__(crb, serializer, dut_id, name,
+                                      alt_session)
         self.testpmd = None
 
     def set_target(self, target, bind_dev=True):
diff --git a/framework/tester.py b/framework/tester.py
index 9b764e8b..73b428d9 100644
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -80,7 +80,7 @@ class Tester(Crb):
     def __init__(self, crb, serializer):
         self.NAME = 'tester'
         self.scapy_session = None
-        super(Tester, self).__init__(crb, serializer, self.NAME)
+        super(Tester, self).__init__(crb, serializer, name=self.NAME)
         # check the python version of tester
         check_crb_python_version(self)
 
diff --git a/framework/virt_base.py b/framework/virt_base.py
index 148a4d18..ef117062 100644
--- a/framework/virt_base.py
+++ b/framework/virt_base.py
@@ -104,7 +104,7 @@ class VirtBase(object):
         """
         Get the virtual type, such as KVM, XEN or LIBVIRT.
         """
-        NotImplemented
+        raise NotImplementedError
 
     def has_virtual_ability(self):
         """
@@ -118,6 +118,30 @@ class VirtBase(object):
         """
         NotImplemented
 
+    def get_vm_login(self):
+        """
+        Get VM credentials.
+        """
+        raise NotImplementedError
+
+    def add_vm_login(self):
+        """
+        Add VM credentials.
+        """
+        raise NotImplementedError
+
+    def _attach_vm(self):
+        """
+        Attach VM.
+        """
+        raise NotImplementedError
+
+    def _quick_start_vm(self):
+        """
+        Quick start VM.
+        """
+        raise NotImplementedError
+
     def load_global_config(self):
         """
         Load global configure in the path DTS_ROOT_PATH/conf.
@@ -230,7 +254,7 @@ class VirtBase(object):
         """
         Get the VM IP.
         """
-        NotImplemented
+        raise NotImplementedError
 
     def get_pci_mappings(self):
         """
diff --git a/framework/virt_dut.py b/framework/virt_dut.py
index ca8c8c6b..7164c522 100644
--- a/framework/virt_dut.py
+++ b/framework/virt_dut.py
@@ -61,16 +61,16 @@ class VirtDut(DPDKdut):
     """
 
     def __init__(self, hyper, crb, serializer, virttype, vm_name, suite, cpu_topo, dut_id):
+        self.vm_ip = crb['IP']
+        self.NAME = 'virtdut' + LOG_NAME_SEP + '%s' % self.vm_ip
+        # do not create addition alt_session
+        super(VirtDut, self).__init__(crb, serializer, dut_id,
+                                      self.NAME, alt_session=False)
         self.vm_name = vm_name
         self.hyper = hyper
         self.cpu_topo = cpu_topo
-        self.dut_id = dut_id
         self.migration_vm = False
 
-        self.vm_ip = crb['IP']
-        self.NAME = 'virtdut' + LOG_NAME_SEP + '%s' % self.vm_ip
-        # do not create addition alt_session
-        super(Dut, self).__init__(crb, serializer, self.NAME, alt_session=False, dut_id=self.dut_id)
         # load port config from suite cfg
         self.suite = suite
 
diff --git a/framework/virt_resource.py b/framework/virt_resource.py
index 8a19f1b2..3163372d 100644
--- a/framework/virt_resource.py
+++ b/framework/virt_resource.py
@@ -70,6 +70,9 @@ class VirtResource(object):
         # save allocated cores and related vm
         self.allocated_info = {}
 
+    def __port_isused(self, pci):
+        return pci in self.used_ports
+
     def __port_used(self, pci):
         index = self.ports.index(pci)
         self.used_ports[index] = pci
@@ -243,8 +246,8 @@ class VirtResource(object):
                     print("Port %s has been used!!!" % pci)
                 else:
                     if self.__port_on_socket(pci, socket) is True:
-                        self.__port_used(core)
-                        ports.append(core)
+                        self.__port_used(pci)
+                        ports.append(pci)
 
         if vm not in self.allocated_info:
             self.allocated_info[vm] = {}
diff --git a/framework/virt_scene.py b/framework/virt_scene.py
index cc36a09b..62748bc2 100644
--- a/framework/virt_scene.py
+++ b/framework/virt_scene.py
@@ -61,6 +61,7 @@ class VirtScene(object):
         self.name = scene_name
         self.host_dut = dut
         self.tester_dut = tester
+        self.vm_dut = None
         self.pre_cmds = []
         self.post_cmds = []
 
@@ -347,7 +348,7 @@ class VirtScene(object):
             elif cmd['type'] == 'host':
                 crb = self.host_dut
             elif cmd['type'] == 'tester':
-                crb = self.tester
+                crb = self.tester_dut
             else:
                 crb = self.host_dut
 
@@ -394,7 +395,7 @@ class VirtScene(object):
             elif cmd['type'] == 'host':
                 crb = self.host_dut
             elif cmd['type'] == 'tester':
-                crb = self.tester
+                crb = self.tester_dut
             else:
                 crb = self.host_dut
 
@@ -456,7 +457,6 @@ class VirtScene(object):
                 topo = self.get_cputopo(scene_params)
                 try:
                     vm_dut = vm.start(load_config=False, set_target=False,
-                                      auto_portmap=self.auto_portmap,
                                       cpu_topo=topo)
                     if vm_dut is None:
                         raise Exception("Set up VM ENV failed!")
@@ -507,16 +507,18 @@ class VirtScene(object):
 
 if __name__ == "__main__":
 
-    class QEMUKvm():
+    class QEMUKvmTmp():
 
         def __init__(self, dut, vm_name, suite_name):
             print(vm_name)
             print(suite_name)
 
         def start(self):
-            print(self.params)
+            print(self.__dict__)
             return True
 
+    QEMUKvm = QEMUKvmTmp
+
     class simple_dev(object):
 
         def __init__(self, pci):
-- 
2.20.1

RE: [PATCH v1 1/2] framework/crb: fix pylama errors

[Dong, JunX]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Wednesday, January 19, 2022 9:41 PM
> To: Dong, JunX <junx.dong@intel.com>; Tu, Lijuan <lijuan.tu@intel.com>;
> ohilyard@iol.unh.edu
> Cc: dts@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v1 1/2] framework/crb: fix pylama errors
> 
> Pylama found the following errors:
> framework/crb.py:68: [E] E1101 Instance of 'Crb' has no 'get_ip_address'
> member [pylint]
> framework/crb.py:69: [E] E1101 Instance of 'Crb' has no 'get_username'
> member [pylint]
> framework/crb.py:70: [E] E1101 Instance of 'Crb' has no 'get_password'
> member [pylint]
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Reviewed-by: Jun Dong <junx.dong@intel.com>

RE: [PATCH v1 2/2] framework/virt_*: fix pylama errors

[Dong, JunX]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Wednesday, January 19, 2022 9:41 PM
> To: Dong, JunX <junx.dong@intel.com>; Tu, Lijuan <lijuan.tu@intel.com>;
> ohilyard@iol.unh.edu
> Cc: dts@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v1 2/2] framework/virt_*: fix pylama errors
> 
> Pylama found the following errors:
> framework/virt_base.py:271: [E] E1101 Instance of 'VirtBase' has no
> 'add_vm_login' member [pylint]
> framework/virt_base.py:274: [E] E1101 Instance of 'VirtBase' has no
> '_attach_vm' member [pylint]
> framework/virt_base.py:319: [E] E1101 Instance of 'VirtBase' has no
> '_quick_start_vm' member [pylint]
> framework/virt_base.py:413: [E] E1111 Assigning result of a function call,
> where the function has no return [pylint]
> framework/virt_base.py:416: [E] E1101 Instance of 'VirtBase' has no
> 'get_vm_login' member [pylint]
> framework/virt_base.py:88: [E] E1111 Assigning result of a function call,
> where the function has no return [pylint]
> framework/virt_dut.py:73: [E] E1003 Bad first argument 'Dut' given to super()
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Reviewed-by: Jun Dong <junx.dong@intel.com>

RE: [PATCH v1 2/2] framework/virt_*: fix pylama errors

[Tu, Lijuan]

> -----Original Message-----
> From: Dong, JunX <junx.dong@intel.com>
> Sent: 2022年1月25日 13:42
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Tu, Lijuan <lijuan.tu@intel.com>;
> ohilyard@iol.unh.edu
> Cc: dts@dpdk.org
> Subject: RE: [PATCH v1 2/2] framework/virt_*: fix pylama errors
> 
> > -----Original Message-----
> > From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Sent: Wednesday, January 19, 2022 9:41 PM
> > To: Dong, JunX <junx.dong@intel.com>; Tu, Lijuan
> > <lijuan.tu@intel.com>; ohilyard@iol.unh.edu
> > Cc: dts@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Subject: [PATCH v1 2/2] framework/virt_*: fix pylama errors
> >
> > Pylama found the following errors:
> > framework/virt_base.py:271: [E] E1101 Instance of 'VirtBase' has no
> > 'add_vm_login' member [pylint]
> > framework/virt_base.py:274: [E] E1101 Instance of 'VirtBase' has no
> > '_attach_vm' member [pylint]
> > framework/virt_base.py:319: [E] E1101 Instance of 'VirtBase' has no
> > '_quick_start_vm' member [pylint]
> > framework/virt_base.py:413: [E] E1111 Assigning result of a function
> > call, where the function has no return [pylint]
> > framework/virt_base.py:416: [E] E1101 Instance of 'VirtBase' has no
> > 'get_vm_login' member [pylint]
> > framework/virt_base.py:88: [E] E1111 Assigning result of a function
> > call, where the function has no return [pylint]
> > framework/virt_dut.py:73: [E] E1003 Bad first argument 'Dut' given to
> > super()
> >
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> Reviewed-by: Jun Dong <junx.dong@intel.com>

Series applied.