[dts] [PATCH V1 0/5] update framework to support vfio-pci and new version of OS

[dts] [PATCH V1 0/5] update framework to support vfio-pci and new version of OS

[Xiao Qimai]

*. conf/virt_global.cfg: add noiommu mode for vfio-pci
*. framework/dut.py: restore fvl vf to iavf when bind i40evf failed, iavf is the default kernel driver on new kernel OS for fvl vf
*. framework/project_dpdk.py: modprobe vfio-pci with noiommu mode in vm
*. framework/qemu_kvm.py: cat command of QMP is disabled by default on redhat related OS
*. framework/ssh_pexpect.py: add retry loop when connect to vm

Xiao Qimai (5):
  vfio-pci should be loaded with noiommu mode in vm
  update method of restore_interfaces_linux
  update method setup_modules_linux for vfio-pci in vm
  remove cat command
  add retry when connect to vm

 conf/virt_global.cfg      |  1 +
 framework/dut.py          | 19 ++++++++++++++++++-
 framework/project_dpdk.py | 26 ++++++++------------------
 framework/qemu_kvm.py     |  9 ---------
 framework/ssh_pexpect.py  | 25 +++++++++++++++++++------
 5 files changed, 46 insertions(+), 34 deletions(-)

-- 
1.8.3.1

[dts] [PATCH V1 1/5]conf/virt_global.cfg: vfio-pci should be loaded with noiommu mode in vm

[Xiao Qimai]

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 conf/virt_global.cfg | 1 +
 1 file changed, 1 insertion(+)

diff --git a/conf/virt_global.cfg b/conf/virt_global.cfg
index 7ab0440..b437a0a 100644
--- a/conf/virt_global.cfg
+++ b/conf/virt_global.cfg
@@ -39,6 +39,7 @@ vnc =
     disable=False;
 def_driver =
     driver_name=vfio-pci;
+    driver_mode=noiommu;
 [XEN]
 cpu = 
     number=4,cpupin=3 4 5 6;
-- 
1.8.3.1

[dts] [PATCH V1 2/5]framework/dut.py: update method of restore_interfaces_linux

[Xiao Qimai]

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 framework/dut.py | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/framework/dut.py b/framework/dut.py
index 73f965e..2545621 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -447,13 +447,30 @@ class Dut(Crb):
                 self.send_expect('echo %s > /sys/bus/pci/drivers/%s/bind'
                                  % (pci_bus, driver), '# ')
                 pull_retries = 5
+                itf = 'N/A'
                 while pull_retries > 0:
                     itf = port.get_interface_name()
-                    if itf == 'N/A':
+                    if not itf or itf == 'N/A':
                         time.sleep(1)
                         pull_retries -= 1
                     else:
                         break
+                else:
+                    # try to bind nic with iavf
+                    if driver == 'i40evf':
+                        driver = 'iavf'
+                        self.send_expect('modprobe %s' % driver, '# ')
+                        self.send_expect('echo %s > /sys/bus/pci/drivers/%s/bind'
+                                         % (pci_bus, driver), '# ')
+                        pull_retries = 5
+                        itf = 'N/A'
+                        while pull_retries > 0:
+                            itf = port.get_interface_name()
+                            if not itf or itf == 'N/A':
+                                time.sleep(1)
+                                pull_retries -= 1
+                            else:
+                                break
                 if itf == 'N/A':
                     self.logger.warning("Fail to bind the device with the linux driver")
                 else:
-- 
1.8.3.1

[dts] [PATCH V1 3/5]framework/project_dpdk: update method setup_modules_linux for vfio-pci in vm

[Xiao Qimai]

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 framework/project_dpdk.py | 26 ++++++++------------------
 1 file changed, 8 insertions(+), 18 deletions(-)

diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
index e8b5d0f..b8cb2bd 100644
--- a/framework/project_dpdk.py
+++ b/framework/project_dpdk.py
@@ -101,25 +101,15 @@ class DPDKdut(Dut):
 
     def setup_modules_linux(self, target, drivername, drivermode):
         if drivername == "vfio-pci":
-            if 'VirtDut' in str(self.__class__):
-                self.send_expect("modprobe -r vfio_iommu_type1", "#")
-                self.send_expect("modprobe -r vfio", "#")
-                self.send_expect("modprobe vfio enable_unsafe_noiommu_mode=1", "#")
-                self.send_expect("modprobe vfio-pci", "#")
-            else:
-                self.send_expect("rmmod vfio_pci", "#", 70)
-                self.send_expect("rmmod vfio_iommu_type1", "#", 70)
-                self.send_expect("rmmod vfio", "#", 70)
-                self.send_expect("modprobe vfio", "#", 70)
-                self.send_expect("modprobe vfio-pci", "#", 70)
-            out = self.send_expect("lsmod | grep vfio_iommu_type1", "#")
-            if not out:
-                out = self.send_expect("ls /sys/module |grep vfio_pci", "#")
-            assert ("vfio_pci" in out), "Failed to insmod vfio_pci"
-
-
+            self.send_expect("rmmod vfio_pci", "#")
+            self.send_expect("rmmod vfio_iommu_type1", "#")
+            self.send_expect("rmmod vfio", "#")
+            self.send_expect("modprobe vfio", "#")
+            self.send_expect("modprobe vfio-pci", "#")
             if drivermode == "noiommu":
-                self.send_expect("echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode", "#", 70)
+                self.send_expect("echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode", "#")
+            out = self.send_expect("ls /sys/module|grep vfio_pci", "#")
+            assert ("vfio_pci" in out), "load vfio_pci failed"
 
         elif drivername == "uio_pci_generic":
             self.send_expect("modprobe uio", "#", 70)
-- 
1.8.3.1

[dts] [PATCH V1 4/5]framework/qemu_kvm: remove cat command

[Xiao Qimai]

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 framework/qemu_kvm.py | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/framework/qemu_kvm.py b/framework/qemu_kvm.py
index 421ee3e..757737e 100644
--- a/framework/qemu_kvm.py
+++ b/framework/qemu_kvm.py
@@ -1829,11 +1829,6 @@ class QEMUKvm(VirtBase):
             if self.control_type == "qga":
                 # wait few seconds for network ready
                 time.sleep(5)
-                # before get the ip, check the ssh service is ok
-                # if the ssh service if ready, the file /run/sshd.pid will store the pid
-                out = self.control_session.send_expect(self.qga_cmd_head + "cat /run/sshd.pid" , "#", timeout=self.OPERATION_TIMEOUT)
-                if out == '':
-                    return "Failed"
                 out = self.control_session.send_expect(self.qga_cmd_head + "ifconfig" , "#", timeout=self.OPERATION_TIMEOUT)
             else:
                 pci = "00:1f.0"
@@ -1843,10 +1838,6 @@ class QEMUKvm(VirtBase):
                     if self.nic_model == "virtio":
                         pci += "/virtio*/"
 
-                # before get the ip, check the ssh service is ok
-                out = self.control_session.send_expect("ls /run/sshd.pid", "# ", timeout=self.OPERATION_TIMEOUT, verify=True)
-                if isinstance(out, int):
-                    return "Failed"
                 intf = self.control_session.send_expect("ls -1 /sys/bus/pci/devices/0000:%s/net" %pci, "#", timeout=self.OPERATION_TIMEOUT)
                 out = self.control_session.send_expect("ifconfig %s" % intf, "#", timeout=self.OPERATION_TIMEOUT)
                 if "10.0.2" not in out:
-- 
1.8.3.1

[dts] [PATCH V1 5/5]framework/ssh_pexpect: add retry when connect to vm

[Xiao Qimai]

Signed-off-by: Xiao Qimai <qimaix.xiao@intel.com>
---
 framework/ssh_pexpect.py | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/framework/ssh_pexpect.py b/framework/ssh_pexpect.py
index 0230207..1f839d4 100644
--- a/framework/ssh_pexpect.py
+++ b/framework/ssh_pexpect.py
@@ -34,15 +34,28 @@ class SSHPexpect:
         threads number is limited to 8 which less than 10. Lock number can
         be modified along with MaxStartups value.
         """
+        retry_times = 10
         try:
-            self.session = pxssh.pxssh(encoding='utf-8')
             if ':' in self.host:
-                self.ip = self.host.split(':')[0]
-                self.port = int(self.host.split(':')[1])
-                self.session.login(self.ip, self.username,
-                                   self.password, original_prompt='[$#>]',
-                                   port=self.port, login_timeout=20)
+                while retry_times:
+                    self.ip = self.host.split(':')[0]
+                    self.port = int(self.host.split(':')[1])
+                    self.session = pxssh.pxssh(encoding='utf-8')
+                    try:
+                        self.session.login(self.ip, self.username,
+                                           self.password, original_prompt='[$#>]',
+                                           port=self.port, login_timeout=20)
+                    except Exception as e:
+                        print(e)
+                        time.sleep(2)
+                        retry_times -= 1
+                        print("retry %d times connecting..." % (10-retry_times))
+                    else:
+                        break
+                else:
+                    raise Exception('connect to %s:%s failed' % (self.ip, self.port))
             else:
+                self.session = pxssh.pxssh(encoding='utf-8')
                 self.session.login(self.host, self.username,
                                    self.password, original_prompt='[$#>]')
             self.send_expect('stty -echo', '#')
-- 
1.8.3.1

Re: [dts] [PATCH V1 0/5] update framework to support vfio-pci and new version of OS

[Xiao, QimaiX]

Tested-by: Xiao, QimaiX <qimaix.xiao@intel.com>

Regards,
Xiao Qimai

> -----Original Message-----
> From: Xiao, QimaiX <qimaix.xiao@intel.com>
> Sent: Tuesday, June 30, 2020 5:47 PM
> To: dts@dpdk.org
> Cc: Xiao, QimaiX <qimaix.xiao@intel.com>
> Subject: [dts][PATCH V1 0/5] update framework to support vfio-pci and new
> version of OS
> 
> *. conf/virt_global.cfg: add noiommu mode for vfio-pci *. framework/dut.py:
> restore fvl vf to iavf when bind i40evf failed, iavf is the default kernel driver
> on new kernel OS for fvl vf *. framework/project_dpdk.py: modprobe vfio-
> pci with noiommu mode in vm *. framework/qemu_kvm.py: cat command of
> QMP is disabled by default on redhat related OS *.
> framework/ssh_pexpect.py: add retry loop when connect to vm
> 
> Xiao Qimai (5):
>   vfio-pci should be loaded with noiommu mode in vm
>   update method of restore_interfaces_linux
>   update method setup_modules_linux for vfio-pci in vm
>   remove cat command
>   add retry when connect to vm
> 
>  conf/virt_global.cfg      |  1 +
>  framework/dut.py          | 19 ++++++++++++++++++-
>  framework/project_dpdk.py | 26 ++++++++------------------
>  framework/qemu_kvm.py     |  9 ---------
>  framework/ssh_pexpect.py  | 25 +++++++++++++++++++------
>  5 files changed, 46 insertions(+), 34 deletions(-)
> 
> --
> 1.8.3.1

Re: [dts] [PATCH V1 0/5] update framework to support vfio-pci and new version of OS

[Tu, Lijuan]

Applied the series, thanks

> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of Xiao Qimai
> Sent: 2020年6月30日 17:47
> To: dts@dpdk.org
> Cc: Xiao, QimaiX <qimaix.xiao@intel.com>
> Subject: [dts] [PATCH V1 0/5] update framework to support vfio-pci and new
> version of OS
> 
> *. conf/virt_global.cfg: add noiommu mode for vfio-pci *. framework/dut.py:
> restore fvl vf to iavf when bind i40evf failed, iavf is the default kernel driver on
> new kernel OS for fvl vf *. framework/project_dpdk.py: modprobe vfio-pci with
> noiommu mode in vm *. framework/qemu_kvm.py: cat command of QMP is
> disabled by default on redhat related OS *. framework/ssh_pexpect.py: add
> retry loop when connect to vm
> 
> Xiao Qimai (5):
>   vfio-pci should be loaded with noiommu mode in vm
>   update method of restore_interfaces_linux
>   update method setup_modules_linux for vfio-pci in vm
>   remove cat command
>   add retry when connect to vm
> 
>  conf/virt_global.cfg      |  1 +
>  framework/dut.py          | 19 ++++++++++++++++++-
>  framework/project_dpdk.py | 26 ++++++++------------------
>  framework/qemu_kvm.py     |  9 ---------
>  framework/ssh_pexpect.py  | 25 +++++++++++++++++++------
>  5 files changed, 46 insertions(+), 34 deletions(-)
> 
> --
> 1.8.3.1