[dts] [PATCH V1 0/6] framework: add dirver/firmware/pkg in result

[dts] [PATCH V1 0/6] framework: add dirver/firmware/pkg in result

[Haiyang Zhao]

For the convenience of debugging and analysis, add the NIC
firmware, drvier, pkg version in excel and json format result file.

Haiyang Zhao (6):
  nics/net_device: add driver/firmware
  framework/dut: get nic driver/firmware info
  framework/test_result: add driver/firmware/package
  framework/dts: add more nic configurations in result
  framework/excel_reporter: add driver/firmware/pkg
  framework/json_reporter: add driver/firmware/pkg

 framework/dts.py            |  4 ++
 framework/dut.py            |  4 +-
 framework/excel_reporter.py | 12 ++++++
 framework/json_reporter.py  |  7 ++++
 framework/test_result.py    | 83 +++++++++++++++++++++++++++++++++++--
 nics/net_device.py          | 35 +++++++++++++++-
 6 files changed, 139 insertions(+), 6 deletions(-)

-- 
2.17.1

[dts] [PATCH V1 1/6] nics/net_device: add driver/firmware

[Haiyang Zhao]

add attribute driver/firmware and the related methods for NetDevice.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 nics/net_device.py | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/nics/net_device.py b/nics/net_device.py
index 6a26d1eb..98d31237 100644
--- a/nics/net_device.py
+++ b/nics/net_device.py
@@ -71,7 +71,9 @@ class NetDevice(object):
         self.intf2_name = None
         self.get_interface_name()
         self.socket = self.get_nic_socket()
-        self.pkg = {}
+        self.driver_version = ''
+        self.firmware = ''
+        self.pkg = None
 
     def stop(self):
         pass
@@ -123,6 +125,7 @@ class NetDevice(object):
         """
         Get the NIC pkg.
         """
+        self.pkg = {'type': '', 'version': ''}
         out = self.__send_expect('dmesg | grep "DDP package" | tail -1', '# ')
         if 'could not load' in out:
             print(RED(out))
@@ -140,6 +143,36 @@ class NetDevice(object):
                 self.pkg['version'] = pkg_info[1].strip()
         return self.pkg
 
+    @nic_has_driver
+    def get_driver_firmware(self):
+        """
+        Get NIC driver and firmware version.
+        """
+        get_driver_firmware = getattr(
+            self, 'get_driver_firmware_%s' %
+                  self.__get_os_type())
+        get_driver_firmware()
+
+    def get_driver_firmware_linux(self):
+        """
+        Get NIC driver and firmware version.
+        """
+        rexp = 'version:\s.+'
+        pattern = re.compile(rexp)
+        out = self.__send_expect('ethtool -i {} | grep version'.format(self.intf_name), '# ')
+        driver_firmware = pattern.findall(out)
+        if len(driver_firmware) > 1:
+            self.driver_version = driver_firmware[0].split(': ')[-1].strip()
+            self.firmware = driver_firmware[1].split(': ')[-1].strip()
+
+        return self.driver_version, self.firmware
+
+    def get_driver_firmware_freebsd(self):
+        """
+        Get the NIC driver and firmware version.
+        """
+        NotImplemented
+
     def get_nic_socket(self):
         """
         Get socket id of specified pci device.
-- 
2.17.1

[dts] [PATCH V1 2/6] framework/dut: get nic driver/firmware info

[Haiyang Zhao]

get nic driver/firmware info in dut prerequisites.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 framework/dut.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/framework/dut.py b/framework/dut.py
index 5bc84fca..d31ba42c 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -424,13 +424,13 @@ class Dut(Crb):
         retry_times = 3
         if self.ports_info:
             self.nic = self.ports_info[0]['port']
-            # TODO: get nic driver/firmware version
+            self.nic.get_driver_firmware()
             if self.nic.default_driver == 'ice':
                 self.get_nic_pkg(retry_times)
 
     def get_nic_pkg(self, retry_times=3):
         self.nic.pkg = self.nic.get_nic_pkg()
-        while not self.nic.pkg and retry_times > 0:
+        while not self.nic.pkg.get('type') and retry_times > 0:
             self.restore_interfaces()
             self.nic.pkg = self.nic.get_nic_pkg()
             retry_times = retry_times - 1
-- 
2.17.1

[dts] [PATCH V1 3/6] framework/test_result: add driver/firmware/package

[Haiyang Zhao]

add driver/firmware/package in result and the related get method,
and also update the index of other elements.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 framework/test_result.py | 83 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 3 deletions(-)

diff --git a/framework/test_result.py b/framework/test_result.py
index 35607451..6dd94451 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -46,6 +46,11 @@ class Result(object):
 
     internals = [
         'dut1', [
+            'kdriver',
+            'firmware',
+            'pkg',
+            'driver',
+            'dpdk_version',
             'target1', 'nic1', [
                 'suite1', [
                     'case1', ['PASSED', ''],
@@ -92,17 +97,85 @@ class Result(object):
         """
         try:
             dut_idx = self.__internals.index(dut)
-            return self.__internals[dut_idx + 1][0]
+            return self.__internals[dut_idx + 1][4]
         except:
             return ''
 
     def __set_dpdk_version(self, dpdk_version):
-        if not self.internals[self.__dut + 1]:
+        if dpdk_version not in self.internals[self.__dut + 1]:
             self.internals[self.__dut + 1].append(dpdk_version)
 
     def __get_dpdk_version(self):
+        return self.internals[self.__dut + 1][3]
+
+    def current_kdriver(self, dut):
+        """
+        Returns the driver version for a given DUT
+        """
+        try:
+            dut_idx = self.__internals.index(dut)
+            return self.__internals[dut_idx + 1][0]
+        except:
+            return ''
+
+    def __set_kdriver(self, driver):
+        if not self.internals[self.__dut + 1]:
+            self.internals[self.__dut + 1].append(driver)
+
+    def __get_kdriver(self):
         return self.internals[self.__dut + 1][0]
 
+    def current_firmware_version(self, dut):
+        """
+        Returns the firmware version for a given DUT
+        """
+        try:
+            dut_idx = self.__internals.index(dut)
+            return self.__internals[dut_idx + 1][1]
+        except:
+            return ''
+
+    def __set_firmware(self, firmware):
+        if firmware not in self.internals[self.__dut + 1]:
+            self.internals[self.__dut + 1].append(firmware)
+
+    def __get_firmware(self):
+        return self.internals[self.__dut + 1][1]
+
+    def current_package_version(self, dut):
+        """
+        Returns the DDP package version for a given DUT
+        """
+        try:
+            dut_idx = self.__internals.index(dut)
+            return self.__internals[dut_idx + 1][2]
+        except:
+            return ''
+
+    def __set_ddp_package(self, package):
+        if package not in self.internals[self.__dut + 1]:
+            self.internals[self.__dut + 1].append(package)
+
+    def __get_ddp_package(self):
+        return self.internals[self.__dut + 1][2]
+
+    def current_driver(self, dut):
+        """
+        Returns the DDP package version for a given DUT
+        """
+        try:
+            dut_idx = self.__internals.index(dut)
+            return self.__internals[dut_idx + 1][3]
+        except:
+            return ''
+
+    def __set_driver(self, package):
+        if package not in self.internals[self.__dut + 1]:
+            self.internals[self.__dut + 1].append(package)
+
+    def __get_driver(self):
+        return self.internals[self.__dut + 1][3]
+
     def __current_targets(self):
         return self.internals[self.__dut + 1]
 
@@ -210,7 +283,7 @@ class Result(object):
             dut_idx = self.__internals.index(dut)
         except:
             return None
-        return self.__internals[dut_idx + 1][1::3]
+        return self.__internals[dut_idx + 1][5::3]
 
     def current_nic(self, dut, target):
         """
@@ -305,6 +378,10 @@ class Result(object):
     """
     dut = property(__get_dut, __set_dut)
     dpdk_version = property(__get_dpdk_version, __set_dpdk_version)
+    kdriver = property(__get_kdriver, __set_kdriver)
+    driver = property(__get_driver, __set_driver)
+    firmware = property(__get_firmware, __set_firmware)
+    package = property(__get_ddp_package, __set_ddp_package)
     target = property(__get_target, __set_target)
     test_suite = property(__get_test_suite, __set_test_suite)
     test_case = property(__get_test_case, __set_test_case)
-- 
2.17.1

[dts] [PATCH V1 4/6] framework/dts: add more nic configurations in result

[Haiyang Zhao]

add nic driver/firmware/package version in result.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 framework/dts.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/framework/dts.py b/framework/dts.py
index d4a8c6e9..6f621a51 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -624,6 +624,10 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
         if dts_run_prerequisties(duts, tester, pkgName, patch, dts_commands, serializer) is False:
             dts_crbs_exit(duts, tester)
             continue
+        result.kdriver = duts[0].nic.default_driver + '-' + duts[0].nic.driver_version
+        result.firmware = duts[0].nic.firmware
+        result.package = duts[0].nic.pkg['type'] + ' ' + duts[0].nic.pkg['version'] if duts[0].nic.pkg else None
+        result.driver = settings.load_global_setting(settings.HOST_DRIVER_SETTING)
         result.dpdk_version = duts[0].dpdk_version
         dts_run_target(duts, tester, targets, test_suites, subtitle)
 
-- 
2.17.1

[dts] [PATCH V1 5/6] framework/excel_reporter: add driver/firmware/pkg

[Haiyang Zhao]

add driver/firmware/pkg info in excel format result file.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 framework/excel_reporter.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/framework/excel_reporter.py b/framework/excel_reporter.py
index 89b9a175..747f577e 100644
--- a/framework/excel_reporter.py
+++ b/framework/excel_reporter.py
@@ -193,8 +193,20 @@ class ExcelReporter(object):
 
     def __write_nic(self, dut, target):
         nic = self.result.current_nic(dut, target)
+        driver = self.result.current_driver(dut)
+        kdriver = self.result.current_kdriver(dut)
+        firmware = self.result.current_firmware_version(dut)
+        pkg = self.result.current_package_version(dut)
         self.col += 1
+        self.sheet.col(self.col).width = 32 * 256  # 32 characters
         self.sheet.write(self.row, self.col, nic, self.title_style)
+        self.sheet.write(self.row+1, self.col, 'driver: ' + driver)
+        self.sheet.write(self.row+2, self.col, 'kdriver: ' + kdriver)
+        self.sheet.write(self.row+3, self.col, 'firmware: ' + firmware)
+        if pkg is not None:
+            self.sheet.write(self.row+4, self.col, 'pkg: ' + pkg)
+            self.row = self.row + 1
+        self.row = self.row + 3
         self.__write_suites(dut, target)
         self.col -= 1
 
-- 
2.17.1

[dts] [PATCH V1 6/6] framework/json_reporter: add driver/firmware/pkg

[Haiyang Zhao]

add driver/firmware/pkg info in json format result file.

Signed-off-by: Haiyang Zhao <haiyangx.zhao@intel.com>
---
 framework/json_reporter.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/framework/json_reporter.py b/framework/json_reporter.py
index 88099cb8..f7025c2c 100644
--- a/framework/json_reporter.py
+++ b/framework/json_reporter.py
@@ -65,8 +65,15 @@ class JSONReporter(object):
             return 'fail'
         target_map = {}
         target_map['dpdk_version'] = result.current_dpdk_version(dut)
+        target_map['nic'] = {}
         for target in result.all_targets(dut):
+            target_map['nic']['name'] = result.current_nic(dut, target)
             target_map[target] = self.__scan_target(result, dut, target)
+            target_map['nic']['kdriver'] = result.current_kdriver(dut)
+            target_map['nic']['driver'] = result.current_driver(dut)
+            target_map['nic']['firmware'] = result.current_firmware_version(dut)
+            if result.current_package_version(dut) is not None:
+                target_map['nic']['pkg'] = result.current_package_version(dut)
         return target_map
 
     def save(self, result):
-- 
2.17.1

Re: [dts] [PATCH V1 0/6] framework: add dirver/firmware/pkg in result

[Zhao, HaiyangX]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]


> -----Original Message-----
> From: Zhao, HaiyangX <haiyangx.zhao@intel.com>
> Sent: Monday, March 22, 2021 10:46
> To: dts@dpdk.org
> Cc: Tu, Lijuan <lijuan.tu@intel.com>; Zhao, HaiyangX
> <haiyangx.zhao@intel.com>
> Subject: [dts][PATCH V1 0/6] framework: add dirver/firmware/pkg in result
> 
> For the convenience of debugging and analysis, add the NIC firmware, drvier,
> pkg version in excel and json format result file.
> 
> Haiyang Zhao (6):
>   nics/net_device: add driver/firmware
>   framework/dut: get nic driver/firmware info
>   framework/test_result: add driver/firmware/package
>   framework/dts: add more nic configurations in result
>   framework/excel_reporter: add driver/firmware/pkg
>   framework/json_reporter: add driver/firmware/pkg
> 
>  framework/dts.py            |  4 ++
>  framework/dut.py            |  4 +-
>  framework/excel_reporter.py | 12 ++++++  framework/json_reporter.py  |
> 7 ++++
>  framework/test_result.py    | 83
> +++++++++++++++++++++++++++++++++++--
>  nics/net_device.py          | 35 +++++++++++++++-
>  6 files changed, 139 insertions(+), 6 deletions(-)
> 
> --
> 2.17.1

Tested-by: Haiyang Zhao <haiyangx.zhao@intel.com>

[-- Attachment #2: TestUnitTestsCmdline.log --]
[-- Type: application/octet-stream, Size: 2351 bytes --]

22/03/2021 10:30:24                            dts: 
TEST SUITE : TestUnitTestsCmdline
22/03/2021 10:30:24                            dts: NIC :        columbiaville_100g
22/03/2021 10:30:24              dut.10.240.183.72: 
22/03/2021 10:30:24                         tester: 
22/03/2021 10:30:24           TestUnitTestsCmdline: Test Case test_cmdline Begin
22/03/2021 10:30:24              dut.10.240.183.72: 
22/03/2021 10:30:24                         tester: 
22/03/2021 10:30:24              dut.10.240.183.72: x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87 -n 4   --file-prefix=dpdk_11658_20210322103002  
22/03/2021 10:30:25              dut.10.240.183.72: EAL: Detected 88 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/dpdk_11658_20210322103002/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:05:00.0 (socket 0)
ice_load_pkg_type(): Active package is: 1.3.20.0, ICE OS Default Package
EAL: Ignore mapping IO port bar(1)
EAL: Ignore mapping IO port bar(4)
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:05:00.1 (socket 0)
ice_load_pkg_type(): Active package is: 1.3.20.0, ICE OS Default Package
APP: HPET is not enabled, using TSC as default timer
22/03/2021 10:30:25              dut.10.240.183.72: cmdline_autotest
22/03/2021 10:30:25              dut.10.240.183.72: 
Testind parsing ethernet addresses...
Testind parsing port lists...
Testind parsing numbers...
Testing parsing IP addresses...
Testing parsing strings...
Testing circular buffer...
Testing library functions...
Test OK
22/03/2021 10:30:25              dut.10.240.183.72: quit
22/03/2021 10:30:26              dut.10.240.183.72: 
22/03/2021 10:30:26           TestUnitTestsCmdline: Test Case test_cmdline Result PASSED:
22/03/2021 10:30:26                            dts: 
TEST SUITE ENDED: TestUnitTestsCmdline

Re: [dts] [PATCH V1 0/6] framework: add dirver/firmware/pkg in result

[Tu, Lijuan]

> -----Original Message-----
> From: Zhao, HaiyangX <haiyangx.zhao@intel.com>
> Sent: 2021年3月22日 10:59
> To: dts@dpdk.org
> Cc: Tu, Lijuan <lijuan.tu@intel.com>
> Subject: RE: [dts][PATCH V1 0/6] framework: add dirver/firmware/pkg in result
> 
> 
> > -----Original Message-----
> > From: Zhao, HaiyangX <haiyangx.zhao@intel.com>
> > Sent: Monday, March 22, 2021 10:46
> > To: dts@dpdk.org
> > Cc: Tu, Lijuan <lijuan.tu@intel.com>; Zhao, HaiyangX
> > <haiyangx.zhao@intel.com>
> > Subject: [dts][PATCH V1 0/6] framework: add dirver/firmware/pkg in
> > result
> >
> > For the convenience of debugging and analysis, add the NIC firmware,
> > drvier, pkg version in excel and json format result file.
> >
> > Haiyang Zhao (6):
> >   nics/net_device: add driver/firmware
> >   framework/dut: get nic driver/firmware info
> >   framework/test_result: add driver/firmware/package
> >   framework/dts: add more nic configurations in result
> >   framework/excel_reporter: add driver/firmware/pkg
> >   framework/json_reporter: add driver/firmware/pkg
> >
> >  framework/dts.py            |  4 ++
> >  framework/dut.py            |  4 +-
> >  framework/excel_reporter.py | 12 ++++++  framework/json_reporter.py
> > |
> > 7 ++++
> >  framework/test_result.py    | 83
> > +++++++++++++++++++++++++++++++++++--
> >  nics/net_device.py          | 35 +++++++++++++++-
> >  6 files changed, 139 insertions(+), 6 deletions(-)
> >
> > --
> > 2.17.1
> 
> Tested-by: Haiyang Zhao <haiyangx.zhao@intel.com>

Applied, thanks