test suite reviews and discussions
 help / color / mirror / Atom feed
From: Jun Dong <junx.dong@intel.com>
To: dts@dpdk.org
Cc: lijuan.tu@intel.com, qingx.sun@intel.com, junx.dong@intel.com
Subject: [dts] [V1] framework/*: fix bug of excel/json report is empty
Date: Tue, 26 Apr 2022 11:07:11 +0800	[thread overview]
Message-ID: <20220426030711.6754-1-junx.dong@intel.com> (raw)

In execution.cfg, assume that we defined multiple section
of configs that they can have different driver,crbs,target
and suite list. If one target seted failure(e.g.: driver
loaded failure ), then the related dut and target will be
recorded in a failure target collect(like a blacklist),
and the excel/json test result report about the dut and
target that be in blacklist will be empty, even if other
section that have the same dut and target executed normally.

so, if one dut and target related section executed normally,
we can try to removing it over from the blacklist, then the
excel/json report will generated normally.

last, if the lastted section's target seted failure, and
framework try to generated the report again in the end
point of the process. the excel/json report also be empty, the
empty report will replace that already normally generated
report, and the end point save operation actually is
meaningless because every suite have saved result to report.

Signed-off-by: Jun Dong <junx.dong@intel.com>
---
 framework/dts.py         |  8 +++++---
 framework/test_result.py | 15 +++++++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/framework/dts.py b/framework/dts.py
index d31beccb..4003d5c3 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -71,7 +71,7 @@ from .utils import (
 imp.reload(sys)
 
 requested_tests = None
-result = None
+result: Result = None
 excel_report = None
 json_report = None
 stats_report = None
@@ -407,6 +407,8 @@ def dts_run_prerequisties(duts, tester, pkgName, patch, dts_commands, serializer
         serializer.discard_cache()
         settings.report_error("DUT_SETUP_ERR")
         return False
+    else:
+        result.remove_failed_dut(duts[0])
 
 
 def dts_run_target(duts, tester, targets, test_suites, subtitle):
@@ -435,6 +437,8 @@ def dts_run_target(duts, tester, targets, test_suites, subtitle):
             log_handler.error(" !!! DEBUG IT: " + traceback.format_exc())
             result.add_failed_target(result.dut, target, str(ex))
             continue
+        else:
+            result.remove_failed_target(result.dut, target)
 
         dts_run_suite(duts, tester, test_suites, target, subtitle)
 
@@ -675,8 +679,6 @@ def run_all(
 
         dts_crbs_exit(duts, tester)
 
-    save_all_results()
-
 
 def show_speedup_options_messages(read_cache, skip_setup):
     if read_cache:
diff --git a/framework/test_result.py b/framework/test_result.py
index abf8edb9..2d7cda4b 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -386,6 +386,13 @@ class Result(object):
         """
         self.__failed_duts[dut] = msg
 
+    def remove_failed_dut(self, dut):
+        """
+        Remove the given DUT from failed duts collection
+        """
+        if dut in self.__failed_duts:
+            self.__failed_duts.pop(dut)
+
     def is_dut_failed(self, dut):
         """
         True if the given DUT was marked as failing
@@ -404,6 +411,14 @@ class Result(object):
         """
         self.__failed_targets[dut + target] = msg
 
+    def remove_failed_target(self, dut, target):
+        """
+        Remove the given DUT, target from failed targets collection
+        """
+        key_word = dut + target
+        if key_word in self.__failed_targets:
+            self.__failed_targets.pop(key_word)
+
     def is_target_failed(self, dut, target):
         """
         True if the given DUT,target were marked as failing
-- 
2.33.1.windows.1


             reply	other threads:[~2022-04-26  3:07 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-26  3:07 Jun Dong [this message]
2022-04-26  3:22 ` lijuan.tu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220426030711.6754-1-junx.dong@intel.com \
    --to=junx.dong@intel.com \
    --cc=dts@dpdk.org \
    --cc=lijuan.tu@intel.com \
    --cc=qingx.sun@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).