test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [V1] framework/*: fix bug of excel/json report is empty
@ 2022-04-26  3:07 Jun Dong
  2022-04-26  3:22 ` lijuan.tu
  0 siblings, 1 reply; 2+ messages in thread
From: Jun Dong @ 2022-04-26  3:07 UTC (permalink / raw)
  To: dts; +Cc: lijuan.tu, qingx.sun, junx.dong

In execution.cfg, assume that we defined multiple section
of configs that they can have different driver,crbs,target
and suite list. If one target seted failure(e.g.: driver
loaded failure ), then the related dut and target will be
recorded in a failure target collect(like a blacklist),
and the excel/json test result report about the dut and
target that be in blacklist will be empty, even if other
section that have the same dut and target executed normally.

so, if one dut and target related section executed normally,
we can try to removing it over from the blacklist, then the
excel/json report will generated normally.

last, if the lastted section's target seted failure, and
framework try to generated the report again in the end
point of the process. the excel/json report also be empty, the
empty report will replace that already normally generated
report, and the end point save operation actually is
meaningless because every suite have saved result to report.

Signed-off-by: Jun Dong <junx.dong@intel.com>
---
 framework/dts.py         |  8 +++++---
 framework/test_result.py | 15 +++++++++++++++
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/framework/dts.py b/framework/dts.py
index d31beccb..4003d5c3 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -71,7 +71,7 @@ from .utils import (
 imp.reload(sys)
 
 requested_tests = None
-result = None
+result: Result = None
 excel_report = None
 json_report = None
 stats_report = None
@@ -407,6 +407,8 @@ def dts_run_prerequisties(duts, tester, pkgName, patch, dts_commands, serializer
         serializer.discard_cache()
         settings.report_error("DUT_SETUP_ERR")
         return False
+    else:
+        result.remove_failed_dut(duts[0])
 
 
 def dts_run_target(duts, tester, targets, test_suites, subtitle):
@@ -435,6 +437,8 @@ def dts_run_target(duts, tester, targets, test_suites, subtitle):
             log_handler.error(" !!! DEBUG IT: " + traceback.format_exc())
             result.add_failed_target(result.dut, target, str(ex))
             continue
+        else:
+            result.remove_failed_target(result.dut, target)
 
         dts_run_suite(duts, tester, test_suites, target, subtitle)
 
@@ -675,8 +679,6 @@ def run_all(
 
         dts_crbs_exit(duts, tester)
 
-    save_all_results()
-
 
 def show_speedup_options_messages(read_cache, skip_setup):
     if read_cache:
diff --git a/framework/test_result.py b/framework/test_result.py
index abf8edb9..2d7cda4b 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -386,6 +386,13 @@ class Result(object):
         """
         self.__failed_duts[dut] = msg
 
+    def remove_failed_dut(self, dut):
+        """
+        Remove the given DUT from failed duts collection
+        """
+        if dut in self.__failed_duts:
+            self.__failed_duts.pop(dut)
+
     def is_dut_failed(self, dut):
         """
         True if the given DUT was marked as failing
@@ -404,6 +411,14 @@ class Result(object):
         """
         self.__failed_targets[dut + target] = msg
 
+    def remove_failed_target(self, dut, target):
+        """
+        Remove the given DUT, target from failed targets collection
+        """
+        key_word = dut + target
+        if key_word in self.__failed_targets:
+            self.__failed_targets.pop(key_word)
+
     def is_target_failed(self, dut, target):
         """
         True if the given DUT,target were marked as failing
-- 
2.33.1.windows.1


^ permalink raw reply	[flat|nested] 2+ messages in thread

* [dts] [V1] framework/*: fix bug of excel/json report is empty
  2022-04-26  3:07 [dts] [V1] framework/*: fix bug of excel/json report is empty Jun Dong
@ 2022-04-26  3:22 ` lijuan.tu
  0 siblings, 0 replies; 2+ messages in thread
From: lijuan.tu @ 2022-04-26  3:22 UTC (permalink / raw)
  To: dts, Jun Dong; +Cc: lijuan.tu, qingx.sun, junx.dong

On Tue, 26 Apr 2022 11:07:11 +0800, Jun Dong <junx.dong@intel.com> wrote:
> In execution.cfg, assume that we defined multiple section
> of configs that they can have different driver,crbs,target
> and suite list. If one target seted failure(e.g.: driver
> loaded failure ), then the related dut and target will be
> recorded in a failure target collect(like a blacklist),
> and the excel/json test result report about the dut and
> target that be in blacklist will be empty, even if other
> section that have the same dut and target executed normally.
> 
> so, if one dut and target related section executed normally,
> we can try to removing it over from the blacklist, then the
> excel/json report will generated normally.
> 
> last, if the lastted section's target seted failure, and
> framework try to generated the report again in the end
> point of the process. the excel/json report also be empty, the
> empty report will replace that already normally generated
> report, and the end point save operation actually is
> meaningless because every suite have saved result to report.
> 
> Signed-off-by: Jun Dong <junx.dong@intel.com>

Acked-by: Lijuan Tu <lijuan.tu@intel.com>
Applied, thanks

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-04-26  3:22 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-26  3:07 [dts] [V1] framework/*: fix bug of excel/json report is empty Jun Dong
2022-04-26  3:22 ` lijuan.tu

test suite reviews and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ http://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git