test suite reviews and discussions
 help / color / mirror / Atom feed
From: Marvin Liu <yong.liu@intel.com>
To: dts@dpdk.org
Cc: Marvin Liu <yong.liu@intel.com>
Subject: [dts] [PATCH v1 3/4] framework: enhance test case execution process
Date: Wed, 26 Jul 2017 04:16:11 -0400	[thread overview]
Message-ID: <1501056972-59668-4-git-send-email-yong.liu@intel.com> (raw)
In-Reply-To: <1501056972-59668-1-git-send-email-yong.liu@intel.com>

1. Support suite and case level configuration in test_case module.
2. Debugger rerun command will call _execute_test_case which can do more
things than just call case object.

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/framework/debugger.py b/framework/debugger.py
index 8375590..07c2bdf 100644
--- a/framework/debugger.py
+++ b/framework/debugger.py
@@ -98,7 +98,7 @@ def rerun_command():
         # re-run specified test case
         for case in suite_obj._get_test_cases(r'%s' % AliveCase):
             if callable(case):
-                case()
+                suite_obj._execute_test_case(case)
 
 
 def exit_command():
diff --git a/framework/test_case.py b/framework/test_case.py
index 31a5eaf..2d1a58d 100644
--- a/framework/test_case.py
+++ b/framework/test_case.py
@@ -45,6 +45,7 @@ from settings import report_error
 from rst import RstReport
 from test_result import ResultTable, Result
 from logger import getLogger
+from config import SuiteConf
 
 class TestCase(object):
 
@@ -254,6 +255,12 @@ class TestCase(object):
         self._suite_result.test_case = case_obj.__name__
 
         self._rst_obj.write_title("Test Case: " + case_name)
+
+        # load suite configuration file here for rerun command
+        self._suite_conf = SuiteConf(self.suite_name)
+        self.case_cfg = self._suite_conf.load_case_config(case_name)
+        del(self._suite_conf)
+
         case_result = True
         if self._check_inst is not None:
             if self._check_inst.case_skip(case_name[len("test_"):]):
@@ -281,15 +288,8 @@ class TestCase(object):
             self.tester.get_session_output(timeout=0.1)
             # run set_up function for each case
             self.set_up()
-            # prepare debugger re-run case environment
-            if self._enable_debug or self._debug_case:
-                debugger.AliveSuite = self
-                debugger.AliveModule = __import__('TestSuite_' + self.suite_name)
-                debugger.AliveCase = case_name
-            if self._debug_case:
-                debugger.keyboard_handle(signal.SIGINT, None)
-            else:
-                case_obj()
+            # run test case
+            case_obj()
 
             self._suite_result.test_case_passed()
 
@@ -328,22 +328,38 @@ class TestCase(object):
         """
         Execute all test cases in one suite.
         """
+        # prepare debugger rerun case environment
+        if self._enable_debug or self._debug_case:
+            debugger.AliveSuite = self
+            debugger.AliveModule = __import__('TestSuite_' + self.suite_name)
 
         if load_global_setting(FUNC_SETTING) == 'yes':
             for case_obj in self._get_functional_cases():
                 for i in range(self.tester.re_run_time + 1):
-                    if self._execute_test_case(case_obj):
+                    if self.execute_test_case(case_obj):
                         break
                     else:
                         for dutobj in self.duts:
                             dutobj.get_session_output(timeout = 0.5 * (i + 1))
                         self.tester.get_session_output(timeout = 0.5 * (i + 1))
                         time.sleep(i + 1)
-                        self.logger.info(" Test case %s re-run %d time" % (case_obj.__name__, i + 1))
+                        self.logger.info(" Test case %s failed and re-run %d time" % (case_obj.__name__, i + 1))
 
         if load_global_setting(PERF_SETTING) == 'yes':
             for case_obj in self._get_performance_cases():
-                self._execute_test_case(case_obj)
+                self.execute_test_case(case_obj)
+
+    def execute_test_case(self, case_obj):
+        """
+        Execute test case or enter into debug mode.
+        """
+        debugger.AliveCase = case_obj.__name__
+
+        if self._debug_case:
+            self.logger.info("Rerun Test Case %s Begin" % debugger.AliveCase)
+            debugger.keyboard_handle(signal.SIGINT, None)
+        else:
+            self._execute_test_case(case_obj)
 
     def get_result(self):
         return self._suite_result
-- 
1.9.3

  parent reply	other threads:[~2017-07-26  8:18 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-07-26  8:16 [dts] [PATCH v1 0/4] support suite and case level configuration Marvin Liu
2017-07-26  8:16 ` [dts] [PATCH v1 1/4] framework setting: support configuration file folder change Marvin Liu
2017-07-26  8:16 ` [dts] [PATCH v1 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-26  8:16 ` Marvin Liu [this message]
2017-07-26  8:16 ` [dts] [PATCH v1 4/4] conf: add suite and case level configuration sample Marvin Liu
2017-07-26  9:20 ` [dts] [PATCH v2 0/4] support suite and case level configuration Marvin Liu
2017-07-26  9:20   ` [dts] [PATCH v2 1/4] framework setting: support configuration file folder change Marvin Liu
2017-07-26  9:20   ` [dts] [PATCH v2 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-26  9:20   ` [dts] [PATCH v2 3/4] framework: enhance test case execution process Marvin Liu
2017-07-26  9:21   ` [dts] [PATCH v2 4/4] conf: add suite and case level configuration sample Marvin Liu
2017-07-27  1:07   ` [dts] [PATCH v3 0/4] support suite and case level configuration Marvin Liu
2017-07-27  1:07     ` [dts] [PATCH v3 1/4] framework setting: support change configuration file folder Marvin Liu
2017-07-27  1:07     ` [dts] [PATCH v3 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-27  1:07     ` [dts] [PATCH v3 3/4] framework: enhance test case execution process Marvin Liu
2017-07-27  1:07     ` [dts] [PATCH v3 4/4] conf: add suite and case level configuration sample Marvin Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1501056972-59668-4-git-send-email-yong.liu@intel.com \
    --to=yong.liu@intel.com \
    --cc=dts@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).