From: Marvin Liu <yong.liu@intel.com>
To: dts@dpdk.org
Cc: Marvin Liu <yong.liu@intel.com>
Subject: [dts] [PATCH v2 3/4] framework: enhance test case execution process
Date: Wed, 26 Jul 2017 05:20:59 -0400 [thread overview]
Message-ID: <1501060860-63777-4-git-send-email-yong.liu@intel.com> (raw)
In-Reply-To: <1501060860-63777-1-git-send-email-yong.liu@intel.com>
1. Support suite and case level configuration in test_case module.
2. Debugger rerun command will call _execute_test_case which can do more
things than just call case object.
Signed-off-by: Marvin Liu <yong.liu@intel.com>
diff --git a/framework/debugger.py b/framework/debugger.py
index 8375590..c3ff33d 100644
--- a/framework/debugger.py
+++ b/framework/debugger.py
@@ -98,7 +98,8 @@ def rerun_command():
# re-run specified test case
for case in suite_obj._get_test_cases(r'%s' % AliveCase):
if callable(case):
- case()
+ suite_obj.logger.info("Rerun Test Case %s Begin" % case.__name__)
+ suite_obj._execute_test_case(case)
def exit_command():
diff --git a/framework/test_case.py b/framework/test_case.py
index 31a5eaf..8671ffa 100644
--- a/framework/test_case.py
+++ b/framework/test_case.py
@@ -45,6 +45,7 @@ from settings import report_error
from rst import RstReport
from test_result import ResultTable, Result
from logger import getLogger
+from config import SuiteConf
class TestCase(object):
@@ -254,6 +255,12 @@ class TestCase(object):
self._suite_result.test_case = case_obj.__name__
self._rst_obj.write_title("Test Case: " + case_name)
+
+ # load suite configuration file here for rerun command
+ self._suite_conf = SuiteConf(self.suite_name)
+ self.case_cfg = self._suite_conf.load_case_config(case_name)
+ del(self._suite_conf)
+
case_result = True
if self._check_inst is not None:
if self._check_inst.case_skip(case_name[len("test_"):]):
@@ -281,15 +288,8 @@ class TestCase(object):
self.tester.get_session_output(timeout=0.1)
# run set_up function for each case
self.set_up()
- # prepare debugger re-run case environment
- if self._enable_debug or self._debug_case:
- debugger.AliveSuite = self
- debugger.AliveModule = __import__('TestSuite_' + self.suite_name)
- debugger.AliveCase = case_name
- if self._debug_case:
- debugger.keyboard_handle(signal.SIGINT, None)
- else:
- case_obj()
+ # run test case
+ case_obj()
self._suite_result.test_case_passed()
@@ -328,22 +328,40 @@ class TestCase(object):
"""
Execute all test cases in one suite.
"""
+ # prepare debugger rerun case environment
+ if self._enable_debug or self._debug_case:
+ debugger.AliveSuite = self
+ debugger.AliveModule = __import__('TestSuite_' + self.suite_name)
if load_global_setting(FUNC_SETTING) == 'yes':
for case_obj in self._get_functional_cases():
for i in range(self.tester.re_run_time + 1):
- if self._execute_test_case(case_obj):
- break
- else:
+ ret = self.execute_test_case(case_obj):
+
+ if ret is False:
for dutobj in self.duts:
dutobj.get_session_output(timeout = 0.5 * (i + 1))
self.tester.get_session_output(timeout = 0.5 * (i + 1))
time.sleep(i + 1)
- self.logger.info(" Test case %s re-run %d time" % (case_obj.__name__, i + 1))
+ self.logger.info(" Test case %s failed and re-run %d time" % (case_obj.__name__, i + 1))
+ else:
+ break
if load_global_setting(PERF_SETTING) == 'yes':
for case_obj in self._get_performance_cases():
- self._execute_test_case(case_obj)
+ self.execute_test_case(case_obj)
+
+ def execute_test_case(self, case_obj):
+ """
+ Execute test case or enter into debug mode.
+ """
+ debugger.AliveCase = case_obj.__name__
+
+ if self._debug_case:
+ self.logger.info("Rerun Test Case %s Begin" % debugger.AliveCase)
+ debugger.keyboard_handle(signal.SIGINT, None)
+ else:
+ return self._execute_test_case(case_obj)
def get_result(self):
return self._suite_result
--
1.9.3
next prev parent reply other threads:[~2017-07-26 9:23 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-26 8:16 [dts] [PATCH v1 0/4] support suite and case level configuration Marvin Liu
2017-07-26 8:16 ` [dts] [PATCH v1 1/4] framework setting: support configuration file folder change Marvin Liu
2017-07-26 8:16 ` [dts] [PATCH v1 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-26 8:16 ` [dts] [PATCH v1 3/4] framework: enhance test case execution process Marvin Liu
2017-07-26 8:16 ` [dts] [PATCH v1 4/4] conf: add suite and case level configuration sample Marvin Liu
2017-07-26 9:20 ` [dts] [PATCH v2 0/4] support suite and case level configuration Marvin Liu
2017-07-26 9:20 ` [dts] [PATCH v2 1/4] framework setting: support configuration file folder change Marvin Liu
2017-07-26 9:20 ` [dts] [PATCH v2 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-26 9:20 ` Marvin Liu [this message]
2017-07-26 9:21 ` [dts] [PATCH v2 4/4] conf: add suite and case level configuration sample Marvin Liu
2017-07-27 1:07 ` [dts] [PATCH v3 0/4] support suite and case level configuration Marvin Liu
2017-07-27 1:07 ` [dts] [PATCH v3 1/4] framework setting: support change configuration file folder Marvin Liu
2017-07-27 1:07 ` [dts] [PATCH v3 2/4] framework config: support suite&case level configuration Marvin Liu
2017-07-27 1:07 ` [dts] [PATCH v3 3/4] framework: enhance test case execution process Marvin Liu
2017-07-27 1:07 ` [dts] [PATCH v3 4/4] conf: add suite and case level configuration sample Marvin Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1501060860-63777-4-git-send-email-yong.liu@intel.com \
--to=yong.liu@intel.com \
--cc=dts@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).