[dts] [PATCH V2 0/5] add check case list for skip the case, when the case can't test

[dts] [PATCH V2 0/5] add check case list for skip the case, when the case can't test

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

this patch for skip case, that case can't work in test

huilong xu (5):
  add check case list
  add N/A result in test report
  add case skip result when case running
  add case skip result when case running
  add check case function in run case module

 conf/dpdk_test_case_checklist.xls |  Bin 0 -> 19456 bytes
 framework/checkCase.py            |  100 +++++++++++++++++++++++++++++++++++++
 framework/dts.py                  |   21 ++++++--
 framework/excel_reporter.py       |    9 ++-
 framework/test_result.py          |    6 ++
 5 files changed, 127 insertions(+), 9 deletions(-)
 create mode 100644 conf/dpdk_test_case_checklist.xls
 create mode 100644 framework/checkCase.py

-- 
1.7.4.4

[dts] [PATCH V2 1/5] add check case list

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

1. add a excel file(excel 1997-2003) for set skip case check list 2. table format
   +---------+---+----+------+----------+--------+
   |case name|os |nic |target|wq request|comments|
   +---------+---+----+------+----------+--------+
   case name is which case need ti skip.
   os is which os need to skip, if mutil os, need used ',' to division. the dut os set in crbs.py, and dut provide function, os_get_type()
   nic is which nic type to skip, if mutil nic type, need used ',' to division
   target as same with nic
   wq request and comments is the message, when case skip.
   if you want ignore some check option, you can set value is "all"

Signed-off-by: huilong xu <huilongx.xu@intel.com>

diff --git a/conf/dpdk_test_case_checklist.xls b/conf/dpdk_test_case_checklist.xls
new file mode 100644
index 0000000000000000000000000000000000000000..ae88705bd0e2b9d1e278e3fdd47893d809134047
GIT binary patch
literal 19456
zcmeHPdvILUc|Uh|rCnLJSMm$v=URTrvMgjt7&|tW9}we4eh{27otVn&)m>}j)vj2r
zjNL(lXp?^m%@DbP1e`KX$_y<jX$&Q#rNwkQqz}<#+E9iRIwTV^A?dWmO{Wb5Y=6IV
z_U^s+?7dgJfoAHCujbsn=ey_pzVCeBcfRwu_v$;}Zhhmz*Oz=>Xva<w7VnpuM5BXl
z;F?JuEkdlq1@rHhN+m@U0+&~hD@X(HLDtnUUKlBY)PO{rN0DMkO-QqlnvrHB%|V)r
z6i0#!h*qR|Nb`{vAT2~%gmewkVx()4mLM%fT86Y7X$8_sqy!RuhiQ%D`U+Ct6))<?
zMIL`e*y3i9!*4;Hg(Ye~7QtgIC4^8d5#w2iOY$r}%AdXIsXw{n+ut$7qd$3zzipsR
zs$$|OxEI7JaYD**O4X|4JWqrS`mH5HEmg^S=MGh}LEI+B#R&eAvhDk!g|w83l%mMs
zIu`)i4*raR>_3ztps4HCUzKwW;wErNNf}IOf%@dx3jS8xYvoUU{jAKtDTZF$*dU%*
z`BL<+5zQNM^ZOzs{wVSS&O-bE>6jShb>f$K?2$4g4a#r{3ck^3gmfY3`8>`oh%3AU
z>*G9Mp1&r`J}1vN;?BRrb}w)Hm6cmjg5%Wfp<S}vZ_9I=JkQ115Z{n>z9%W05uIK`
z`IvZsW(|oYQDZMWAq%t<@Q{r7zgD7YMoAaUP{I&>;yEhPx3vbwf#4D$MEsdF)*4u%
z6a8pwAeYFDC0>IygE45^oq=+&MBj`hcFa)15dDE|8TM$)NZG@ZWgj!(g~3|q*zBkE
z6_I6#%@~K<*0ineT)nz`aKrXv?N;X4h82-j80nX}?tk**=FlDPLT#7|V9>Veb#=RO
zkSEl7h}x!?($LC9Zx9`#M~Fd+(vh$vD`$_jOTw`Y+slNuO>K~SuzlAcxO9mP_(N4^
zy^e&lQb)p7qhEB2kGPqs7P$FI(Ht!M`)#IM#75Y|W@^>1@U<#S)^^P|<a|ho|12fo
zQPy!OK{{tySLq*~9;jUdbC}8z()9dv`A+0fxBe^iKlGvhfe-yjANnIc^so8Qzv@H(
zk`MhpAG)7Be)vyG9M1HmA6E46yFPR!Pjz`EPc{Abd~m+*Lx0+b{^vgQ-}0e<(}!NK
zJbx^4qE-4)e!`WJ)Z6d0wm#JSP_u|UD9dxTr0D7N^DCcUDI(vH^ajKbN&jyYkcZ`;
z8+_=UKJ*SxI#*~C-<sA5h0j%+q*v^#{Fy5`N&l`mj7u!WRC%uM9CTDybguZy^aL)k
z_|xbbU#<>i`HFs&|8vDy#yO5lEDRM6SCNulk@JbN-pcgzt*wii7eUVtOFCDa7QWK6
z(tiWyDUz<~nX6Mt*W_Ga(Z7o+p4eH@16Q$n`MYBa#N9M}<=82fK7aY`iP$DF;i9uZ
z2rK0Ua#g@-BnL?hFCB9c`9swe)J(IE3Tm=eM+GiV>Jo`-gt{tlNUN&?V@6#SqV-mw
zQR?Er?ps#{_Pe?&utU~WfjzaZ3hcslRbbz)s{%WBV1?aUe_*c<`}c>4>?mfME~jRC
ziNr)q2Vsb)0fcVo55l-n0|;HyAB4fA1`xWdKL{gF4Ip%3e-MVL8bIjQ{vh0~n-OGs
zAlN$G?yCVL*gA`A0|~YcHymq_B-lFKv8(|k*gD+utN|q0I#{1MK#*~dC93cA^afjp
zo36Gbz97NY;jV2BAi>t*wr>p}!PZIC1`=$YRkeWxTL(+tnVX|F*8Mx6+J3hNb0B(|
zSnbUFgRH4dl3?qstqmmDI_qi!3AWDq+CYM>(_R7LeCfPsEV<@k>^r<s`qtl-Y{h;~
z8@x84L{tu5Ec?!c@CLbj`Ems$S_Z!7o_k7(@)7f(H92U&md7j)+C)qjssm+}P@LHD
z@NC&*k3Ci;RDXYel~7KaB@}52nwv~YQ%+j-EOsM`2hI!hd2(mH6*kpdp{w2s*HtRe
z+BBd`vDMR(ZWL1ijTNwpa96N=?8Sd``j|eq!S<3PjL@VmuJj!egM&Qw)Mf@L^#_CW
z)&imrH@aQ>0Af2rRdcG{`Fd2a`&fVfX>TQ#roUL_&L3VM(~YvFq6*KBZ*}O>q)`#7
z+C6S?ft)ve+@ti<^9~Tbw{NYehq|m<Pg*`!Dos~FtPphZ;>D`o?xb10oit#}!|Lr$
zTD8r#p{&zp{oXe7ii^&0W2w<evxIVn8%roBtvUo@tJmRqRhp`<u-#QbcU5B|_6cn`
z*ntWW|FsPDu6I>u_qB$DYDsRaNJ4jmG2Hq*7n+BsURd1qbgxZpO6*Rh%v^EvRP2D5
z;z(LPjL3@89PF-li?t#lHX-fCv&EE9PZm{}G4LEe8Zp!V^yb^823}K5V7Hcg6J*n>
zv+1d3(-y!+1tQ6&!AxI%{WWhk^K>@7)oj)Uuu-8(vS~EaKY9CgZ#MIFHhtA>Is(`*
zn1#{lps1*se)Elg@n*9?XLEfuo2~#h41r}fG1%r`|HGTjLY>W)YBoIqY*bK_t!XmT
z@4fY^H=9Ld(--V%E5W9(22YzFcy{`S-fXVX*?4%`>Hs!QPn#b2(Kny<X0upl<Kb!R
z1K2n{ZF=C9A3o>J=31SNho@}}VB_?(>4E7>-}7d(L}%mSY1aj?aeCVHz%#G^*qhB#
zosEa5^#-tUdfN2ByDvZO&1M<dYzg+X1lVk;!P9zrKmXJ#iVXuAj!H~huCwv*v^4>2
zoSxRx`|3lV_hz#~XXD{%?E!3@p4QX*7ytOMH=C6@8xK$G3}EB*w4UCJ7ry1qCZV(O
z@U-p#Hcn6L={@&SIYR5US*5e_@U*@FHcn6L>3!$Y1759}jmOhiw^qi@-P*XhycG2v
zD}^|8@1J>tuC_tpQ99@<4b<sHc=U1jfH&wG8x;PdgRa#;ojxPPBY*m<-k@u3P<V?D
zx<LbVdI(<D8v3#~=sFt|zM+F|(m<X5AjHSM`l2`JdK(m;po4DKK%LeX;=dmJs26BV
zv~yavCpk0~+vc{sv9uHxd$4dDs*J!5oPA+afU_fBbLXHONm}s!0rv3sCbOyWY_gci
z=cZ!!x&cFrO3gyOGQ`)qvVgW<Ba6zJ1VK45wis|%K{9B>yOq4jRgD2PICC*quE~`<
z7mDMp3OtS2J8Y&-CH9)x>{RR?O|EN7vmn<nCW9%waF@Wf3F)$Ewk;KR4CJKH*-0&_
zIQMR>NTqU?R4QjlrE=#&5wUSHT$+j{+@dsu!DKJqdsFY-ftO&}&dElUZP&^+;w@ym
zY(jhzBf%Z4BI|6LjG)dstxlvAL!EKFuT_w9-&FanMsn$zjG{)DRwG)PT~)))YoWM~
zZ@k`oW;C13C5!pO*~Ag^OmQl<&kYs2zSM%3DbL`gIodua?a!AS&qC)%AkG>11D7e5
z^=a&~do`hLSfKCA=i#W^-MGe*Qbg>7((;<Ca&N)H0y7J1F5X(f{@5*MawwBKnZQ~>
z6Exfc&E5im==&K6dJ+PvHG*^jbeNU<ClnlAF$&34AUqeEi$#V59I;OaZ}k9I@y`dm
z)c^RUe?~(H-s=QM4AjBrd4Q`p=mV~%1|bRF=LAPA)B-|yz6Uttp|V)isg`r0S>k%W
z&9gr@Iu2JDcbhD<0PW!$rK9jg8r|+_+{>_x#tn}T*!^2DK|)*_$mC9$L*@SeF-@4o
zh)V;~(5GPZAy)(lSG=(ji>&Cw4UR@AZnn4q@{5D!c(IT~c+mp0u?&IvAb{D@0(^>r
zMXhLLJjD)&)!y^i1}Qr{TxwFFJNCL|H3V25gr^tei@Ag<Yu~`7y;yC++kSvJm@k@B
zu@i2L&@A*9ViX}1x6irik)mLq#0qKL<FEs*%2^>z<*bmVa_2(LVw=D&`l0cnwD?^b
z(}jROBy~`2b7#nBgml@Q1X~i7vm{YDOA?i1DEr7{19Z2>EkZ*B3~?B3E>sr&R2f}$
zJ6-iirkI7cHoI$u<1o(=ct;V2qrR%W-5|sc36U?-1GK$|@U;?x3Rl*TNT+h`FHrAR
z98s;ca&)m)aU*ObjwTB^hAb6c``v<unh~jwf<poQ%8FWEVU>fp)bM9xw~Nm@Hc1{p
z>ss-51Ry75Yj1<dXXVuaw4K3_uY_-d7Bk|s^dP$gd(cUYqD6$tE#kO*b@(%A)%}c|
z@QmArO|!U}lpb!8S(F}9p5^|OMd^JwwpcvukW0C?`t0H>crE)Yj(e<kKV03w-wQ?z
z37=B@)DZ8A-teus$A@!>$EoC)naCwaOza^Yx($));QqaE)?%`7(kucll^+?wzR?(F
z3Y&Yj4&FI#j+<jBwtHZpRfu%vv^iKDAI+MB=}fk077ht<B3UTrbHkZzcC2$Wf0tP}
zjZ3pLoi7yS)u5R?>B^6dW(t|2qktj|j=`c?6EP~CHP4js_Y0BBBy+_~s`FIxE;E}P
z>~mDC4;e%Ar^ZK4<OkCQv>)t#-Ne4>FDmCkAx6ft#f+_svBAyV`}W)}L`BjMSv3p1
zSnk`i%hu5^+89`S>};;AbG*G#yesdt@;o6<V@fkPRxD&j?;K3!bLq@Us8bS)qnRO%
zbZaJeYZ(lW>8cFqtrp0T#oz=c$>JD1_2{jMgD{s_Kz|=A=0^t$CT<mqaKjYmuw&yR
zgYp7ygU8oJ+#MYmO62mz#Mt=gDBLem9L|g-c!T$c;rE$T^?mjTG<md;$-^zrmgO#C
zH5e0ri|0vTFP7sm(XZk60-pZ-03B<*TZm`SgPLRWkc3!|aqKwmzx5tYA{ASN#4B|y
z!VktE1k4A~X&-dl+4H6$9s%tU2aTQle$eiB(1-*52=y&Si|bpA<`Dd%e848+%a}tw
z4#%DYejLz32v>1=-HJ26$%=`0fB)^jKXBr3|DdFF$n!>d{tcWBjJP)QwHQW(=rr!d
zofjhH1=kn9DNBD^p4XvtS{^^@=WE{9b$bQn$Nd`|*B78$A~IAN&<wTEE02HZw|=pv
z!kqoQEe2}1j~$%O$>S4{A|@uJr3^8$9M;6ad<cJQZer_IKFl><%+8}_hE-l*R>^}p
zWC%vVkagXuBCipbG-ethKg)?SjU03ucvaD;OT8S1+$tLR>*X-Fs-jUrdO38SDjK~<
zFNaWDMWeUt<*@%&MWe3ta@4z%_MY@v^(CMXDrdkKsW_DjNzb%$VdSVYbuWV4;!3Uo
zIqHu0SU=8LMma}J>Py`trkx|Eog)fgx?xFLEYA6AIp;*7-@w+DdrlNid*bp-MWRsN
zI0sgV!r83NFDy~0c~#Ph!s)b)!j{G57nCf?LEJ`(NEF&d-E*RF{$!(2BXMq($&#GT
z+9-H(gB-0!lm^J{yyrxrw&M6xCHYO2U9wT4uvZYJ(S_3Jl7)JW%WpYZlB*gUCF*KX
z)P)jtq0j<x`2{IaVlI@J3nk`4iMdc{ow)o4l_*UvlqQK{*{I2d(&R#6nYjFll_>N=
zyG8V-xQsvQo>RNjwrwN!xi}sHDh)QfP?}vR%`RD*T`2U1xQxT(H`|56s27)C*s6O@
zS+Klu$U;wv%kOYma*hjyqlew1IWAe|xKQX*arqT5QRcc(`1PEPGS`K|H=S+0(BI<n
zJ7A(PM%dbAytYvoBb@gvlvl(W@g%xHr*Q-ymHPkf!~@J{=;UG-Pvzr2jJWr?Mw<}4
z7rZ#w@savMsqQaAl#%5pW7mk2j=~r+`79xPQ_v`_#*#8SaT6x`SI!8{23jMt^;`c4
zZH=&&4)8ubZ6UL}Y$EL7V+&PAjk^LC_`p{98$8{=YK8wzD>%%0_410;0J|IKN}TEN
z|1hZ8Dt9VXo}(-2SC9Xn8h9VuWQ}qY3x;+a^out>^XTQD9~y3X;&U;vas5-Tc7yf>
zB>D#L??U3-_7)_Lmq(DWnIMiMaSn3^iL;PTBh5#80Exc#IGl|eR;<fIzvftaAd@QO
z$MWf7;^s3cGpk1POLxy%{pyRxF5I7dXF~oj3Ht2^IfB|BdikZ7&vz|tIrk9S-}&<=
z+5W#oVq4jMwwoV597f{I?{*~0ejJJI9Y*5rz$g;+Kz&dz7mzGHDEVj0`XKxMt^hF0
zQuk_$(Z1}lFMGvo6eKv6N}-PC3#Z1oCpjkT>-sXv&(=n<Ykfj|3Ki-*_#YUYJ)@S|
z)L)^Nf;E<@mw%10H5a>*`5TDtVJv>w2K|4xtb+#Zet0OI&ZJD$9TPp>-Mxv5YsoeE
ho=iHuRjxs0kL{9P+O<{uY}>^<YgXmeg1=-9{12Nhvy1=$

literal 0
HcmV?d00001

-- 
1.7.4.4

[dts] [PATCH V2 2/5] add N/A result in test report

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

Signed-off-by: huilong xu <huilongx.xu@intel.com>

diff --git a/framework/excel_reporter.py b/framework/excel_reporter.py
index 809dead..2030d87 100644
--- a/framework/excel_reporter.py
+++ b/framework/excel_reporter.py
@@ -85,13 +85,15 @@ class ExcelReporter(object):
         self.sheet.write(0, 7, 'Pass', self.header_style)
         self.sheet.write(0, 8, 'Fail', self.header_style)
         self.sheet.write(0, 9, 'Blocked', self.header_style)
-        self.sheet.write(0, 10, 'Not Run', self.header_style)
-        self.sheet.write(0, 11, 'Total', self.header_style)
+        self.sheet.write(0, 10, 'N/A', self.header_style)
+        self.sheet.write(0, 11, 'Not Run', self.header_style)
+        self.sheet.write(0, 12, 'Total', self.header_style)
 
         self.sheet.write(1, 7, Formula('COUNTIF(F2:F2000,"PASSED")'))
         self.sheet.write(1, 8, Formula('COUNTIF(F2:F2000,"FAILED*") + COUNTIF(F2:F2000,"IXA*")'))
         self.sheet.write(1, 9, Formula('COUNTIF(F2:F2000,"BLOCKED*")'))
-        self.sheet.write(1, 11, Formula('H2+I2+J2+K2'))
+        self.sheet.write(1, 10, Formula('COUNTIF(F2:F2000,"N/A*")'))
+        self.sheet.write(1, 12, Formula('H2+I2+J2+K2+L2'))
 
         self.sheet.col(0).width = 4000
         self.sheet.col(1).width = 7500
@@ -105,6 +107,7 @@ class ExcelReporter(object):
         self.sheet.col(9).width = 3000
         self.sheet.col(10).width = 3000
         self.sheet.col(11).width = 3000
+        self.sheet.col(12).width = 3000
 
     def __styles(self):
         header_pattern = xlwt.Pattern()
-- 
1.7.4.4

[dts] [PATCH V2 3/5] add case skip result when case running

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

Signed-off-by: huilong xu <huilongx.xu@intel.com>

diff --git a/framework/test_result.py b/framework/test_result.py
index 79faee1..9cef34c 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -157,6 +157,12 @@ class Result(object):
         """
         self.__set_test_case_result(result='PASSED', message='')
 
+    def test_case_skip(self,message):
+        """
+        set last test case add as N/A
+        """
+        self.__set_test_case_result(result='N/A', message=message)
+
     def test_case_failed(self, message):
         """
         Set last test case added as FAILED
-- 
1.7.4.4

[dts] [PATCH V2 4/5] add case skip result when case running

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

1. add parse_file class for get check case info form execl file, and send check function list from tabel head.
2. add case_skip class for check case is skip or running.
   a)from talbe head get check function list. excep "case name", "wq request", "comments"
   b)when add a new check rule in excel file, you must add a check function in this clase, the function name is "check_rule" and parameter is a list.
     eg, add kernel rule in excel table head. you need add function check_kernerl(self,kernel_type) if you not add the function, it will ignore this rule.

Signed-off-by: huilong xu <huilongx.xu@intel.com>

diff --git a/framework/checkCase.py b/framework/checkCase.py
new file mode 100644
index 0000000..b2e79a0
--- /dev/null
+++ b/framework/checkCase.py
@@ -0,0 +1,100 @@
+import xlrd
+
+from settings import nic_name_from_type
+
+filter_file = r'./conf/dpdk_test_case_checklist.xls'
+filter_case = []
+check_function_dict = {}
+
+class parse_file():
+
+    def __init__(self):
+        try:
+            self.book = xlrd.open_workbook(filter_file)
+            self.sheet = self.book.sheet_by_index(0)
+            self.init_check_function_dict()
+        except:
+            pass
+
+    def init_check_function_dict(self):
+        '''
+        init check case functio, and skip case message.
+        '''
+        row_data  = self.sheet.row_values(0)
+        for i in range(1,len(row_data)):
+            if  row_data[i].lower() in ['wq number', 'comments']:
+                if 'message' not in check_function_dict:
+                    check_function_dict['message'] = [i]
+                else:
+                    check_function_dict['message'].append(i)
+            else:
+                check_function_dict[row_data[i].lower()] = i       
+
+    def set_filter_case(self):
+        for row in range(self.sheet.nrows):   
+            row_data = self.sheet.row_values(row)
+            # add case name
+            tmp_filter = [row_data[0]]
+            for i in range(1,len(row_data) - 2):
+                tmp_filter.append(row_data[i].split(','))
+
+            tmp_filter.append(row_data[-2])
+            tmp_filter.append(row_data[-1])
+
+            filter_case.append(tmp_filter)
+
+class check_case_skip():
+    def __init__(self, Dut):
+        self.dut = Dut
+        self.comments = ''
+
+    def check_os(self,os_type):
+        if 'all' == os_type[0].lower():
+            return True
+        dut_os_type = self.dut.get_os_type()
+        if dut_os_type in os_type:
+            return True
+        else:
+            return False
+
+    def check_nic(self, nic_type):
+        if 'all' == nic_type[0].lower():
+            return True
+        dut_nic_type = nic_name_from_type(self.dut.ports_info[0]['type'])
+        if dut_nic_type in nic_type:
+            return True
+        else:
+            return False
+
+    def check_target(self,target):
+        if 'all' == target[0].lower():
+            return True
+        if self.dut.target in target:
+            return True
+        else:
+            return False
+
+    def case_skip(self, case_name):
+        skip_flage = False
+        for rule in filter_case[1:]:
+            # check case name
+            if case_name == rule[0]:
+                for key in check_function_dict.keys():
+                    try:
+                        if 'message' == key:
+                            continue 
+                        check_function = getattr(self, 'check_%s' % key)
+                    except:
+                        print "can't check %s type" % key
+                    if check_function(rule[check_function_dict[key]]):
+                        skip_flage = True
+                    else:
+                        skip_flage = False
+                        break
+                if skip_flage: 
+                    if 'message' in check_function_dict:
+                        for i in check_function_dict['message']:
+                            self.comments += '%s,' % rule[i]
+                    return skip_flage
+                
+        return skip_flage
-- 
1.7.4.4

[dts] [PATCH V2 5/5] add check case function in run case module

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

Signed-off-by: huilong xu <huilongx.xu@intel.com>

diff --git a/framework/dts.py b/framework/dts.py
index e9513c6..613926c 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -57,7 +57,7 @@ from logger import getLogger
 import logger
 import debugger
 from virt_scene import VirtScene
-
+from checkCase import *
 import sys
 reload(sys)
 sys.setdefaultencoding('UTF8')
@@ -291,6 +291,8 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
     """
     Run each target in execution targets.
     """
+    global skip_case_mode
+    skip_case_mode = check_case_skip(dut)
     if scenario != '':
         scene = VirtScene(dut, tester, scenario)
     else:
@@ -299,7 +301,7 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
     if scene:
         scene.load_config()
         scene.create_scene()
-
+    
     for target in targets:
         log_handler.info("\nTARGET " + target)
         result.target = target
@@ -402,11 +404,12 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
     global Package
     global Patches
     global scenario
-
+    global check_case_inst
     # save global variable
     Package = pkgName
     Patches = patch
-
+    check_case = parse_file()
+    check_case.set_filter_case()
     # prepare the output folder
     if output_dir == '':
         output_dir = FOLDERS['Output']
@@ -459,7 +462,6 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
 
         # verify if the delimiter is good if the lists are vertical
         dutIP, targets, test_suites, nics, scenario = dts_parse_config(section)
-
         log_handler.info("\nDUT " + dutIP)
 
         # look up in crbs - to find the matching IP
@@ -479,6 +481,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
         # init dut, tester crb
         dts_crbs_init(crbInst, skip_setup, read_cache, project, base_dir, nics, virttype)
 
+        check_case_inst = check_case_skip(dut)
         # Run DUT prerequisites
         if dts_run_prerequisties(pkgName, patch) is False:
             dts_crbs_exit()
@@ -591,8 +594,14 @@ def execute_test_case(test_suite, test_case):
     global debug_case
     global module
     result.test_case = test_case.__name__
-
     rst.write_title("Test Case: " + test_case.__name__)
+    if check_case_inst.case_skip(test_case.__name__[len("test_"):]):
+       log_handler.info('Test Case %s Result SKIPED:' % test_case.__name__)
+       rst.write_result("N/A")
+       result.test_case_skip(skip_case_mode.comments)
+       save_all_results()
+       return
+
     if performance_only:
         rst.write_annex_title("Annex: " + test_case.__name__)
     try:
-- 
1.7.4.4

Re: [dts] [PATCH V2 0/5] add check case list for skip the case, when the case can't test

[Liu, Yong]

Applied into 1.1 branch. Thanks.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 3:45 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V2 0/5] add check case list for skip the case, when
> the case can't test
> 
> From: huilong xu <huilongx.xu@intel.com>
> 
> this patch for skip case, that case can't work in test
> 
> huilong xu (5):
>   add check case list
>   add N/A result in test report
>   add case skip result when case running
>   add case skip result when case running
>   add check case function in run case module
> 
>  conf/dpdk_test_case_checklist.xls |  Bin 0 -> 19456 bytes
>  framework/checkCase.py            |  100
> +++++++++++++++++++++++++++++++++++++
>  framework/dts.py                  |   21 ++++++--
>  framework/excel_reporter.py       |    9 ++-
>  framework/test_result.py          |    6 ++
>  5 files changed, 127 insertions(+), 9 deletions(-)
>  create mode 100644 conf/dpdk_test_case_checklist.xls
>  create mode 100644 framework/checkCase.py
> 
> --
> 1.7.4.4