[dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test

[dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

*** BLURB HERE ***

huilong xu (5):
  add check case list
  add N/A result in test report
  add case skip result when case running
  add checkCase module
  add check case function in run case module

 conf/dpdk_test_case_checklist.xls |  Bin 0 -> 19456 bytes
 framework/checkCase.py            |  100 +++++++++++++++++++++++++++++++++++++
 framework/dts.py                  |   21 ++++++--
 framework/excel_reporter.py       |    9 ++-
 framework/test_result.py          |    5 ++
 5 files changed, 126 insertions(+), 9 deletions(-)
 create mode 100644 conf/dpdk_test_case_checklist.xls
 create mode 100644 framework/checkCase.py

-- 
1.7.4.4

[dts] [PATCH V1 1/5] add check case list

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

1. add a excel file(excel 1997-2003) for set skip case check list
2. table format
   +---------+---+----+------+----------+--------+
   |case name|os |nic |target|wq request|comments|
   +---------+---+----+------+----------+--------+
   case name is which case need ti skip.
   os is which os need to skip, if mutil os, need used ',' to division. the dut os set in crbs.py, and dut provide function, os_get_type()
   nic is which nic type to skip, if mutil nic type, need used ',' to division
   target as same with nic
   wq request and comments is the message, when case skip.
   if you want ignore some check option, you can set value is "all"

Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
 conf/dpdk_test_case_checklist.xls |  Bin 0 -> 19456 bytes
 1 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 conf/dpdk_test_case_checklist.xls

diff --git a/conf/dpdk_test_case_checklist.xls b/conf/dpdk_test_case_checklist.xls
new file mode 100644
index 0000000000000000000000000000000000000000..ae88705bd0e2b9d1e278e3fdd47893d809134047
GIT binary patch
literal 19456
zcmeHPdvILUc|Uh|rCnLJSMm$v=URTrvMgjt7&|tW9}we4eh{27otVn&)m>}j)vj2r
zjNL(lXp?^m%@DbP1e`KX$_y<jX$&Q#rNwkQqz}<#+E9iRIwTV^A?dWmO{Wb5Y=6IV
z_U^s+?7dgJfoAHCujbsn=ey_pzVCeBcfRwu_v$;}Zhhmz*Oz=>Xva<w7VnpuM5BXl
z;F?JuEkdlq1@rHhN+m@U0+&~hD@X(HLDtnUUKlBY)PO{rN0DMkO-QqlnvrHB%|V)r
z6i0#!h*qR|Nb`{vAT2~%gmewkVx()4mLM%fT86Y7X$8_sqy!RuhiQ%D`U+Ct6))<?
zMIL`e*y3i9!*4;Hg(Ye~7QtgIC4^8d5#w2iOY$r}%AdXIsXw{n+ut$7qd$3zzipsR
zs$$|OxEI7JaYD**O4X|4JWqrS`mH5HEmg^S=MGh}LEI+B#R&eAvhDk!g|w83l%mMs
zIu`)i4*raR>_3ztps4HCUzKwW;wErNNf}IOf%@dx3jS8xYvoUU{jAKtDTZF$*dU%*
z`BL<+5zQNM^ZOzs{wVSS&O-bE>6jShb>f$K?2$4g4a#r{3ck^3gmfY3`8>`oh%3AU
z>*G9Mp1&r`J}1vN;?BRrb}w)Hm6cmjg5%Wfp<S}vZ_9I=JkQ115Z{n>z9%W05uIK`
z`IvZsW(|oYQDZMWAq%t<@Q{r7zgD7YMoAaUP{I&>;yEhPx3vbwf#4D$MEsdF)*4u%
z6a8pwAeYFDC0>IygE45^oq=+&MBj`hcFa)15dDE|8TM$)NZG@ZWgj!(g~3|q*zBkE
z6_I6#%@~K<*0ineT)nz`aKrXv?N;X4h82-j80nX}?tk**=FlDPLT#7|V9>Veb#=RO
zkSEl7h}x!?($LC9Zx9`#M~Fd+(vh$vD`$_jOTw`Y+slNuO>K~SuzlAcxO9mP_(N4^
zy^e&lQb)p7qhEB2kGPqs7P$FI(Ht!M`)#IM#75Y|W@^>1@U<#S)^^P|<a|ho|12fo
zQPy!OK{{tySLq*~9;jUdbC}8z()9dv`A+0fxBe^iKlGvhfe-yjANnIc^so8Qzv@H(
zk`MhpAG)7Be)vyG9M1HmA6E46yFPR!Pjz`EPc{Abd~m+*Lx0+b{^vgQ-}0e<(}!NK
zJbx^4qE-4)e!`WJ)Z6d0wm#JSP_u|UD9dxTr0D7N^DCcUDI(vH^ajKbN&jyYkcZ`;
z8+_=UKJ*SxI#*~C-<sA5h0j%+q*v^#{Fy5`N&l`mj7u!WRC%uM9CTDybguZy^aL)k
z_|xbbU#<>i`HFs&|8vDy#yO5lEDRM6SCNulk@JbN-pcgzt*wii7eUVtOFCDa7QWK6
z(tiWyDUz<~nX6Mt*W_Ga(Z7o+p4eH@16Q$n`MYBa#N9M}<=82fK7aY`iP$DF;i9uZ
z2rK0Ua#g@-BnL?hFCB9c`9swe)J(IE3Tm=eM+GiV>Jo`-gt{tlNUN&?V@6#SqV-mw
zQR?Er?ps#{_Pe?&utU~WfjzaZ3hcslRbbz)s{%WBV1?aUe_*c<`}c>4>?mfME~jRC
ziNr)q2Vsb)0fcVo55l-n0|;HyAB4fA1`xWdKL{gF4Ip%3e-MVL8bIjQ{vh0~n-OGs
zAlN$G?yCVL*gA`A0|~YcHymq_B-lFKv8(|k*gD+utN|q0I#{1MK#*~dC93cA^afjp
zo36Gbz97NY;jV2BAi>t*wr>p}!PZIC1`=$YRkeWxTL(+tnVX|F*8Mx6+J3hNb0B(|
zSnbUFgRH4dl3?qstqmmDI_qi!3AWDq+CYM>(_R7LeCfPsEV<@k>^r<s`qtl-Y{h;~
z8@x84L{tu5Ec?!c@CLbj`Ems$S_Z!7o_k7(@)7f(H92U&md7j)+C)qjssm+}P@LHD
z@NC&*k3Ci;RDXYel~7KaB@}52nwv~YQ%+j-EOsM`2hI!hd2(mH6*kpdp{w2s*HtRe
z+BBd`vDMR(ZWL1ijTNwpa96N=?8Sd``j|eq!S<3PjL@VmuJj!egM&Qw)Mf@L^#_CW
z)&imrH@aQ>0Af2rRdcG{`Fd2a`&fVfX>TQ#roUL_&L3VM(~YvFq6*KBZ*}O>q)`#7
z+C6S?ft)ve+@ti<^9~Tbw{NYehq|m<Pg*`!Dos~FtPphZ;>D`o?xb10oit#}!|Lr$
zTD8r#p{&zp{oXe7ii^&0W2w<evxIVn8%roBtvUo@tJmRqRhp`<u-#QbcU5B|_6cn`
z*ntWW|FsPDu6I>u_qB$DYDsRaNJ4jmG2Hq*7n+BsURd1qbgxZpO6*Rh%v^EvRP2D5
z;z(LPjL3@89PF-li?t#lHX-fCv&EE9PZm{}G4LEe8Zp!V^yb^823}K5V7Hcg6J*n>
zv+1d3(-y!+1tQ6&!AxI%{WWhk^K>@7)oj)Uuu-8(vS~EaKY9CgZ#MIFHhtA>Is(`*
zn1#{lps1*se)Elg@n*9?XLEfuo2~#h41r}fG1%r`|HGTjLY>W)YBoIqY*bK_t!XmT
z@4fY^H=9Ld(--V%E5W9(22YzFcy{`S-fXVX*?4%`>Hs!QPn#b2(Kny<X0upl<Kb!R
z1K2n{ZF=C9A3o>J=31SNho@}}VB_?(>4E7>-}7d(L}%mSY1aj?aeCVHz%#G^*qhB#
zosEa5^#-tUdfN2ByDvZO&1M<dYzg+X1lVk;!P9zrKmXJ#iVXuAj!H~huCwv*v^4>2
zoSxRx`|3lV_hz#~XXD{%?E!3@p4QX*7ytOMH=C6@8xK$G3}EB*w4UCJ7ry1qCZV(O
z@U-p#Hcn6L={@&SIYR5US*5e_@U*@FHcn6L>3!$Y1759}jmOhiw^qi@-P*XhycG2v
zD}^|8@1J>tuC_tpQ99@<4b<sHc=U1jfH&wG8x;PdgRa#;ojxPPBY*m<-k@u3P<V?D
zx<LbVdI(<D8v3#~=sFt|zM+F|(m<X5AjHSM`l2`JdK(m;po4DKK%LeX;=dmJs26BV
zv~yavCpk0~+vc{sv9uHxd$4dDs*J!5oPA+afU_fBbLXHONm}s!0rv3sCbOyWY_gci
z=cZ!!x&cFrO3gyOGQ`)qvVgW<Ba6zJ1VK45wis|%K{9B>yOq4jRgD2PICC*quE~`<
z7mDMp3OtS2J8Y&-CH9)x>{RR?O|EN7vmn<nCW9%waF@Wf3F)$Ewk;KR4CJKH*-0&_
zIQMR>NTqU?R4QjlrE=#&5wUSHT$+j{+@dsu!DKJqdsFY-ftO&}&dElUZP&^+;w@ym
zY(jhzBf%Z4BI|6LjG)dstxlvAL!EKFuT_w9-&FanMsn$zjG{)DRwG)PT~)))YoWM~
zZ@k`oW;C13C5!pO*~Ag^OmQl<&kYs2zSM%3DbL`gIodua?a!AS&qC)%AkG>11D7e5
z^=a&~do`hLSfKCA=i#W^-MGe*Qbg>7((;<Ca&N)H0y7J1F5X(f{@5*MawwBKnZQ~>
z6Exfc&E5im==&K6dJ+PvHG*^jbeNU<ClnlAF$&34AUqeEi$#V59I;OaZ}k9I@y`dm
z)c^RUe?~(H-s=QM4AjBrd4Q`p=mV~%1|bRF=LAPA)B-|yz6Uttp|V)isg`r0S>k%W
z&9gr@Iu2JDcbhD<0PW!$rK9jg8r|+_+{>_x#tn}T*!^2DK|)*_$mC9$L*@SeF-@4o
zh)V;~(5GPZAy)(lSG=(ji>&Cw4UR@AZnn4q@{5D!c(IT~c+mp0u?&IvAb{D@0(^>r
zMXhLLJjD)&)!y^i1}Qr{TxwFFJNCL|H3V25gr^tei@Ag<Yu~`7y;yC++kSvJm@k@B
zu@i2L&@A*9ViX}1x6irik)mLq#0qKL<FEs*%2^>z<*bmVa_2(LVw=D&`l0cnwD?^b
z(}jROBy~`2b7#nBgml@Q1X~i7vm{YDOA?i1DEr7{19Z2>EkZ*B3~?B3E>sr&R2f}$
zJ6-iirkI7cHoI$u<1o(=ct;V2qrR%W-5|sc36U?-1GK$|@U;?x3Rl*TNT+h`FHrAR
z98s;ca&)m)aU*ObjwTB^hAb6c``v<unh~jwf<poQ%8FWEVU>fp)bM9xw~Nm@Hc1{p
z>ss-51Ry75Yj1<dXXVuaw4K3_uY_-d7Bk|s^dP$gd(cUYqD6$tE#kO*b@(%A)%}c|
z@QmArO|!U}lpb!8S(F}9p5^|OMd^JwwpcvukW0C?`t0H>crE)Yj(e<kKV03w-wQ?z
z37=B@)DZ8A-teus$A@!>$EoC)naCwaOza^Yx($));QqaE)?%`7(kucll^+?wzR?(F
z3Y&Yj4&FI#j+<jBwtHZpRfu%vv^iKDAI+MB=}fk077ht<B3UTrbHkZzcC2$Wf0tP}
zjZ3pLoi7yS)u5R?>B^6dW(t|2qktj|j=`c?6EP~CHP4js_Y0BBBy+_~s`FIxE;E}P
z>~mDC4;e%Ar^ZK4<OkCQv>)t#-Ne4>FDmCkAx6ft#f+_svBAyV`}W)}L`BjMSv3p1
zSnk`i%hu5^+89`S>};;AbG*G#yesdt@;o6<V@fkPRxD&j?;K3!bLq@Us8bS)qnRO%
zbZaJeYZ(lW>8cFqtrp0T#oz=c$>JD1_2{jMgD{s_Kz|=A=0^t$CT<mqaKjYmuw&yR
zgYp7ygU8oJ+#MYmO62mz#Mt=gDBLem9L|g-c!T$c;rE$T^?mjTG<md;$-^zrmgO#C
zH5e0ri|0vTFP7sm(XZk60-pZ-03B<*TZm`SgPLRWkc3!|aqKwmzx5tYA{ASN#4B|y
z!VktE1k4A~X&-dl+4H6$9s%tU2aTQle$eiB(1-*52=y&Si|bpA<`Dd%e848+%a}tw
z4#%DYejLz32v>1=-HJ26$%=`0fB)^jKXBr3|DdFF$n!>d{tcWBjJP)QwHQW(=rr!d
zofjhH1=kn9DNBD^p4XvtS{^^@=WE{9b$bQn$Nd`|*B78$A~IAN&<wTEE02HZw|=pv
z!kqoQEe2}1j~$%O$>S4{A|@uJr3^8$9M;6ad<cJQZer_IKFl><%+8}_hE-l*R>^}p
zWC%vVkagXuBCipbG-ethKg)?SjU03ucvaD;OT8S1+$tLR>*X-Fs-jUrdO38SDjK~<
zFNaWDMWeUt<*@%&MWe3ta@4z%_MY@v^(CMXDrdkKsW_DjNzb%$VdSVYbuWV4;!3Uo
zIqHu0SU=8LMma}J>Py`trkx|Eog)fgx?xFLEYA6AIp;*7-@w+DdrlNid*bp-MWRsN
zI0sgV!r83NFDy~0c~#Ph!s)b)!j{G57nCf?LEJ`(NEF&d-E*RF{$!(2BXMq($&#GT
z+9-H(gB-0!lm^J{yyrxrw&M6xCHYO2U9wT4uvZYJ(S_3Jl7)JW%WpYZlB*gUCF*KX
z)P)jtq0j<x`2{IaVlI@J3nk`4iMdc{ow)o4l_*UvlqQK{*{I2d(&R#6nYjFll_>N=
zyG8V-xQsvQo>RNjwrwN!xi}sHDh)QfP?}vR%`RD*T`2U1xQxT(H`|56s27)C*s6O@
zS+Klu$U;wv%kOYma*hjyqlew1IWAe|xKQX*arqT5QRcc(`1PEPGS`K|H=S+0(BI<n
zJ7A(PM%dbAytYvoBb@gvlvl(W@g%xHr*Q-ymHPkf!~@J{=;UG-Pvzr2jJWr?Mw<}4
z7rZ#w@savMsqQaAl#%5pW7mk2j=~r+`79xPQ_v`_#*#8SaT6x`SI!8{23jMt^;`c4
zZH=&&4)8ubZ6UL}Y$EL7V+&PAjk^LC_`p{98$8{=YK8wzD>%%0_410;0J|IKN}TEN
z|1hZ8Dt9VXo}(-2SC9Xn8h9VuWQ}qY3x;+a^out>^XTQD9~y3X;&U;vas5-Tc7yf>
zB>D#L??U3-_7)_Lmq(DWnIMiMaSn3^iL;PTBh5#80Exc#IGl|eR;<fIzvftaAd@QO
z$MWf7;^s3cGpk1POLxy%{pyRxF5I7dXF~oj3Ht2^IfB|BdikZ7&vz|tIrk9S-}&<=
z+5W#oVq4jMwwoV597f{I?{*~0ejJJI9Y*5rz$g;+Kz&dz7mzGHDEVj0`XKxMt^hF0
zQuk_$(Z1}lFMGvo6eKv6N}-PC3#Z1oCpjkT>-sXv&(=n<Ykfj|3Ki-*_#YUYJ)@S|
z)L)^Nf;E<@mw%10H5a>*`5TDtVJv>w2K|4xtb+#Zet0OI&ZJD$9TPp>-Mxv5YsoeE
ho=iHuRjxs0kL{9P+O<{uY}>^<YgXmeg1=-9{12Nhvy1=$

literal 0
HcmV?d00001

-- 
1.7.4.4

[dts] [PATCH V1 2/5] add N/A result in test report

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>


Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
 framework/excel_reporter.py |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/framework/excel_reporter.py b/framework/excel_reporter.py
index 809dead..2030d87 100644
--- a/framework/excel_reporter.py
+++ b/framework/excel_reporter.py
@@ -85,13 +85,15 @@ class ExcelReporter(object):
         self.sheet.write(0, 7, 'Pass', self.header_style)
         self.sheet.write(0, 8, 'Fail', self.header_style)
         self.sheet.write(0, 9, 'Blocked', self.header_style)
-        self.sheet.write(0, 10, 'Not Run', self.header_style)
-        self.sheet.write(0, 11, 'Total', self.header_style)
+        self.sheet.write(0, 10, 'N/A', self.header_style)
+        self.sheet.write(0, 11, 'Not Run', self.header_style)
+        self.sheet.write(0, 12, 'Total', self.header_style)
 
         self.sheet.write(1, 7, Formula('COUNTIF(F2:F2000,"PASSED")'))
         self.sheet.write(1, 8, Formula('COUNTIF(F2:F2000,"FAILED*") + COUNTIF(F2:F2000,"IXA*")'))
         self.sheet.write(1, 9, Formula('COUNTIF(F2:F2000,"BLOCKED*")'))
-        self.sheet.write(1, 11, Formula('H2+I2+J2+K2'))
+        self.sheet.write(1, 10, Formula('COUNTIF(F2:F2000,"N/A*")'))
+        self.sheet.write(1, 12, Formula('H2+I2+J2+K2+L2'))
 
         self.sheet.col(0).width = 4000
         self.sheet.col(1).width = 7500
@@ -105,6 +107,7 @@ class ExcelReporter(object):
         self.sheet.col(9).width = 3000
         self.sheet.col(10).width = 3000
         self.sheet.col(11).width = 3000
+        self.sheet.col(12).width = 3000
 
     def __styles(self):
         header_pattern = xlwt.Pattern()
-- 
1.7.4.4

[dts] [PATCH V1 3/5] add case skip result when case running

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>


Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
 framework/test_result.py |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/framework/test_result.py b/framework/test_result.py
index 79faee1..2e62d9e 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -157,6 +157,11 @@ class Result(object):
         """
         self.__set_test_case_result(result='PASSED', message='')
 
+    def test_case_skip(self,message):
+        """
+        set last test case add as N/A
+        """
+        self.__set_test_case_result(result='N/A', message=message)
     def test_case_failed(self, message):
         """
         Set last test case added as FAILED
-- 
1.7.4.4

[dts] [PATCH V1 4/5] add checkCase module

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>

1. add parse_file class for get check case info form execl file, and send check function list from tabel head.
2. add case_skip class for check case is skip or running.
   a)from talbe head get check function list. excep "case name", "wq request", "comments"
   b)when add a new check rule in excel file, you must add a check function in this clase, the function name is "check_rule" and parameter is a list.
     eg, add kernel rule in excel table head. you need add function check_kernerl(self,kernel_type) if you not add the function, it will ignore this rule.

Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
 framework/checkCase.py |  100 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 100 insertions(+), 0 deletions(-)
 create mode 100644 framework/checkCase.py

diff --git a/framework/checkCase.py b/framework/checkCase.py
new file mode 100644
index 0000000..b2e79a0
--- /dev/null
+++ b/framework/checkCase.py
@@ -0,0 +1,100 @@
+import xlrd
+
+from settings import nic_name_from_type
+
+filter_file = r'./conf/dpdk_test_case_checklist.xls'
+filter_case = []
+check_function_dict = {}
+
+class parse_file():
+
+    def __init__(self):
+        try:
+            self.book = xlrd.open_workbook(filter_file)
+            self.sheet = self.book.sheet_by_index(0)
+            self.init_check_function_dict()
+        except:
+            pass
+
+    def init_check_function_dict(self):
+        '''
+        init check case functio, and skip case message.
+        '''
+        row_data  = self.sheet.row_values(0)
+        for i in range(1,len(row_data)):
+            if  row_data[i].lower() in ['wq number', 'comments']:
+                if 'message' not in check_function_dict:
+                    check_function_dict['message'] = [i]
+                else:
+                    check_function_dict['message'].append(i)
+            else:
+                check_function_dict[row_data[i].lower()] = i       
+
+    def set_filter_case(self):
+        for row in range(self.sheet.nrows):   
+            row_data = self.sheet.row_values(row)
+            # add case name
+            tmp_filter = [row_data[0]]
+            for i in range(1,len(row_data) - 2):
+                tmp_filter.append(row_data[i].split(','))
+
+            tmp_filter.append(row_data[-2])
+            tmp_filter.append(row_data[-1])
+
+            filter_case.append(tmp_filter)
+
+class check_case_skip():
+    def __init__(self, Dut):
+        self.dut = Dut
+        self.comments = ''
+
+    def check_os(self,os_type):
+        if 'all' == os_type[0].lower():
+            return True
+        dut_os_type = self.dut.get_os_type()
+        if dut_os_type in os_type:
+            return True
+        else:
+            return False
+
+    def check_nic(self, nic_type):
+        if 'all' == nic_type[0].lower():
+            return True
+        dut_nic_type = nic_name_from_type(self.dut.ports_info[0]['type'])
+        if dut_nic_type in nic_type:
+            return True
+        else:
+            return False
+
+    def check_target(self,target):
+        if 'all' == target[0].lower():
+            return True
+        if self.dut.target in target:
+            return True
+        else:
+            return False
+
+    def case_skip(self, case_name):
+        skip_flage = False
+        for rule in filter_case[1:]:
+            # check case name
+            if case_name == rule[0]:
+                for key in check_function_dict.keys():
+                    try:
+                        if 'message' == key:
+                            continue 
+                        check_function = getattr(self, 'check_%s' % key)
+                    except:
+                        print "can't check %s type" % key
+                    if check_function(rule[check_function_dict[key]]):
+                        skip_flage = True
+                    else:
+                        skip_flage = False
+                        break
+                if skip_flage: 
+                    if 'message' in check_function_dict:
+                        for i in check_function_dict['message']:
+                            self.comments += '%s,' % rule[i]
+                    return skip_flage
+                
+        return skip_flage
-- 
1.7.4.4

[dts] [PATCH V1 5/5] add check case function in run case module

[huilong,xu]

From: huilong xu <huilongx.xu@intel.com>


Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
 framework/dts.py |   21 +++++++++++++++------
 1 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/framework/dts.py b/framework/dts.py
index e9513c6..cb8f26e 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -57,7 +57,7 @@ from logger import getLogger
 import logger
 import debugger
 from virt_scene import VirtScene
-
+from checkCase import *
 import sys
 reload(sys)
 sys.setdefaultencoding('UTF8')
@@ -291,6 +291,8 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
     """
     Run each target in execution targets.
     """
+    global skip_case_mode
+    skip_case_mode = check_case_skip(dut)
     if scenario != '':
         scene = VirtScene(dut, tester, scenario)
     else:
@@ -299,7 +301,7 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
     if scene:
         scene.load_config()
         scene.create_scene()
-
+    
     for target in targets:
         log_handler.info("\nTARGET " + target)
         result.target = target
@@ -402,11 +404,12 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
     global Package
     global Patches
     global scenario
-
+    global skip_case_mode
     # save global variable
     Package = pkgName
     Patches = patch
-
+    check_case = parse_file()
+    check_case.set_filter_case()
     # prepare the output folder
     if output_dir == '':
         output_dir = FOLDERS['Output']
@@ -459,7 +462,6 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
 
         # verify if the delimiter is good if the lists are vertical
         dutIP, targets, test_suites, nics, scenario = dts_parse_config(section)
-
         log_handler.info("\nDUT " + dutIP)
 
         # look up in crbs - to find the matching IP
@@ -479,6 +481,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
         # init dut, tester crb
         dts_crbs_init(crbInst, skip_setup, read_cache, project, base_dir, nics, virttype)
 
+        skip_case_mode = check_case_skip(dut)
         # Run DUT prerequisites
         if dts_run_prerequisties(pkgName, patch) is False:
             dts_crbs_exit()
@@ -576,6 +579,7 @@ def execute_all_test_cases(test_suite):
     """
     if functional_only:
         for test_case in get_functional_test_cases(test_suite):
+            skip_case_mode.case_skip(test_case)
             execute_test_case(test_suite, test_case)
     if performance_only:
         for test_case in get_performance_test_cases(test_suite):
@@ -591,8 +595,13 @@ def execute_test_case(test_suite, test_case):
     global debug_case
     global module
     result.test_case = test_case.__name__
-
     rst.write_title("Test Case: " + test_case.__name__)
+    if skip_case_mode.case_skip(test_case.__name__[len("test_"):]):
+       log_handler.info('Test Case %s Result SKIPED:' % test_case.__name__)
+       rst.write_result("N/A")
+       result.test_case_skip(skip_case_mode.comments)
+       save_all_results()
+       return
     if performance_only:
         rst.write_annex_title("Annex: " + test_case.__name__)
     try:
-- 
1.7.4.4

Re: [dts] [PATCH V1 3/5] add case skip result when case running

[Liu, Yong]

One comment for add blank line.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 3/5] add case skip result when case running
> 
> From: huilong xu <huilongx.xu@intel.com>
> 
> 
> Signed-off-by: huilong xu <huilongx.xu@intel.com>
> ---
>  framework/test_result.py |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
> 
> diff --git a/framework/test_result.py b/framework/test_result.py
> index 79faee1..2e62d9e 100644
> --- a/framework/test_result.py
> +++ b/framework/test_result.py
> @@ -157,6 +157,11 @@ class Result(object):
>          """
>          self.__set_test_case_result(result='PASSED', message='')
> 
> +    def test_case_skip(self,message):
> +        """
> +        set last test case add as N/A
> +        """
> +        self.__set_test_case_result(result='N/A', message=message)

Add blank line here.

>      def test_case_failed(self, message):
>          """
>          Set last test case added as FAILED
> --
> 1.7.4.4

Re: [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test

[Liu, Yong]

Please change " BLURB " to description of this patch set.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 0/5] add check case list for skip the case, when
> the case can't test
> 
> From: huilong xu <huilongx.xu@intel.com>
> 
> *** BLURB HERE ***
> 
> huilong xu (5):
>   add check case list
>   add N/A result in test report
>   add case skip result when case running
>   add checkCase module
>   add check case function in run case module
> 
>  conf/dpdk_test_case_checklist.xls |  Bin 0 -> 19456 bytes
>  framework/checkCase.py            |  100
> +++++++++++++++++++++++++++++++++++++
>  framework/dts.py                  |   21 ++++++--
>  framework/excel_reporter.py       |    9 ++-
>  framework/test_result.py          |    5 ++
>  5 files changed, 126 insertions(+), 9 deletions(-)
>  create mode 100644 conf/dpdk_test_case_checklist.xls
>  create mode 100644 framework/checkCase.py
> 
> --
> 1.7.4.4

Re: [dts] [PATCH V1 3/5] add case skip result when case running

[Xu, HuilongX]

Ok, I will update it at v2 version.

> -----Original Message-----
> From: Liu, Yong
> Sent: Friday, August 07, 2015 2:06 PM
> To: Xu, HuilongX; dts@dpdk.org
> Subject: RE: [dts] [PATCH V1 3/5] add case skip result when case running
> 
> One comment for add blank line.
> 
> > -----Original Message-----
> > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> > Sent: Friday, August 07, 2015 2:03 PM
> > To: dts@dpdk.org
> > Subject: [dts] [PATCH V1 3/5] add case skip result when case running
> >
> > From: huilong xu <huilongx.xu@intel.com>
> >
> >
> > Signed-off-by: huilong xu <huilongx.xu@intel.com>
> > ---
> >  framework/test_result.py |    5 +++++
> >  1 files changed, 5 insertions(+), 0 deletions(-)
> >
> > diff --git a/framework/test_result.py b/framework/test_result.py
> > index 79faee1..2e62d9e 100644
> > --- a/framework/test_result.py
> > +++ b/framework/test_result.py
> > @@ -157,6 +157,11 @@ class Result(object):
> >          """
> >          self.__set_test_case_result(result='PASSED', message='')
> >
> > +    def test_case_skip(self,message):
> > +        """
> > +        set last test case add as N/A
> > +        """
> > +        self.__set_test_case_result(result='N/A', message=message)
> 
> Add blank line here.
> 
> >      def test_case_failed(self, message):
> >          """
> >          Set last test case added as FAILED
> > --
> > 1.7.4.4

Re: [dts] [PATCH V1 5/5] add check case function in run case module

[Liu, Yong]

One question about skip_case_mode.case_skip.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 5/5] add check case function in run case module
> 
> From: huilong xu <huilongx.xu@intel.com>
> 
> 
> Signed-off-by: huilong xu <huilongx.xu@intel.com>
> ---
>  framework/dts.py |   21 +++++++++++++++------
>  1 files changed, 15 insertions(+), 6 deletions(-)
> 
> diff --git a/framework/dts.py b/framework/dts.py
> index e9513c6..cb8f26e 100644
> --- a/framework/dts.py
> +++ b/framework/dts.py
> @@ -57,7 +57,7 @@ from logger import getLogger
>  import logger
>  import debugger
>  from virt_scene import VirtScene
> -
> +from checkCase import *
>  import sys
>  reload(sys)
>  sys.setdefaultencoding('UTF8')
> @@ -291,6 +291,8 @@ def dts_run_target(crbInst, targets, test_suites, nic,
> scenario):
>      """
>      Run each target in execution targets.
>      """
> +    global skip_case_mode
> +    skip_case_mode = check_case_skip(dut)
>      if scenario != '':
>          scene = VirtScene(dut, tester, scenario)
>      else:
> @@ -299,7 +301,7 @@ def dts_run_target(crbInst, targets, test_suites, nic,
> scenario):
>      if scene:
>          scene.load_config()
>          scene.create_scene()
> -
> +
>      for target in targets:
>          log_handler.info("\nTARGET " + target)
>          result.target = target
> @@ -402,11 +404,12 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
>      global Package
>      global Patches
>      global scenario
> -
> +    global skip_case_mode
>      # save global variable
>      Package = pkgName
>      Patches = patch
> -
> +    check_case = parse_file()
> +    check_case.set_filter_case()
>      # prepare the output folder
>      if output_dir == '':
>          output_dir = FOLDERS['Output']
> @@ -459,7 +462,6 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
> 
>          # verify if the delimiter is good if the lists are vertical
>          dutIP, targets, test_suites, nics, scenario =
> dts_parse_config(section)
> -
>          log_handler.info("\nDUT " + dutIP)
> 
>          # look up in crbs - to find the matching IP
> @@ -479,6 +481,7 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
>          # init dut, tester crb
>          dts_crbs_init(crbInst, skip_setup, read_cache, project, base_dir,
> nics, virttype)
> 
> +        skip_case_mode = check_case_skip(dut)
>          # Run DUT prerequisites
>          if dts_run_prerequisties(pkgName, patch) is False:
>              dts_crbs_exit()
> @@ -576,6 +579,7 @@ def execute_all_test_cases(test_suite):
>      """
>      if functional_only:
>          for test_case in get_functional_test_cases(test_suite):
> +            skip_case_mode.case_skip(test_case)

Why call this function here? In function execute_test_case also check whether skip test case.

>              execute_test_case(test_suite, test_case)
>      if performance_only:
>          for test_case in get_performance_test_cases(test_suite):
> @@ -591,8 +595,13 @@ def execute_test_case(test_suite, test_case):
>      global debug_case
>      global module
>      result.test_case = test_case.__name__
> -
>      rst.write_title("Test Case: " + test_case.__name__)
> +    if skip_case_mode.case_skip(test_case.__name__[len("test_"):]):
> +       log_handler.info('Test Case %s Result SKIPED:' %
> test_case.__name__)
> +       rst.write_result("N/A")
> +       result.test_case_skip(skip_case_mode.comments)
> +       save_all_results()
> +       return
>      if performance_only:
>          rst.write_annex_title("Annex: " + test_case.__name__)
>      try:
> --
> 1.7.4.4