* [dts] [PATCH V1 1/5] add check case list
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
@ 2015-08-07 6:03 ` huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 2/5] add N/A result in test report huilong,xu
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: huilong,xu @ 2015-08-07 6:03 UTC (permalink / raw)
To: dts
From: huilong xu <huilongx.xu@intel.com>
1. add a excel file(excel 1997-2003) for set skip case check list
2. table format
+---------+---+----+------+----------+--------+
|case name|os |nic |target|wq request|comments|
+---------+---+----+------+----------+--------+
case name is which case need ti skip.
os is which os need to skip, if mutil os, need used ',' to division. the dut os set in crbs.py, and dut provide function, os_get_type()
nic is which nic type to skip, if mutil nic type, need used ',' to division
target as same with nic
wq request and comments is the message, when case skip.
if you want ignore some check option, you can set value is "all"
Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
conf/dpdk_test_case_checklist.xls | Bin 0 -> 19456 bytes
1 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 conf/dpdk_test_case_checklist.xls
diff --git a/conf/dpdk_test_case_checklist.xls b/conf/dpdk_test_case_checklist.xls
new file mode 100644
index 0000000000000000000000000000000000000000..ae88705bd0e2b9d1e278e3fdd47893d809134047
GIT binary patch
literal 19456
zcmeHPdvILUc|Uh|rCnLJSMm$v=URTrvMgjt7&|tW9}we4eh{27otVn&)m>}j)vj2r
zjNL(lXp?^m%@DbP1e`KX$_y<jX$&Q#rNwkQqz}<#+E9iRIwTV^A?dWmO{Wb5Y=6IV
z_U^s+?7dgJfoAHCujbsn=ey_pzVCeBcfRwu_v$;}Zhhmz*Oz=>Xva<w7VnpuM5BXl
z;F?JuEkdlq1@rHhN+m@U0+&~hD@X(HLDtnUUKlBY)PO{rN0DMkO-QqlnvrHB%|V)r
z6i0#!h*qR|Nb`{vAT2~%gmewkVx()4mLM%fT86Y7X$8_sqy!RuhiQ%D`U+Ct6))<?
zMIL`e*y3i9!*4;Hg(Ye~7QtgIC4^8d5#w2iOY$r}%AdXIsXw{n+ut$7qd$3zzipsR
zs$$|OxEI7JaYD**O4X|4JWqrS`mH5HEmg^S=MGh}LEI+B#R&eAvhDk!g|w83l%mMs
zIu`)i4*raR>_3ztps4HCUzKwW;wErNNf}IOf%@dx3jS8xYvoUU{jAKtDTZF$*dU%*
z`BL<+5zQNM^ZOzs{wVSS&O-bE>6jShb>f$K?2$4g4a#r{3ck^3gmfY3`8>`oh%3AU
z>*G9Mp1&r`J}1vN;?BRrb}w)Hm6cmjg5%Wfp<S}vZ_9I=JkQ115Z{n>z9%W05uIK`
z`IvZsW(|oYQDZMWAq%t<@Q{r7zgD7YMoAaUP{I&>;yEhPx3vbwf#4D$MEsdF)*4u%
z6a8pwAeYFDC0>IygE45^oq=+&MBj`hcFa)15dDE|8TM$)NZG@ZWgj!(g~3|q*zBkE
z6_I6#%@~K<*0ineT)nz`aKrXv?N;X4h82-j80nX}?tk**=FlDPLT#7|V9>Veb#=RO
zkSEl7h}x!?($LC9Zx9`#M~Fd+(vh$vD`$_jOTw`Y+slNuO>K~SuzlAcxO9mP_(N4^
zy^e&lQb)p7qhEB2kGPqs7P$FI(Ht!M`)#IM#75Y|W@^>1@U<#S)^^P|<a|ho|12fo
zQPy!OK{{tySLq*~9;jUdbC}8z()9dv`A+0fxBe^iKlGvhfe-yjANnIc^so8Qzv@H(
zk`MhpAG)7Be)vyG9M1HmA6E46yFPR!Pjz`EPc{Abd~m+*Lx0+b{^vgQ-}0e<(}!NK
zJbx^4qE-4)e!`WJ)Z6d0wm#JSP_u|UD9dxTr0D7N^DCcUDI(vH^ajKbN&jyYkcZ`;
z8+_=UKJ*SxI#*~C-<sA5h0j%+q*v^#{Fy5`N&l`mj7u!WRC%uM9CTDybguZy^aL)k
z_|xbbU#<>i`HFs&|8vDy#yO5lEDRM6SCNulk@JbN-pcgzt*wii7eUVtOFCDa7QWK6
z(tiWyDUz<~nX6Mt*W_Ga(Z7o+p4eH@16Q$n`MYBa#N9M}<=82fK7aY`iP$DF;i9uZ
z2rK0Ua#g@-BnL?hFCB9c`9swe)J(IE3Tm=eM+GiV>Jo`-gt{tlNUN&?V@6#SqV-mw
zQR?Er?ps#{_Pe?&utU~WfjzaZ3hcslRbbz)s{%WBV1?aUe_*c<`}c>4>?mfME~jRC
ziNr)q2Vsb)0fcVo55l-n0|;HyAB4fA1`xWdKL{gF4Ip%3e-MVL8bIjQ{vh0~n-OGs
zAlN$G?yCVL*gA`A0|~YcHymq_B-lFKv8(|k*gD+utN|q0I#{1MK#*~dC93cA^afjp
zo36Gbz97NY;jV2BAi>t*wr>p}!PZIC1`=$YRkeWxTL(+tnVX|F*8Mx6+J3hNb0B(|
zSnbUFgRH4dl3?qstqmmDI_qi!3AWDq+CYM>(_R7LeCfPsEV<@k>^r<s`qtl-Y{h;~
z8@x84L{tu5Ec?!c@CLbj`Ems$S_Z!7o_k7(@)7f(H92U&md7j)+C)qjssm+}P@LHD
z@NC&*k3Ci;RDXYel~7KaB@}52nwv~YQ%+j-EOsM`2hI!hd2(mH6*kpdp{w2s*HtRe
z+BBd`vDMR(ZWL1ijTNwpa96N=?8Sd``j|eq!S<3PjL@VmuJj!egM&Qw)Mf@L^#_CW
z)&imrH@aQ>0Af2rRdcG{`Fd2a`&fVfX>TQ#roUL_&L3VM(~YvFq6*KBZ*}O>q)`#7
z+C6S?ft)ve+@ti<^9~Tbw{NYehq|m<Pg*`!Dos~FtPphZ;>D`o?xb10oit#}!|Lr$
zTD8r#p{&zp{oXe7ii^&0W2w<evxIVn8%roBtvUo@tJmRqRhp`<u-#QbcU5B|_6cn`
z*ntWW|FsPDu6I>u_qB$DYDsRaNJ4jmG2Hq*7n+BsURd1qbgxZpO6*Rh%v^EvRP2D5
z;z(LPjL3@89PF-li?t#lHX-fCv&EE9PZm{}G4LEe8Zp!V^yb^823}K5V7Hcg6J*n>
zv+1d3(-y!+1tQ6&!AxI%{WWhk^K>@7)oj)Uuu-8(vS~EaKY9CgZ#MIFHhtA>Is(`*
zn1#{lps1*se)Elg@n*9?XLEfuo2~#h41r}fG1%r`|HGTjLY>W)YBoIqY*bK_t!XmT
z@4fY^H=9Ld(--V%E5W9(22YzFcy{`S-fXVX*?4%`>Hs!QPn#b2(Kny<X0upl<Kb!R
z1K2n{ZF=C9A3o>J=31SNho@}}VB_?(>4E7>-}7d(L}%mSY1aj?aeCVHz%#G^*qhB#
zosEa5^#-tUdfN2ByDvZO&1M<dYzg+X1lVk;!P9zrKmXJ#iVXuAj!H~huCwv*v^4>2
zoSxRx`|3lV_hz#~XXD{%?E!3@p4QX*7ytOMH=C6@8xK$G3}EB*w4UCJ7ry1qCZV(O
z@U-p#Hcn6L={@&SIYR5US*5e_@U*@FHcn6L>3!$Y1759}jmOhiw^qi@-P*XhycG2v
zD}^|8@1J>tuC_tpQ99@<4b<sHc=U1jfH&wG8x;PdgRa#;ojxPPBY*m<-k@u3P<V?D
zx<LbVdI(<D8v3#~=sFt|zM+F|(m<X5AjHSM`l2`JdK(m;po4DKK%LeX;=dmJs26BV
zv~yavCpk0~+vc{sv9uHxd$4dDs*J!5oPA+afU_fBbLXHONm}s!0rv3sCbOyWY_gci
z=cZ!!x&cFrO3gyOGQ`)qvVgW<Ba6zJ1VK45wis|%K{9B>yOq4jRgD2PICC*quE~`<
z7mDMp3OtS2J8Y&-CH9)x>{RR?O|EN7vmn<nCW9%waF@Wf3F)$Ewk;KR4CJKH*-0&_
zIQMR>NTqU?R4QjlrE=#&5wUSHT$+j{+@dsu!DKJqdsFY-ftO&}&dElUZP&^+;w@ym
zY(jhzBf%Z4BI|6LjG)dstxlvAL!EKFuT_w9-&FanMsn$zjG{)DRwG)PT~)))YoWM~
zZ@k`oW;C13C5!pO*~Ag^OmQl<&kYs2zSM%3DbL`gIodua?a!AS&qC)%AkG>11D7e5
z^=a&~do`hLSfKCA=i#W^-MGe*Qbg>7((;<Ca&N)H0y7J1F5X(f{@5*MawwBKnZQ~>
z6Exfc&E5im==&K6dJ+PvHG*^jbeNU<ClnlAF$&34AUqeEi$#V59I;OaZ}k9I@y`dm
z)c^RUe?~(H-s=QM4AjBrd4Q`p=mV~%1|bRF=LAPA)B-|yz6Uttp|V)isg`r0S>k%W
z&9gr@Iu2JDcbhD<0PW!$rK9jg8r|+_+{>_x#tn}T*!^2DK|)*_$mC9$L*@SeF-@4o
zh)V;~(5GPZAy)(lSG=(ji>&Cw4UR@AZnn4q@{5D!c(IT~c+mp0u?&IvAb{D@0(^>r
zMXhLLJjD)&)!y^i1}Qr{TxwFFJNCL|H3V25gr^tei@Ag<Yu~`7y;yC++kSvJm@k@B
zu@i2L&@A*9ViX}1x6irik)mLq#0qKL<FEs*%2^>z<*bmVa_2(LVw=D&`l0cnwD?^b
z(}jROBy~`2b7#nBgml@Q1X~i7vm{YDOA?i1DEr7{19Z2>EkZ*B3~?B3E>sr&R2f}$
zJ6-iirkI7cHoI$u<1o(=ct;V2qrR%W-5|sc36U?-1GK$|@U;?x3Rl*TNT+h`FHrAR
z98s;ca&)m)aU*ObjwTB^hAb6c``v<unh~jwf<poQ%8FWEVU>fp)bM9xw~Nm@Hc1{p
z>ss-51Ry75Yj1<dXXVuaw4K3_uY_-d7Bk|s^dP$gd(cUYqD6$tE#kO*b@(%A)%}c|
z@QmArO|!U}lpb!8S(F}9p5^|OMd^JwwpcvukW0C?`t0H>crE)Yj(e<kKV03w-wQ?z
z37=B@)DZ8A-teus$A@!>$EoC)naCwaOza^Yx($));QqaE)?%`7(kucll^+?wzR?(F
z3Y&Yj4&FI#j+<jBwtHZpRfu%vv^iKDAI+MB=}fk077ht<B3UTrbHkZzcC2$Wf0tP}
zjZ3pLoi7yS)u5R?>B^6dW(t|2qktj|j=`c?6EP~CHP4js_Y0BBBy+_~s`FIxE;E}P
z>~mDC4;e%Ar^ZK4<OkCQv>)t#-Ne4>FDmCkAx6ft#f+_svBAyV`}W)}L`BjMSv3p1
zSnk`i%hu5^+89`S>};;AbG*G#yesdt@;o6<V@fkPRxD&j?;K3!bLq@Us8bS)qnRO%
zbZaJeYZ(lW>8cFqtrp0T#oz=c$>JD1_2{jMgD{s_Kz|=A=0^t$CT<mqaKjYmuw&yR
zgYp7ygU8oJ+#MYmO62mz#Mt=gDBLem9L|g-c!T$c;rE$T^?mjTG<md;$-^zrmgO#C
zH5e0ri|0vTFP7sm(XZk60-pZ-03B<*TZm`SgPLRWkc3!|aqKwmzx5tYA{ASN#4B|y
z!VktE1k4A~X&-dl+4H6$9s%tU2aTQle$eiB(1-*52=y&Si|bpA<`Dd%e848+%a}tw
z4#%DYejLz32v>1=-HJ26$%=`0fB)^jKXBr3|DdFF$n!>d{tcWBjJP)QwHQW(=rr!d
zofjhH1=kn9DNBD^p4XvtS{^^@=WE{9b$bQn$Nd`|*B78$A~IAN&<wTEE02HZw|=pv
z!kqoQEe2}1j~$%O$>S4{A|@uJr3^8$9M;6ad<cJQZer_IKFl><%+8}_hE-l*R>^}p
zWC%vVkagXuBCipbG-ethKg)?SjU03ucvaD;OT8S1+$tLR>*X-Fs-jUrdO38SDjK~<
zFNaWDMWeUt<*@%&MWe3ta@4z%_MY@v^(CMXDrdkKsW_DjNzb%$VdSVYbuWV4;!3Uo
zIqHu0SU=8LMma}J>Py`trkx|Eog)fgx?xFLEYA6AIp;*7-@w+DdrlNid*bp-MWRsN
zI0sgV!r83NFDy~0c~#Ph!s)b)!j{G57nCf?LEJ`(NEF&d-E*RF{$!(2BXMq($&#GT
z+9-H(gB-0!lm^J{yyrxrw&M6xCHYO2U9wT4uvZYJ(S_3Jl7)JW%WpYZlB*gUCF*KX
z)P)jtq0j<x`2{IaVlI@J3nk`4iMdc{ow)o4l_*UvlqQK{*{I2d(&R#6nYjFll_>N=
zyG8V-xQsvQo>RNjwrwN!xi}sHDh)QfP?}vR%`RD*T`2U1xQxT(H`|56s27)C*s6O@
zS+Klu$U;wv%kOYma*hjyqlew1IWAe|xKQX*arqT5QRcc(`1PEPGS`K|H=S+0(BI<n
zJ7A(PM%dbAytYvoBb@gvlvl(W@g%xHr*Q-ymHPkf!~@J{=;UG-Pvzr2jJWr?Mw<}4
z7rZ#w@savMsqQaAl#%5pW7mk2j=~r+`79xPQ_v`_#*#8SaT6x`SI!8{23jMt^;`c4
zZH=&&4)8ubZ6UL}Y$EL7V+&PAjk^LC_`p{98$8{=YK8wzD>%%0_410;0J|IKN}TEN
z|1hZ8Dt9VXo}(-2SC9Xn8h9VuWQ}qY3x;+a^out>^XTQD9~y3X;&U;vas5-Tc7yf>
zB>D#L??U3-_7)_Lmq(DWnIMiMaSn3^iL;PTBh5#80Exc#IGl|eR;<fIzvftaAd@QO
z$MWf7;^s3cGpk1POLxy%{pyRxF5I7dXF~oj3Ht2^IfB|BdikZ7&vz|tIrk9S-}&<=
z+5W#oVq4jMwwoV597f{I?{*~0ejJJI9Y*5rz$g;+Kz&dz7mzGHDEVj0`XKxMt^hF0
zQuk_$(Z1}lFMGvo6eKv6N}-PC3#Z1oCpjkT>-sXv&(=n<Ykfj|3Ki-*_#YUYJ)@S|
z)L)^Nf;E<@mw%10H5a>*`5TDtVJv>w2K|4xtb+#Zet0OI&ZJD$9TPp>-Mxv5YsoeE
ho=iHuRjxs0kL{9P+O<{uY}>^<YgXmeg1=-9{12Nhvy1=$
literal 0
HcmV?d00001
--
1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread* [dts] [PATCH V1 2/5] add N/A result in test report
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 1/5] add check case list huilong,xu
@ 2015-08-07 6:03 ` huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 3/5] add case skip result when case running huilong,xu
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: huilong,xu @ 2015-08-07 6:03 UTC (permalink / raw)
To: dts
From: huilong xu <huilongx.xu@intel.com>
Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
framework/excel_reporter.py | 9 ++++++---
1 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/framework/excel_reporter.py b/framework/excel_reporter.py
index 809dead..2030d87 100644
--- a/framework/excel_reporter.py
+++ b/framework/excel_reporter.py
@@ -85,13 +85,15 @@ class ExcelReporter(object):
self.sheet.write(0, 7, 'Pass', self.header_style)
self.sheet.write(0, 8, 'Fail', self.header_style)
self.sheet.write(0, 9, 'Blocked', self.header_style)
- self.sheet.write(0, 10, 'Not Run', self.header_style)
- self.sheet.write(0, 11, 'Total', self.header_style)
+ self.sheet.write(0, 10, 'N/A', self.header_style)
+ self.sheet.write(0, 11, 'Not Run', self.header_style)
+ self.sheet.write(0, 12, 'Total', self.header_style)
self.sheet.write(1, 7, Formula('COUNTIF(F2:F2000,"PASSED")'))
self.sheet.write(1, 8, Formula('COUNTIF(F2:F2000,"FAILED*") + COUNTIF(F2:F2000,"IXA*")'))
self.sheet.write(1, 9, Formula('COUNTIF(F2:F2000,"BLOCKED*")'))
- self.sheet.write(1, 11, Formula('H2+I2+J2+K2'))
+ self.sheet.write(1, 10, Formula('COUNTIF(F2:F2000,"N/A*")'))
+ self.sheet.write(1, 12, Formula('H2+I2+J2+K2+L2'))
self.sheet.col(0).width = 4000
self.sheet.col(1).width = 7500
@@ -105,6 +107,7 @@ class ExcelReporter(object):
self.sheet.col(9).width = 3000
self.sheet.col(10).width = 3000
self.sheet.col(11).width = 3000
+ self.sheet.col(12).width = 3000
def __styles(self):
header_pattern = xlwt.Pattern()
--
1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread* [dts] [PATCH V1 3/5] add case skip result when case running
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 1/5] add check case list huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 2/5] add N/A result in test report huilong,xu
@ 2015-08-07 6:03 ` huilong,xu
2015-08-07 6:05 ` Liu, Yong
2015-08-07 6:03 ` [dts] [PATCH V1 4/5] add checkCase module huilong,xu
` (2 subsequent siblings)
5 siblings, 1 reply; 10+ messages in thread
From: huilong,xu @ 2015-08-07 6:03 UTC (permalink / raw)
To: dts
From: huilong xu <huilongx.xu@intel.com>
Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
framework/test_result.py | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/framework/test_result.py b/framework/test_result.py
index 79faee1..2e62d9e 100644
--- a/framework/test_result.py
+++ b/framework/test_result.py
@@ -157,6 +157,11 @@ class Result(object):
"""
self.__set_test_case_result(result='PASSED', message='')
+ def test_case_skip(self,message):
+ """
+ set last test case add as N/A
+ """
+ self.__set_test_case_result(result='N/A', message=message)
def test_case_failed(self, message):
"""
Set last test case added as FAILED
--
1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [dts] [PATCH V1 3/5] add case skip result when case running
2015-08-07 6:03 ` [dts] [PATCH V1 3/5] add case skip result when case running huilong,xu
@ 2015-08-07 6:05 ` Liu, Yong
2015-08-07 6:29 ` Xu, HuilongX
0 siblings, 1 reply; 10+ messages in thread
From: Liu, Yong @ 2015-08-07 6:05 UTC (permalink / raw)
To: Xu, HuilongX, dts
One comment for add blank line.
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 3/5] add case skip result when case running
>
> From: huilong xu <huilongx.xu@intel.com>
>
>
> Signed-off-by: huilong xu <huilongx.xu@intel.com>
> ---
> framework/test_result.py | 5 +++++
> 1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/framework/test_result.py b/framework/test_result.py
> index 79faee1..2e62d9e 100644
> --- a/framework/test_result.py
> +++ b/framework/test_result.py
> @@ -157,6 +157,11 @@ class Result(object):
> """
> self.__set_test_case_result(result='PASSED', message='')
>
> + def test_case_skip(self,message):
> + """
> + set last test case add as N/A
> + """
> + self.__set_test_case_result(result='N/A', message=message)
Add blank line here.
> def test_case_failed(self, message):
> """
> Set last test case added as FAILED
> --
> 1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dts] [PATCH V1 3/5] add case skip result when case running
2015-08-07 6:05 ` Liu, Yong
@ 2015-08-07 6:29 ` Xu, HuilongX
0 siblings, 0 replies; 10+ messages in thread
From: Xu, HuilongX @ 2015-08-07 6:29 UTC (permalink / raw)
To: Liu, Yong, dts
Ok, I will update it at v2 version.
> -----Original Message-----
> From: Liu, Yong
> Sent: Friday, August 07, 2015 2:06 PM
> To: Xu, HuilongX; dts@dpdk.org
> Subject: RE: [dts] [PATCH V1 3/5] add case skip result when case running
>
> One comment for add blank line.
>
> > -----Original Message-----
> > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> > Sent: Friday, August 07, 2015 2:03 PM
> > To: dts@dpdk.org
> > Subject: [dts] [PATCH V1 3/5] add case skip result when case running
> >
> > From: huilong xu <huilongx.xu@intel.com>
> >
> >
> > Signed-off-by: huilong xu <huilongx.xu@intel.com>
> > ---
> > framework/test_result.py | 5 +++++
> > 1 files changed, 5 insertions(+), 0 deletions(-)
> >
> > diff --git a/framework/test_result.py b/framework/test_result.py
> > index 79faee1..2e62d9e 100644
> > --- a/framework/test_result.py
> > +++ b/framework/test_result.py
> > @@ -157,6 +157,11 @@ class Result(object):
> > """
> > self.__set_test_case_result(result='PASSED', message='')
> >
> > + def test_case_skip(self,message):
> > + """
> > + set last test case add as N/A
> > + """
> > + self.__set_test_case_result(result='N/A', message=message)
>
> Add blank line here.
>
> > def test_case_failed(self, message):
> > """
> > Set last test case added as FAILED
> > --
> > 1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* [dts] [PATCH V1 4/5] add checkCase module
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
` (2 preceding siblings ...)
2015-08-07 6:03 ` [dts] [PATCH V1 3/5] add case skip result when case running huilong,xu
@ 2015-08-07 6:03 ` huilong,xu
2015-08-07 6:03 ` [dts] [PATCH V1 5/5] add check case function in run case module huilong,xu
2015-08-07 6:06 ` [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test Liu, Yong
5 siblings, 0 replies; 10+ messages in thread
From: huilong,xu @ 2015-08-07 6:03 UTC (permalink / raw)
To: dts
From: huilong xu <huilongx.xu@intel.com>
1. add parse_file class for get check case info form execl file, and send check function list from tabel head.
2. add case_skip class for check case is skip or running.
a)from talbe head get check function list. excep "case name", "wq request", "comments"
b)when add a new check rule in excel file, you must add a check function in this clase, the function name is "check_rule" and parameter is a list.
eg, add kernel rule in excel table head. you need add function check_kernerl(self,kernel_type) if you not add the function, it will ignore this rule.
Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
framework/checkCase.py | 100 ++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 100 insertions(+), 0 deletions(-)
create mode 100644 framework/checkCase.py
diff --git a/framework/checkCase.py b/framework/checkCase.py
new file mode 100644
index 0000000..b2e79a0
--- /dev/null
+++ b/framework/checkCase.py
@@ -0,0 +1,100 @@
+import xlrd
+
+from settings import nic_name_from_type
+
+filter_file = r'./conf/dpdk_test_case_checklist.xls'
+filter_case = []
+check_function_dict = {}
+
+class parse_file():
+
+ def __init__(self):
+ try:
+ self.book = xlrd.open_workbook(filter_file)
+ self.sheet = self.book.sheet_by_index(0)
+ self.init_check_function_dict()
+ except:
+ pass
+
+ def init_check_function_dict(self):
+ '''
+ init check case functio, and skip case message.
+ '''
+ row_data = self.sheet.row_values(0)
+ for i in range(1,len(row_data)):
+ if row_data[i].lower() in ['wq number', 'comments']:
+ if 'message' not in check_function_dict:
+ check_function_dict['message'] = [i]
+ else:
+ check_function_dict['message'].append(i)
+ else:
+ check_function_dict[row_data[i].lower()] = i
+
+ def set_filter_case(self):
+ for row in range(self.sheet.nrows):
+ row_data = self.sheet.row_values(row)
+ # add case name
+ tmp_filter = [row_data[0]]
+ for i in range(1,len(row_data) - 2):
+ tmp_filter.append(row_data[i].split(','))
+
+ tmp_filter.append(row_data[-2])
+ tmp_filter.append(row_data[-1])
+
+ filter_case.append(tmp_filter)
+
+class check_case_skip():
+ def __init__(self, Dut):
+ self.dut = Dut
+ self.comments = ''
+
+ def check_os(self,os_type):
+ if 'all' == os_type[0].lower():
+ return True
+ dut_os_type = self.dut.get_os_type()
+ if dut_os_type in os_type:
+ return True
+ else:
+ return False
+
+ def check_nic(self, nic_type):
+ if 'all' == nic_type[0].lower():
+ return True
+ dut_nic_type = nic_name_from_type(self.dut.ports_info[0]['type'])
+ if dut_nic_type in nic_type:
+ return True
+ else:
+ return False
+
+ def check_target(self,target):
+ if 'all' == target[0].lower():
+ return True
+ if self.dut.target in target:
+ return True
+ else:
+ return False
+
+ def case_skip(self, case_name):
+ skip_flage = False
+ for rule in filter_case[1:]:
+ # check case name
+ if case_name == rule[0]:
+ for key in check_function_dict.keys():
+ try:
+ if 'message' == key:
+ continue
+ check_function = getattr(self, 'check_%s' % key)
+ except:
+ print "can't check %s type" % key
+ if check_function(rule[check_function_dict[key]]):
+ skip_flage = True
+ else:
+ skip_flage = False
+ break
+ if skip_flage:
+ if 'message' in check_function_dict:
+ for i in check_function_dict['message']:
+ self.comments += '%s,' % rule[i]
+ return skip_flage
+
+ return skip_flage
--
1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread* [dts] [PATCH V1 5/5] add check case function in run case module
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
` (3 preceding siblings ...)
2015-08-07 6:03 ` [dts] [PATCH V1 4/5] add checkCase module huilong,xu
@ 2015-08-07 6:03 ` huilong,xu
2015-08-07 6:52 ` Liu, Yong
2015-08-07 6:06 ` [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test Liu, Yong
5 siblings, 1 reply; 10+ messages in thread
From: huilong,xu @ 2015-08-07 6:03 UTC (permalink / raw)
To: dts
From: huilong xu <huilongx.xu@intel.com>
Signed-off-by: huilong xu <huilongx.xu@intel.com>
---
framework/dts.py | 21 +++++++++++++++------
1 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/framework/dts.py b/framework/dts.py
index e9513c6..cb8f26e 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -57,7 +57,7 @@ from logger import getLogger
import logger
import debugger
from virt_scene import VirtScene
-
+from checkCase import *
import sys
reload(sys)
sys.setdefaultencoding('UTF8')
@@ -291,6 +291,8 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
"""
Run each target in execution targets.
"""
+ global skip_case_mode
+ skip_case_mode = check_case_skip(dut)
if scenario != '':
scene = VirtScene(dut, tester, scenario)
else:
@@ -299,7 +301,7 @@ def dts_run_target(crbInst, targets, test_suites, nic, scenario):
if scene:
scene.load_config()
scene.create_scene()
-
+
for target in targets:
log_handler.info("\nTARGET " + target)
result.target = target
@@ -402,11 +404,12 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
global Package
global Patches
global scenario
-
+ global skip_case_mode
# save global variable
Package = pkgName
Patches = patch
-
+ check_case = parse_file()
+ check_case.set_filter_case()
# prepare the output folder
if output_dir == '':
output_dir = FOLDERS['Output']
@@ -459,7 +462,6 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
# verify if the delimiter is good if the lists are vertical
dutIP, targets, test_suites, nics, scenario = dts_parse_config(section)
-
log_handler.info("\nDUT " + dutIP)
# look up in crbs - to find the matching IP
@@ -479,6 +481,7 @@ def run_all(config_file, pkgName, git, patch, skip_setup,
# init dut, tester crb
dts_crbs_init(crbInst, skip_setup, read_cache, project, base_dir, nics, virttype)
+ skip_case_mode = check_case_skip(dut)
# Run DUT prerequisites
if dts_run_prerequisties(pkgName, patch) is False:
dts_crbs_exit()
@@ -576,6 +579,7 @@ def execute_all_test_cases(test_suite):
"""
if functional_only:
for test_case in get_functional_test_cases(test_suite):
+ skip_case_mode.case_skip(test_case)
execute_test_case(test_suite, test_case)
if performance_only:
for test_case in get_performance_test_cases(test_suite):
@@ -591,8 +595,13 @@ def execute_test_case(test_suite, test_case):
global debug_case
global module
result.test_case = test_case.__name__
-
rst.write_title("Test Case: " + test_case.__name__)
+ if skip_case_mode.case_skip(test_case.__name__[len("test_"):]):
+ log_handler.info('Test Case %s Result SKIPED:' % test_case.__name__)
+ rst.write_result("N/A")
+ result.test_case_skip(skip_case_mode.comments)
+ save_all_results()
+ return
if performance_only:
rst.write_annex_title("Annex: " + test_case.__name__)
try:
--
1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [dts] [PATCH V1 5/5] add check case function in run case module
2015-08-07 6:03 ` [dts] [PATCH V1 5/5] add check case function in run case module huilong,xu
@ 2015-08-07 6:52 ` Liu, Yong
0 siblings, 0 replies; 10+ messages in thread
From: Liu, Yong @ 2015-08-07 6:52 UTC (permalink / raw)
To: Xu, HuilongX, dts
One question about skip_case_mode.case_skip.
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 5/5] add check case function in run case module
>
> From: huilong xu <huilongx.xu@intel.com>
>
>
> Signed-off-by: huilong xu <huilongx.xu@intel.com>
> ---
> framework/dts.py | 21 +++++++++++++++------
> 1 files changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/framework/dts.py b/framework/dts.py
> index e9513c6..cb8f26e 100644
> --- a/framework/dts.py
> +++ b/framework/dts.py
> @@ -57,7 +57,7 @@ from logger import getLogger
> import logger
> import debugger
> from virt_scene import VirtScene
> -
> +from checkCase import *
> import sys
> reload(sys)
> sys.setdefaultencoding('UTF8')
> @@ -291,6 +291,8 @@ def dts_run_target(crbInst, targets, test_suites, nic,
> scenario):
> """
> Run each target in execution targets.
> """
> + global skip_case_mode
> + skip_case_mode = check_case_skip(dut)
> if scenario != '':
> scene = VirtScene(dut, tester, scenario)
> else:
> @@ -299,7 +301,7 @@ def dts_run_target(crbInst, targets, test_suites, nic,
> scenario):
> if scene:
> scene.load_config()
> scene.create_scene()
> -
> +
> for target in targets:
> log_handler.info("\nTARGET " + target)
> result.target = target
> @@ -402,11 +404,12 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
> global Package
> global Patches
> global scenario
> -
> + global skip_case_mode
> # save global variable
> Package = pkgName
> Patches = patch
> -
> + check_case = parse_file()
> + check_case.set_filter_case()
> # prepare the output folder
> if output_dir == '':
> output_dir = FOLDERS['Output']
> @@ -459,7 +462,6 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
>
> # verify if the delimiter is good if the lists are vertical
> dutIP, targets, test_suites, nics, scenario =
> dts_parse_config(section)
> -
> log_handler.info("\nDUT " + dutIP)
>
> # look up in crbs - to find the matching IP
> @@ -479,6 +481,7 @@ def run_all(config_file, pkgName, git, patch,
> skip_setup,
> # init dut, tester crb
> dts_crbs_init(crbInst, skip_setup, read_cache, project, base_dir,
> nics, virttype)
>
> + skip_case_mode = check_case_skip(dut)
> # Run DUT prerequisites
> if dts_run_prerequisties(pkgName, patch) is False:
> dts_crbs_exit()
> @@ -576,6 +579,7 @@ def execute_all_test_cases(test_suite):
> """
> if functional_only:
> for test_case in get_functional_test_cases(test_suite):
> + skip_case_mode.case_skip(test_case)
Why call this function here? In function execute_test_case also check whether skip test case.
> execute_test_case(test_suite, test_case)
> if performance_only:
> for test_case in get_performance_test_cases(test_suite):
> @@ -591,8 +595,13 @@ def execute_test_case(test_suite, test_case):
> global debug_case
> global module
> result.test_case = test_case.__name__
> -
> rst.write_title("Test Case: " + test_case.__name__)
> + if skip_case_mode.case_skip(test_case.__name__[len("test_"):]):
> + log_handler.info('Test Case %s Result SKIPED:' %
> test_case.__name__)
> + rst.write_result("N/A")
> + result.test_case_skip(skip_case_mode.comments)
> + save_all_results()
> + return
> if performance_only:
> rst.write_annex_title("Annex: " + test_case.__name__)
> try:
> --
> 1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test
2015-08-07 6:03 [dts] [PATCH V1 0/5] add check case list for skip the case, when the case can't test huilong,xu
` (4 preceding siblings ...)
2015-08-07 6:03 ` [dts] [PATCH V1 5/5] add check case function in run case module huilong,xu
@ 2015-08-07 6:06 ` Liu, Yong
5 siblings, 0 replies; 10+ messages in thread
From: Liu, Yong @ 2015-08-07 6:06 UTC (permalink / raw)
To: Xu, HuilongX, dts
Please change " BLURB " to description of this patch set.
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of huilong,xu
> Sent: Friday, August 07, 2015 2:03 PM
> To: dts@dpdk.org
> Subject: [dts] [PATCH V1 0/5] add check case list for skip the case, when
> the case can't test
>
> From: huilong xu <huilongx.xu@intel.com>
>
> *** BLURB HERE ***
>
> huilong xu (5):
> add check case list
> add N/A result in test report
> add case skip result when case running
> add checkCase module
> add check case function in run case module
>
> conf/dpdk_test_case_checklist.xls | Bin 0 -> 19456 bytes
> framework/checkCase.py | 100
> +++++++++++++++++++++++++++++++++++++
> framework/dts.py | 21 ++++++--
> framework/excel_reporter.py | 9 ++-
> framework/test_result.py | 5 ++
> 5 files changed, 126 insertions(+), 9 deletions(-)
> create mode 100644 conf/dpdk_test_case_checklist.xls
> create mode 100644 framework/checkCase.py
>
> --
> 1.7.4.4
^ permalink raw reply [flat|nested] 10+ messages in thread