From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 6262E3252 for ; Fri, 23 Jun 2017 11:30:32 +0200 (CEST) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP; 23 Jun 2017 02:30:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.39,377,1493708400"; d="scan'208";a="118410710" Received: from irsmsx103.ger.corp.intel.com ([163.33.3.157]) by fmsmga006.fm.intel.com with ESMTP; 23 Jun 2017 02:30:29 -0700 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.26]) by IRSMSX103.ger.corp.intel.com ([169.254.3.9]) with mapi id 14.03.0319.002; Fri, 23 Jun 2017 10:30:28 +0100 From: "Richardson, Bruce" To: "Xu, Qian Q" , Thomas Monjalon , "Wei, FangfangX" CC: "ci@dpdk.org" , "O'Driscoll, Tim" , Eugene Voronov Thread-Topic: [dpdk-ci] script to determine target repo (was DPDK Lab) Thread-Index: AQHStCGpMpJWjq4ArUimVAHX1/Hj4aHVn+UAgAAHowCAWaRFAIAAGZEQ///zlgCAAyepAIAAGwoQ Date: Fri, 23 Jun 2017 09:30:28 +0000 Message-ID: <59AF69C657FD0841A61C55336867B5B072181A27@IRSMSX104.ger.corp.intel.com> References: <26FA93C7ED1EAA44AB77D62FBE1D27BA722C837C@IRSMSX108.ger.corp.intel.com> <067B569323FEB248B5CB480E1954F4346F4174FD@SHSMSX101.ccr.corp.intel.com> <59AF69C657FD0841A61C55336867B5B0721805ED@IRSMSX104.ger.corp.intel.com> <1814490.xy7qWLUraa@xps> <82F45D86ADE5454A95A89742C8D1410E3B698B4E@shsmsx102.ccr.corp.intel.com> In-Reply-To: <82F45D86ADE5454A95A89742C8D1410E3B698B4E@shsmsx102.ccr.corp.intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOWM2ZjkyMTAtZjFlYy00ZjVhLThkYTktYjZlNzAyOWI5MTNjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IjB2dHJKOHFhcU9wRGtYaUVkcnl4TjBhc3ZZZWpuc1RaMTRsYzRPRjc0cUE9In0= x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 10.0.102.7 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab) X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2017 09:30:33 -0000 > -----Original Message----- > From: Xu, Qian Q > Sent: Friday, June 23, 2017 9:44 AM > To: Thomas Monjalon ; Wei, FangfangX > > Cc: ci@dpdk.org; Richardson, Bruce ; > O'Driscoll, Tim ; Eugene Voronov > > Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab) >=20 > Thomas/Bruce > 1. For determining the repo tree to target, I don't believe that we can > ever > > come up with a 100% accurate rule, as the tree to which a set is to be > > applied can be difficult to determine, so it may be done on the basis o= f > on-list discussion. > > A 90% accurate rule it what we may have to accept. >=20 > -- Then if we find the performance issue, then maybe it's a false alarm > due to apply to the wrong repo. So, we may face many false alarms > according with the time. > Then people may not treat the performance issue as a problem, so I still > think we need to try 100% accurate to have a more trustable result when w= e > send out the alarm. I find that rather improbable, and not worth considering. For that to per a= problem multiple unlikely events have to occur: 1) we mis-identify the tree on which the set is to be applied (we should be= able to get to 90% accuracy here) 2) the patchset must apply cleanly to the "wrong" tree (this is reasonably = likely, but it's still another condition that has to be met for us to have = a problem) 3) the patchset has to cause a performance regression in the "wrong" tree 4) but NOT cause a regression when in the right tree. If we assume 90% accuracy of tree identification, optimistically that 90% o= f patches will apply to the wrong tree, that 5% of patches cause a performa= nce regression (an overestimate IMHO), and that even 1/3 of those won't cau= se a performance regression in the right tree (a very overestimate IMHO, I = would expect just about none of them to even have this), it still means tha= t only about 1 patch in 1000 will show as a false positive performance regr= ession. 0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regres= sion) =3D 0.0015, or 0.15% So worst case, I still don't think we have a problem for the scenario you d= escribe. /Bruce >=20 >=20 > > -----Original Message----- > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > Sent: Wednesday, June 21, 2017 4:33 PM > > To: Wei, FangfangX > > Cc: ci@dpdk.org; Richardson, Bruce ; Xu, > > Qian Q ; O'Driscoll, Tim > > ; Eugene Voronov > > Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab) > > > > I agree with Bruce. > > > > Sorry for not having written the scripts yet. > > Someone else in Mellanox should do it in July. > > In the meantime, do not hesitate to share your code if it speed up > things. > > > > Thanks a lot > > > > > > 21/06/2017 10:20, Richardson, Bruce: > > > Hi Fangfang, > > > > > > My comments on the questions you asked: > > > > > > 1. For determining the repo tree to target, I don't believe that we > > > can ever > > come up with a 100% accurate rule, as the tree to which a set is to be > > applied can be difficult to determine, so it may be done on the basis o= f > on-list discussion. > > A 90% accurate rule it what we may have to accept. However, since > > applying a patchset to a tree should not be a time-consuming > > operation, I suggest any script produce a list of possible trees in > > priority order to try. If not net, then try main, etc. etc. > > > > > > 2. Using the order from patch titles is correct. > > > > > > /Bruce