From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id C3AC12BAF for ; Thu, 16 Aug 2018 11:33:37 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Aug 2018 02:33:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,246,1531810800"; d="scan'208,217";a="80719249" Received: from irsmsx153.ger.corp.intel.com ([163.33.192.75]) by fmsmga004.fm.intel.com with ESMTP; 16 Aug 2018 02:33:34 -0700 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.40]) by IRSMSX153.ger.corp.intel.com ([169.254.9.126]) with mapi id 14.03.0319.002; Thu, 16 Aug 2018 10:33:34 +0100 From: "Yigit, Ferruh" To: "ren@iol.unh.edu" CC: "O'Driscoll, Tim" , "'ci@dpdk.org'" Thread-Topic: Minutes of DPDK Lab Meeting, July 17th Thread-Index: AdQfcBBqBFhXj01EQGKYgN4ojL8dugV0GBaQ Date: Thu, 16 Aug 2018 09:33:32 +0000 Message-ID: <9A3E4DCF947BAB4FBECB1FA88D4D67891F33B018@IRSMSX101.ger.corp.intel.com> References: <26FA93C7ED1EAA44AB77D62FBE1D27BAB7721682@IRSMSX107.ger.corp.intel.com> In-Reply-To: <26FA93C7ED1EAA44AB77D62FBE1D27BAB7721682@IRSMSX107.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTY3M2QxYmEtYjcwYi00NGI4LWFjNWItODhlMWYxNmUyZDEyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoia25TbzVCQXR5a20yak5cL3NqSGp4dno0eGY4bzdPamJuUEJXVHJCcmF2ZXZzVXY0MnNFRWpuV3lmTE9nVzRJSjgifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: multipart/alternative; boundary="_000_9A3E4DCF947BAB4FBECB1FA88D4D67891F33B018IRSMSX101gercor_" MIME-Version: 1.0 Subject: Re: [dpdk-ci] Minutes of DPDK Lab Meeting, July 17th X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 09:33:38 -0000 --_000_9A3E4DCF947BAB4FBECB1FA88D4D67891F33B018IRSMSX101gercor_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Bob, A few comments/questions on "UNH Policies and Procedures Doc": - Objective and Scope of Project - 2: "DPDK-enabled applications", what are these applicatio= ns, are they refer to testpmd/l2fwd like test applications, or OVS/VPP like= other products using DPDK? And if these are other products, will they run = for all vendors or for the vendor requested it? - 3: "Demonstrate any new feature performance of DPDK", is = this via updating test scripts, if not how these new test will be run and h= ow results will be shared? - What is the plan to use Lab for Continuous Integration, a= dding more sanity checks than performance checks by time, as far as I know = this was one of the initial plans for a lab. - DPDK Branch(s) to test - 5.1.1: "master", a little background, in dpdk development= there are multiple sub-trees, some specific patches targets specific trees= , these sub-trees are merged into main tree before release candidate, and t= here is a target to do regular integration from these sub-trees. As a result of this process, for example a patch sent for next-net sub-tree= may not apply cleanly to main repo, so it won't be tested. But that patch = will be applied to next-net tree and a week later next-net tree will be mer= ged into main tree, this patch can be something affects the performance, bu= t it won't be detected. Later, when a patch arrives that can be applied on = main tree, it will reveal the performance issue, but suspect will be the wr= ong patch and the problematic patch will be already merged. We need a solut= ion for this. There are 5 sub-trees merged into main tree and more than hal= f of the patches are coming to main repo through them. - Private DPDK-Member only Dashboard Specification - 5.6.1.2: "The delta-values of the script output, per test performed.", in= member-only dashboard, why not show base value too, since it will be updat= ed regularly, via "--update-expected argument", it would be good to see bot= h current baseline and the diff. - I am for defining a change management system, there are multiple vendor a= nd multiple requests, it would be good to trace, discuss and record the res= ult for all of them systematically. Thanks, ferruh From: O'Driscoll, Tim Sent: Thursday, July 19, 2018 3:55 PM To: 'ci@dpdk.org' Cc: 'Bob Noseworthy' ; Mcnamara, John ; 'Shepard Siegel' ; 'Thomas Monjalon'= ; 'Erez Scop' ; 'Shreyansh Jain'= ; Xu, Qian Q ; 'pmacarth@iol.= unh.edu' ; 'Matt Spencer' ; 'Ge= orge Zhao' ; 'Mishra, Shishir' ; 'Lixuming' ; Tkachuk, Georgii ; 'Trishan de Lanerolle' ;= 'Sean Campbell' ; 'Ali Alnubani' ; 'May Chen' ; 'Lodha, Nishant' ; Zhang, Chun ; 'Malla, Malathi' ; 'khemendra kumar' ; 'grae= me.gregory@linaro.org' ; Yigit, Ferruh ; Tu, Lijuan Subject: Minutes of DPDK Lab Meeting, July 17th UNH Policies and Procedures Doc: Document is available at: https://docs.google.com/document/d/1rtSwpNKltVNyD= KNWgeTV5gaYeDoAPlK-sfm8XE7o_5s/edit?usp=3Dsharing I reviewed it and it looks good to me. Please add any comments to the google doc, or send them directly to Bob (re= n@iol.unh.edu). Dashboard: Dashboard is in much better shape now and most tests are passing. We do need to monitor the results and determine if the tolerance is right. = If it's too high then all tests will pass and we'll miss any genuine issues= . If it's too low then we'll have too many tests failing when there isn't r= eally a problem. Agreed that tuning the tolerance is the responsibility of = each vendor. For Intel, we need to add the specific test config to the results page. We = need to confirm the config details to Patrick, and he'll add this to the re= sults pages. Other vendors may want to do something similar. Hardware: Shreyansh is almost ready to ship the NXP hardware. It should be in UNH in = 2-3 weeks. --_000_9A3E4DCF947BAB4FBECB1FA88D4D67891F33B018IRSMSX101gercor_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Bob,

 

A few comments/questio= ns on “UNH Policies and Procedures Doc”:

 

- Objective and Scope = of Project

   &nbs= p;            - 2: &= #8220;DPDK-enabled applications”, what are these applications, are th= ey refer to testpmd/l2fwd like test applications, or OVS/VPP like other pro= ducts using DPDK? And if these are other products, will they run for all vendors or for the vendor requested it?

   &nbs= p;            - 3: &= #8220;Demonstrate any new feature performance of DPDK”, is this via u= pdating test scripts, if not how these new test will be run and how results= will be shared?

   &nbs= p;            - What= is the plan to use Lab for Continuous Integration, adding more sanity chec= ks than performance checks by time, as far as I know this was one of the in= itial plans for a lab.

 

- DPDK Branch(s) to te= st

   &nbs= p;            - 5.1.= 1: “master”, a little background, in dpdk development there are= multiple sub-trees, some specific patches targets specific trees, these su= b-trees are merged into main tree before release candidate, and there is a target to do regular integration from these sub-= trees.

As a result of this pr= ocess, for example a patch sent for next-net sub-tree may not apply cleanly= to main repo, so it won’t be tested. But that patch will be applied = to next-net tree and a week later next-net tree will be merged into main tree, this patch can be something affects th= e performance, but it won’t be detected. Later, when a patch arrives = that can be applied on main tree, it will reveal the performance issue, but= suspect will be the wrong patch and the problematic patch will be already merged. We need a solution for this.<= /b> There are 5 sub-trees merged into main tree and more than half of the p= atches are coming to main repo through them.

 

- Private DPDK-Member = only Dashboard Specification

- 5.6.1.2: “The delta-values of the script output, per test perf= ormed.”, in member-only dashboard, why not show base value too, since= it will be updated regularly, via “--update-expected argument”, it would be good to see both current baseline and the dif= f.

 

- I am for defining a = change management system, there are multiple vendor and multiple requests, = it would be good to trace, discuss and record the result for all of them sy= stematically.

 

Thanks,

ferruh

 

From: O'D= riscoll, Tim
Sent: Thursday, July 19, 2018 3:55 PM
To: 'ci@dpdk.org' <ci@dpdk.org>
Cc: 'Bob Noseworthy' <ren@iol.unh.edu>; Mcnamara, John <joh= n.mcnamara@intel.com>; 'Shepard Siegel' <shepard.siegel@atomicrules.c= om>; 'Thomas Monjalon' <thomas@monjalon.net>; 'Erez Scop' <erez= sc@mellanox.com>; 'Shreyansh Jain' <shreyansh.jain@nxp.com>; Xu, Qian Q <qian.q.xu@intel.com>; 'pmacarth@iol.unh.edu' <pmacart= h@iol.unh.edu>; 'Matt Spencer' <Matt.Spencer@arm.com>; 'George Zha= o' <George.Y.Zhao@huawei.com>; 'Mishra, Shishir' <Shishir.Mishra@s= pirent.com>; 'Lixuming' <lixuming@huawei.com>; Tkachuk, Georgii <georgii.tkachuk@intel.com>; 'Trishan de Lanerolle' <tdelanerolle= @linuxfoundation.org>; 'Sean Campbell' <scampbel@qti.qualcomm.com>= ; 'Ali Alnubani' <alialnu@mellanox.com>; 'May Chen' <May.Chen@huaw= ei.com>; 'Lodha, Nishant' <Nishant.Lodha@cavium.com>; Zhang, Chun <chun.zhang@intel.com>; 'Malla, Malathi' <Malathi.Malla@spir= ent.com>; 'khemendra kumar' <khemendra.kumar13@gmail.com>; 'graeme= .gregory@linaro.org' <graeme.gregory@linaro.org>; Yigit, Ferruh <f= erruh.yigit@intel.com>; Tu, Lijuan <lijuan.tu@intel.com>
Subject: Minutes of DPDK Lab Meeting, July 17th

 

UNH Policies and Procedures Doc:

Document is available at:= https://docs.google.com/document/d/1rtSwpNKltVNyDKNWgeTV5gaYeDoAPlK-sfm8XE7= o_5s/edit?usp=3Dsharing

I reviewed it and it look= s good to me.

Please add any comments t= o the google doc, or send them directly to Bob (ren@iol.unh.edu).

 

Dashboard:

Dashboard is in much bett= er shape now and most tests are passing.

We do need to monitor the= results and determine if the tolerance is right. If it’s too high th= en all tests will pass and we’ll miss any genuine issues. If it’= ;s too low then we’ll have too many tests failing when there isn’t really a problem. Agreed that tuning the tolerance is th= e responsibility of each vendor.

For Intel, we need to add= the specific test config to the results page. We need to confirm the confi= g details to Patrick, and he’ll add this to the results pages. Other = vendors may want to do something similar.

 

Hardware:

Shreyansh is almost ready= to ship the NXP hardware. It should be in UNH in 2-3 weeks.

 

 

 

 

--_000_9A3E4DCF947BAB4FBECB1FA88D4D67891F33B018IRSMSX101gercor_--