From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id DD2781BF54 for ; Wed, 4 Jul 2018 16:22:46 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 04 Jul 2018 07:22:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,306,1526367600"; d="scan'208";a="54049510" Received: from irsmsx153.ger.corp.intel.com ([163.33.192.75]) by orsmga007.jf.intel.com with ESMTP; 04 Jul 2018 07:22:41 -0700 Received: from irsmsx108.ger.corp.intel.com ([169.254.11.139]) by IRSMSX153.ger.corp.intel.com ([169.254.9.60]) with mapi id 14.03.0319.002; Wed, 4 Jul 2018 15:22:41 +0100 From: "O'Driscoll, Tim" To: "'ci@dpdk.org'" CC: 'Bob Noseworthy' , "Mcnamara, John" , 'Shepard Siegel' , 'Thomas Monjalon' , 'Erez Scop' , 'Shreyansh Jain' , "Xu, Qian Q" , "'pmacarth@iol.unh.edu'" , 'Matt Spencer' , 'George Zhao' , "Mishra, Shishir" , 'Lixuming' , "Tkachuk, Georgii" , 'Trishan de Lanerolle' , Sean Campbell , Ali Alnubani , 'May Chen' , "Lodha, Nishant" , "Zhang, Chun" , "'Malla, Malathi'" , 'khemendra kumar' , "'graeme.gregory@linaro.org'" , "Yigit, Ferruh" , "Tu, Lijuan" Thread-Topic: Minutes of DPDK Lab Meeting, July 3rd Thread-Index: AdQToiFDxe+XaaxDQpmX/RXD5jik4Q== Date: Wed, 4 Jul 2018 14:22:40 +0000 Message-ID: <26FA93C7ED1EAA44AB77D62FBE1D27BAB7708B46@IRSMSX108.ger.corp.intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNzY0ZWUxNWEtZjMzYy00Nzk5LWFjZGItMjIyMGJhOTczNjA5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiNzl0SytHUEFycm96Z2E5dG0zUFQyT25FdDdFOUFIeUFxUFltRjhuQnZrYk1pXC96MDlKQ2RidXJTRlF6dXhydDYifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.200.100 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 04 Jul 2018 22:26:27 +0200 Subject: [dpdk-ci] Minutes of DPDK Lab Meeting, July 3rd X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2018 14:22:47 -0000 Lab Policies and Procedures: - Bob's draft doc is at: =A0https://docs.google.com/document/d/1rtSwpNKltVN= yDKNWgeTV5gaYeDoAPlK-sfm8XE7o_5s/edit?usp=3Dsharing. Please review and comm= ent. - Ferruh suggested adding some form of change control for the lab. This can= be included in the Policies and Procedures doc. Test Setup: - Current dashboard is at: https://dpdkdashboard.iol.unh.edu/results/dashbo= ard/ - At the moment, all test cases are reporting errors (either Possible Regre= ssion or Apply Error). We need to reduce the number of failures to a manage= able level, so that we're only highlighting genuine problems. - At the moment, test results are compared with a hardcoded value provided = by each vendor to determine whether a test passes or not. Agreed that we sh= ould be comparing performance versus master to determine if a patch passes = or fails. Each vendor needs to update their scripts so that test results ar= e compared against master. Vendors also need to review the tolerance (% dev= iation allowed for test to pass) for the tests to make sure it's correct. - Vendors also need to confirm if they're happy for the relative performanc= e data (what % patch improved/degraded performance by compared to master) f= or their platforms to be made public. This will allow the next level of tes= t results to be accessible to anybody, so they can see which platform tests= are failing on, and how much degradation in performance there is. - When the previous two items are complete (reduce false positives, make ne= xt level of performance data public), we'll be able to link to the dashboar= d from DPDK.org. The dashboard can also be integrated with Patchwork at tha= t time.