From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f47.google.com (mail-it0-f47.google.com [209.85.214.47]) by dpdk.org (Postfix) with ESMTP id 753091DB1 for ; Mon, 7 Nov 2016 11:47:19 +0100 (CET) Received: by mail-it0-f47.google.com with SMTP id q124so14523790itd.1 for ; Mon, 07 Nov 2016 02:47:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qwilt-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=4nYIpOVK0fmItxRMxmfNVAXAA8SEjf8/qaws4LAhwr0=; b=Olko7G6q6JABdRFeD2P1NgHCadwD7E3x5whBvCVKB7qh3iSrYZ12Iz4lL399gMBosX +U7Rcf/x8G+UgVEY6WRdqf0FnYQqmlsE6dzN270RD0y8ZajlF301sb6X6weIwJk7JYTc +kxOT6EKUJzZFhs4octE5ZymacQtzA+4MiSyVMPXydr1WQ8I8C+h+r5bnOjB+QSIeFNB wRmRBHa6gzud8RYe4avGUh/wJETHkp4sgcZDRymTtUMM6LWKVD0prhEroHZfLoDml3el zdsxUB9+LK+9q1BW+E7vAxxZqVbaXatu/wFSxozvz1t3A3t/qCXFj8hJ4Lmw7RREhbYA TJmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4nYIpOVK0fmItxRMxmfNVAXAA8SEjf8/qaws4LAhwr0=; b=SqLcWeIvSVbPmJtwJx2EGrWZnHDglv30X/75KCHu3cOAlqXi0u1xB1enapu1iFRwSU 2C43y+d7JsE/wejv3o4uW1KtbcgyPvzRg/DGrCxGMVxYevwOiyY9ORoMGPtj0jMRM+2I 6AcITcc+yLSKCm69p2+iwABv6Y8kA7vIP2VyAvv5dUCATyxc687s0jgJ+NXPJwYGbUY6 BH1QZcALQ+XpthQxiKxYQLzpaGtZJINRhQhERYDROFaixMylxbH1FmOjuWpluGhuikMM TBHEpXBibv9yYdWhBoMSak843ALDh1596IL5JR9GCas69RomY/wgh8mmu3pWiAGgIHxX sCnA== X-Gm-Message-State: ABUngveaPZBc1bQhelUFM0A0MaK+tA2vdNC8qKzp8WPP5obW08wQVySBetP5IuWmXyBBx4X3tUDvI8GeZUE4Nw== X-Received: by 10.36.76.22 with SMTP id a22mr4576381itb.44.1478515638046; Mon, 07 Nov 2016 02:47:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.137.71 with HTTP; Mon, 7 Nov 2016 02:47:17 -0800 (PST) In-Reply-To: <26FA93C7ED1EAA44AB77D62FBE1D27BA6760D97B@IRSMSX108.ger.corp.intel.com> References: <86228AFD5BCD8E4EBFD2B90117B5E81E60310FA1@SHSMSX103.ccr.corp.intel.com> <3804736.OkjAMiHs6v@xps13> <82F45D86ADE5454A95A89742C8D1410E3923B784@shsmsx102.ccr.corp.intel.com> <1689822.FXvyOjK9nz@xps13> <26FA93C7ED1EAA44AB77D62FBE1D27BA6760D97B@IRSMSX108.ger.corp.intel.com> From: Arnon Warshavsky Date: Mon, 7 Nov 2016 12:47:17 +0200 Message-ID: To: "O'Driscoll, Tim" Cc: "Jerome Tollet (jtollet)" , Thomas Monjalon , "Xu, Qian Q" , "moving@dpdk.org" , "Liu, Yong" , "ci@dpdk.org" Content-Type: multipart/alternative; boundary=001a11447d866889800540b3bf25 Subject: Re: [dpdk-ci] [dpdk-moving] proposal for DPDK CI improvement X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 10:47:19 -0000 --001a11447d866889800540b3bf25 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Mon, Nov 7, 2016 at 12:34 PM, O'Driscoll, Tim wrote: > > > -----Original Message----- > > From: moving [mailto:moving-bounces@dpdk.org] On Behalf Of Jerome Tolle= t > > (jtollet) > > Sent: Monday, November 7, 2016 10:27 AM > > To: Thomas Monjalon ; Xu, Qian Q > > > > Cc: moving@dpdk.org; Liu, Yong ; ci@dpdk.org > > Subject: Re: [dpdk-moving] proposal for DPDK CI improvement > > > > Hi Thomas & Qian, > > IMHO, performance results should be centralized and executed in a > > trusted & controlled environment. > > If official DPDK numbers are coming from private lab=E2=80=99s vendors, > > perception might be that they are not 100% neutral. That would probably > > not help DPDK community to be seen open & transparent. > > +1 > > Somebody (Jan Blunck I think) also said on last week's call that > performance testing was a higher priority than CI for a centralized lab. = A > model where we have centralized performance test and distributed CI might > work well. +1 to the above approach , yet I still see value in publishing both types of performance results as long as they are clearly separated. This might might need a way to retroactively mark some results as "proved invalid" but otoh encourage a cycle of propagating distributed tests proved beneficial correct and unbiased to the central tests. /Arnon --001a11447d866889800540b3bf25 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Mon, Nov 7, 2016 at 12:34 PM, O'Driscoll, Tim = <tim.odrisc= oll@intel.com> wrote:

> -----Original Message-----
> From: moving [mailto:moving= -bounces@dpdk.org] On Behalf Of Jerome Tollet
> (jtollet)
> Sent: Monday, November 7, 2016 10:27 AM
> To: Thomas Monjalon <t= homas.monjalon@6wind.com>; Xu, Qian Q
> <qian.q.xu@intel.com>=
> Cc: moving@dpdk.org; Liu, Yong = <yong.liu@intel.com>; ci@dpdk.org
> Subject: Re: [dpdk-moving] proposal for DPDK CI improvement
>
> Hi Thomas & Qian,
> IMHO, performance results should be centralized and executed in a
> trusted & controlled environment.
> If official DPDK numbers are coming from private lab=E2=80=99s vendors= ,
> perception might be that they are not 100% neutral. That would probabl= y
> not help DPDK community to be seen open & transparent.

+1

Somebody (Jan Blunck I think) also said on last week's call that perfor= mance testing was a higher priority than CI for a centralized lab. A model = where we have centralized performance test and distributed CI might work we= ll.


+1 to the ab= ove approach , yet I still see value in publishing both types of performanc= e results as long as they are clearly separated.
This might might need a way to retroactively mark some results as= "proved invalid" but otoh encourage a cycle of propagating distr= ibuted tests proved beneficial correct and unbiased to the central tests.
/Arnon

--001a11447d866889800540b3bf25--