On Mon, Nov 7, 2016 at 12:34 PM, O'Driscoll, Tim <tim.odriscoll@intel.com>
wrote:

>
> > -----Original Message-----
> > From: moving [mailto:moving-bounces@dpdk.org] On Behalf Of Jerome Tollet
> > (jtollet)
> > Sent: Monday, November 7, 2016 10:27 AM
> > To: Thomas Monjalon <thomas.monjalon@6wind.com>; Xu, Qian Q
> > <qian.q.xu@intel.com>
> > Cc: moving@dpdk.org; Liu, Yong <yong.liu@intel.com>; ci@dpdk.org
> > Subject: Re: [dpdk-moving] proposal for DPDK CI improvement
> >
> > Hi Thomas & Qian,
> > IMHO, performance results should be centralized and executed in a
> > trusted & controlled environment.
> > If official DPDK numbers are coming from private lab’s vendors,
> > perception might be that they are not 100% neutral. That would probably
> > not help DPDK community to be seen open & transparent.
>
> +1
>
> Somebody (Jan Blunck I think) also said on last week's call that
> performance testing was a higher priority than CI for a centralized lab. A
> model where we have centralized performance test and distributed CI might
> work well.



+1 to the above approach , yet I still see value in publishing both types
of performance results as long as they are clearly separated.
This might might need a way to retroactively mark some results as "proved
invalid" but otoh encourage a cycle of propagating distributed tests proved
beneficial correct and unbiased to the central tests.

/Arnon