On Mon, Nov 7, 2016 at 12:34 PM, O'Driscoll, Tim <tim.odriscoll@intel.com> wrote:

> -----Original Message-----
> From: moving [mailto:moving-bounces@dpdk.org] On Behalf Of Jerome Tollet
> (jtollet)
> Sent: Monday, November 7, 2016 10:27 AM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>; Xu, Qian Q
> <qian.q.xu@intel.com>
> Cc: moving@dpdk.org; Liu, Yong <yong.liu@intel.com>; ci@dpdk.org
> Subject: Re: [dpdk-moving] proposal for DPDK CI improvement
>
> Hi Thomas & Qian,
> IMHO, performance results should be centralized and executed in a
> trusted & controlled environment.
> If official DPDK numbers are coming from private lab’s vendors,
> perception might be that they are not 100% neutral. That would probably
> not help DPDK community to be seen open & transparent.

+1

Somebody (Jan Blunck I think) also said on last week's call that performance testing was a higher priority than CI for a centralized lab. A model where we have centralized performance test and distributed CI might work well.

+1 to the above approach , yet I still see value in publishing both types of performance results as long as they are clearly separated.

This might might need a way to retroactively mark some results as "proved invalid" but otoh encourage a cycle of propagating distributed tests proved beneficial correct and unbiased to the central tests.

/Arnon