[dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[O'Driscoll, Tim]

Dashboard Status:
- Jeremy is working on setting up the GA performance testing (compare patch performance versus previous GA release as well as with master), and other dashboard features.
- There are fewer apply errors now. This is mostly because master is more up to date now because of the stage 18.11 is at. We need to monitor this to see if issues occur again during the 19.02 release cycle. The Maintainers have been aiming to have sub-trees merged every week, but that's never happened. Jeremy has also made an update to fix the exclusion of the docs directory, so we're no longer seeing apply errors because of documentation issues.

Public Results:
- Mellanox test configuration details have been provided to UNH and linked from the results page.
- Mellanox legal disclaimers have been provided to UNH and added to the test results page.
- Agreed that we're all happy to make the relative performance results public. Jeremy will make this change and notify ci@dpdk.org when this is complete.

Patchwork Integration:
- To make the results more visible, and to ensure that developers and maintainers act on them, we agreed that results should be integrated into Patchwork, similar to the way that build tests are handled. Jeremy will review the scripts in the ci repository to determine required email format etc. Ali can help with any issues.

DPDK Summit:
- Jeremy's talk was accepted. This will cover the latest dashboard status (including public relative performance results), and performance measurements over time. Abstract for the presentation is:
"The DPDK Open Lab is a performance based continuous integration system, supported by the DPDK project. When a patch is submitted, it is automatically sent to our CI to be applied and built. Once the patch is compiled and installed, it is ran against each of the bare metal environments hosted in the lab. This is to check for performance degradations or speed ups within DPDK on various hardware platforms. This talk will explore how the this system supports the development community, such as accepting patches based on performance and tracking how performance has changed in DPDK over time. We will go over how to navigate and use the Dashboard. We will show how the performance has changed in DPDK over the past six months, looking at relative numbers and graphs of various platforms. Finally, we will also talk about the future of the Open Lab, such as running more test cases, running unit tests for DPDK, additional capabilities for the dashboard, and making the systems more accessible to the development community."

Re: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[Thomas Monjalon]

06/11/2018 16:19, O'Driscoll, Tim:
> Patchwork Integration:
> - To make the results more visible, and to ensure that developers and maintainers act on them, we agreed that results should be integrated into Patchwork, similar to the way that build tests are handled. Jeremy will review the scripts in the ci repository to determine required email format etc. Ali can help with any issues.

The script to use is:
	http://git.dpdk.org/tools/dpdk-ci/tree/tools/send-patch-report.sh
The sender must be whitelisted in order to appear in patchwork.

Re: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[Tu, Lijuan]

Glad to see that the performance results public. 
Can we have a portal for dashboard in dpdk.org? Currently we must know the accurate address and we can get to the dashboard: https://lab.dpdk.org/results/dashboard/
It's helpful if there is an icon in http://core.dpdk.org/perf-lab/, and we can get to the dashboard just clicking the icon, it's more convenience.

> -----Original Message-----
> From: ci [mailto:ci-bounces@dpdk.org] On Behalf Of O'Driscoll, Tim
> Sent: Tuesday, November 6, 2018 11:19 PM
> To: 'ci@dpdk.org' <ci@dpdk.org>
> Subject: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th
> 
> Dashboard Status:
> - Jeremy is working on setting up the GA performance testing (compare
> patch performance versus previous GA release as well as with master), and
> other dashboard features.
> - There are fewer apply errors now. This is mostly because master is more up
> to date now because of the stage 18.11 is at. We need to monitor this to see
> if issues occur again during the 19.02 release cycle. The Maintainers have
> been aiming to have sub-trees merged every week, but that's never
> happened. Jeremy has also made an update to fix the exclusion of the docs
> directory, so we're no longer seeing apply errors because of documentation
> issues.
> 
> Public Results:
> - Mellanox test configuration details have been provided to UNH and linked
> from the results page.
> - Mellanox legal disclaimers have been provided to UNH and added to the
> test results page.
> - Agreed that we're all happy to make the relative performance results public.
> Jeremy will make this change and notify ci@dpdk.org when this is complete.
> 
> Patchwork Integration:
> - To make the results more visible, and to ensure that developers and
> maintainers act on them, we agreed that results should be integrated into
> Patchwork, similar to the way that build tests are handled. Jeremy will
> review the scripts in the ci repository to determine required email format etc.
> Ali can help with any issues.
> 
> DPDK Summit:
> - Jeremy's talk was accepted. This will cover the latest dashboard status
> (including public relative performance results), and performance
> measurements over time. Abstract for the presentation is:
> "The DPDK Open Lab is a performance based continuous integration system,
> supported by the DPDK project. When a patch is submitted, it is
> automatically sent to our CI to be applied and built. Once the patch is
> compiled and installed, it is ran against each of the bare metal environments
> hosted in the lab. This is to check for performance degradations or speed ups
> within DPDK on various hardware platforms. This talk will explore how the
> this system supports the development community, such as accepting patches
> based on performance and tracking how performance has changed in DPDK
> over time. We will go over how to navigate and use the Dashboard. We will
> show how the performance has changed in DPDK over the past six months,
> looking at relative numbers and graphs of various platforms. Finally, we will
> also talk about the future of the Open Lab, such as running more test cases,
> running unit tests for DPDK, additional capabilities for the dashboard, and
> making the systems more accessible to the development community."
> 
> 

Re: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[O'Driscoll, Tim]

Great idea. Since http://core.dpdk.org/perf-lab/ is part of the core site, it needs a patch to update it. Is there somebody who can do that?

> -----Original Message-----
> From: Tu, Lijuan
> Sent: Wednesday, November 7, 2018 5:29 AM
> To: O'Driscoll, Tim <tim.odriscoll@intel.com>; 'ci@dpdk.org'
> <ci@dpdk.org>
> Subject: RE: Minutes of DPDK Lab Meeting, November 6th
> 
> Glad to see that the performance results public.
> Can we have a portal for dashboard in dpdk.org? Currently we must know
> the accurate address and we can get to the dashboard:
> https://lab.dpdk.org/results/dashboard/
> It's helpful if there is an icon in http://core.dpdk.org/perf-lab/, and
> we can get to the dashboard just clicking the icon, it's more
> convenience.
> 
> > -----Original Message-----
> > From: ci [mailto:ci-bounces@dpdk.org] On Behalf Of O'Driscoll, Tim
> > Sent: Tuesday, November 6, 2018 11:19 PM
> > To: 'ci@dpdk.org' <ci@dpdk.org>
> > Subject: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th
> >
> > Dashboard Status:
> > - Jeremy is working on setting up the GA performance testing (compare
> > patch performance versus previous GA release as well as with master),
> and
> > other dashboard features.
> > - There are fewer apply errors now. This is mostly because master is
> more up
> > to date now because of the stage 18.11 is at. We need to monitor this
> to see
> > if issues occur again during the 19.02 release cycle. The Maintainers
> have
> > been aiming to have sub-trees merged every week, but that's never
> > happened. Jeremy has also made an update to fix the exclusion of the
> docs
> > directory, so we're no longer seeing apply errors because of
> documentation
> > issues.
> >
> > Public Results:
> > - Mellanox test configuration details have been provided to UNH and
> linked
> > from the results page.
> > - Mellanox legal disclaimers have been provided to UNH and added to
> the
> > test results page.
> > - Agreed that we're all happy to make the relative performance results
> public.
> > Jeremy will make this change and notify ci@dpdk.org when this is
> complete.
> >
> > Patchwork Integration:
> > - To make the results more visible, and to ensure that developers and
> > maintainers act on them, we agreed that results should be integrated
> into
> > Patchwork, similar to the way that build tests are handled. Jeremy
> will
> > review the scripts in the ci repository to determine required email
> format etc.
> > Ali can help with any issues.
> >
> > DPDK Summit:
> > - Jeremy's talk was accepted. This will cover the latest dashboard
> status
> > (including public relative performance results), and performance
> > measurements over time. Abstract for the presentation is:
> > "The DPDK Open Lab is a performance based continuous integration
> system,
> > supported by the DPDK project. When a patch is submitted, it is
> > automatically sent to our CI to be applied and built. Once the patch
> is
> > compiled and installed, it is ran against each of the bare metal
> environments
> > hosted in the lab. This is to check for performance degradations or
> speed ups
> > within DPDK on various hardware platforms. This talk will explore how
> the
> > this system supports the development community, such as accepting
> patches
> > based on performance and tracking how performance has changed in DPDK
> > over time. We will go over how to navigate and use the Dashboard. We
> will
> > show how the performance has changed in DPDK over the past six months,
> > looking at relative numbers and graphs of various platforms. Finally,
> we will
> > also talk about the future of the Open Lab, such as running more test
> cases,
> > running unit tests for DPDK, additional capabilities for the
> dashboard, and
> > making the systems more accessible to the development community."
> >
> >

Re: [dpdk-ci] Minutes of DPDK Lab Meeting, November 6th

[Ferruh Yigit]

On 11/6/2018 3:19 PM, O'Driscoll, Tim wrote:
> Patchwork Integration:
> - To make the results more visible, and to ensure that developers and maintainers act on them, we agreed that results should be integrated into Patchwork, similar to the way that build tests are handled. Jeremy will review the scripts in the ci repository to determine required email format etc. Ali can help with any issues.

We start getting performance test reports to test-report@dpdk.org mail list,
this is good.