DPDK CI discussions
 help / color / Atom feed
* [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th
@ 2019-01-29 18:58 O'Driscoll, Tim
  2019-01-29 21:06 ` Jeremy Plsek
  0 siblings, 1 reply; 3+ messages in thread
From: O'Driscoll, Tim @ 2019-01-29 18:58 UTC (permalink / raw)
  To: ci

Red Hat Test Cases:
- We had a separate meeting with Red Hat after our lab meeting today. Aaron and Rashid explained that they have a suite of DPDK tests, some of which use OVS and some of which test DPDK is isolation. These can be easily set up in the DPDK lab.
- We agreed to proceed with this. It's a great way to get additional test coverage which we've been struggling with for a while. Aaron will work with Jeremy on the implementation.
- The initial goal is to get tests running and results into the database. After that, we can decide how to represent the results in the dashboard and patchwork.
- We will need to make sure that running these tests has not impact on the current test cases. The assumption is that current utilization is low enough that this should not be a problem.

DTS Improvements:
- Thomas will solicit community feedback on this. Community help will also be required to implement any changes.

Applying Patches to Sub-Trees:
- Plan is to apply patch sets to master first. If that fails, then we'll use a script to determine which sub-tree to apply the patch set to. Thomas and Ali hope to provide a script this week.
- If a patch set is applied to a sub-tree, then we need to agree whether the baseline for performance comparison should be the master or the sub-tree. Agreed that we should compare to master.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th
  2019-01-29 18:58 [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th O'Driscoll, Tim
@ 2019-01-29 21:06 ` Jeremy Plsek
  2019-01-30  6:43   ` Tu, Lijuan
  0 siblings, 1 reply; 3+ messages in thread
From: Jeremy Plsek @ 2019-01-29 21:06 UTC (permalink / raw)
  To: ci

[-- Attachment #1: Type: text/plain, Size: 2983 bytes --]

It was also discussed in this meeting, after Tim had to leave for another
meeting, that we should be saving absolute results instead of delta results.

In past discussions, it was agreed to only save the delta result to the
UNH-IOL database that drives the dashboard, with the baseline (absolute
result) kept on the respective system (without keeping old absolute
baselines).

Additionally, keeping absolute data in the database could be used to
produce accurate charts of performance over time and better show trends.
However, that will require some agreements to be reached for what data
could be made public, compared to what is available to logged in users
(i.e. seeing their own absolute performance numbers).  At this time, the
only change would be first storing the numbers into the database, with only
UNH-IOL having access to the data. Nothing else with the systems would
change with this first step.

The public view will still remain showing a delta for patches, until we
decide later on, on whether to show these absolute numbers in private or
also in public views.

If there is any issues with going forward with this decision, I'd like this
thread to be a public discussion for said issues. Ideally, we would like to
reach a definite go / no-go on this change for the next team meeting on
February 12. No reply or participation in this thread will be considered as
accepting the proposal to store the absolute numbers in the database.

Thanks.

On Tue, Jan 29, 2019 at 1:58 PM O'Driscoll, Tim <tim.odriscoll@intel.com>
wrote:

> Red Hat Test Cases:
> - We had a separate meeting with Red Hat after our lab meeting today.
> Aaron and Rashid explained that they have a suite of DPDK tests, some of
> which use OVS and some of which test DPDK is isolation. These can be easily
> set up in the DPDK lab.
> - We agreed to proceed with this. It's a great way to get additional test
> coverage which we've been struggling with for a while. Aaron will work with
> Jeremy on the implementation.
> - The initial goal is to get tests running and results into the database.
> After that, we can decide how to represent the results in the dashboard and
> patchwork.
> - We will need to make sure that running these tests has not impact on the
> current test cases. The assumption is that current utilization is low
> enough that this should not be a problem.
>
> DTS Improvements:
> - Thomas will solicit community feedback on this. Community help will also
> be required to implement any changes.
>
> Applying Patches to Sub-Trees:
> - Plan is to apply patch sets to master first. If that fails, then we'll
> use a script to determine which sub-tree to apply the patch set to. Thomas
> and Ali hope to provide a script this week.
> - If a patch set is applied to a sub-tree, then we need to agree whether
> the baseline for performance comparison should be the master or the
> sub-tree. Agreed that we should compare to master.
>


-- 
Jeremy Plsek
UNH InterOperability Laboratory

[-- Attachment #2: Type: text/html, Size: 3680 bytes --]

<div dir="ltr"><div dir="ltr">It was also discussed in this meeting, after Tim had to leave for another meeting, that we should be saving absolute results instead of delta results.<br><br>In past discussions, it was agreed to only save the delta result to the UNH-IOL database that drives the dashboard, with the baseline (absolute result) kept on the respective system (without keeping old absolute baselines).</div><div dir="ltr"><br>Additionally, keeping absolute data in the database could be used to produce accurate charts of performance over time and better show trends. However, that will require some agreements to be reached for what data could be made public, compared to what is available to logged in users (i.e. seeing their own absolute performance numbers).  At this time, the only change would be first storing the numbers into the database, with only UNH-IOL having access to the data. Nothing else with the systems would change with this first step.<br><br>The public view will still remain showing a delta for patches, until we decide later on, on whether to show these absolute numbers in private or also in public views.<br><br>If there is any issues with going forward with this decision, I&#39;d like this thread to be a public discussion for said issues. Ideally, we would like to reach a definite go / no-go on this change for the next team meeting on February 12. No reply or participation in this thread will be considered as accepting the proposal to store the absolute numbers in the database.<br><br>Thanks.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail-m_1219576548744510424gmail_attr">On Tue, Jan 29, 2019 at 1:58 PM O&#39;Driscoll, Tim &lt;<a href="mailto:tim.odriscoll@intel.com" target="_blank">tim.odriscoll@intel.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Red Hat Test Cases:<br>
- We had a separate meeting with Red Hat after our lab meeting today. Aaron and Rashid explained that they have a suite of DPDK tests, some of which use OVS and some of which test DPDK is isolation. These can be easily set up in the DPDK lab.<br>
- We agreed to proceed with this. It&#39;s a great way to get additional test coverage which we&#39;ve been struggling with for a while. Aaron will work with Jeremy on the implementation.<br>
- The initial goal is to get tests running and results into the database. After that, we can decide how to represent the results in the dashboard and patchwork.<br>
- We will need to make sure that running these tests has not impact on the current test cases. The assumption is that current utilization is low enough that this should not be a problem.<br>
<br>
DTS Improvements:<br>
- Thomas will solicit community feedback on this. Community help will also be required to implement any changes.<br>
<br>
Applying Patches to Sub-Trees:<br>
- Plan is to apply patch sets to master first. If that fails, then we&#39;ll use a script to determine which sub-tree to apply the patch set to. Thomas and Ali hope to provide a script this week.<br>
- If a patch set is applied to a sub-tree, then we need to agree whether the baseline for performance comparison should be the master or the sub-tree. Agreed that we should compare to master.<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail-m_1219576548744510424gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div></div>Jeremy Plsek<br></div><div>UNH InterOperability Laboratory<br></div></div></div></div></div></div></div></div></div></div>

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th
  2019-01-29 21:06 ` Jeremy Plsek
@ 2019-01-30  6:43   ` Tu, Lijuan
  0 siblings, 0 replies; 3+ messages in thread
From: Tu, Lijuan @ 2019-01-30  6:43 UTC (permalink / raw)
  To: Jeremy Plsek, ci

[-- Attachment #1: Type: text/plain, Size: 3708 bytes --]

As we got an agreement on removing the absolute numbers in the dts, I have submit a patch for this, and you will not see any absolute performance numbers in the dts, but instead of this, you only see 0.00. Every single user of DTS could keep their numbers in private if they’d like to.

There is no action required for UNH-IOL for this change, because the input and output don’t change, and it’s still easy to get the baseline through the argument “—update-expected”, and UNO has done this to get baseline.

From: ci [mailto:ci-bounces@dpdk.org] On Behalf Of Jeremy Plsek
Sent: Wednesday, January 30, 2019 5:07 AM
To: ci@dpdk.org
Subject: Re: [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th

It was also discussed in this meeting, after Tim had to leave for another meeting, that we should be saving absolute results instead of delta results.

In past discussions, it was agreed to only save the delta result to the UNH-IOL database that drives the dashboard, with the baseline (absolute result) kept on the respective system (without keeping old absolute baselines).

Additionally, keeping absolute data in the database could be used to produce accurate charts of performance over time and better show trends. However, that will require some agreements to be reached for what data could be made public, compared to what is available to logged in users (i.e. seeing their own absolute performance numbers).  At this time, the only change would be first storing the numbers into the database, with only UNH-IOL having access to the data. Nothing else with the systems would change with this first step.

The public view will still remain showing a delta for patches, until we decide later on, on whether to show these absolute numbers in private or also in public views.

If there is any issues with going forward with this decision, I'd like this thread to be a public discussion for said issues. Ideally, we would like to reach a definite go / no-go on this change for the next team meeting on February 12. No reply or participation in this thread will be considered as accepting the proposal to store the absolute numbers in the database.

Thanks.

On Tue, Jan 29, 2019 at 1:58 PM O'Driscoll, Tim <tim.odriscoll@intel.com<mailto:tim.odriscoll@intel.com>> wrote:
Red Hat Test Cases:
- We had a separate meeting with Red Hat after our lab meeting today. Aaron and Rashid explained that they have a suite of DPDK tests, some of which use OVS and some of which test DPDK is isolation. These can be easily set up in the DPDK lab.
- We agreed to proceed with this. It's a great way to get additional test coverage which we've been struggling with for a while. Aaron will work with Jeremy on the implementation.
- The initial goal is to get tests running and results into the database. After that, we can decide how to represent the results in the dashboard and patchwork.
- We will need to make sure that running these tests has not impact on the current test cases. The assumption is that current utilization is low enough that this should not be a problem.

DTS Improvements:
- Thomas will solicit community feedback on this. Community help will also be required to implement any changes.

Applying Patches to Sub-Trees:
- Plan is to apply patch sets to master first. If that fails, then we'll use a script to determine which sub-tree to apply the patch set to. Thomas and Ali hope to provide a script this week.
- If a patch set is applied to a sub-tree, then we need to agree whether the baseline for performance comparison should be the master or the sub-tree. Agreed that we should compare to master.


--
Jeremy Plsek
UNH InterOperability Laboratory

[-- Attachment #2: Type: text/html, Size: 7081 bytes --]

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:SimSun;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@SimSun";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0cm;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-family:"Calibri",sans-serif;}
@page WordSection1
	{size:612.0pt 792.0pt;
	margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">As we got an agreement on removing the absolute numbers in the dts, I have submit a patch for this, and you will not see any absolute performance numbers in the dts, but instead of this, you only see 0.00. Every single user of DTS could
 keep their numbers in private if they’d like to.<o:p></o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">There is no action required for UNH-IOL for this change, because the input and output don’t change, and it’s still easy to get the baseline through the argument “—update-expected”, and UNO has done this to get baseline.<o:p></o:p></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif;color:#1F497D"><o:p>&nbsp;</o:p></span></a></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> ci [mailto:ci-bounces@dpdk.org]
<b>On Behalf Of </b>Jeremy Plsek<br>
<b>Sent:</b> Wednesday, January 30, 2019 5:07 AM<br>
<b>To:</b> ci@dpdk.org<br>
<b>Subject:</b> Re: [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<div>
<p class="MsoNormal">It was also discussed in this meeting, after Tim had to leave for another meeting, that we should be saving absolute results instead of delta results.<br>
<br>
In past discussions, it was agreed to only save the delta result to the UNH-IOL database that drives the dashboard, with the baseline (absolute result) kept on the respective system (without keeping old absolute baselines).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
Additionally, keeping absolute data in the database could be used to produce accurate charts of performance over time and better show trends. However, that will require some agreements to be reached for what data could be made public, compared to what is available
 to logged in users (i.e. seeing their own absolute performance numbers).&nbsp; At this time, the only change would be first storing the numbers into the database, with only UNH-IOL having access to the data. Nothing else with the systems would change with this
 first step.<br>
<br>
The public view will still remain showing a delta for patches, until we decide later on, on whether to show these absolute numbers in private or also in public views.<br>
<br>
If there is any issues with going forward with this decision, I'd like this thread to be a public discussion for said issues. Ideally, we would like to reach a definite go / no-go on this change for the next team meeting on February 12. No reply or participation
 in this thread will be considered as accepting the proposal to store the absolute numbers in the database.<br>
<br>
Thanks.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Jan 29, 2019 at 1:58 PM O'Driscoll, Tim &lt;<a href="mailto:tim.odriscoll@intel.com" target="_blank">tim.odriscoll@intel.com</a>&gt; wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Red Hat Test Cases:<br>
- We had a separate meeting with Red Hat after our lab meeting today. Aaron and Rashid explained that they have a suite of DPDK tests, some of which use OVS and some of which test DPDK is isolation. These can be easily set up in the DPDK lab.<br>
- We agreed to proceed with this. It's a great way to get additional test coverage which we've been struggling with for a while. Aaron will work with Jeremy on the implementation.<br>
- The initial goal is to get tests running and results into the database. After that, we can decide how to represent the results in the dashboard and patchwork.<br>
- We will need to make sure that running these tests has not impact on the current test cases. The assumption is that current utilization is low enough that this should not be a problem.<br>
<br>
DTS Improvements:<br>
- Thomas will solicit community feedback on this. Community help will also be required to implement any changes.<br>
<br>
Applying Patches to Sub-Trees:<br>
- Plan is to apply patch sets to master first. If that fails, then we'll use a script to determine which sub-tree to apply the patch set to. Thomas and Ali hope to provide a script this week.<br>
- If a patch set is applied to a sub-tree, then we need to agree whether the baseline for performance comparison should be the master or the sub-tree. Agreed that we should compare to master.<o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jeremy Plsek<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">UNH InterOperability Laboratory<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-29 18:58 [dpdk-ci] Minutes of DPDK Lab Meeting, January 29th O'Driscoll, Tim
2019-01-29 21:06 ` Jeremy Plsek
2019-01-30  6:43   ` Tu, Lijuan

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox