DPDK CI discussions
 help / color / Atom feed
* [dpdk-ci] DPDK lab meeting agenda items 2018-07-17
@ 2018-07-16 17:48 Patrick MacArthur
  0 siblings, 0 replies; only message in thread
From: Patrick MacArthur @ 2018-07-16 17:48 UTC (permalink / raw)
  To: ci; +Cc: dpdklab

[-- Attachment #1: Type: text/plain, Size: 5563 bytes --]

Hi, all,

I would like to discuss the following issues:

* Discuss the policies and procedures document that was sent out as part of
the minutes from the last meeting.
* How will the feature request process work? I would like to propose that
we create a new project in bugs.dpdk.org and use that as the public-facing
portal to track bugs and feature requests at a high level. UNH-IOL will
decide in which order to work on feature requests and bug fixes, with input
from the group. UNH-IOL may make improvements to the dashboard over time as
we see fit, but the group must approve any changes that result in more
vendor-specific information being made publicly available through the
dashboard before it is made live.
* What will the process be for making the relative numbers for each vendor
public? Here are the scenarios that I am envisioning:
  - A vendor may choose to make results for a device all public or all
private.
  - A vendor may choose to make results public starting at date X.
  - A vendor may choose to make future results private leaving current
results public (to support experimental changes on a platform).
  - Anything missing from this list?
* Any immediate thoughts on the proposed functionality below that require
group discussion, or any other feature requests? You can also e-mail
dpdklab@iol.unh.edu with any feedback.


Proposed functionality for the dashboard:

* (from DPDK 2018-06-19 meeting) Distinguish between error with the patch
and patch just not applying to master.
* (from DPDK 2018-06-19 meeting) Show setup configuration, topology, and
hardware/CPU/motherboard environment information.
  - Requires approval from every Member with hardware in the lab.
* (external request) Show description of tests performed.
  - This should ideally link to an external site describing the test cases.
Is there somewhere where the DTS test_plans documentation is published that
we can link to?
* (external request) Show absolute values in the dashboard detail view if
they were provided by the vendor, and a user from that company is logged in.
  - Requires explicit approval from the individual Members for us to store
absolute/expected values in our database.
* Adding the functionality for results to be made public as discussed above.
* Ability for users to request accounts on the dashboard using a Web form.
  - Need to define approval process for this. This likely needs to wait on
a primary contact from the appropriate company. Do we allow login by users
who are not part of any company (it won't do anything for now, but maybe
there will be a reason in the future for patch submitters to have accounts)?
* Add the patch apply or build output text to the dashboard detail page if
the tarball was not created.
* Add a download link for the test logs, in the form of a ZIP file
containing the contents of the DTS output directory.
* Add the ability for a vendor to share the test log zip file. I propose
that this be done as follows, though we could go with a different scheme if
the group desires:
  - The log file URLs will be contain a unique, unpredictable >= 40
character random string so that any shared logs are unlikely to be leaked
to users who were not given the link.
  - Each URL will require authentication by default, but vendors will have
the option to share the link, which removes the authentication requirement
(maybe with an expiration time).
  - Vendors will also have the option to revoke access, thus again
requiring authentication as an employee of that company to access the log.
* A Platforms tab which lists vendor systems along with a button to request
downtime for each system. During a downtime window, the Jenkins agent on
that node is suspended and no jobs will run until the end of the downtime
window. This makes it easier for vendors to make updates to their systems
without interfering with live test runs.
* Show entire history of test runs on the detail page if a test was re-run.


Functionality added/bugs fixed in the past 2 weeks (for informational
purposes; we don't need to spend time during the meeting discussing these
unless anyone has strong feedback):

* A preferences page was added allowing individual users to turn on or off
e-mail notifications of test results for their organization's devices.
* A password change form was added to the preferences page.
* Tooltips were added to the front page with definitions of each status, as
well as a legend that appears if you click the ? next to the Status column.
* Error pages were added for the common error pages to preserve the site
navigation should they occur. This also includes graceful notification if
the dashboard becomes unavailable due to a power outage, etc.
* Timestamps are shown for each patchset and for each test run.
* Pagination was added to the dashboard front page; the dashboard will show
the most recent 50 patchsets that are currently active in Patchwork on the
first page.
* Patch submitter e-mail addresses are no longer shown to anonymous users;
they will be shown if a user is logged in.
* Official NIC product names (if known to us) are shown in the dashboard
instead of the codenames used by DTS.
* Result on front page shows as "Incomplete" if not all devices have
completed testing on a specific patch.
* Newly added devices will no longer affect the overall result show on the
front page of the dashboard until the vendor is satisfied that the results
for that device are stable.

-- 
Patrick MacArthur
Research and Development, High Performance Networking and Storage
UNH InterOperability Laboratory

[-- Attachment #2: Type: text/html, Size: 10667 bytes --]

<div dir="ltr"><span style="font-size:small;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Hi, all,</span><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">I would like to discuss the following issues:</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Discuss the policies and procedures document that was sent out as part of the minutes from the last meeting.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* How will the feature request process work? I would like to propose that we create a new project in<span> bugs.</span><a href="http://dpdk.org">dpdk.org</a> and use that as the public-facing portal to track bugs and feature requests at a high level. UNH-IOL will decide in which order to work on feature requests and bug fixes, with input from the group. UNH-IOL may make improvements to the dashboard over time as we see fit, but the group must approve any changes that result in more vendor-specific information being made publicly available through the dashboard before it is made live.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* What will the process be for making the relative numbers for each vendor public? Here are the scenarios that I am envisioning:</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">  - A vendor may choose to make results for a device all public or all private.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">  - A vendor may choose to make results public starting at date X.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">  - A vendor may choose to make future results private leaving current results public (to support experimental changes on a platform).</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">  - Anything missing from this list?</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Any immediate thoughts on the proposed functionality below that require group discussion, or any other feature requests? You can also e-mail <a href="mailto:dpdklab@iol.unh.edu">dpdklab@iol.unh.edu</a> with any feedback.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><div style="text-decoration-style:initial;text-decoration-color:initial">Proposed functionality for the dashboard:</div><div style="text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="text-decoration-style:initial;text-decoration-color:initial">* (from DPDK 2018-06-19 meeting) Distinguish between error with the patch and patch just not applying to master. </div><div style="text-decoration-style:initial;text-decoration-color:initial">* (from DPDK 2018-06-19 meeting) Show setup configuration, topology, and hardware/CPU/motherboard environment information.</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - Requires approval from every Member with hardware in the lab.</div><div style="text-decoration-style:initial;text-decoration-color:initial">* (external request) Show description of tests performed.</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - This should ideally link to an external site describing the test cases. Is there somewhere where the DTS test_plans documentation is published that we can link to?</div><div style="text-decoration-style:initial;text-decoration-color:initial">* (external request) Show absolute values in the dashboard detail view if they were provided by the vendor, and a user from that company is logged in.</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - Requires explicit approval from the individual Members for us to store absolute/expected values in our database.<br></div><div style="text-decoration-style:initial;text-decoration-color:initial">* Adding the functionality for results to be made public as discussed above.</div><div style="text-decoration-style:initial;text-decoration-color:initial">* Ability for users to request accounts on the dashboard using a Web form.</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - Need to define approval process for this. This likely needs to wait on a primary contact from the appropriate company. Do we allow login by users who are not part of any company (it won&#39;t do anything for now, but maybe there will be a reason in the future for patch submitters to have accounts)?</div><div style="text-decoration-style:initial;text-decoration-color:initial">* Add the patch apply or build output text to the dashboard detail page if the tarball was not created.<br></div><div style="text-decoration-style:initial;text-decoration-color:initial">* Add a download link for the test logs, in the form of a ZIP file containing the contents of the DTS output directory.<br></div><div style="text-decoration-style:initial;text-decoration-color:initial">* Add the ability for a vendor to share the test log zip file. I propose that this be done as follows, though we could go with a different scheme if the group desires:<br></div><div style="text-decoration-style:initial;text-decoration-color:initial">  - The log file URLs will be contain a unique, unpredictable &gt;= 40 character random string so that any shared logs are unlikely to be leaked to users who were not given the link.</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - Each URL will require authentication by default, but vendors will have the option to share the link, which removes the authentication requirement (maybe with an expiration time).</div><div style="text-decoration-style:initial;text-decoration-color:initial">  - Vendors will also have the option to revoke access, thus again requiring authentication as an employee of that company to access the log.</div><div style="text-decoration-style:initial;text-decoration-color:initial"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">* A Platforms tab which lists vendor systems along with a button to request downtime for each system. During a downtime window, the Jenkins agent on that node is suspended and no jobs will run until the end of the downtime window. This makes it easier for vendors to make updates to their systems without interfering with live test runs.</span><br></div><div style="text-decoration-style:initial;text-decoration-color:initial"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="font-size:small;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">* Show entire history of test runs on the detail page if a test was re-run.</span></span></div></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">Functionality added/bugs fixed in the past 2 weeks (for informational purposes; we don&#39;t need to spend time during the meeting discussing these unless anyone has strong feedback):</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* A preferences page was added allowing individual users to turn on or off e-mail notifications of test results for their organization&#39;s devices.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* A password change form was added to the preferences page.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Tooltips were added to the front page with definitions of each status, as well as a legend that appears if you click the ? next to the Status column.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Error pages were added for the common error pages to preserve the site navigation should they occur. This also includes graceful notification if the dashboard becomes unavailable due to a power outage, etc.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Timestamps are shown for each patchset and for each test run.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Pagination was added to the dashboard front page; the dashboard will show the most recent 50 patchsets that are currently active in Patchwork on the first page.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Patch submitter e-mail addresses are no longer shown to anonymous users; they will be shown if a user is logged in.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Official NIC product names (if known to us) are shown in the dashboard instead of the codenames used by DTS.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Result on front page shows as &quot;Incomplete&quot; if not all devices have completed testing on a specific patch.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">* Newly added devices will no longer affect the overall result show on the front page of the dashboard until the vendor is satisfied that the results for that device are stable.</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Patrick MacArthur<br>Research and Development, High Performance Networking and Storage<br>UNH InterOperability Laboratory</div></div>

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, back to index

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-16 17:48 [dpdk-ci] DPDK lab meeting agenda items 2018-07-17 Patrick MacArthur

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox