From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f45.google.com (mail-oi0-f45.google.com [209.85.218.45]) by dpdk.org (Postfix) with ESMTP id 13110200 for ; Mon, 16 Jul 2018 19:48:45 +0200 (CEST) Received: by mail-oi0-f45.google.com with SMTP id k12-v6so76390954oiw.8 for ; Mon, 16 Jul 2018 10:48:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:from:date:message-id:subject:to:cc; bh=kVMSO33xcP6IguLfWNjmVIfiP75cj48QBwOPvjUDozk=; b=Holw+lasY7pdvrr8BENgiaYQ6oCVbL/HB4jQZmOA56I3vbGqhErpMO16R5kPUCgzEa SKfFmbENz6TUjVg163RHV1gRKI8b+gNnH+FsE9mGDGPFjz6wgEDhpr0skPxnIcyovnvx wITfIWkcuRzEaprlsNUI3kNF/xx3kYamaqyg0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=kVMSO33xcP6IguLfWNjmVIfiP75cj48QBwOPvjUDozk=; b=dpbmTnOHtgWsdfQ04hGXl/BL6N0IG5/Xl98XoZoRmsiIrDwMyMRBww8YYwE1hYGrlE jnqkXnECzX5z8JWji58MlGr5Pnz5dUuZXOViCbD7qjCPEM4MAecUWxPpGdQmxh3dIQr8 sw0XhEUEXWre6G4vCeYlloWbNqjUHeWSK7t8MbNHAqVTCjg9MmX4/r3mGfO42Yk9f0Mp NJ7g3aK0UM/cHMqbTJfOIqHrS3QyZDBesfkSEcz8hK6RwcPTzQA2xCGnFEPPTyR8el0A toqNWtilPxUMOt9yRO9cfUbe82lWEV+pBPnXFoUuk3ub3kr+uleg2Gx2VgTEDWUlBpC0 5WVg== X-Gm-Message-State: AOUpUlHq4kE2Nc+0GKtS3KCU8XhbsNLDZx9AWSkp/fyZY4mfY/qWlD0b wrxV+ob5nZ5g5hThg/2Nk6p2aGBs5xntsODBikPrrZwI X-Google-Smtp-Source: AAOMgpd4FoP1AHdmEBi4BaJYnQCYPJ6/wfev1Lkd3jooW7s+qatbYZPSIzVFJ3CBjWd5KBzZw6uP2aHdrFhkgc7CV40= X-Received: by 2002:aca:dc55:: with SMTP id t82-v6mr316221oig.159.1531763324988; Mon, 16 Jul 2018 10:48:44 -0700 (PDT) MIME-Version: 1.0 From: Patrick MacArthur Date: Mon, 16 Jul 2018 13:48:33 -0400 Message-ID: To: ci@dpdk.org Cc: dpdklab@iol.unh.edu Content-Type: multipart/alternative; boundary="000000000000ddc9a8057121702e" Subject: [dpdk-ci] DPDK lab meeting agenda items 2018-07-17 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2018 17:48:46 -0000 --000000000000ddc9a8057121702e Content-Type: text/plain; charset="UTF-8" Hi, all, I would like to discuss the following issues: * Discuss the policies and procedures document that was sent out as part of the minutes from the last meeting. * How will the feature request process work? I would like to propose that we create a new project in bugs.dpdk.org and use that as the public-facing portal to track bugs and feature requests at a high level. UNH-IOL will decide in which order to work on feature requests and bug fixes, with input from the group. UNH-IOL may make improvements to the dashboard over time as we see fit, but the group must approve any changes that result in more vendor-specific information being made publicly available through the dashboard before it is made live. * What will the process be for making the relative numbers for each vendor public? Here are the scenarios that I am envisioning: - A vendor may choose to make results for a device all public or all private. - A vendor may choose to make results public starting at date X. - A vendor may choose to make future results private leaving current results public (to support experimental changes on a platform). - Anything missing from this list? * Any immediate thoughts on the proposed functionality below that require group discussion, or any other feature requests? You can also e-mail dpdklab@iol.unh.edu with any feedback. Proposed functionality for the dashboard: * (from DPDK 2018-06-19 meeting) Distinguish between error with the patch and patch just not applying to master. * (from DPDK 2018-06-19 meeting) Show setup configuration, topology, and hardware/CPU/motherboard environment information. - Requires approval from every Member with hardware in the lab. * (external request) Show description of tests performed. - This should ideally link to an external site describing the test cases. Is there somewhere where the DTS test_plans documentation is published that we can link to? * (external request) Show absolute values in the dashboard detail view if they were provided by the vendor, and a user from that company is logged in. - Requires explicit approval from the individual Members for us to store absolute/expected values in our database. * Adding the functionality for results to be made public as discussed above. * Ability for users to request accounts on the dashboard using a Web form. - Need to define approval process for this. This likely needs to wait on a primary contact from the appropriate company. Do we allow login by users who are not part of any company (it won't do anything for now, but maybe there will be a reason in the future for patch submitters to have accounts)? * Add the patch apply or build output text to the dashboard detail page if the tarball was not created. * Add a download link for the test logs, in the form of a ZIP file containing the contents of the DTS output directory. * Add the ability for a vendor to share the test log zip file. I propose that this be done as follows, though we could go with a different scheme if the group desires: - The log file URLs will be contain a unique, unpredictable >= 40 character random string so that any shared logs are unlikely to be leaked to users who were not given the link. - Each URL will require authentication by default, but vendors will have the option to share the link, which removes the authentication requirement (maybe with an expiration time). - Vendors will also have the option to revoke access, thus again requiring authentication as an employee of that company to access the log. * A Platforms tab which lists vendor systems along with a button to request downtime for each system. During a downtime window, the Jenkins agent on that node is suspended and no jobs will run until the end of the downtime window. This makes it easier for vendors to make updates to their systems without interfering with live test runs. * Show entire history of test runs on the detail page if a test was re-run. Functionality added/bugs fixed in the past 2 weeks (for informational purposes; we don't need to spend time during the meeting discussing these unless anyone has strong feedback): * A preferences page was added allowing individual users to turn on or off e-mail notifications of test results for their organization's devices. * A password change form was added to the preferences page. * Tooltips were added to the front page with definitions of each status, as well as a legend that appears if you click the ? next to the Status column. * Error pages were added for the common error pages to preserve the site navigation should they occur. This also includes graceful notification if the dashboard becomes unavailable due to a power outage, etc. * Timestamps are shown for each patchset and for each test run. * Pagination was added to the dashboard front page; the dashboard will show the most recent 50 patchsets that are currently active in Patchwork on the first page. * Patch submitter e-mail addresses are no longer shown to anonymous users; they will be shown if a user is logged in. * Official NIC product names (if known to us) are shown in the dashboard instead of the codenames used by DTS. * Result on front page shows as "Incomplete" if not all devices have completed testing on a specific patch. * Newly added devices will no longer affect the overall result show on the front page of the dashboard until the vendor is satisfied that the results for that device are stable. -- Patrick MacArthur Research and Development, High Performance Networking and Storage UNH InterOperability Laboratory --000000000000ddc9a8057121702e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi, all,=

I would like to discuss the follo= wing issues:

* Discuss the p= olicies and procedures document that was sent out as part of the minutes fr= om the last meeting.
* How will the feature request pr= ocess work? I would like to propose that we create a new project in= =C2=A0bugs.dpdk.org=C2=A0and use tha= t as the public-facing portal to track bugs and feature requests at a high = level. UNH-IOL will decide in which order to work on feature requests and b= ug fixes, with input from the group. UNH-IOL may make improvements to the d= ashboard over time as we see fit, but the group must approve any changes th= at result in more vendor-specific information being made publicly available= through the dashboard before it is made live.
* What = will the process be for making the relative numbers for each vendor public?= Here are the scenarios that I am envisioning:
=C2=A0 = - A vendor may choose to make results for a device all public or all privat= e.
=C2=A0 - A vendor may choose to make results public= starting at date X.
=C2=A0 - A vendor may choose to m= ake future results private leaving current results public (to support exper= imental changes on a platform).
=C2=A0 - Anything miss= ing from this list?
* Any immediate thoughts on the pr= oposed functionality below that require group discussion, or any other feat= ure requests? You can also e-mail dp= dklab@iol.unh.edu with any feedback.


Proposed functionality for the dashb= oard:

* (from DPDK 2018-06-19 meeting) Distinguish between e= rror with the patch and patch just not applying to master.=C2=A0
* (fr= om DPDK 2018-06-19 meeting) Show setup configuration, topology, and hardwar= e/CPU/motherboard environment information.
=C2=A0 - Requires approval = from every Member with hardware in the lab.
* (external request) Show = description of tests performed.
=C2=A0 - This should ideally link to a= n external site describing the test cases. Is there somewhere where the DTS= test_plans documentation is published that we can link to?
* (externa= l request) Show absolute values in the dashboard detail view if they were p= rovided by the vendor, and a user from that company is logged in.
=C2= =A0 - Requires explicit approval from the individual Members for us to stor= e absolute/expected values in our database.
* Adding the functiona= lity for results to be made public as discussed above.
* Ability for u= sers to request accounts on the dashboard using a Web form.
=C2=A0 - N= eed to define approval process for this. This likely needs to wait on a pri= mary contact from the appropriate company. Do we allow login by users who a= re not part of any company (it won't do anything for now, but maybe the= re will be a reason in the future for patch submitters to have accounts)?
* Add the patch apply or build output text to the dashboard detail page= if the tarball was not created.
* Add a download link for the tes= t logs, in the form of a ZIP file containing the contents of the DTS output= directory.
* Add the ability for a vendor to share the test log z= ip file. I propose that this be done as follows, though we could go with a = different scheme if the group desires:
=C2=A0 -=C2=A0The log file = URLs will be contain a unique, unpredictable >=3D 40 character random st= ring so that any shared logs are unlikely to be leaked to users who were no= t given the link.
=C2=A0 - Each URL will require authentication by def= ault, but vendors will have the option to share the link, which removes the= authentication requirement (maybe with an expiration time).
=C2=A0 - = Vendors will also have the option to revoke access, thus again requiring au= thentication as an employee of that company to access the log.
* A Platforms tab whi= ch lists vendor systems along with a button to request downtime for each sy= stem. During a downtime window, the Jenkins agent on that node is suspended= and no jobs will run until the end of the downtime window. This makes it e= asier for vendors to make updates to their systems without interfering with= live test runs.
* Show entire= history of test runs on the detail page if a test was re-run.


Functionality added/bugs fixed in the past 2 weeks (for informational= purposes; we don't need to spend time during the meeting discussing th= ese unless anyone has strong feedback):

* A preferences page was added allowing individual users to tur= n on or off e-mail notifications of test results for their organization'= ;s devices.
* A password change form was added to the = preferences page.
* Tooltips were added to the front p= age with definitions of each status, as well as a legend that appears if yo= u click the ? next to the Status column.
* Error pages= were added for the common error pages to preserve the site navigation shou= ld they occur. This also includes graceful notification if the dashboard be= comes unavailable due to a power outage, etc.
* Timest= amps are shown for each patchset and for each test run.
* Patch submitter e-mail addresses are = no longer shown to anonymous users; they will be shown if a user is logged = in.
* Official NIC product names (if known to us) are = shown in the dashboard instead of the codenames used by DTS.
* Result on front page shows as "Incomplete" if not all de= vices have completed testing on a specific patch.
* Ne= wly added devices will no longer affect the overall result show on the fron= t page of the dashboard until the vendor is satisfied that the results for = that device are stable.

--
Patrick Ma= cArthur
Research and Development, High Performance Networking and Storag= e
UNH InterOperability Laboratory
--000000000000ddc9a8057121702e--