From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E65AEA0A02 for ; Thu, 25 Mar 2021 15:36:09 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B828E140DA1; Thu, 25 Mar 2021 15:36:09 +0100 (CET) Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by mails.dpdk.org (Postfix) with ESMTP id 65A2E4067B for ; Thu, 25 Mar 2021 15:36:08 +0100 (CET) Received: by mail-ed1-f43.google.com with SMTP id bf3so2668476edb.6 for ; Thu, 25 Mar 2021 07:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:from:date:message-id:subject:to; bh=hCaUOqrFvUbmq2amnNFr3+sxCNk3Wmrz16T6cQJWgK0=; b=b+AJEdTmK++d9WWbTmevkZnsVmTOqn3vdlf2UyepCSHn05mD0O1wIN+NCOcYu0aXDs 2oOnHmZTn9tnzkdFbOWGAtrEf+Ziy7OPQlKBUYFVvOIIKWzszjkc6kEGcOzekMi7jpDz bxT5fQApGBN9WMNOpL/hM6mI2ryX2WLRNt9e4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hCaUOqrFvUbmq2amnNFr3+sxCNk3Wmrz16T6cQJWgK0=; b=Ck4AIX3zeYKmZr1s/7Tn7ymcI7QlsAIGOCzvVykEDejeR4DGuTDE/0CvICY8GfkBn2 8bVbzn9mxYNQVNYP1qrCQmGctGwgrcFbxeEcOa8JV18Nma7RnU4Lx7FV5TK/3JXpMays KJMzBT8B2QdKp8+mPu+l5JKuyr27a4qeGEt+FYyS9pzz1ZeMGFbGTUsm6o92V6qDVPI2 U0DhDFXxmHdozk5NFWCSvJ5scSeI3QMmTMtMzGeNSsykVCpQMeR/SfnjpmWhNSA3+NtJ u1bkLXnSjvGKH9JLQYZSh55z5dunmUIy4Klij6SOQjDvd5xU/88+Qe9KaybtOmPHy6PC Sbvw== X-Gm-Message-State: AOAM531I9JDhGpH5aQuqwJKRIg2tq9VjNTFZ/cUy1/UBQwRXYZjXd7PP QRoYtR6qHMZdLdDcY9tT6yzZcid+BBWTC2Ndo9eUQCCJvhifKA== X-Google-Smtp-Source: ABdhPJymLEKZh6doc5jBmtmIXclzXKcxMTKpNrB2+LAk9iUjG3SLF0gMuG1wDW/1v1vKR9OW41bX2WCoAnPEa5C2d8M= X-Received: by 2002:a50:fa92:: with SMTP id w18mr9388398edr.172.1616682967561; Thu, 25 Mar 2021 07:36:07 -0700 (PDT) MIME-Version: 1.0 From: Lincoln Lavoie Date: Thu, 25 Mar 2021 10:33:11 -0400 Message-ID: To: ci@dpdk.org Content-Type: multipart/alternative; boundary="000000000000ff114105be5d55a6" Subject: [dpdk-ci] March 25, 2021 Community CI Meeting Minutes X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" --000000000000ff114105be5d55a6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ########################################################## Attendees 1. Lincoln Lavoie 2. Brandon Lo 3. Aaron Conole 4. Ali Alnubani 5. Ashley Weltz 6. David Liu 7. Juraj Linke=C5=A1 8. Michael Santana 9. Lijuan 10. Thomas Monjalon ########################################################## Agenda 1. Performance Testing Metrics 2. CI Status 3. Bugzilla Status 4. Test Development 5. Any other business ########################################################## Minutes =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D Performance Testing Metrics * UNH-IOL has tested the DTS patch on most of the bare-metal systems in the lab. Couple of systems require python updates. Requests / feedback made on output format, to align with data type expectations, etc. * Lijuan is planning to send the V2 patch for next week. Migration won=E2= =80=99t start until after the next DPDK CI meeting, to allow everyone time to review and test the changes and latest DTS. * Migration is estimated for about 2-3 days, including taking systems offline, making updates, and putting back into production, to move all of the systems to reporting performance numbers. Other testing should be able to remain running, except for during the database migration processes (this is for safety, so we have a stable backup copy if necessary). * Database migration will be run to create percentage results where possible. Data prior to January 2020 was not storing the full absolute results in the database (only the delta), so it=E2=80=99s not possible to c= onvert to the percentage, so that data will not show up in the Grafana charts, etc. If the group wants, we can maintain the earlier (current) carts for that data with the delta values. Performance numbers for LTS19.11 may be worth recreating into the new system (percentage). We will not maintain the graphs / access to the older data. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D CI Status ---------------------------------------------------------- UNH-IOL Community Lab * Reviewed and tested DTS changes for migration to percentage, along with development of the migration planning (i.e. review of script changes, database migration, etc.). * Continued migration process of compile testing to containers. ** New containers for Ubuntu 20.04 and CentOS Stream 8 ** Existing virtual machines are now being migrated over to containers ** Unit testing is being investigated for implementation into the dashboard * Checked SPDK failures, fixed through changes on DPDK main branch. * Tracked down performance failures on arm to a change made while debugging the intermittent timeout failures. This has been rolled back, but we have not been able to track down the other intermittent timeout failure. Aaron will help coordinate a session to debug with the arm team on this issue. Mellanox requests to rerun the patches that are waiting in patchwork, due to the previous failure on the performance run. * Aaron / Lincoln to update bugzilla to track Q1 SOW items. * How to request / schedule a rerun of a patch? What is the process? ** Add patches to a list (i.e. google doc), remove them when the rerun is either started or completed. How could this be automated? Button in patchwork? ** In the meantime, email the lab or the ci list to make rerun requests. ---------------------------------------------------------- Intel Lab * Submitting test results per patch for DPDK, failures were being missed by patchwork, this has been fixed. * Results being sent for DTS to patchworks, reports do show up within DTS patchwork, but the reports show up in the same email list as DPDK reports with the same tags, so it may be confusing to the developers. Thomas / Ali will look into solutions in April. ---------------------------------------------------------- Travis CI / Github Actions / OBS * Aaron is still working on the OBS monitor script. Before this can be turned on, we need to clear the backlog from the OBS system (might be easiest to delete / recreate the account), so the storage is cleared out. * Where do we want to store the scripts for github / obs? Currently separate and used by multiple projects (i.e. open vSwtich, etc.). Could this be handled as documentation as part of the ci repo (i.e. a markdown readme, etc). =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D Bugzilla Status * https://bugs.dpdk.org/buglist.cgi?bug_status=3DUNCONFIRMED&bug_status=3DCON= FIRMED&bug_status=3DIN_PROGRESS&columnlist=3Dproduct%2Ccomponent%2Cpriority= %2Cbug_status%2Cassigned_to%2Cshort_desc%2Cchangeddate&component=3Djob%20sc= ripts&component=3DUNH%20infra&component=3DIntel%20Lab&component=3DTravis%20= CI&list_id=3D3419&order=3Dpriority%2Cchangeddate%20DESC&product=3Dlab&query= _format=3Dadvanced&resolution=3D--- =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D Test Development * Request to share the SOW priorities for 2021, this could be handled as a part of the Q1 blog as well. * Q1 Priorities in the UNH-IOL Community Lab * Migration of compile / unit jobs into containers, increase unit testing coverage / runs where possible, increase ABI coverage to all OS types except Windows * Roll out the percentage performance changes * Migrate to templated pipelines (i.e. utilize patch categorization to direct testing) * Investigate Coverity alternative (build list for tech board review) * Provide report on job run times to techboard (i.e. how long testing jobs take) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D Any other business * April 8, 2021 --=20 *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m) --000000000000ff114105be5d55a6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
###= #######################################################
Attendees
1. Lincoln Lavoie
2. Brandon Lo
3. Aaron Conole
4. Ali Alnubani5. Ashley Weltz
6. David Liu
7. Juraj Linke=C5=A1
8. Michael San= tana
9. Lijuan
10. Thomas Monjalon

###########################= ###############################
Agenda

1. Performance Testing Met= rics
2. CI Status
3. Bugzilla Status
4. Test Development
5. Any= other business

####################################################= ######
Minutes

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Performance Testing = Metrics

* UNH-IOL has tested the DTS patch on most of the bare-metal= systems in the lab.=C2=A0 Couple of systems require python updates.=C2=A0 = Requests / feedback made on output format, to align with data type expectat= ions, etc.
* Lijuan is planning to send the V2 patch for next week.=C2= =A0 Migration won=E2=80=99t start until after the next DPDK CI meeting, to = allow everyone time to review and test the changes and latest DTS.
* Mig= ration is estimated for about 2-3 days, including taking systems offline, m= aking updates, and putting back into production, to move all of the systems= to reporting performance numbers. Other testing should be able to remain r= unning, except for during the database migration processes (this is for saf= ety, so we have a stable backup copy if necessary).
* Database migration= will be run to create percentage results where possible.=C2=A0 Data prior = to January 2020 was not storing the full absolute results in the database (= only the delta), so it=E2=80=99s not possible to convert to the percentage,= so that data will not show up in the Grafana charts, etc.=C2=A0 If the gro= up wants, we can maintain the earlier (current) carts for that data with th= e delta values.=C2=A0 Performance numbers for LTS19.11 may be worth recreat= ing into the new system (percentage).=C2=A0 We will not maintain the graphs= / access to the older data.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
CI Statu= s

----------------------------------------------------------
UNH-= IOL Community Lab

* Reviewed and tested DTS changes for migration to= percentage, along with development of the migration planning (i.e. review = of script changes, database migration, etc.).
* Continued migration proc= ess of compile testing to containers. =C2=A0
** New containers for Ubunt= u 20.04 and CentOS Stream 8
** Existing virtual machines are now being m= igrated over to containers
** Unit testing is being investigated for imp= lementation into the dashboard
* Checked SPDK failures, fixed through ch= anges on DPDK main branch.
* Tracked down performance failures on arm to= a change made while debugging the intermittent timeout failures.=C2=A0 Thi= s has been rolled back, but we have not been able to track down the other i= ntermittent timeout failure.=C2=A0 Aaron will help coordinate a session to = debug with the arm team on this issue.=C2=A0 Mellanox requests to rerun the= patches that are waiting in patchwork, due to the previous failure on the = performance run.
* Aaron / Lincoln to update bugzilla to track Q1 SOW it= ems.
* How to request / schedule a rerun of a patch?=C2=A0 What is the p= rocess?
** Add patches to a list (i.e. google doc), remove them when the= rerun is either started or completed.=C2=A0 How could this be automated?= =C2=A0 Button in patchwork?
** In the meantime, email the lab or the ci = list to make rerun requests.


-----------------------------------= -----------------------
Intel Lab

* Submitting test results per p= atch for DPDK, failures were being missed by patchwork, this has been fixed= .
* Results being sent for DTS to patchworks, reports do show up within = DTS patchwork, but the reports show up in the same email list as DPDK repor= ts with the same tags, so it may be confusing to the developers.=C2=A0 Thom= as / Ali will look into solutions in April.

------------------------= ----------------------------------
Travis CI / Github Actions / OBS
<= br>* Aaron is still working on the OBS monitor script.=C2=A0 Before this ca= n be turned on, we need to clear the backlog from the OBS system (might be = easiest to delete / recreate the account), so the storage is cleared out.* Where do we want to store the scripts for github / obs?=C2=A0 Currently= separate and used by multiple projects (i.e. open vSwtich, etc.).=C2=A0 Co= uld this be handled as documentation as part of the ci repo (i.e. a markdow= n readme, etc).

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Bugzilla Status

= * https://bugs.dpdk.org/buglist.cgi?bug_status=3DUNCON= FIRMED&bug_status=3DCONFIRMED&bug_status=3DIN_PROGRESS&columnli= st=3Dproduct%2Ccomponent%2Cpriority%2Cbug_status%2Cassigned_to%2Cshort_desc= %2Cchangeddate&component=3Djob%20scripts&component=3DUNH%20infra&am= p;component=3DIntel%20Lab&component=3DTravis%20CI&list_id=3D3419&am= p;order=3Dpriority%2Cchangeddate%20DESC&product=3Dlab&query_format= =3Dadvanced&resolution=3D---

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Te= st Development

* Request to share the SOW priorities for 2021, this = could be handled as a part of the Q1 blog as well.
* Q1 Priorities in th= e UNH-IOL Community Lab
=C2=A0 =C2=A0* Migration of compile / unit jobs = into containers, increase unit testing coverage / runs where possible, incr= ease ABI coverage to all OS types except Windows
=C2=A0 =C2=A0* Roll out= the percentage performance changes
=C2=A0 =C2=A0* Migrate to templated= pipelines (i.e. utilize patch categorization to direct testing)
=C2=A0 = =C2=A0* Investigate Coverity alternative (build list for tech board review)=
=C2=A0 =C2=A0* Provide report on job run times to techboard (i.e. how l= ong testing jobs take)

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Any other busines= s

* April 8, 2021

--
Lincoln Lavoie
Principal Engineer, Broadband Te= chnologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
<= a href=3D"mailto:lylavoie@iol.unh.edu" target=3D"_blank">lylavoie@iol.unh.e= du
+1-603-674-2755 (m)

--000000000000ff114105be5d55a6--