February 18, 2021 ############################################################# Attendees 1. Lincoln Lavoie 2. Brandon Lo 3. Thomas Monjalon 4. Lijuan Tu 5. Ali Alnubani 6. David Liu 7. Juraj Linkeš 8. Michael Santana 9. Tomasz Zawadzki 10. Michael Santana 11. Owen Hilyard ############################################################# Agenda 1. CI Status 2. Bugzilla Status 3. Test Development 4. Any other business ############################################################# Minutes ============================================================= CI Status ------------------------------------------------------------- UNH-IOL Community CI Lab * SPDK was failing due to changes in DPDK / SPDK and new dependence on vhost. Vhost has been enabled and is now in production. * ABI Reference - Needs to be updated to 21.02 release - this will be handled today. * Performance and functional testing for Intel and Mellanox NICs on aarch64 are now running and should be stable, working on bringing Broadcom 25G NIC online. * Containers for RHEL7 and RHEL8 have been created, working on getting these into production. This is being done as part of a larger effort to move as many of the compile / unit test jobs into containers as possible, which will allow for more “workers.” The goal is to keep the backlog of the compile and unit testing as short as possible. Need to look into c-cache enabled for the containers via an external mount / volume. * Completed development on pipeline changes for the performance and functional testing (i.e. hardware dependent testing), that will leverage the patch categorization contribution submitted to dpdk-ci. This is intended to allow for more targeted testing, i.e. on test and build what is necessary. Note, this dpdk-ci patch still needs review and merge. * Update so failures on the dashboard that are not the main compilation test cases (i.e. downstream projects) will show up as a warning (orange, not red). This is not yet deployed in production * Performance test emails now send out percent differences instead of raw value differences. Question to the group, do you want this change on the stats (grafana instances as well)? * Upcoming maintenance - Lab will send out notices to the dpdk-ci as this work happens, etc. * Migration to the containers - planning to migrate one job at a time (i.e. swapping to containers one at a time). * Migration to the new pipeline structure for hardware dependent testing. This will require some down time (stop new jobs from starting), for the migration. That will only impact the functional / performance testing. ------------------------------------------------------------- Intel Lab * Working on a change to allow systems to continue to run test cases for systems where the build succeeds, dropping / stopping only on the systems where the build failed. This will probably be ready in the next two weeks. ------------------------------------------------------------- Travis / Github Actions / OBS * GitHub Actions & Travis * Reports should now be flowing, we've pushed the latest set of scripts to the pw-ci repo * ARM folks worked out a deal with the travis people and we have credits to run the DPDK jobs on travis-ci, so they are going * Currently, some jobs are intermittently failing due to a bad version.map file - see http://mails.dpdk.org/archives/dev/2021-February/199761.html * OBS * We have a few mechanisms now to declare branches as no longer needed, so we will use those to clear branch status on the OBS builds. * Plan is to adapt the OBS scripts to the rest of the pw-ci framework. Maybe 3 weeks time. * "Pw-ci" * Bigger question about whether to fold all of pw-ci into the dpdk CI scripts and abandon the pw-ci repo we have. That's a separate discussion, really, but it might make sense since it includes scripts to detect changes on patchwork, submit to different CI backends (like jenkins, obs, travis, etc.), poll status, and report that status. * Group wants to move the discussion over to the dpdk ci mailing list. Lincoln will send out a mail to kick things off. ============================================================= Bugzilla Status * https://bugs.dpdk.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&columnlist=product%2Ccomponent%2Cpriority%2Cbug_status%2Cassigned_to%2Cshort_desc%2Cchangeddate&component=job%20scripts&component=UNH%20infra&component=Intel%20Lab&component=Travis%20CI&list_id=3419&order=priority%2Cchangeddate%20DESC&product=lab&query_format=advanced&resolution=--- * Reviewed and updated bugs in bugzilla. ============================================================= Test Development * Aaron, Trishan, and Lincoln are working to finalize the 2021 priorities and should have that list available to this group for the next meeting. ============================================================= Any other business * Next Meeting: February 25, 2021, then 2 weeks after that (back to the original schedule) * Patchworks is going to be updated on Sunday, February 21, at 4am PST. It will be down for a few minutes during this time. * Performance testing failures - change to failure if fluctuation is larger than x%. Should open a bug for this, Ali will do the first write up. -- *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m)