Community CI Meeting Minutes - August 17, 2023

Community CI Meeting Minutes - August 17, 2023

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 7884 bytes --]

August 17, 2023

#####################################################################
Attendees
1. Patrick Robb
2. Bruce Richardson
3. Honnappa Nagarahalli
4. Aaron Conole
5. David Marchand
6. Manit Mahajan
7. Lincoln Lavoie

#####################################################################
Agenda
1. General Announcements
2. CI Status
3. DTS Improvements & Test Development
4. Any other business

#####################################################################
Minutes

=====================================================================
General Announcements
* DPDK Summit is Sept 12-13
   * Should we hold the scheduled Sept 14th ci meeting? Which ci meeting
regulars will be at the summit? Aaron and Patrick can’t make it, so let’s
cancel this one.
      * Will suggest an off normal schedule meeting to replace this
* Patchwork v3:
   * Bruce reported incorrect series titles and missing cover letters for
some newer patchseries
   * Don’t CC the ci mailing list with dpdk patches, as it may get diverted
to the ci project on patchwork.
      * A temporary workaround is to cc just relevant people in the ci
community
      * Aaron is supposed to have admin permissions for the ci project on
pathwork, but he needs to ping Ali about this
* Unit test suites: Bruce Richardson is working on a patch which eliminates
the old approach of setting the DPDK unit testuites from
/app/test/meson.build, and instead dynamically builds the lists based on
build configuration
   * This raised one issue with UNH CI, which is that we are filtering out
the eal_flags_file_prefix_autotest on our ARM containers, and this will no
longer be possible following Bruce’s rework. Bruce has suggested
implementing meson functionality allowing for disabling specific tests,
which is a better version of how we filter currently.
      * Testlogs for the failure are now shared with David who is going to
try and help
      * David has reproduced and has seen a leak when there is 1 (and only
1) device
      * UNH will send lspci output from the VMs which host the containers
         * ARM vms are on KVM, so that might be a reason
   * Bruce pushed a new patch where you can set an env variable of tests to
be skipped
      * This is better in that the tests show up as skipped instead of just
not showing up at all - in this way it is clear to people looking at the
logs that the test was not run, but usually is run for this testsuite
   * For 32 bit aarch testing, which cannot run via meson test and is
dependent on the dpdk-test binary path, we will have to coordinate the
merge of this patchseries with Bruce so we can modify our script, or Bruce
can put a symlink connecting the old path to the new
      * Cross compile a version of meson to run on that? Honnappa does not
think this is needed, but he is going to talk to arm folks about it.

=====================================================================
CI Status

---------------------------------------------------------------------
UNH-IOL Community Lab
* Mellanox perf testing:
   * The Mellanox DUT is upgraded to 22.04 and is running perf testing, but
with no reporting until Mellanox people assess the new results. So far,
results appear consistent with the performance seen when the DUT was on
18.04, and with an older version of the MLNX OFED driver.
      * Patrick will send a report to Ali today with the results from the
previous 24 hours
   * Hardware Refresh
      * The CX6 NIC is installed. Ali created a TREX config for this nic.
Patrick will complete the remaining setup for perf testing. That will
require a patch to DTS adding NIC info. CX7 is in the same boat. Ali shared
an example from when the cx5 was added, which I will use as a reference:
b6b0a575d34f ("tests/nic_single_core_perf: add support for Mellanox's nics")
      * CX7: The SKU we ordered was discontinued, so our order has been
canceled. We put in another order for a different model:MCX713106AC-CEAT
   * Running DTS within a VM as a security measure?
      * Will require pci passthrough, cant be done with virtio
      * Does this added level of complexity justify the benefits?
* Intel 8970 QAT Accelerator card:
   * We still need to rebuild the kernel with the patch Ruifeng shared:
https://lkml.org/lkml/2022/6/17/328
   * Are there security issues or instability introduced by building the
custom kernel and not changing it?
   * There may be an official PPA we can use with the latest official kernel
      * It should be in kernel 6.0 and newer, so it may be as simple as
upgrading to the latest kernel
* Retesting framework roadmap - UNH:
   * 1. Database migrations - done
      * Storing retest request relations to patchseries tarballs, and the
retest datetimes, to be used as parameters for future comment requests
      * Will limit retest requests to x per patchseries tarball (two seems
reasonable, but open to a group decision)
         * Limit to 1 at first
   * 2. Script for parsing comments for api calls to pw, contexts/labels,
producing list, writing json file. - done.
      * This is not UNH specific and we can upstream this to the dpdk-ci
repo if there’s interest from other labs
   * 3. Write jenkinsfile script for requesting comments and triggering
testing pipelines according to retest requests - in progress:
      * What is an acceptable frequency for us to run this (which may
involve many patchwork API calls)
   * 4. Contact community with the rules and format
* Reporting: UNH rolled out “tail reporting” which moves reporting from
result aggregating reporting stages at the end of our CI process, to
reporting from the individual jobs which run testing. This should decrease
our time to delivery for reporting, and increase our reliability, as it
essentially moves us to one source of truth for reporting.
* TS-Factory: We are running this on a Dev server, but need some help with
setting the config files for the DPDK PMD testsuite. Adam has emailed
Konstantin from Oktet Labs for clarification on this.
   * Consider test capacity
* If Intel lab is downsizing, UNH may need to reassess our goal of adding a
framework for other labs to submit results and artifacts to the UNH
dashboard, since that was an Intel lab request.
* Aaron is interested in maintainers getting their own branch that
maintainers can push to to get test results
   * Similar to LTS-staging* branches
   * What policies would have to be set regarding how maintainers use this
ability?

---------------------------------------------------------------------
Intel Lab
* No reports since August 12th

---------------------------------------------------------------------
Loongarch Lab
* none

---------------------------------------------------------------------
Github Actions
* Robot’s poller will be offline for some time in sept-october due to
systems being physically moved
   * The poller is run in a VM - if this can be migrated, downtime may be a
day or less, but it’s unclear whether we can
* Retesting framework: work on this is ongoing, and will be sent to the ci
mailing list
* Going to be re-assessing some APIs on pw and GHA, and reverting old
workaround for polling via GHA

=====================================================================
DTS Improvements & Test Development
* DTS presentation (work in progress):
https://docs.google.com/presentation/d/1fm8EtbzEQHrFyHoHiy0PNQz3MYY2NsQRatR_eC3SvHw/edit#slide=id.g260b440c69d_0_331
   * Honnappa, Juraj, and Patrick
* Jeremy submitted an RFC for porting over the Scatter Testsuite
* 9AM eastern next week we will have a meeting about the DTS roadmap

=====================================================================
Any other business
* Next meeting is August 31, 2023

[-- Attachment #2: Type: text/html, Size: 8389 bytes --]