Community CI Meeting Minutes - August 31, 2023

Community CI Meeting Minutes - August 31, 2023

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 8322 bytes --]

August 31, 2023

#####################################################################
Attendees
1. Patrick Robb
2. Adam Hassick
3. Aaron Conole
4. Bruce Richardson
5. Juraj Linkeš
6. Paul Szczepenek
7. Ali Alnubani

#####################################################################
Agenda
1. General Announcements
2. CI Status
3. DTS Improvements & Test Development
4. Any other business

#####################################################################
Minutes

=====================================================================
General Announcements
* DPDK Summit is Sept 12-13
   * The next CI meeting will be rescheduled from September 14th to
September 21st, to avoid a clash with DPDK summit travel. The following
meeting on September 28th will return us to our normal cycle.
   * There is a gov board and tech board session on the 11th with some ci
discussion in both meetings
* Unit test suites: Bruce’s patch for dynamically building the unit test
suites has hit mainline
   * David’s patch fixing the memory leak from PCI device probing (arm64)
is still pending
   * V3 of patch for skipping specific tests based on an env variable is
submitted
      * Opting to use an environment variable to skip the tests based on
command line parsing concerns
      * Patrick should ack this

=====================================================================
CI Status

---------------------------------------------------------------------
UNH-IOL Community Lab
* Mellanox perf testing:
   * cx5 is back online
   * Hardware Refresh
      * The CX6 NIC is running with no reporting. It is testing at line
rates for performance test runs with frame sizes 256-1518, but is falling
below line rate for the test when run for 64B and 128B frames.
         * Is this due to packet overhead?
         * Currently this is running on a gen3.0 x8 pci slot
         * Ali is going to remote onto the testbed soon to take a look
      * CX7: backordered
   * Running DTS within a VM as a security measure?
      * Will require pci passthrough, cant be done with virtio
      * Does this added level of complexity justify the benefits?
      * Connection to host and rest of network should be blocked, and
dispose of vm after each run
* Intel 8970 QAT Accelerator card:
   * The custom patch doesn’t cleanly apply on the kernel after checking
out to 5.4.0-155(currently running), so I’d like to just rebuild the kernel
from 5.15 or 6.0. But, I don’t want to do it without ARM people authorizing
it, so I’ll proceed once I have the go-ahead from them.
      * Juraj: ubuntu versions should be uniform across the lab (so 22.04
for all systems)
* Retesting framework roadmap - UNH:
   * This is online, and an email explaining the process has been sent to
the dev mailing list:
https://inbox.dpdk.org/dev/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
   * We will add some basic instructions to the DPDK website:
https://inbox.dpdk.org/web/20230831031834.9271-2-probb@iol.unh.edu/T/#u
   * We’ll also put something on the community lab dashboard about page
* TS-Factory: Using our dev testbed, Adam has attempted to run the ethdev
testsuite on a few nics (MLNX cx5, Intel x710, Intel E810). We have only
gotten the test suite to run on our CX5s so far. Oktet lab is communicating
with us to resolve some issues we’ve run into and provide guidance
regarding how this can be used in CI.
   * Adam discovered the bug that the RCF (remote control) implementation
for ts-factory required that DNS returned v4 ip addresses to the test
engine when initiating connections to the tester and DUT systems. Andrew at
Oktet labs hotfixed this for the testing branch we are using it, and this
bugfix will hit ts-factory mainline soon
   * New changes (as of a few days ago) to ts-factory are causing –werror
builds of the testsuite to fail, which has been reported to Oktet labs and
they are working on this week
   * How to use this in a ci context?
      * The DPDK testsuite has approx 6000 test cases, which presently
takes 9 hours to run, and none of the NICs supported by ts-factory will
pass all of them. So, there is not an expectation that all tests will pass.
The “Bublik” tool used by the framework allows for comparison of pass/fails
from the previous run to the current run, but Oktet says this output is
aimed for human readability and is not designed for automation.
      * There is a flag for cutting down the testsuite to “sanity check”
testcases, which we could more reasonably expect to pass 100%. This would
mean we could more reasonably report CI results, expecting 100% passes, and
also it would mean a much shorter runtime. but my guess is this is throwing
the baby out with the bathwater, as those sanity checks results won’t be
very valuable.
      * Do we need to run this periodically, as opposed to on every patch,
due to the test duration?
      * Should we find a way to report some kind of result based on what
tests pass? I don’t yet know if this is feasible. Or, we can simply run it
and store the human readable artifact on the dashboard at an easy to find
place.
      * Does not compile on ARM, but we can reach out to Oktet/ARM to
resolve these issues
         * Test engine only has to compile on a non-worker node, and that
node could be x86
         * Need to figure out what exactly has to be compiled on the (arm)
worker node and communicate with oktet labs if there are issues
* Patrick and Aaron talked about the UNH possibly doing more redundancy
testing for the Intel lab. So, running some of the testsuites they’re
running which we aren’t.
   * It looks like Intel is reporting results again (woohoo!)
   * Patrick will determine the coverage gap between UNH and the Intel lab
before Dublin so that he can discuss it with any interested parties in
person
* Last meeting Aaron asked about maintainers for next-* branches getting
immediate CI runs on an “on-push” basis, like with the LTS-staging
branches. There is nothing preventing this except A. There needs to be a
github mirror so we can use the github API (like we do with LTS-staging),
and B. to Aaron’s point from last time, we need to agree on testrun
frequency.
   * 2024 SOW item?

---------------------------------------------------------------------
Intel Lab
* They are reporting results again

---------------------------------------------------------------------
Loongarch Lab
* none

---------------------------------------------------------------------
Github Actions
* Retest framework: currently testing this internally, and will soon submit
it for review.
* Physical server move: downtime will occur at the end of the year or
beginning of next year
   * Figuring out if there’s a way to migrate the VM to another system so
that downtime is reduced. Otherwise, it will be about a week of downtime

=====================================================================
DTS Improvements & Test Development
* DTS presentation (work in progress):
https://docs.google.com/presentation/d/1fm8EtbzEQHrFyHoHiy0PNQz3MYY2NsQRatR_eC3SvHw/edit#slide=id.g260b440c69d_0_331
   * Honnappa, Juraj, and Patrick
* Paul Szczepanek will be working on DTS
   * Should be included in any conversation for DTS improvement group
* Group met last week to discuss DTS roadmap for 23.11, and Honnappa is
sending that out today
* Jeremy is porting over the scatter testsuite and packet module for packet
comparison/other packet related functions
* Juraj - Documentation
   * What tools to generate the API docs? Sphinx (developed for python, is
a natural choice) or Doxygen.
   * We need to agree on the format of the documentation
      * Juraj likes Google docs format (very readable)
   * Jeremy will review this patch
* DTS roadmap
   * 1)        Documentation
   * 2)        TG related code (Packet manipulation and verification
module, Support for TREX which require Non-packet-capture method
enhancements)
   * 3)        Scatter test suite
   * 4)        Merge pending patches

=====================================================================
Any other business
* Next meeting is September 21, 2023

[-- Attachment #2: Type: text/html, Size: 8999 bytes --]