Community CI Meeting Minutes - September 28, 2023

Community CI Meeting Minutes - September 28, 2023

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 8718 bytes --]

September 21, 2023

#####################################################################
Attendees
1. Patrick Robb
2. David Marchand
3. Paul Szczepanek
4. Aaron Conole
5. Thomas Monjalon
6. Honnappa Nagarahalli
7. Adam Hassick
8. Juraj Linkeš

#####################################################################
Agenda
1. General Announcements
2. CI Status
3. DTS Improvements & Test Development
4. Any other business

#####################################################################
Minutes

=====================================================================
General Announcements
* DPDK Summit was Sept 12-13
   * Patrick went to gov board and tech board dinners to give a lab update
and talk about future plans for the lab/SOW talks
      * Add next-* staging branches for maintainers, like exists currently
for LTS releases
         * “Test on push” function desired, like how 21.11-staging works
now?
      * “Reporting completeness” by submitting PENDING to patchwork
initially during a testrun, then overwriting later with PASS/FAIL (or
similar solution also accomplishing showing reporting completeness)
      * Redundancy testing with other public labs?
         * Patrick produced a document showing what Intel lab tests which
the community lab doesn’t:
https://docs.google.com/document/d/1dDVhUFlk3DoQzf0AvyqaVYzJiCDFx5uAnU6_qbivxe4/edit?usp=sharing
         * TLDR: For DTS, we have basically the same hardware and could
likely add the functional tests and functional-smoke tests Intel Lab is
using to our testruns. For compile testing, we mostly use the same distros,
but a few gaps there can be easily bridged. The most interesting gap in
compile testing is we almost exclusively compile with GCC and static
library linking, whereas Intel Lab has more diversity like more extensive
use of Clang than us, more extensive use of shared libraries vs static, and
some use of debug build type.
      * Investigate testing DPDK using cloud platforms?
      * Honnappa asked about adding another DTS developer at the lab. This
is probably possible, but it would require additional funding to the lab of
course. Will have to discuss further with tech board and bring it to gov
board if desired.
      * Poll the tech board to determine what CI testing improvements are
needed in 2024

=====================================================================
CI Status

---------------------------------------------------------------------
UNH-IOL Community Lab
* Nvidia perf testing:
   * Hardware Refresh
      * The cx6 NIC is running with no reporting. It is testing at line
rates for performance test runs with frame sizes 256-1518, but is falling
below line rate for the test when run for 64B and 128B frames. The tx is
not the bottleneck, it is dropping packets on the DUT side.
         * Ali will discuss with team
      * CX7: installed in NVIDIA DUT server, and Intel tester TG server.
         * Need to verify OFED will not disrupt the Intel testing
         * Need to aggregate info regarding any changes required for the
system and send to Intel contacts
         * Do NOT install OFED - just need to use most recent distribution
      * Are there functional testsuites which should be run on these cards?
* Intel 8970 QAT Accelerator card:
   * Have built a custom kernel which Ampere is running now, with QAT
drivers statically built in
      * Seg fault when echoing a VF number into (device
address)/sriov_numvfs.
* TS-Factory ethdev testsuite has been run on cx5, xl710, and E810 NICS
   * Per conversation with the tech board at DPDK summit
      * We can run this testing periodically, and will be running the full
testsuite (for now, no use of the new “dial” feature which reduces the
scope from 6000 testcases to a lower number)
      * We will publish a human readable testing report to a “Bublik”
website instance on our dashboard
      * We won’t be comparing current runs against previous runs to
determine a pass/fail/etc based on how current runs compare to older runs.
Those human cycles would better be put into DTS.
* We are reviewing/updating our apply patchset script(s). This is to remove
dependencies on our internal infrastructure, so it is upstreamable, and
make improvements like “depends on patch” support. We are reviewing our
assumptions regarding the apply process on the CI mailing list, but some
which can be discussed are:
   * 1. RFC series should be included in series to apply and run ci testing
on
   * 2. Always check out to the current head of tree when applying a patch,
regardless of whether the tree state has changed between patch submission
and patch application in CI.
   * 3. If the cover letter contains “depends-on,” extract the dependency
series id(s), apply those, then attempt to apply the patch
   * 4. If patch does not cleanly apply to the branch supplied by
pw_maintainers_cli.py, attempt to apply on dpdk main. If this also fails,
report an apply failure.
   * If we can use pw-client and upstream the depends-on support, that
would be a benefit for other projects too
      * Aaron: other projects may have no need for this
      * Git-pw can do a similar thing, and should be used instead of
pwclient

---------------------------------------------------------------------
Intel Lab
* Patrick is reaching out figure out who is active in this lab
   * I have emailed John, but should email Robin Giller too
* Issue with testing cryptodev on centos. John is going to reach out to
Intel lab to see who is responsible for addressing this.

---------------------------------------------------------------------
Loongarch Lab
* Has been working well

---------------------------------------------------------------------
Github Actions
* Moving servers physically from MA. There is an internal security audit
which will cause some jobs to be disrupted.
   * Security process will require spinning up new instances and redeploying
   * Working with IT infra to have a server ready to go at the new
location, so the VM can be quickly migrated and downtime can be reduced.
But, the downtime may still be up to 1 week.
* Aaron is reviewing and will merge get_reruns.py
   * Should integrate with the robot fine

=====================================================================
DTS Improvements & Test Development
* Dperf is a new DPDK application which can act as a traffic generator. The
maintainer gave a talk at DPDK Summit and Honnappa and I chatted with him
about whether we can create a traffic generator subclass for dperf, as an
alternative to using Trex, which currently is our only option. We should
sync on this as we get closer to 23.11 release and start thinking about
24.03 roadmap.
   * https://github.com/baidu/dperf
   * Perf testing examples:
https://dperf.org/doc/html/dperf-performance-testing-basic
   * Can you send l2 traffic?
      * Patrick will reach out to the maintainer
   * Can the maintainer be asked to subclass the traffic gen in the new DTS
framework
* Is 23.11 roadmap emailed out?
* How to inject ssh keys into the container running DTS?
   * It may be best to just bake it in from the Dockerfile, and people can
just destroy the image and rebuild if they need a new key
* Scatter testsuite: Will not write a class to wrap scapy functions, but
may write a module for some packet building/manipulation functions which
will be very common in writing DTS testsuites
   * Thomas: We can build up the module for packet manipulation as we go,
according to needs which pop up as new testsuites are being written
* Devtools update got merged. Jeremy should use the updated linter script
for development going forward.
* How to select a list of testsuites to run?
   * Sections of DPDK project touched in a patch
   * Environment information and NIC feature set info
   * Do we want to evaluate on a testsuite basis, or more granular, like
testcases. Modules are associated with testcases currently.
      * 1. Does it make sense to have more than 1 testsuite class per
module?
      * 2. Do we want to add relations between modules and testsuites,
instead of just modules-testcases?
      * 3. Can we build a hierarchy for determining
      * 4. EXAMPLE: For ethdev, we could have 1 module associated with a
list of testsuites
      * 5. Honnappa: organization should align with how code is organized.
So, 1 module per library.
         * Can divide testsuites according to subdirs

=====================================================================
Any other business
* Next meeting is October 12, 2023

[-- Attachment #2: Type: text/html, Size: 9326 bytes --]