September 21, 2023
#####################################################################
Attendees
1. Patrick Robb
2. David Marchand
3. Paul Szczepanek
4. Aaron Conole
5. Thomas Monjalon
6. Honnappa Nagarahalli
7. Adam Hassick
8. Juraj Linkeš
#####################################################################
Agenda
1. General Announcements
2. CI Status
3. DTS Improvements & Test Development
4. Any other business
#####################################################################
Minutes
=====================================================================
General Announcements
* DPDK Summit was Sept 12-13
* Patrick went to gov board and tech board dinners to give a lab update and talk about future plans for the lab/SOW talks
* Add next-* staging branches for maintainers, like exists currently for LTS releases
* “Test on push” function desired, like how 21.11-staging works now?
* “Reporting completeness” by submitting PENDING to patchwork initially during a testrun, then overwriting later with PASS/FAIL (or similar solution also accomplishing showing reporting completeness)
* Redundancy testing with other public labs?
* Patrick produced a document showing what Intel lab tests which the community lab doesn’t: https://docs.google.com/document/d/1dDVhUFlk3DoQzf0AvyqaVYzJiCDFx5uAnU6_qbivxe4/edit?usp=sharing
* TLDR: For DTS, we have basically the same hardware and could likely add the functional tests and functional-smoke tests Intel Lab is using to our testruns. For compile testing, we mostly use the same distros, but a few gaps there can be easily bridged. The most interesting gap in compile testing is we almost exclusively compile with GCC and static library linking, whereas Intel Lab has more diversity like more extensive use of Clang than us, more extensive use of shared libraries vs static, and some use of debug build type.
* Investigate testing DPDK using cloud platforms?
* Honnappa asked about adding another DTS developer at the lab. This is probably possible, but it would require additional funding to the lab of course. Will have to discuss further with tech board and bring it to gov board if desired.
* Poll the tech board to determine what CI testing improvements are needed in 2024
=====================================================================
CI Status
---------------------------------------------------------------------
UNH-IOL Community Lab
* Nvidia perf testing:
* Hardware Refresh
* The cx6 NIC is running with no reporting. It is testing at line rates for performance test runs with frame sizes 256-1518, but is falling below line rate for the test when run for 64B and 128B frames. The tx is not the bottleneck, it is dropping packets on the DUT side.
* Ali will discuss with team
* CX7: installed in NVIDIA DUT server, and Intel tester TG server.
* Need to verify OFED will not disrupt the Intel testing
* Need to aggregate info regarding any changes required for the system and send to Intel contacts
* Do NOT install OFED - just need to use most recent distribution
* Are there functional testsuites which should be run on these cards?
* Intel 8970 QAT Accelerator card:
* Have built a custom kernel which Ampere is running now, with QAT drivers statically built in
* Seg fault when echoing a VF number into (device address)/sriov_numvfs.
* TS-Factory ethdev testsuite has been run on cx5, xl710, and E810 NICS
* Per conversation with the tech board at DPDK summit
* We can run this testing periodically, and will be running the full testsuite (for now, no use of the new “dial” feature which reduces the scope from 6000 testcases to a lower number)
* We will publish a human readable testing report to a “Bublik” website instance on our dashboard
* We won’t be comparing current runs against previous runs to determine a pass/fail/etc based on how current runs compare to older runs. Those human cycles would better be put into DTS.
* We are reviewing/updating our apply patchset script(s). This is to remove dependencies on our internal infrastructure, so it is upstreamable, and make improvements like “depends on patch” support. We are reviewing our assumptions regarding the apply process on the CI mailing list, but some which can be discussed are:
* 1. RFC series should be included in series to apply and run ci testing on
* 2. Always check out to the current head of tree when applying a patch, regardless of whether the tree state has changed between patch submission and patch application in CI.
* 3. If the cover letter contains “depends-on,” extract the dependency series id(s), apply those, then attempt to apply the patch
* 4. If patch does not cleanly apply to the branch supplied by pw_maintainers_cli.py, attempt to apply on dpdk main. If this also fails, report an apply failure.
* If we can use pw-client and upstream the depends-on support, that would be a benefit for other projects too
* Aaron: other projects may have no need for this
* Git-pw can do a similar thing, and should be used instead of pwclient
---------------------------------------------------------------------
Intel Lab
* Patrick is reaching out figure out who is active in this lab
* I have emailed John, but should email Robin Giller too
* Issue with testing cryptodev on centos. John is going to reach out to Intel lab to see who is responsible for addressing this.
---------------------------------------------------------------------
Loongarch Lab
* Has been working well
---------------------------------------------------------------------
Github Actions
* Moving servers physically from MA. There is an internal security audit which will cause some jobs to be disrupted.
* Security process will require spinning up new instances and redeploying
* Working with IT infra to have a server ready to go at the new location, so the VM can be quickly migrated and downtime can be reduced. But, the downtime may still be up to 1 week.
* Aaron is reviewing and will merge get_reruns.py
* Should integrate with the robot fine
=====================================================================
DTS Improvements & Test Development
* Dperf is a new DPDK application which can act as a traffic generator. The maintainer gave a talk at DPDK Summit and Honnappa and I chatted with him about whether we can create a traffic generator subclass for dperf, as an alternative to using Trex, which currently is our only option. We should sync on this as we get closer to 23.11 release and start thinking about 24.03 roadmap.
* https://github.com/baidu/dperf
* Perf testing examples: https://dperf.org/doc/html/dperf-performance-testing-basic
* Can you send l2 traffic?
* Patrick will reach out to the maintainer
* Can the maintainer be asked to subclass the traffic gen in the new DTS framework
* Is 23.11 roadmap emailed out?
* How to inject ssh keys into the container running DTS?
* It may be best to just bake it in from the Dockerfile, and people can just destroy the image and rebuild if they need a new key
* Scatter testsuite: Will not write a class to wrap scapy functions, but may write a module for some packet building/manipulation functions which will be very common in writing DTS testsuites
* Thomas: We can build up the module for packet manipulation as we go, according to needs which pop up as new testsuites are being written
* Devtools update got merged. Jeremy should use the updated linter script for development going forward.
* How to select a list of testsuites to run?
* Sections of DPDK project touched in a patch
* Environment information and NIC feature set info
* Do we want to evaluate on a testsuite basis, or more granular, like testcases. Modules are associated with testcases currently.
* 1. Does it make sense to have more than 1 testsuite class per module?
* 2. Do we want to add relations between modules and testsuites, instead of just modules-testcases?
* 3. Can we build a hierarchy for determining
* 4. EXAMPLE: For ethdev, we could have 1 module associated with a list of testsuites
* 5. Honnappa: organization should align with how code is organized. So, 1 module per library.
* Can divide testsuites according to subdirs
=====================================================================
Any other business
* Next meeting is October 12, 2023