Community CI Meeting Minutes - August 8, 2024

Community CI Meeting Minutes - August 8, 2024

[Patrick Robb]

#####################################################################
August 8, 2024
Attendees
1. Patrick Robb
2. Juraj Linkeš
3. Aaron Conole
4. Alex Chapman
5. Dean Marx
6. Jeremy Spewock
7. Luca Vizzarro
8. Nicholas Pratte
9. Paul Szczepanek
10. Manit Mahajan

#####################################################################
Minutes

=====================================================================
General Announcements
* Testing presentations at DPDK Summit:
   * UNH Lab Update and CI tooling updates talk was approved. Patrick
and Adam Hassick will give this talk
   * Luca’s proposal for a DTS talk was approved, but the remote
presentation option was rejected. So, Patrick and (Nick or Dean) will
give this presentation.
* AWS CI Testing initiative status?
   * What testing tooling/frameworks can AWS people use, for CI
testing and reporting to the mailing list?
      * 1. They mentioned they have some internal DPDK test tools
      * 2. dpdk-test/meson test from the DPDK repo
      * 3. DTS (would be most appropriate in a couple months once more
testsuites are merged to DPDK and we can figure out running the
framework in a cloud environment)
   * What utilities do they need in order to setup CI?
      * Each lab has a different automation framework setup (UNH uses
Jenkins, for instance) but the dpdk-ci repo
(https://git.dpdk.org/tools/dpdk-ci/) contains bash/python scripts
which can be used to stand up the CI process regardless of the
automation tool in use:
         * downloading patches from DPDK patchwork (download-patch.sh)
            * UNH also wrote a python equivalent for downloading
patchseries, which we can publish to dpdk-ci if there is interest.
         * Making a guess of what the correct DPDK branch to apply on
is, based on files modified
(https://git.dpdk.org/tools/dpdk-ci/tree/tools/pw_maintainers_cli.py)
         * (if needed) Apply patches to DPDK with git-pw and create a
tarball of the resulting DPDK artifact to test on
(https://git.dpdk.org/tools/dpdk-ci/tree/tools/create_series_artifact.py)
         * Send test report to the DPDK test-report mailing list
(https://git.dpdk.org/tools/dpdk-ci/tree/tools/send-patch-report.sh)
         * And more utilities, which may be needed depending on the
specific infra/process you decide to stand up
   * There are some people wanting to visit the UNH lab. Need to
resurrect the mailing list conversation.
      * Most people will probably try to leave the morning of the 26th
and drive to UNH (5 hours), and can arrive early afternoon at UNH

=====================================================================
CI Status

---------------------------------------------------------------------
UNH-IOL Community Lab
* Windows Server testbeds: Tyler from Microsoft let us know that we
can discontinue usage of the 2019 server. So, we will only run from
server 2022 now, we just need to verify we are running all 3 compile
toolchains on server 2022. As a reminder, they are:
   * LLVM 14.0.0 (or later) and Microsoft MSVC linker.
   * Microsoft Visual Studio 2022 (any edition).
   * The MinGW-w64 10.0 (or later) toolchain (native).
      * One more note about Windows compile jobs is that the prebuilt
mingw64 binaries are not longer available at Sourceforge. It is
necessary to download them at https://winlibs.com/
* Depends-on support on Patchwork: submitted a v2 for the series
   * https://patchwork.ozlabs.org/project/patchwork/list/?submitter=88738
* Pending reporting:
   * This is working, but we have gotten complaints about spamming the
mailing list with too many reports. The reason is that we send a new
report every time a testing job finishes for a particular environment
(so we add a new row to the results table for a patchwork context).
The community has requested that we change to a model of “send a
report when testing for a patchwork context begins, then as tests
finish, only send a new report if the overall test status for the
context has changed, then send a report when all testing for the
patchwork context is complete.” We are making these updates now.
* Testbed maintenance: Updating baremetal systems, firmware, VMs,
containers definitions via dockerfiles, distro upgrades. We do this at
the start of every release cycle.
* DPDK Dashboard: Have made some improvements to how we query for test
results, and started caching some requests we make to our results REST
API, which have improved performance quite a bit. Still not lightning
fast, but will no longer get timeout errors trying to load the results
detail view for a patchseries etc.

---------------------------------------------------------------------
Intel Lab
* Juraj’s DTS API doc generation patch failed Intel’s CI, due to “--
exclude=doc/* *” build parameter being used. They have stated this is
done to provide CI stability, and suggested in this case maintainers
make an exception and merge the patch without a pass from Intel lab.
Conversation is ongoing on the mailing list here, but Juraj/Patrick
trying to figure out why this option is needed:
https://inbox.dpdk.org/ci/CAJvnSUB_Ff_vw5p_VRoVstSrkXts8O9bWEuOSXvmTnMyBj27sQ@mail.gmail.com/T/#t
   * This issue is no longer relevant (in terms of Juraj’s patch) as
he has moved .rst sources to a different location and the series is
now patching
   * We should still get a resolution regarding their build process,
how changes can be authorized etc.
   * They appear to have an apply process in which they skip a patch
if the patch only contains changes within the /doc directory. If a
patch contains change in /doc, and also elsewhere, it is applied.

---------------------------------------------------------------------
Github Actions
* Aaron does intend to adjust his depends-on code once that support is
added to Patchwork and DPDK patchwork upgrades to the latest version.

---------------------------------------------------------------------
Loongarch Lab
* None, Patrick Robb still needs to work with Min Zhou on retests

=====================================================================
DTS Improvements & Test Development
* DTS 24.11 roadmap:
https://docs.google.com/document/d/1Rcp1-gZWzGGCCSkbEsigrd0-NoQmknv6ZS7V2CPdgFo/edit
   * First 4 items will be taken by Juraj And Tomas
* Filtering noise packets within testpmd verbose output
   * Need to identify packets via verbose output based on some unique identifier
   * Option 1:
      * Could possibly use the SRC mac address
      * Can also check the packet layers, to see whether they match,
adding a 2nd check beyond the SRC mac address check
   * Option 2:
      * Can use checksums
      * Will DPDK always report packets with a bad checksum?
         * NOTE: We have noticed that there is a bug for mlx5, which
does not differentiate between a bad checksum and no checksum.
Bugzilla ticket is here: https://bugs.dpdk.org/show_bug.cgi?id=1514
   * Option 3:
      * Add support to testpmd code to show the port number
      * This appears to be the best candidate which will give us a
high confidence in its stability, and allow for the greatest test
coverage going forward
* Test cases for hardware reported values, like Speed capabilities,
RSS Key Update, etc.
   * Requires a config file for the testsuite registering the expected
output to be returned by testpmd
* Release notes:
   * The release notes updates should come in with the patchseries
which add a particular feature
(/doc/release_notes/24.11_release_notes.rst or something similar)
   * Would be good to add a list of bulleted testsuites which are
added once per cycle
* API Docs Generation:
   * Juraj has majorly refactored the suite for v16. So, he is needing
new reviews.
   * Thomas has provided some comments, which Juraj has now addressed
   * There may be minor changes ahead, but it’s essentially done (so,
ready for reviews again)
   * We no longer require that DTS dependencies are satisfied in order
to generate documentation. There is a sphinx option which allows you
to flag dependencies as not required to build docs.
      * This is so there is less hassle to building documentation
   * Where do we put the source .rst files? It was previously in the
dts folder. Now moved to where DPDK API docs files are located
(/doc/api). Now all the docs files are located in one place.
* Ethertype testsuite RFC was posted, but the logic behind the
testcases needs to be validated (original testplan is not valid), so
we should provide an explanation of any new testcases on the mailing
list

=====================================================================
Any other business
* Next Meeting Aug 22, 2024