Hi,

> * Test reports mailing list was created for DTS, but patchwork needs to be set up to monitor this list.  Lijuan will send an example email report to Ali.

> * http://mails.dpdk.org/archives/dts-test-report/2021-June/000005.html

> * http://mails.dpdk.org/archives/dts-test-report/2021-June/subject.html

I figured out why the checks weren’t being added to patchwork, and I’ll look into a solution with Thomas.

Regards,
Ali

From: ci <ci-bounces@dpdk.org> On Behalf Of Lincoln Lavoie
Sent: Thursday, June 17, 2021 6:47 PM
To: ci@dpdk.org
Subject: [dpdk-ci] June 17 Community CI Meeting Minutes

June 17, 2021

#####################################################################
Attendees

1. Lincoln Lavoie
2. Brandon Lo
3. Owen Hilyard
4. Aaron Conole
5. Ali Alnubani
6. Juraj Linkeš
7. Lijuan Tu
8. Ashley Weltz
9. Thomas Monjalon
10. Michael Santana
11. David Marchand

#####################################################################
Agenda

1. Performance Metric Change to Percentage
2. Coverity Testing
3. CI Lab Status
4. Test Development
5. Any Other Business

#####################################################################
Minutes

=====================================================================
Performance Metric Change to Percentage

* Patch series is still pending merge: https://patchwork.dpdk.org/project/dts/list/?series=15990
* Blocked by: https://bugs.dpdk.org/show_bug.cgi?id=670
* Blocks any upgrade to the latest DTS versions, which depend on the latest TRex.
* A work around could be to skip testing specific frame sizes that cause that issue. Ali will confirm internally and reply back to the lab on this way forward.

=====================================================================
Coverity Testing

* https://bugs.dpdk.org/show_bug.cgi?id=719
* Need to investigate if the lab can set up a way to desktop analysis, which would run local to the systems.  Request is to run the test per patch, to check for additional issues being introduced by the patch.  Only reports new defects.
* https://community.synopsys.com/s/topic/0TO34000000LJdaGAG/coverity-desktop-analysis I think this is not an article (just Q&A), you might have to check the documentation in your local installation
* Need to verify the requirements for running the tool, and if it required a paid account, etc.

=====================================================================
CI Lab Status

---------------------------------------------------------------------
UNH-IOL Community Lab

* Currently looking into 40G Intel machine issues.
   * Stats checks disabled due to the requirement for VM images for the test suite in newer DTS versions
   * MTU update test suite disabled due to issue of unknown origin causing packets to appear as received without appearing as sent by the tester.
   * UNH to look at enabling an SSH jump host server, to work around VPN issues.
* Dynamic Config DTS test case is still disabled
   *  https://bugs.dpdk.org/show_bug.cgi?id=639
   * Need to confirm the patch status within DTS, looks like it was merged 8-days ago.
* Mellanox testing disabled
   * DTS seems to be sending packets out the wrong ports, flipping TX/RX ports in code seems to fix this.  Root cause is unknown.  Can not upgrade DTS / TRex because of the blocking issue listed above.
* Unit testing failures
   * Still seeing random failures on the Func_reentrancy_autotest unit test.  3 of the last 5 unit test failures are this test.  Failures do not seem related to OS, Virtual or Bare-metal, or architecture.
   * Aaron is going to create a bug for this, and will look into removing the test case from the suite for now.
* ASAN deployment
   * Various memory leak fixes in patchworks
      * rte_pci_scan
         * http://patches.dpdk.org/project/dpdk/patch/20210616065257.16044-1-david.marchand@redhat.com/
      * ribv6
         * http://patches.dpdk.org/project/dpdk/patch/20210616181833.356159-1-ohilyard@iol.unh.edu/
      * cmdline autotest
         * http://patches.dpdk.org/project/dpdk/patch/20210616180724.355217-1-ohilyard@iol.unh.edu/
      * test_eal_flags
         * http://patches.dpdk.org/project/dpdk/patch/20210616162453.349614-1-ohilyard@iol.unh.edu/
      * flow_classify
         * http://patches.dpdk.org/project/dpdk/patch/20210616195724.366103-1-ohilyard@iol.unh.edu/
   * Still more leaks in unit tests, which prevent deployment.
* Periodic Testing
   * Run daily, but only if the git HEAD has moved.
   * Branches (pointing to the default of the repos):
      * dpdk main branch
      * dpdk-next-net
      * dpdk-next-virtio
      * dpdk-next-eventdev
      * dpdk-next-crypto
      * LTS current and previous (i.e. DPDK release plan) (e.g. 20.11 and 19.11 currently)
   * Test cases:
      * mtu_update
      * scatter
      * unit_tests_mbuf
      * stats_checks
      * dynamic_config
         * Planned, but disabled due to https://bugs.dpdk.org/show_bug.cgi?id=639
      * mac_filter
      * link_status_interrupt
      * nic_single_core_perf
   * Should the lab run ABI/Compile/Unit testing periodically?
      * Daily, same triggers as described above
   * When and where should the lab send an email?
      * Every run (both pass & fail)
      * On failure
      * Something else?
      * To maintainers, devs mailing list, somewhere else?
      * Send the reports to the test-reports mailing list.  In the future we will need to align the report formatting between labs.  CC the maintainers for what was tested..
   * How often should these be run?
      * Daily if there are any changes (Current)
   * rte flow
      * Will be deployed alongside periodic testing, to provide a way to test systems while failures are addressed by the maintainers
* Clang Cross-compile Testing
   * Failure to build. Related to event/cnxk/cnxk_tim_worker.
   * Bug submitted to https://bugs.dpdk.org/show_bug.cgi?id=740
   * Production pipeline is set up to enable this test once this bug is resolved
* FreeBSD 13
   * Now doing compile and ABI testing
   * Unit testing blocked by contigmem-related issue
* Clock skew issue resolved
   * Patch applied to IOL infrastructure and merged into DTS
* ABI False failures
   * Last week (June 4), a couple of false failures were caused by compiler updates (i.e. updating container images after the ABI reference was generated).  The lab saves off the reference as an optimization.  The lab systems were updated to regenerate the ABI references if container images are updated.
* aarch64 SVE testing
   * With Ccache enabled, builds take about 5 to 7 minutes within the emulated environment.  This will keep the compile and binarys in  the same environment (i.e. within a single container).

---------------------------------------------------------------------
Intel Lab

* Patchwork Series https://bugs.dpdk.org/show_bug.cgi?id=718
   * It’s on-going, and we have found the root cause. The solution is a little complicated in CI script, and needs more time to fix them.
* Some of the Intel test bed is off-line (for valid reasons, i.e. debugging or development), this can block the reporting. Some workaround have been made, will disable the test-bed if there is a need to offline it. And if it is offline unexpectedly, CI will notify CI maintainers.
* Test reports mailing list was created for DTS, but patchwork needs to be set up to monitor this list.  Lijuan will send an example email report to Ali.
* http://mails.dpdk.org/archives/dts-test-report/2021-June/000005.html
* http://mails.dpdk.org/archives/dts-test-report/2021-June/subject.html
* Also need to add the email list onto the webpage for https://www.dpdk.org/contribute/#mailing-lists

---------------------------------------------------------------------
Github Actions / OBS

* Github no changes.
* OBS no changes.

=====================================================================
CI / Test Development

* DTS Improvement
   * Discussions are progressing. Work is being tracked here: https://docs.google.com/document/d/1c5S0_mZzFvzZfYkqyORLT2-qNvUb-fBdjA6DGusy4yM/edit
   * Next Meeting: TBD (working around vacation schedules)
* Developer Retesting Request
   * https://mails.dpdk.org/archives/ci/2021-May/001189.html
   * Seeking feedback on the “command syntax.”  Still need to implement the tracking / mechanics for things like patches and message ID tracking.
* Community Lab Objectives
   * Q1 Update to the techboard: https://docs.google.com/document/d/1v0VKtZdsMXg35WNDawdsnqj5J4Xl9Egu_4180ukKD2o
   * Status tracking is also being partly carried out in Bugzilla.  Epics have been created for each major objective. This will span 2021, do not expect them to be closed quickly. Updates posted to each objective will track story completion, etc.  This is a compromise of how to easily track the long term objectives.
* Open Requests (outside of planned objectives)
   * Add meson syntax check testing (https://bugs.dpdk.org/show_bug.cgi?id=696)
      * Will leave this open for now for folks to give feedback on.

=====================================================================
Any Other Business

* Next CI Meeting: June 30, 1pm UTC

--

Lincoln Lavoie

Principal Engineer, Broadband Technologies

21 Madbury Rd., Ste. 100, Durham, NH 03824

lylavoie@iol.unh.edu

https://www.iol.unh.edu

+1-603-674-2755 (m)