DTS WG Meeting Minutes - January 17, 2024

DTS WG Meeting Minutes - January 17, 2024

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 2643 bytes --]

January 17, 2024

#####################################################################
Attendees
* Juraj Linkeš
* Gregory Etelson
* Honnappa Nagarahalli
* Jeremy Spewock
* Luca Vizzarro
* Paul Szczepanek

#####################################################################
Agenda
* Additions to the agenda
* Patch discussions
* DTS Developer documentation
* 24.03 roadmap

#####################################################################
Minutes

=====================================================================
Additions to the agenda
* Nothing

=====================================================================
YAML test suites
* Greg wanted to automate his testing; started with test writtens in
Python, but was not scalable; easily understandable by newcomers.
   * The idea is to take an application, send commands (interactive input),
collect output and compare with expected strings.
* The code was available as of two months ago, but no longer is (private on
GitHub). Greg may be able to share it once taking care of it in his company.
* Gregory submitted an idea for writing test suites in yaml, which just
passes values into a templated testpmd testsuite.
   * Do we want to support a secondary way of writing test suites?
      * Will this be usable for both functional and performance testing?
   * Will this coexist well with the current method?
      * The current method also aims to be minimalistic and intuitive
      * Coexistence makes sense as the yaml approach may not be able to
cover more complicated cases
   * What are any limitations which this places on the testing framework?
If there aren’t major downsides, then the benefits in terms of quickly
adding new testpmd testsuites seems significant.
      * The traffic generator can't be configured here, we need an
abstraction that works for all traffic generators; we can mark the test
cases as functional/performance though, which could be enough
      * We can only specify test-specific testpmd cmdline options;
shouldn't be a problem, but we have to keep in mind that configuration such
as cores and pci addresses are configured elsewhere (the testbed
configuration)
      * Using specific strings in testpmd is harder to maintain (if the
same config is used in multiple places)
   * Are the phases for both setup/teardown and test cases? This could
complicate results recording
   * Can we easily specify multiple test cases? I.e. we have a test method
and we want to test different input combinations (the inputs could just be
the number of cores/packet size for performance tests)

[-- Attachment #2: Type: text/html, Size: 2768 bytes --]