January 17, 2024

#####################################################################
Attendees
* Juraj Linkeš
* Gregory Etelson
* Honnappa Nagarahalli
* Jeremy Spewock
* Luca Vizzarro
* Paul Szczepanek

#####################################################################
Agenda
* Additions to the agenda
* Patch discussions
* DTS Developer documentation
* 24.03 roadmap

#####################################################################
Minutes

=====================================================================
Additions to the agenda
* Nothing

=====================================================================
YAML test suites
* Greg wanted to automate his testing; started with test writtens in Python, but was not scalable; easily understandable by newcomers.
   * The idea is to take an application, send commands (interactive input), collect output and compare with expected strings.
* The code was available as of two months ago, but no longer is (private on GitHub). Greg may be able to share it once taking care of it in his company.
* Gregory submitted an idea for writing test suites in yaml, which just passes values into a templated testpmd testsuite.
   * Do we want to support a secondary way of writing test suites?
      * Will this be usable for both functional and performance testing?
   * Will this coexist well with the current method?
      * The current method also aims to be minimalistic and intuitive
      * Coexistence makes sense as the yaml approach may not be able to cover more complicated cases
   * What are any limitations which this places on the testing framework? If there aren’t major downsides, then the benefits in terms of quickly adding new testpmd testsuites seems significant.
      * The traffic generator can't be configured here, we need an abstraction that works for all traffic generators; we can mark the test cases as functional/performance though, which could be enough
      * We can only specify test-specific testpmd cmdline options; shouldn't be a problem, but we have to keep in mind that configuration such as cores and pci addresses are configured elsewhere (the testbed configuration)
      * Using specific strings in testpmd is harder to maintain (if the same config is used in multiple places)
   * Are the phases for both setup/teardown and test cases? This could complicate results recording
   * Can we easily specify multiple test cases? I.e. we have a test method and we want to test different input combinations (the inputs could just be the number of cores/packet size for performance tests)