Hi all,
Now that we've written a significant number of tests and developed a
consistent usage pattern, I think it's a good time to take the next step
toward project maturity: establishing a stable, well-defined API that
developers can rely on across releases.
Agree
We got some comments from the devs present at the last DPDK summit and
they are valuable guidance on what direction you'd like the framework to
go and we're taking them onboard to shape our roadmap. Very much related
to that we'd appreciate any comments on what you'd like to see in the API.
API proposal goal
Split framework internals from functions intended for use by test writers.
This API would offer:
- Discoverability of available functionality
- Clear boundaries between public/test-facing APIs and internal
framework logic
- A stable interface for writing and maintaining tests
- A DPDK-style release cadence and compatibility guarantee (i.e., write
once, run future-proof)
The notes about making the API stable make sense:
1. So, is there a suggestion here that the API can be broken once per year, and that would be a new policy? Would that happen at a .11 release with your proposal?
2. I assume I can add functions to a DTS API module, but cannot remove/change existing ones during a stable period?
3. I guess we can either have reviews and maintainer review enforce the API stability, or we can develop some checks that will try to enforce the API stability. Do you have a preference?
With this split, DTS becomes effectively self-documenting — users can
explore and use only what’s in the api.
Technical Summary
Introduce a dedicated api/ directory that contains a small, focused set
of modules exposing all the functionality needed to write tests.
All test execution logic and framework internals will remain private and
continue to live in framework/, where we can iterate freely without
breaking user tests.
Sounds good.
The new api/ modules will be stateless, accessing test context via
get_ctx().
Users will import only what they need:
> from api import dts, tg, sut
and use it as module functions
> tg.send_packet(...)
> dts.verify(...)
> testpmd = sut.testpmd_create(...)
Because the modules are stateless and the context is already managed by
the runner, the changes to the existing framework will be minimal.
Initially, we’ll have three modules in api/: dts, tg, and sut. This can
grow as needed.
Makes sense, although I think (and hope) it will not need to grow much. Keeping in mind that we we want the journey from "not knowing DTS" to "having written a testsuite" to be as short as possible.
The goal of this approach is to provide a clean, intuitive developer
experience when doing API calls, with logical namespacing and good IDE
support for discoverability and usability.
Next Steps
Feedback is very welcome — feel free to reply here or on our slack
channel #dts-dev. If you would like to join an upcoming DTS meeting on
Thursday to discuss this live send us and email (or ask on slack) to get
an invite.
After initial discussions I will send a small RFC patch with framework
changes only allowing further comments.
I will follow up with a V1 patch introducing the new API and updating
all existing tests to use it.
Thanks for reading! I'm excited about getting DTS in more hands and hope
this will make it even easier for contributors and users to write
robust, maintainable tests.
Kindest regards,
Paul