Re: [RFC] Proposal for stable developer-friendly DTS API

[Patrick Robb <probb@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 3784 bytes --]

On Tue, Jul 15, 2025 at 2:57 AM Paul Szczepanek <paul.szczepanek@arm.com>
wrote:

> Hi all,
>
> Now that we've written a significant number of tests and developed a
> consistent usage pattern, I think it's a good time to take the next step
> toward project maturity: establishing a stable, well-defined API that
> developers can rely on across releases.
>

Agree


>
> We got some comments from the devs present at the last DPDK summit and
> they are valuable guidance on what direction you'd like the framework to
> go and we're taking them onboard to shape our roadmap. Very much related
> to that we'd appreciate any comments on what you'd like to see in the API.
>
> API proposal goal
>
> Split framework internals from functions intended for use by test writers.
>
> This API would offer:
> - Discoverability of available functionality
> - Clear boundaries between public/test-facing APIs and internal
> framework logic
> - A stable interface for writing and maintaining tests
> - A DPDK-style release cadence and compatibility guarantee (i.e., write
> once, run future-proof)
>

The notes about making the API stable make sense:
1. So, is there a suggestion here that the API can be broken once per year,
and that would be a new policy? Would that happen at a .11 release with
your proposal?
2. I assume I can add functions to a DTS API module, but cannot
remove/change existing ones during a stable period?
3. I guess we can either have reviews and maintainer review enforce the API
stability, or we can develop some checks that will try to enforce the API
stability. Do you have a preference?


>
> With this split, DTS becomes effectively self-documenting — users can
> explore and use only what’s in the api.
>
> Technical Summary
>
> Introduce a dedicated api/ directory that contains a small, focused set
> of modules exposing all the functionality needed to write tests.
>
> All test execution logic and framework internals will remain private and
> continue to live in framework/, where we can iterate freely without
> breaking user tests.
>

Sounds good.


>
> The new api/ modules will be stateless, accessing test context via
> get_ctx().
>
> Users will import only what they need:
>
> > from api import dts, tg, sut
>
> and use it as module functions
>
> > tg.send_packet(...)
> > dts.verify(...)
> > testpmd = sut.testpmd_create(...)
>
> Because the modules are stateless and the context is already managed by
> the runner, the changes to the existing framework will be minimal.
>
> Initially, we’ll have three modules in api/: dts, tg, and sut. This can
> grow as needed.
>

Makes sense, although I think (and hope) it will not need to grow much.
Keeping in mind that we we want the journey from "not knowing DTS" to
"having written a testsuite" to be as short as possible.


>
> The goal of this approach is to provide a clean, intuitive developer
> experience when doing API calls, with logical namespacing and good IDE
> support for discoverability and usability.
>
> Next Steps
>
> Feedback is very welcome — feel free to reply here or on our slack
> channel #dts-dev. If you would like to join an upcoming DTS meeting on
> Thursday to discuss this live send us and email (or ask on slack) to get
> an invite.
>
> After initial discussions I will send a small RFC patch with framework
> changes only allowing further comments.
>
> I will follow up with a V1 patch introducing the new API and updating
> all existing tests to use it.
>
> Thanks for reading! I'm excited about getting DTS in more hands and hope
> this will make it even easier for contributors and users to write
> robust, maintainable tests.
>
> Kindest regards,
> Paul
>

[-- Attachment #2: Type: text/html, Size: 5017 bytes --]