[RFC] Proposal for stable developer-friendly DTS API

[RFC] Proposal for stable developer-friendly DTS API

[Paul Szczepanek]

Hi all,

Now that we've written a significant number of tests and developed a
consistent usage pattern, I think it's a good time to take the next step
toward project maturity: establishing a stable, well-defined API that
developers can rely on across releases.

We got some comments from the devs present at the last DPDK summit and
they are valuable guidance on what direction you'd like the framework to
go and we're taking them onboard to shape our roadmap. Very much related
to that we'd appreciate any comments on what you'd like to see in the API.

API proposal goal

Split framework internals from functions intended for use by test writers.

This API would offer:
- Discoverability of available functionality
- Clear boundaries between public/test-facing APIs and internal
framework logic
- A stable interface for writing and maintaining tests
- A DPDK-style release cadence and compatibility guarantee (i.e., write
once, run future-proof)

With this split, DTS becomes effectively self-documenting — users can
explore and use only what’s in the api.

Technical Summary

Introduce a dedicated api/ directory that contains a small, focused set
of modules exposing all the functionality needed to write tests.

All test execution logic and framework internals will remain private and
continue to live in framework/, where we can iterate freely without
breaking user tests.

The new api/ modules will be stateless, accessing test context via
get_ctx().

Users will import only what they need:

> from api import dts, tg, sut

and use it as module functions

> tg.send_packet(...)
> dts.verify(...)
> testpmd = sut.testpmd_create(...)

Because the modules are stateless and the context is already managed by
the runner, the changes to the existing framework will be minimal.

Initially, we’ll have three modules in api/: dts, tg, and sut. This can
grow as needed.

The goal of this approach is to provide a clean, intuitive developer
experience when doing API calls, with logical namespacing and good IDE
support for discoverability and usability.

Next Steps

Feedback is very welcome — feel free to reply here or on our slack
channel #dts-dev. If you would like to join an upcoming DTS meeting on
Thursday to discuss this live send us and email (or ask on slack) to get
an invite.

After initial discussions I will send a small RFC patch with framework
changes only allowing further comments.

I will follow up with a V1 patch introducing the new API and updating
all existing tests to use it.

Thanks for reading! I'm excited about getting DTS in more hands and hope
this will make it even easier for contributors and users to write
robust, maintainable tests.

Kindest regards,
Paul

Re: [RFC] Proposal for stable developer-friendly DTS API

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 3784 bytes --]

On Tue, Jul 15, 2025 at 2:57 AM Paul Szczepanek <paul.szczepanek@arm.com>
wrote:

> Hi all,
>
> Now that we've written a significant number of tests and developed a
> consistent usage pattern, I think it's a good time to take the next step
> toward project maturity: establishing a stable, well-defined API that
> developers can rely on across releases.
>

Agree


>
> We got some comments from the devs present at the last DPDK summit and
> they are valuable guidance on what direction you'd like the framework to
> go and we're taking them onboard to shape our roadmap. Very much related
> to that we'd appreciate any comments on what you'd like to see in the API.
>
> API proposal goal
>
> Split framework internals from functions intended for use by test writers.
>
> This API would offer:
> - Discoverability of available functionality
> - Clear boundaries between public/test-facing APIs and internal
> framework logic
> - A stable interface for writing and maintaining tests
> - A DPDK-style release cadence and compatibility guarantee (i.e., write
> once, run future-proof)
>

The notes about making the API stable make sense:
1. So, is there a suggestion here that the API can be broken once per year,
and that would be a new policy? Would that happen at a .11 release with
your proposal?
2. I assume I can add functions to a DTS API module, but cannot
remove/change existing ones during a stable period?
3. I guess we can either have reviews and maintainer review enforce the API
stability, or we can develop some checks that will try to enforce the API
stability. Do you have a preference?


>
> With this split, DTS becomes effectively self-documenting — users can
> explore and use only what’s in the api.
>
> Technical Summary
>
> Introduce a dedicated api/ directory that contains a small, focused set
> of modules exposing all the functionality needed to write tests.
>
> All test execution logic and framework internals will remain private and
> continue to live in framework/, where we can iterate freely without
> breaking user tests.
>

Sounds good.


>
> The new api/ modules will be stateless, accessing test context via
> get_ctx().
>
> Users will import only what they need:
>
> > from api import dts, tg, sut
>
> and use it as module functions
>
> > tg.send_packet(...)
> > dts.verify(...)
> > testpmd = sut.testpmd_create(...)
>
> Because the modules are stateless and the context is already managed by
> the runner, the changes to the existing framework will be minimal.
>
> Initially, we’ll have three modules in api/: dts, tg, and sut. This can
> grow as needed.
>

Makes sense, although I think (and hope) it will not need to grow much.
Keeping in mind that we we want the journey from "not knowing DTS" to
"having written a testsuite" to be as short as possible.


>
> The goal of this approach is to provide a clean, intuitive developer
> experience when doing API calls, with logical namespacing and good IDE
> support for discoverability and usability.
>
> Next Steps
>
> Feedback is very welcome — feel free to reply here or on our slack
> channel #dts-dev. If you would like to join an upcoming DTS meeting on
> Thursday to discuss this live send us and email (or ask on slack) to get
> an invite.
>
> After initial discussions I will send a small RFC patch with framework
> changes only allowing further comments.
>
> I will follow up with a V1 patch introducing the new API and updating
> all existing tests to use it.
>
> Thanks for reading! I'm excited about getting DTS in more hands and hope
> this will make it even easier for contributors and users to write
> robust, maintainable tests.
>
> Kindest regards,
> Paul
>

[-- Attachment #2: Type: text/html, Size: 5017 bytes --]