Currently, DTS tests DPDK via the example applications (/examples) and the apps (/app) included with DPDK. The DTS Working group has been trying to move to primarily using Testpmd, since it is the most complete and only using one application causes less maintenance work. Since Testpmd provides a text interface into DPDK, it must be continually updated to expose this new functionality, sometimes it may lag behind the current feature set of DPDK. For example, Testpmd currently does not support all of the options available for rte flow, which limits the testing coverage that can be provided by DTS for RTE flow. Because DTS is a Python application, it is not able to directly interface with DPDK. This means that whenever Testpmd falls behind DPDK, those unexposed parts of DPDK are functionally untestable as far as DTS is concerned. DTS is the tool used by the CI and labs to functionally test DPDK in a working configuration, compared to the unit testing, that may test individual components or parts of the stack. The DTS improvements WG and the Community Lab have a goal to continue the expansion of the test functional and performance test coverage of DPDK. This means that there must be some way for everything that the DPDK community wants to be tested to be exposed to DTS. I have a few possible solutions to this problem:
Continue to update Testpmd every time new functionality is added to DPDK to expose that functionality
Use a parser generator or some other method to make adding new options to Testpmd much easier, updating this every time new functionality is added to testpmd
Create a dedicated testing application for DPDK that uses a binding generator and Python’s XMLRPC to allow more programmatic access to DPDK on the DUT, adding new RPC calls to expose new functionality.
I personally think that option 3 is the best because it would involve a bit of work up front to make it much easier to expose parts of DPDK to DTS. This is because after the initial work is done, we would just need to write a wrapper function in C to expose the functionality we want, and then expose the wrapper function via RPC.
If you have thoughts or suggestions, we will be discussing this at the DTS working group meeting.