Re: [PATCH v1 0/4] [RFC] Testpmd RPC API

[Owen Hilyard <ohilyard@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 4673 bytes --]

>
> scheme is probably over-engineered


I tried my hardest to keep this as simple as possible. The requirements
imposed by DTS being a distributed system in Python restricted what I could
do a lot. Needing to be compatible with DPDK's license also got rid of a
lot of options. Binding generators are made for simple projects, and DPDK
is not a simple project. There were some other options related to choice in
the RPC framework, but very few RPC protocols seem to work well with C and
be usable from Python, which is why I ended up using C++ with gRPC. Most of
the code in api_impl.cc is taken from /app/test-acl/main.c, and the new
part is mostly the C++ class at the bottom. Overall, this proposal comes
out to ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and
100 lines of Meson. gRPC may be able to do a lot more than I need it to for
the proof of concept, but many of the features that are not used, like
bi-directional streaming, become very useful in writing more complicated
tests. Overall, this solution is probably more capable than we need it to
be, but I think that those extra capabilities don't come at a very large
cost.


> Now that, Test code is also part of DPDK.
>

DTS is pure python. I tried to use FFI to call directly into DPDK from
Python and then use xmlrpc from the python standard library. As mentioned
in the writeup, I couldn't find a binding generator that would properly
handle DPDK's allocators, which made it so that anything passed to DPDK
using python was allocated using the system malloc. I don't think it is
wise to attempt to programmatically re-write the generated code to allow
for custom allocators. The original reason for needing to have DTS and DPDK
in the same repository was so that tests could be committed and run
alongside the feature patch.

Interactive - Testpmd one, I believe, Feeding stdin programmatically would
> suffice to test all the combinations.
>

One of the issues this is trying to address is that human-readable strings
are a poor way to pass complex information between two programs. DTS is a
distributed system, and it can have up to 3 physical servers involved in
any given test. This means that it's not stdin via a pipe, it's an entire
SSH session. This adds a noticeable amount of overhead when trying to send
and verify the result of sending 1,000+ packets, since the lack of
structured output means each packet must be checked before the next can be
sent. This might be solvable by adding a structured output mode to testpmd,
but that would involve committing to writing output twice for every
function in testpmd forever.

We need to add all test cases in this model and we need to maintain two
> sets of programs.(Traditional tests and gRPC model-based tests).
>

Assuming by traditional tests you mean the unit tests run by Meson, I would
argue that we are already maintaining 2 kinds of tests. The unit tests, and
the python-based DTS tests. My intention is to create a thin wrapper around
DPDK that would be exposed via gRPC, like you see here, and use that as
midware. Then, we would have two front-ends. Testpmd, which takes text and
then calls midware as it does now, and the gRPC frontend, which parses
messages from the RPC server and runs the midware. This would enable
testpmd to still be used to sanity check a DPDK installation, but we would
not need to continually expand Testpmd. The primary issue is that, right
now, anything not included in Testpmd is not really testable by DTS. This
includes portions of the RTE Flow API, which was part of my reason for
proposing this. The RTE Flow API would, in my estimation, if fully
implemented into Testpmd, probably add at least another 10,000 lines of
code. As mentioned in my proposal, Testpmd already does more parsing and
lexing than it does interaction with DPDK by line count. Also, since I am
proposing making this a separate application, we would be able to gradually
migrate the tests inside of DTS. This would have no effect on anything
except for Testpmd, the new application and the addition of a flag to
toggle the use of a C++ compiler.

I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses
classes to model services, but for this usecase we are essentially using it
to transfer function arguments across the internet and then pass the return
value back. Any RPC framework would function similarly if I ignored the
restrictions of which languages to use, and the choice is not important to
how tests are conducted. Put another way, how you write a test for DTS will
not change much if you are using this or testpmd, it's just how you
transfer data and get it back that I want to change.

[-- Attachment #2: Type: text/html, Size: 5463 bytes --]