Hi Stephen, it would definitely not be worthwhile to repeat everything that's already tested with testpmd. I was thinking that given that there already is a "flow_parse" function that does almost everything needed, something like that could be exposed. If we think of the testpmd flow string as a sort of "IR" for string flow specification, that would allow others to implement higher-level transform of a schema like JSON or YAML into the testpmd language. Due to the complexity of testpmd and how it's the source of true for testing flows, I think it's too great of an ask to have testpmd support a new type of parsing. My only suggestion would be to take what already exists and expose it in a public API that is included in a DPDK install. 

If you look at the "flow_classify" example in DPDK you can already see that for that application someone had to write another flow text parser for a format they made up. Instead, that example could be converted over to this other API as well.

On Fri, Apr 28, 2023 at 11:09 AM Stephen Hemminger <stephen@networkplumber.org> wrote:
On Fri, 28 Apr 2023 17:36:51 +0000
Tom Barbette <tom.barbette@uclouvain.be> wrote:

> Hi all!
>
> I totally agree with this.
>
> In FastClick we link against a copy of the test-pmd source code to call the parser externally. We just have to patch a bit some files (see https://github.com/tbarbette/fastclick/blob/main/userlevel/rte_parse.mk, and used here : https://github.com/tbarbette/fastclick/blob/main/lib/flowruleparser.cc). It actually worked fairly well until a structure named "template" appeared, which is a registered keyword in C++, and prevent compilation now even under extern "C". This can be patched too but did not find the time yet.
>
> So a clean solution would be more than nice. It's not only the 12K lines of codes, it's also the "testpmd syntax" which is known, and appears in a lot of examples here and there.
>
> Given the relatively easy (but hacky) integration we have, a clean library wouldn't probably be very difficult.
>
>
> Tom
>
> Le 27/04/23 à 15:19, Cliff Burdick a écrit :
> Hi Thomas, testpmd has a 12,000 line parser just for taking in strings and converting it to flow rules. This is obviously useful for testing flows, but it also is an interface for any type of flow imaginable since this is where they're tested.
>
> Now imagine you're developing an application that allows the user to specify custom flows in a config. Your only option is to make your own flow string input (json, etc) and convert that to to the flow spec. This is reinventing almost all of what testpmd already does, and it's extremely error-prone. I think it would be very useful to have this as an API call rather than a user constructing each flow by hand so that all these other applications can benefit and not be worries about bugs during conversions.
>
>
>
> On Thu, Apr 27, 2023, 01:37 Thomas Monjalon <thomas@monjalon.net<mailto:thomas@monjalon.net>> wrote:
> 26/04/2023 07:47, David Marchand:
> > On Wed, Apr 26, 2023 at 6:47 AM Cliff Burdick <shaklee3@gmail.com<mailto:shaklee3@gmail.com>> wrote: 
> > >
> > > Does anyone know if a generic parser for flow strings exists anywhere? The one inside of testpmd is ideal, but unfortunately it's self-contained and not distributed as part of a normal DPDK install. This seems like something that is likely reinvented over and over and it would be useful if there was a single API to take in strings and generate flows. 
> >
> > I heard this same question in the past, but I don't remember the answer.
> > Copying Thomas and Ori who might know. 
>
> I'm not sure how the testpmd code could help another application.
> And in general, if your application has a CLI,
> you need to integrate the flow commands in a broader context.

Exposing the parser for use, would require some renaming of functions, documentation
and a test suite. The testing would be the hardest part.