From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 956AA429EE for ; Fri, 28 Apr 2023 21:13:40 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0DA3C406B5; Fri, 28 Apr 2023 21:13:40 +0200 (CEST) Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by mails.dpdk.org (Postfix) with ESMTP id 4A5B840698 for ; Fri, 28 Apr 2023 21:13:39 +0200 (CEST) Received: by mail-io1-f53.google.com with SMTP id ca18e2360f4ac-76655852953so5357839f.0 for ; Fri, 28 Apr 2023 12:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682709218; x=1685301218; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WH3z6eLI5YDuByZgVq3b8Kqy1w0/a/ejhnp4MeynYOk=; b=qyMhNlDS4OdzuCoPH7NfpMXWzcqwNqul2fSgvnZFkEODZmwQLR4YopQw+Q+iP+r8e4 D4b19ZmKMMPm6xqT+bXB4PKcRPTLetiHqgg990Q299vukUN1JD/Gh/Q3Q8H5jaouJ9Rc yWjgmSsO7jLIC5QqUPZNvFVbG3r5pVQdShh8nq5AEM5zKx5n8TCFAIN547yOrCyFdlkd VKWsJFVK47oHuB1gwIkK7geCa4uxfm2grp8iLdtkuHWYG1Ckpd9t4v7mNKERT/XrbfJM lQ2+LrfyvZdN3mo75AO6iymQeDkd0nw7Nigfn3bk9XeWe1Kh+o+JhskplKc3JQOAqooS W5SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682709218; x=1685301218; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WH3z6eLI5YDuByZgVq3b8Kqy1w0/a/ejhnp4MeynYOk=; b=BDpLJYCndW7nr2lcbediEVAyr8Mv9I649a43bmdP5Odi0Ukk+QseuSPuhLr+9f7bMd DOSFoN5D9RbTAlzMpuzI0SA1z9uj0CBTSSNK1neNikCZIqt4S+SG3Ye4YqvqVoD/0HyD JhG9zi65t12iMZWiGZBTQUPEiy2mWdn8uCEe/ogBbdLoVynu/MBVWUVrCoSsSle0E/fG EtUzxSJt8vlhW4OeOf1l7IYf7h0ZRsl+4CU6qRyusLQg6NAkEIDlyDqaIgMcJvVjCM2W PnBt769DJqvYv4d6oNkaNhD4M4wGFmpCk5n7KiAplYb18pbk9TmQk0rE/5vgouoFfDko Qxuw== X-Gm-Message-State: AC+VfDyByfL2YBW1hpHi156KVmf7gSFTRw06he6De2LN3QXLqkIcdffD Lf8L1Qv1TbX7mn+RHUNGTVrKV1j0QjlvKYi+oww= X-Google-Smtp-Source: ACHHUZ4JLl+Y6/QhltIWBFNEbBH37WI9hZNBhWGhvLAc9JkBU2SzF85IzAr+80i10YZW4Wo/Q/RcKN001Ib3u42aZCU= X-Received: by 2002:a6b:d119:0:b0:760:a3c7:1a86 with SMTP id l25-20020a6bd119000000b00760a3c71a86mr3701174iob.16.1682709218398; Fri, 28 Apr 2023 12:13:38 -0700 (PDT) MIME-Version: 1.0 References: <1814734.QZUTf85G27@thomas> <771fa709-563f-1ec2-a997-084b7709edce@uclouvain.be> <20230428110935.00311cd8@hermes.local> In-Reply-To: <20230428110935.00311cd8@hermes.local> From: Cliff Burdick Date: Fri, 28 Apr 2023 12:13:26 -0700 Message-ID: Subject: Re: Generic flow string parser To: Stephen Hemminger Cc: Tom Barbette , Thomas Monjalon , David Marchand , users , Ori Kam Content-Type: multipart/alternative; boundary="000000000000391a4805fa6a457d" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000391a4805fa6a457d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Stephen, it would definitely not be worthwhile to repeat everything that's already tested with testpmd. I was thinking that given that there already is a "flow_parse" function that does almost everything needed, something like that could be exposed. If we think of the testpmd flow string as a sort of "IR" for string flow specification, that would allow others to implement higher-level transform of a schema like JSON or YAML into the testpmd language. Due to the complexity of testpmd and how it's the source of true for testing flows, I think it's too great of an ask to have testpmd support a new type of parsing. My only suggestion would be to take what already exists and expose it in a public API that is included in a DPDK install. If you look at the "flow_classify" example in DPDK you can already see that for that application someone had to write another flow text parser for a format they made up. Instead, that example could be converted over to this other API as well. On Fri, Apr 28, 2023 at 11:09=E2=80=AFAM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Fri, 28 Apr 2023 17:36:51 +0000 > Tom Barbette wrote: > > > Hi all! > > > > I totally agree with this. > > > > In FastClick we link against a copy of the test-pmd source code to call > the parser externally. We just have to patch a bit some files (see > https://github.com/tbarbette/fastclick/blob/main/userlevel/rte_parse.mk, > and used here : > https://github.com/tbarbette/fastclick/blob/main/lib/flowruleparser.cc). > It actually worked fairly well until a structure named "template" appeare= d, > which is a registered keyword in C++, and prevent compilation now even > under extern "C". This can be patched too but did not find the time yet. > > > > So a clean solution would be more than nice. It's not only the 12K line= s > of codes, it's also the "testpmd syntax" which is known, and appears in a > lot of examples here and there. > > > > Given the relatively easy (but hacky) integration we have, a clean > library wouldn't probably be very difficult. > > > > > > Tom > > > > Le 27/04/23 =C3=A0 15:19, Cliff Burdick a =C3=A9crit : > > Hi Thomas, testpmd has a 12,000 line parser just for taking in strings > and converting it to flow rules. This is obviously useful for testing > flows, but it also is an interface for any type of flow imaginable since > this is where they're tested. > > > > Now imagine you're developing an application that allows the user to > specify custom flows in a config. Your only option is to make your own fl= ow > string input (json, etc) and convert that to to the flow spec. This is > reinventing almost all of what testpmd already does, and it's extremely > error-prone. I think it would be very useful to have this as an API call > rather than a user constructing each flow by hand so that all these other > applications can benefit and not be worries about bugs during conversions= . > > > > > > > > On Thu, Apr 27, 2023, 01:37 Thomas Monjalon thomas@monjalon.net>> wrote: > > 26/04/2023 07:47, David Marchand: > > > On Wed, Apr 26, 2023 at 6:47=E2=80=AFAM Cliff Burdick > wrote: > > > > > > > > Does anyone know if a generic parser for flow strings exists > anywhere? The one inside of testpmd is ideal, but unfortunately it's > self-contained and not distributed as part of a normal DPDK install. This > seems like something that is likely reinvented over and over and it would > be useful if there was a single API to take in strings and generate flows= . > > > > > > I heard this same question in the past, but I don't remember the > answer. > > > Copying Thomas and Ori who might know. > > > > I'm not sure how the testpmd code could help another application. > > And in general, if your application has a CLI, > > you need to integrate the flow commands in a broader context. > > Exposing the parser for use, would require some renaming of functions, > documentation > and a test suite. The testing would be the hardest part. > --000000000000391a4805fa6a457d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stephen, it would definitely not be worthwhile to repea= t everything that's already tested with testpmd. I was thinking that gi= ven that there already is a "flow_parse" function that does almos= t everything needed, something like that could be exposed. If we think of t= he testpmd flow string as a sort of "IR" for string flow specific= ation, that would allow others to implement higher-level transform of a sch= ema like JSON or YAML into the testpmd language. Due to the complexity of t= estpmd and how it's the source of true for testing flows, I think it= 9;s too great of an ask to have testpmd support a new type=C2=A0of parsing.= My only suggestion would be to take what already exists and expose it in a= public API that is included in a DPDK install.=C2=A0

If= you look at the "flow_classify" example in DPDK you can already = see that for that application someone had to write another flow text parser= for a format they made up. Instead, that example could be converted over t= o this other API as well.

On Fri, Apr 28, 2023 at 11:09=E2=80=AFAM Ste= phen Hemminger <stephen@ne= tworkplumber.org> wrote:
On Fri, 28 Apr 2023 17:36:51 +0000
Tom Barbette <tom.barbette@uclouvain.be> wrote:

> Hi all!
>
> I totally agree with this.
>
> In FastClick we link against a copy of the test-pmd source code to cal= l the parser externally. We just have to patch a bit some files (see https://github.com/tbarbette/fastcl= ick/blob/main/userlevel/rte_parse.mk, and used here : https://github.com/tbarbette/fastclick/blob/mai= n/lib/flowruleparser.cc). It actually worked fairly well until a struct= ure named "template" appeared, which is a registered keyword in C= ++, and prevent compilation now even under extern "C". This can b= e patched too but did not find the time yet.
>
> So a clean solution would be more than nice. It's not only the 12K= lines of codes, it's also the "testpmd syntax" which is know= n, and appears in a lot of examples here and there.
>
> Given the relatively easy (but hacky) integration we have, a clean lib= rary wouldn't probably be very difficult.
>
>
> Tom
>
> Le 27/04/23 =C3=A0 15:19, Cliff Burdick a =C3=A9crit :
> Hi Thomas, testpmd has a 12,000 line parser just for taking in strings= and converting it to flow rules. This is obviously useful for testing flow= s, but it also is an interface for any type of flow imaginable since this i= s where they're tested.
>
> Now imagine you're developing an application that allows the user = to specify custom flows in a config. Your only option is to make your own f= low string input (json, etc) and convert that to to the flow spec. This is = reinventing almost all of what testpmd already does, and it's extremely= error-prone. I think it would be very useful to have this as an API call r= ather than a user constructing each flow by hand so that all these other ap= plications can benefit and not be worries about bugs during conversions. >
>
>
> On Thu, Apr 27, 2023, 01:37 Thomas Monjalon <thomas@monjalon.net<mailto:thomas@monjalon.net&= gt;> wrote:
> 26/04/2023 07:47, David Marchand:
> > On Wed, Apr 26, 2023 at 6:47=E2=80=AFAM Cliff Burdick <shaklee3@gmail.com<= mailto:shaklee3@gma= il.com>> wrote:=C2=A0
> > >
> > > Does anyone know if a generic parser for flow strings exists= anywhere? The one inside of testpmd is ideal, but unfortunately it's s= elf-contained and not distributed as part of a normal DPDK install. This se= ems like something that is likely reinvented over and over and it would be = useful if there was a single API to take in strings and generate flows.=C2= =A0
> >
> > I heard this same question in the past, but I don't remember = the answer.
> > Copying Thomas and Ori who might know.=C2=A0
>
> I'm not sure how the testpmd code could help another application.<= br> > And in general, if your application has a CLI,
> you need to integrate the flow commands in a broader context.

Exposing the parser for use, would require some renaming of functions, docu= mentation
and a test suite. The testing would be the hardest part.
--000000000000391a4805fa6a457d--