From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dts-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 7A5E0A034E;
Wed, 9 Feb 2022 16:34:15 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 95E94426DF;
Wed, 9 Feb 2022 16:34:14 +0100 (CET)
Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com
[209.85.210.54]) by mails.dpdk.org (Postfix) with ESMTP id F200F41143
for <dts@dpdk.org>; Wed, 9 Feb 2022 16:34:11 +0100 (CET)
Received: by mail-ot1-f54.google.com with SMTP id
w27-20020a9d5a9b000000b005a17d68ae89so1741371oth.12
for <dts@dpdk.org>; Wed, 09 Feb 2022 07:34:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=GOBmHbPM+qW5nRTXdeO0ctX968Ez9ZhMj61bEDIczfg=;
b=MR+qtN0vyhnqCeSv6VPO8JU0Hy8A6FanB+xMLQstP441T7h/EoZNNVd5lqydP5PE0R
+/cjI/rufMXvSyemF9LLrrn5RElT42roXXQgnkgI4Kmppk/Nj+Xe/9LAz+QRwxy3ozil
U0uAOLQf5yV93vjzXI8dWAiii+v6iAYqkbdj8=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=GOBmHbPM+qW5nRTXdeO0ctX968Ez9ZhMj61bEDIczfg=;
b=Gx58pE1Q7pTjt/R/Dr6mg6PN7OjxVzCuUxPmcQh9DXX7LiDESvvkLSISyelwZlEQ2y
KUR33uF+y1EXElYg4OfAvVlxHm6YPjgJd0pb2cCBB0ENprYqI3itgusAvYYGPz06yyHy
gMREh371wQhEzpAQviOZQDhTFnEpyH6EkpVQ/w6VXHuG9dc5665FKmj2aS3GdZWNwJom
ihAX5kY2G5REubZsYNoGRmQfPHDCRqcbit3rd7FalqjRtt3xnzUWyC65uAbUt47Y1J3l
QDujXnpPqIP6NomerYaASjjwfaYut7wHp7253TtUEQPaXsleBjKtqHbPFaSy6ALjJOIw
ZeTw==
X-Gm-Message-State: AOAM533+5Xt02dfuMitq5/a9RYXAerLUBrck/J+Zd0LWtXmzd+t5PBtp
4HWcAGevdfJvGbF2pSMIul/8pN8Vo7QqHpU1+qL+dA==
X-Google-Smtp-Source: ABdhPJwMROwXyMEPnz4UFAW3HUlB1bq2DPMTXiexLY7/oTuDh/76kKep+wQla1CzEyRRUPigDNGYnqKGnhFIb+WfQas=
X-Received: by 2002:a05:6830:310b:: with SMTP id
b11mr1117816ots.322.1644420850928;
Wed, 09 Feb 2022 07:34:10 -0800 (PST)
MIME-Version: 1.0
References: <CAHx6DYBFV_rO6BafdrFNgyN5YHA78fg83MDw625qptTJDiWCiA@mail.gmail.com>
<5238984.Sb9uPGUboI@thomas>
In-Reply-To: <5238984.Sb9uPGUboI@thomas>
From: Owen Hilyard <ohilyard@iol.unh.edu>
Date: Wed, 9 Feb 2022 10:33:35 -0500
Message-ID: <CAHx6DYBXnGo8Ze1KR1838U7AN0Gj3vEERVepxi8C0WaursgtfQ@mail.gmail.com>
Subject: Re: [RFC] Testpmd Maintenance affecting DTS
To: Thomas Monjalon <thomas@monjalon.net>
Cc: dts@dpdk.org, dev <dev@dpdk.org>, ci@dpdk.org, rasland@nvidia.com
Content-Type: multipart/alternative; boundary="000000000000ae644d05d797906f"
X-BeenThere: dts@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: test suite reviews and discussions <dts.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dts>,
<mailto:dts-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dts/>
List-Post: <mailto:dts@dpdk.org>
List-Help: <mailto:dts-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dts>,
<mailto:dts-request@dpdk.org?subject=subscribe>
Errors-To: dts-bounces@dpdk.org
--000000000000ae644d05d797906f
Content-Type: text/plain; charset="UTF-8"
>
> Yes, any feature in ethdev should have a testpmd entry.
> Which part of rte_flow is not testable with testpmd?
> In general, any device API should be testable with the app/ directory.
There are currently a lot of fields that appear in the documentation for
rte_flow that are not exposed in testpmd. For example, 5/9 fields
in ARP_ETH_IPV4 are not exposed via testpmd. I could compile a full list if
you want me to.
Please could you elaborate how [a parser generator] would work?
Instead of the current approach of hand-rolling a parser, something like
Yacc (Yet Another Compiler Compiler) or Bison (GNU Yacc) could be used to
make adding new items much easier. Although they are both designed to aid
in creating compilers, they also work very well as parsers which output a
parsed structure. When I used Bison in the past to write an AWK
implementation, what I did is have it return a struct. In the case of
testpmd, an array of a tagged union might be easier to handle, so you'd get
something like this back:
[
{
type: ITEM_ETH,
inner: {
eth: {
dst_is_present: true,
dst: "00:00:5e:00:53:af",
src_is_present: true,
src: "00:00:5e:00:53:af",
type_is_present: false,
type: 0
}
}
},
{
type: ITEM_IPV4,
inner: {
ipv4: {
dst_is_present: true,
dst: "127.0.0.1",
src_is_present: true,
src: "127.0.0.1",
}
}
}
]
Once you have the tagged union and the inner structs set up, adding a new
field or item should be fairly easy. Also, the <field>_is_present could be
replaced with specified "not present" values (NULL for strings, etc), this
is just emulating the Option/Optional type from higher-level languages more
directly. The process for adding a new field would involve first adding it
to the bison grammar, then adding the field to the output struct.
Why do you need RPC when a binding is generated?
I guess that this is two ideas. First, have C expose an XMLRPC api. How
hard this would be to do would depend greatly on the availability and
quality of libraries, and it would add a fairly large dependency to this
proposed app. The other idea is to generate Python bindings and then use
the RPC server built into Python. This might make adding the RPC layer
easier, and it would mean that we could more easily send Python types to
the DUT. I think that the second one might be easier to work with, if a bit
slower.
performance is not good
All of the performance-sensitive things, like packet generation, would
still be happening entirely in C code. Also, the last time I profiled DTS
it spent most of its time inside of the SSH library, so moving to Python
RPC API may still increase performance, although not as much as a C RPC
API. A good chunk of the rest of the time was spent either in timed
performance tests (nic_single_core_perf) or waiting for testpmd to start up.
> it is testing a binding, not the real API
Python was built to be glue code for C libraries, so it has a very mature
ecosystem in that area. If we use well-tested binding generators, then most
of the underlying behavior would still be DPDK. As is, DTS doesn't really
test DPDK directly, it tests testpmd, and assumes that testpmd interacts
with DPDK correctly. Making a Python RPC API or Python Bindings would mean
moving from a human-oriented API wrapper to a machine-oriented one.
> it is a huge maintenance effort for everybody
I am not advocating hand-writing Python bindings, that would be way too
much work. I am advocating using a binding generator like SWIG to consume
high-level DPDK header files and create python modules from them. Once this
is set up, it should only need occasional maintenance due to build system
changes. It might still require some work to plug new stuff into the RPC
framework, but that would probably be about the same amount of work as
adding the features to testpmd.
I am also open to using languages other than Python, it's just that C
strikes me as a bad choice for building something like this. It's possible
that we could make use of gRPC or something similar that is designed to
perform cross-language RPC easily, in which case directly exposing RPC in C
might be much easier.
Owen Hilyard
--000000000000ae644d05d797906f
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div dir=3D"ltr"><blockquote class=3D"gmail_quote" style=
=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding=
-left:1ex">Yes, any feature in ethdev should have a testpmd entry.<br>Which=
part of rte_flow is not testable with testpmd?</blockquote><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><br></blockquote><blockquote class=3D"gm=
ail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,=
204,204);padding-left:1ex">In general, any device API should be testable wi=
th the app/ directory.</blockquote><div><br></div><div>=C2=A0There are curr=
ently a lot of fields that appear in the documentation for rte_flow that ar=
e not exposed in testpmd. For example, 5/9 fields in=C2=A0ARP_ETH_IPV4 are =
not exposed via testpmd. I could compile a full list if you want me to.=C2=
=A0</div><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0=
px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">P=
lease could you elaborate how [a parser generator] would work?</blockquote>=
<div><br></div><div>=C2=A0Instead of the current approach of hand-rolling a=
parser, something like Yacc (Yet Another Compiler Compiler) or Bison (GNU =
Yacc) could be used to make adding new items much easier. Although they are=
both designed to aid in creating compilers, they also work very well as pa=
rsers which output a parsed structure. When I used Bison in the past to wri=
te an AWK implementation, what I did is have it return a struct. In the cas=
e of testpmd, an array of a tagged union might be easier to handle, so you&=
#39;d get something like this back:<br></div><div><font face=3D"monospace">=
[<br>=C2=A0 {<br>=C2=A0 =C2=A0 type: ITEM_ETH,<br>=C2=A0 =C2=A0 inner: {<br=
>=C2=A0 =C2=A0 =C2=A0 eth: {<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 dst_is_present:=
true,<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 dst: "00:00:5e:00:53:af",<b=
r>=C2=A0 =C2=A0 =C2=A0 =C2=A0 src_is_present: true,<br>=C2=A0 =C2=A0 =C2=A0=
=C2=A0 src: "00:00:5e:00:53:af",<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
type_is_present: false,<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 type: 0<br>=C2=A0 =
=C2=A0 =C2=A0 }<br>=C2=A0 =C2=A0 }<br>=C2=A0 },<br>=C2=A0 {<br>=C2=A0 =C2=
=A0 type: ITEM_IPV4,<br>=C2=A0 =C2=A0 inner: {<br>=C2=A0 =C2=A0 =C2=A0 ipv4=
: {<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 dst_is_present: true,<br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 dst: "127.0.0.1",<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 sr=
c_is_present: true,<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 src: "127.0.0.1&quo=
t;,<br>=C2=A0 =C2=A0 =C2=A0 }<br>=C2=A0 =C2=A0 }<br>=C2=A0 }<br>]<br><br></=
font></div>Once you have the tagged union and the inner structs set up, add=
ing a new field or item should be fairly easy. Also, the <field>_is_p=
resent could be replaced with specified "not present" values (NUL=
L for strings, etc), this is just emulating the Option/Optional type from h=
igher-level languages more directly. The process for adding a new field wou=
ld involve first adding it to the bison grammar, then adding the field to t=
he output struct.=C2=A0</div><div dir=3D"ltr"><br></div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">Why do you need RPC when a binding is gene=
rated?</blockquote><div><br></div><div>I guess that this is=C2=A0two ideas.=
First, have C expose an XMLRPC api. How hard this would be to do would dep=
end greatly on the availability and quality of libraries, and it would add =
a fairly large dependency to this proposed app. The other idea is to genera=
te Python bindings and then use the RPC server built into Python. This migh=
t make adding the RPC layer easier, and it would mean that we could more ea=
sily send Python types to the DUT. I think that the second one might be eas=
ier to work with, if a bit slower.=C2=A0</div><div><br></div><div><blockquo=
te class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex">performance is not good</blockquot=
e><div><br></div><div>All of the performance-sensitive things, like packet =
generation, would still be happening entirely in C code. Also, the last tim=
e I profiled DTS it spent most of its time inside of the SSH library, so mo=
ving to Python RPC API may still increase performance, although not as much=
as a C RPC API. A good chunk of the rest of the time was spent either in t=
imed performance tests (nic_single_core_perf) or waiting for testpmd to sta=
rt up.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1=
ex">it is testing a binding, not the real API</blockquote><div><br></div><d=
iv>Python was built to be glue code for C libraries, so it has a very matur=
e ecosystem in that area. If we use well-tested binding generators, then mo=
st of the underlying behavior would still be DPDK. As is, DTS doesn't r=
eally test DPDK directly, it tests testpmd, and assumes that testpmd intera=
cts with DPDK correctly. Making a Python RPC API or Python Bindings would m=
ean moving from a human-oriented API wrapper to a machine-oriented one.=C2=
=A0</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin=
:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"=
>it is a huge maintenance effort for everybody</blockquote><div><br></div><=
div>I am not advocating hand-writing Python bindings, that would be way too=
much work. I am advocating using a binding generator like SWIG to consume =
high-level DPDK header files and create python modules from them. Once this=
is set up, it should only need occasional maintenance due to build system =
changes. It might still require some work to plug new stuff into the RPC fr=
amework, but that would probably be about the same amount of work as adding=
the features to testpmd.=C2=A0</div></div><div><br></div><div>I am also op=
en to using languages other than Python, it's just that C strikes me as=
a bad choice for building something like this. It's possible that we c=
ould make use of gRPC or something similar that is designed to perform cros=
s-language RPC easily, in which case directly exposing RPC in C might be mu=
ch easier.=C2=A0</div><div><br></div><div>Owen Hilyard</div></div>
--000000000000ae644d05d797906f--