From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4BC4CA0093; Mon, 11 Apr 2022 16:27:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 316DB427ED; Mon, 11 Apr 2022 16:27:58 +0200 (CEST) Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by mails.dpdk.org (Postfix) with ESMTP id BD9BD415D7 for ; Mon, 11 Apr 2022 16:27:56 +0200 (CEST) Received: by mail-io1-f53.google.com with SMTP id y65so2247050iof.1 for ; Mon, 11 Apr 2022 07:27:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=w34R9dBvyxua9IdGCAQcyqAxCRQ8HzIfVfKvSoLmYI4=; b=JqXfNB4VgEsoIdWS63O+j/gUy1kjsURfZrsan6qYGJ7y+HgsnsaR6ryfofen6l7OVr lOES6uJF3DaQ6CE9ud3D4XKm49xh8/NhYnZ1q9xkOWzYMcN4owpOxJmpKo5BfFTCipIs w1KCUo7snHy3QGoXlADfVHEIysm+di4QNpQ7Ru/H/buSdSAa0vW9UvG/ykkfKxW+9FCW f9uqF9cHCin5puqgWZUsYVJJKjf/bpBkmfWb6K9JrvrSU4/s5w0YxBNq6pvLCzF42V8V //2lfgfWk+Vte7XOIw8FBhWQW0PvGrk63yQ3VazHNJBepTd6Wk0ve33JvKBLY4d180wm QsoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=w34R9dBvyxua9IdGCAQcyqAxCRQ8HzIfVfKvSoLmYI4=; b=fUShNXMAKViIyvG+de5Yv8gQeenks9F3gaUR5MNeIv6zGKM3+cBtqp+BAC9rxqFNSk QLyJs7r4v36WGDwBIcCsOct9Vub8ZnBUNtlHf8TO56X7sZFMPyuUHfGU/ut0GnSzQxIz +fJvHsC/JUolBTZdPTf8iOEd2nimSpzt20aDQisZC+2UtQ6dUKSRa4LyKwwQoAuWoG6I DKZo8N8H3R5wBjFD9USFyYlz6n8AKarFIJNzWAXB5XWqK+Vq63m7e67/Ap14ILTZNo8t 3dGVNhCGtmWmdA1aJE60zpDoDDxOL5VrA/d7x5yLq50ozTEoVJwLFN0gRLvLo0WJ+wSC XEhQ== X-Gm-Message-State: AOAM531D0M0ZMcxIFl7AJPD8VDwxY5jo+hCl1Jd37frNOjESGT/ZC6Jv +rAeKOg0f6hw7TjcQJhpsrED45LOdFFiPHOg+NU= X-Google-Smtp-Source: ABdhPJxBryiSzPc2TwwaRyS8Tm2j2P/ulJVD7cJcYxjS/dGBRegYA4xyp1B005VRj4rFRuF8T4KpDeLua5cIvv8dn1s= X-Received: by 2002:a05:6602:13d4:b0:649:934f:e957 with SMTP id o20-20020a05660213d400b00649934fe957mr13352247iov.25.1649687275739; Mon, 11 Apr 2022 07:27:55 -0700 (PDT) MIME-Version: 1.0 References: <20220407214707.29730-1-ohilyard@iol.unh.edu> In-Reply-To: <20220407214707.29730-1-ohilyard@iol.unh.edu> From: Jerin Jacob Date: Mon, 11 Apr 2022 19:57:29 +0530 Message-ID: Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API To: ohilyard@iol.unh.edu Cc: dpdk-dev , Honnappa Nagarahalli , Thomas Monjalon Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Fri, Apr 8, 2022 at 3:17 AM wrote: > > From: Owen Hilyard > > Currently, DTS uses Testpmd for most of its testing. This has been su= ccessful in reducing the need to create more test apps, but it has a few dr= awbacks. First, if some part of DPDK is not exposed via Testpmd or one of t= he example applications, for the purposes of DTS it is not testable. This i= s a situation I=E2=80=99d like to avoid. However, adding new functionality = to Testpmd is labor-intensive. Testpmd currently uses a hand-written LL(1) = parser (https://en.wikipedia.org/wiki/LL_parser) to parse command line opti= ons. This makes adding new functionality difficult since the parser is stor= ed as a series of several thousand line long lookup tables. To look at it a= nother way, 64% of the 52238 lines in Testpmd are related to command line i= nput in some way. The command line interface of testpmd also presents sever= al challenges for the underlying implementation, since it requires that eve= rything a user might want to reference is identified via something that is = reasonable to ask a user to type. As of right now, this is handled via eith= er strings or integers. This can be handled by creating a global registry f= or objects, but it is still extra work that I think can be avoided. In addi= tion, this leads to more places where things can go wrong. > > This is what DTS running a single command in testpmd looks like right now= : > https://drive.google.com/file/d/1hvTcjfVdh8-I3CUNoq6bx82EuNQSK6qW/view?us= p=3Dsharing > > This approach has a number of disadvantages. First, it requires assem= bling all commands as strings inside of the test suite and sending them thr= ough a full round trip of SSH. This means that any non-trivial command, suc= h as creating an RTE flow, will involve a lot of string templating. This no= rmally wouldn=E2=80=99t be a big issue, except that some of the test suites= are designed to hundreds of commands over the course of a test, paying the= cost of an SSH round trip for each. Once Testpmd has the commands, it will= then call the appropriate functions inside of DPDK, and then print out all= of the state to standard out. All of this is sent back to DTS, where the a= uthor of the test case then needs to handle all possible outputs of Trex, o= ften by either declaring the presence of a single word or short phrase in t= he output as meaning success or failure. In my opinion, this is something t= hat is perfectly fine for humans to interact with, but it causes a lot of i= ssues with automation due to its inherent inflexibility and the less-than-i= deal methods of information transfer. This is why I am proposing the creati= on of an automation-oriented pmd, with a focus on exposing as much. > > https://drive.google.com/file/d/1wj4-RnFPVERCzM8b68VJswAOEI9cg-X8/view?us= p=3Dsharing > > That diagram is a high-level overview of the design, which explic= itly excludes implementation details. However, it already has some benefits= . First, making DPDK do something is a normal method call, instead of needi= ng to format things into a string. This provides a much better interface fo= r people working in both DTS and DPDK. Second, the ability to return struct= ured data means that there won=E2=80=99t be parsers on both sides of commun= ication anymore. Structured data also allows much more verbosity, since it = is no longer an interface designed for humans. If a test case author needs = to return the bytes of every received packet back to DTS for comparison wit= h the expected value, they can. If you need to return a pointer for DTS to = use later, that becomes reasonable. Simply moving to shuffling structured d= ata around and using RPC already provides a lot of benefits. > The next obvious question would be what to use for the implementa= tion. The initial attempt was made using Python on both sides and the stand= ard library xmlrpc module. The RPC aspect of this approach worked very well= , with the ability to send arbitrary python objects back and forth between = DTS and app. However, having Python interacting with DPDK has a few issues.= First, DPDK is generally very multi-threaded and the most common implement= ation of Python, CPython, does not have concurrency. It has something known= as the global interpretr lock, which is a global mutex. This makes it very= difficult to interact with blocking, multi-threaded code. The other issue = is that I was not able to find a binding generator that I feel would be suf= ficient for DPDK. Many generators assumed sizeof(int) =3D=3D 4 or had other= portability issues such as assuming GCC or Clang as a C compiler. Others f= ocused on some subset of C, meaning they would throw errors on alignment an= notations. > Given this, I decided to look for cross-language RPC libraries. Altho= ugh libraries exist for performing xmlrpc in C, they generally appeared qui= te difficult to use and required a lot of manual work. The next best option= was gRPC. gRPC allows using a simple language, protobuf, with a language e= xtension for rpc. It provides code generation to make it easy to use multip= le languages together, since it was developed to make polyglot microservice= interaction easier. The only drawback is that it considers C++ good enough= for C support. In this case, I was able to easily integrate DPDK with C++,= so that isn=E2=80=99t much of a concern. I used C++17 in the attached patc= hes, but the minimum requirements are C++11. If there is concern about mode= rn C++ causing too much mental overhead, a =E2=80=9CC with classes=E2=80=9D= subset of C++ could easily be used. I also added an on-by-default option t= o use a C++ compiler, allowing anyone who does not have a C++ compiler avai= lable to them to turn off everything that uses C++. This disables the appli= cation written for this RFC. > One of the major benefits of gRPC is the asynchronous API. This allow= s streaming data on both sides of an RPC call. This allows streaming logs b= ack to DTS, streaming large amounts of data from low-memory systems back to= DTS for processing, and would allow DTS to easily feed DPDK data, ending t= he test quickly on a failure. Currently, due to the overhead of sending dat= a to Testpmd, it is common to just send all of the commands over and run ev= erything since that will be much faster when the test passes, but it can co= st a lot of time in the event of a failure. There are also optional securit= y features for requiring authentication before allowing code execution. I t= hink that a discussion on whether using them for DTS is necessary is warran= ted, although I personally think that it=E2=80=99s not worth the effort giv= en the type of environment this sample application is expected to run in. > For this RFC, I ported test-acl because it was mostly self-contained = and was something I could run on my laptop. It should be fairly easy to see= how you would expand this proof of concept to cover more of DPDK, and I th= ink that most of the functions currently used in testpmd could be ported ov= er to this approach, saving a lot of development time. However, I would lik= e to see some more interest before I take on a task like that. This will re= quire a lot of work on the DTS side to implement, but it will make it much = easier to add new features to DTS. Thanks, Owen for POC. In my view, May using this scheme is probably over-engineered. The reason for thinking so is -Now that, Test code is also part of DPDK, Exposing as services may not be required. -Now in DPDK, we have two types of existing test cases to verify the API -- Noninteractive - These test cases, can simply run over ssh with bash invocation and return the test from a remote PC, -- Interactive - Testpmd one, I believe, Feeding stdin programmatically would suffice to test all the combinations. -We need to add all test cases in this model and we need to maintain two sets of programs.(Traditional tests and gRPC model-based tests). Just my 2c. > > Owen Hilyard (4): > app/test-pmd-api: Add C++ Compiler > app/test-pmd-api: Add POC with gRPC deps > app/test-pmd-api: Add protobuf file > app/test-pmd-api: Implementation files for the API > > app/meson.build | 17 + > app/test-pmd-api/api.proto | 12 + > app/test-pmd-api/api_impl.cc | 1160 ++++++++++++++++++++++++++++++++++ > app/test-pmd-api/api_impl.h | 10 + > app/test-pmd-api/main.c | 11 + > app/test-pmd-api/meson.build | 96 +++ > meson.build | 3 + > meson_options.txt | 2 + > 8 files changed, 1311 insertions(+) > create mode 100644 app/test-pmd-api/api.proto > create mode 100644 app/test-pmd-api/api_impl.cc > create mode 100644 app/test-pmd-api/api_impl.h > create mode 100644 app/test-pmd-api/main.c > create mode 100644 app/test-pmd-api/meson.build > > -- > 2.30.2 >