From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 95DA7A050C; Thu, 7 Apr 2022 23:47:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3473E4068B; Thu, 7 Apr 2022 23:47:29 +0200 (CEST) Received: from mail-oa1-f98.google.com (mail-oa1-f98.google.com [209.85.160.98]) by mails.dpdk.org (Postfix) with ESMTP id 7743040689 for ; Thu, 7 Apr 2022 23:47:28 +0200 (CEST) Received: by mail-oa1-f98.google.com with SMTP id 586e51a60fabf-df22f50e0cso7872337fac.3 for ; Thu, 07 Apr 2022 14:47:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=49Qsuo66QudCF9+O3yvnpNnqmRwjSC89Ze0SeGDuum8=; b=Hh/BBYr3BtXQtf3gndLDdyzZ+g96MwX39CPLEAGC/U/xN0HLe7DOfj8HGw23loNlPo jyry/1if/C7oNSvqMnhrSOf+Kvh8MCeMXn3TKqD5CLMtTkjs1sz9MqKF8sV8Znm5BlkK AkCX8lsRUxMZUjCAjxPPptazriUKN+Yskn/7s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=49Qsuo66QudCF9+O3yvnpNnqmRwjSC89Ze0SeGDuum8=; b=jGZXgLGrMXzCRwKhRnpxCF7CH73iEAVXcjVKuQxA60mE5HZjIvEzev+8PnTVdF1+/s g8G0cSUbTYfkPMW1rJd2zj9iiHCkU1LGypKugkHN0DI+plcUuaW3qu9CxJLj0ENrmSJV nWi20l+ch/tFLdufdeAF/1ldZKP8VGNj+ShMbtOgfEc5Fzv7FyusiFS50X/YL8Tx7mlc j2Yx3xPmtCx7vFsWctBIgRiFfNgqjdkc+WOh16zwKYDD2CQ4GNkoHtU5PsY201BDQXpl yILqGbhTHPh9SG0xYrp69DVD0hTPU1cuyBJuXSM+0QRNpgNBj8xPhqrMmFXAquQGOjKR Z7TQ== X-Gm-Message-State: AOAM530xlJvOlKJsZt04dBF5qhFnD87P99uApoHVY5B+7TRTt3wgBHAz 8489Q16k39cCDOVk7iG0BmmD655IT7JJLHyINvFRs0mhuBluLUYnj7xuRru1PW2IzA== X-Google-Smtp-Source: ABdhPJwH5ZuhDs+bbAGpMu1cywnAQwLkSyw3ClHylnzUPCGLMB3UUfnPDW+qWLlvPGojWAFL+HA0rznalteT X-Received: by 2002:a05:6870:d20a:b0:e2:182d:47c7 with SMTP id g10-20020a056870d20a00b000e2182d47c7mr7466299oac.244.1649368047769; Thu, 07 Apr 2022 14:47:27 -0700 (PDT) Received: from postal.iol.unh.edu (postal.iol.unh.edu. [2606:4100:3880:1234::84]) by smtp-relay.gmail.com with ESMTPS id g3-20020a056870e1c300b000da371afbe1sm1379586oab.41.2022.04.07.14.47.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Apr 2022 14:47:27 -0700 (PDT) X-Relaying-Domain: iol.unh.edu Received: from iol.unh.edu (unknown [IPv6:2606:4100:3880:1257::105d]) by postal.iol.unh.edu (Postfix) with ESMTP id 310CB6051838; Thu, 7 Apr 2022 17:47:27 -0400 (EDT) From: ohilyard@iol.unh.edu To: dev@dpdk.org Cc: Honnappa.Nagarahalli@arm.com, thomas@monjalon.net, Owen Hilyard Subject: [PATCH v1 0/4] [RFC] Testpmd RPC API Date: Thu, 7 Apr 2022 17:47:04 -0400 Message-Id: <20220407214707.29730-1-ohilyard@iol.unh.edu> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Owen Hilyard Currently, DTS uses Testpmd for most of its testing. This has been succ= essful in reducing the need to create more test apps, but it has a few draw= backs. First, if some part of DPDK is not exposed via Testpmd or one of the= example applications, for the purposes of DTS it is not testable. This is = a situation I=E2=80=99d like to avoid. However, adding new functionality to= Testpmd is labor-intensive. Testpmd currently uses a hand-written LL(1) pa= rser (https://en.wikipedia.org/wiki/LL_parser) to parse command line option= s. This makes adding new functionality difficult since the parser is stored= as a series of several thousand line long lookup tables. To look at it ano= ther way, 64% of the 52238 lines in Testpmd are related to command line inp= ut in some way. The command line interface of testpmd also presents several= challenges for the underlying implementation, since it requires that every= thing a user might want to reference is identified via something that is re= asonable to ask a user to type. As of right now, this is handled via either= strings or integers. This can be handled by creating a global registry for= objects, but it is still extra work that I think can be avoided. In additi= on, this leads to more places where things can go wrong.=20 This is what DTS running a single command in testpmd looks like right now: https://drive.google.com/file/d/1hvTcjfVdh8-I3CUNoq6bx82EuNQSK6qW/view?usp= =3Dsharing This approach has a number of disadvantages. First, it requires assembl= ing all commands as strings inside of the test suite and sending them throu= gh a full round trip of SSH. This means that any non-trivial command, such = as creating an RTE flow, will involve a lot of string templating. This norm= ally wouldn=E2=80=99t be a big issue, except that some of the test suites a= re designed to hundreds of commands over the course of a test, paying the c= ost of an SSH round trip for each. Once Testpmd has the commands, it will t= hen call the appropriate functions inside of DPDK, and then print out all o= f the state to standard out. All of this is sent back to DTS, where the aut= hor of the test case then needs to handle all possible outputs of Trex, oft= en by either declaring the presence of a single word or short phrase in the= output as meaning success or failure. In my opinion, this is something tha= t is perfectly fine for humans to interact with, but it causes a lot of iss= ues with automation due to its inherent inflexibility and the less-than-ide= al methods of information transfer. This is why I am proposing the creation= of an automation-oriented pmd, with a focus on exposing as much. https://drive.google.com/file/d/1wj4-RnFPVERCzM8b68VJswAOEI9cg-X8/view?usp= =3Dsharing=20 That diagram is a high-level overview of the design, which explicitly excl= udes implementation details. However, it already has some benefits. First, = making DPDK do something is a normal method call, instead of needing to for= mat things into a string. This provides a much better interface for people = working in both DTS and DPDK. Second, the ability to return structured data= means that there won=E2=80=99t be parsers on both sides of communication a= nymore. Structured data also allows much more verbosity, since it is no lon= ger an interface designed for humans. If a test case author needs to return= the bytes of every received packet back to DTS for comparison with the exp= ected value, they can. If you need to return a pointer for DTS to use later= , that becomes reasonable. Simply moving to shuffling structured data aroun= d and using RPC already provides a lot of benefits.=20 The next obvious question would be what to use for the implementation. The= initial attempt was made using Python on both sides and the standard libra= ry xmlrpc module. The RPC aspect of this approach worked very well, with th= e ability to send arbitrary python objects back and forth between DTS and a= pp. However, having Python interacting with DPDK has a few issues. First, D= PDK is generally very multi-threaded and the most common implementation of = Python, CPython, does not have concurrency. It has something known as the g= lobal interpretr lock, which is a global mutex. This makes it very difficul= t to interact with blocking, multi-threaded code. The other issue is that I= was not able to find a binding generator that I feel would be sufficient f= or DPDK. Many generators assumed sizeof(int) =3D=3D 4 or had other portabil= ity issues such as assuming GCC or Clang as a C compiler. Others focused on= some subset of C, meaning they would throw errors on alignment annotations= .=20 Given this, I decided to look for cross-language RPC libraries. Althoug= h libraries exist for performing xmlrpc in C, they generally appeared quite= difficult to use and required a lot of manual work. The next best option w= as gRPC. gRPC allows using a simple language, protobuf, with a language ext= ension for rpc. It provides code generation to make it easy to use multiple= languages together, since it was developed to make polyglot microservice i= nteraction easier. The only drawback is that it considers C++ good enough f= or C support. In this case, I was able to easily integrate DPDK with C++, s= o that isn=E2=80=99t much of a concern. I used C++17 in the attached patche= s, but the minimum requirements are C++11. If there is concern about modern= C++ causing too much mental overhead, a =E2=80=9CC with classes=E2=80=9D s= ubset of C++ could easily be used. I also added an on-by-default option to = use a C++ compiler, allowing anyone who does not have a C++ compiler availa= ble to them to turn off everything that uses C++. This disables the applica= tion written for this RFC. One of the major benefits of gRPC is the asynchronous API. This allows = streaming data on both sides of an RPC call. This allows streaming logs bac= k to DTS, streaming large amounts of data from low-memory systems back to D= TS for processing, and would allow DTS to easily feed DPDK data, ending the= test quickly on a failure. Currently, due to the overhead of sending data = to Testpmd, it is common to just send all of the commands over and run ever= ything since that will be much faster when the test passes, but it can cost= a lot of time in the event of a failure. There are also optional security = features for requiring authentication before allowing code execution. I thi= nk that a discussion on whether using them for DTS is necessary is warrante= d, although I personally think that it=E2=80=99s not worth the effort given= the type of environment this sample application is expected to run in.=20 For this RFC, I ported test-acl because it was mostly self-contained an= d was something I could run on my laptop. It should be fairly easy to see h= ow you would expand this proof of concept to cover more of DPDK, and I thin= k that most of the functions currently used in testpmd could be ported over= to this approach, saving a lot of development time. However, I would like = to see some more interest before I take on a task like that. This will requ= ire a lot of work on the DTS side to implement, but it will make it much ea= sier to add new features to DTS.=20 Owen Hilyard (4): app/test-pmd-api: Add C++ Compiler app/test-pmd-api: Add POC with gRPC deps app/test-pmd-api: Add protobuf file app/test-pmd-api: Implementation files for the API app/meson.build | 17 + app/test-pmd-api/api.proto | 12 + app/test-pmd-api/api_impl.cc | 1160 ++++++++++++++++++++++++++++++++++ app/test-pmd-api/api_impl.h | 10 + app/test-pmd-api/main.c | 11 + app/test-pmd-api/meson.build | 96 +++ meson.build | 3 + meson_options.txt | 2 + 8 files changed, 1311 insertions(+) create mode 100644 app/test-pmd-api/api.proto create mode 100644 app/test-pmd-api/api_impl.cc create mode 100644 app/test-pmd-api/api_impl.h create mode 100644 app/test-pmd-api/main.c create mode 100644 app/test-pmd-api/meson.build --=20 2.30.2