From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0D1CBA04FF; Tue, 12 Apr 2022 08:07:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9E20B40E2D; Tue, 12 Apr 2022 08:07:55 +0200 (CEST) Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) by mails.dpdk.org (Postfix) with ESMTP id 695EF40DF7 for ; Tue, 12 Apr 2022 08:07:54 +0200 (CEST) Received: by mail-io1-f44.google.com with SMTP id b16so21221424ioz.3 for ; Mon, 11 Apr 2022 23:07:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=xOFUB0Y7xot6kB2SY6568H5EV6Z/sHD4FzMdAnxvMUM=; b=KsygPac0GpzbuohsLAzV1zc95oEKITfOKec4CeS9p6+fjmNcmFYLV4mnPKn+K6UMAK YMhgKndVAjzdgaGxYzOMO4BON6C0IK829cRmvm8OBmVUUQ5Ef9NgYU6zj07GgNgR3A65 hYO5do8JRZQOwy9jGWWExT72QJ9JEoIA5pIpcFPLqe8ByCXVg6/ZbwRmfsikvwnLMISD 5RfrPXUOVU5kIzFVJjJqmXt/e2MnLsOJtiaYS0vfT93DKSH/caATsU4/pwbZDOqX0kww YH7SmIKYthkXsO9dB/FGXkhfsyIdZOpYmr+B0wnDPDulbSrhpmYBUBj1kpGUd/IthciV GhMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=xOFUB0Y7xot6kB2SY6568H5EV6Z/sHD4FzMdAnxvMUM=; b=0+yvWyWwTE+Y9dYVCZlocQwvPHj80Giuh4BBUKtNfdEAoZa97Izj8qSGGIt/cJ61ze trcuN4yZSMyWKsXHswRyhId9/zxL8uX7gqp1LhTRJJSw2Zfk6NRQteqPFfKjeIUclFQb KEoUfsXg2nu3+VAOyZ/OdIWabUv12cnzfono30p8N9JcfPJhURsxipt9o+/y4mgAYWrs R1WqCJ68nsfEEWgnYI1OQasg8Q5A4kp2jo09WBXoQj8kIgBCvHBMWLTCBtpDVqmBRHJ5 gQ7Yg89EMvIxn6IHdBg8tbuNWIJeO4ntGCvAah77GsyEzuGEyGHA6n1WlgCq0qT1ZfJJ oaXg== X-Gm-Message-State: AOAM532T4rf3Ik/XLH52iyGX0gHWMBDSiw5PFeSC7dg4kKIy9ggchN3T YHKdwK8XW5LSpirDSc3+GDCgI+3r+wjfrI9bcljFsMgPIj4= X-Google-Smtp-Source: ABdhPJwL9FMA5dFtEh9PEtPWc7dryfJ5CAgvM3Fikpa53GcsT0h3WR2ZVJIHO9pYj4DLD9IsIcHA+fr1SejI29N8kdw= X-Received: by 2002:a02:604f:0:b0:30f:e6f1:3883 with SMTP id d15-20020a02604f000000b0030fe6f13883mr18260226jaf.266.1649743673717; Mon, 11 Apr 2022 23:07:53 -0700 (PDT) MIME-Version: 1.0 References: <20220407214707.29730-1-ohilyard@iol.unh.edu> In-Reply-To: From: Jerin Jacob Date: Tue, 12 Apr 2022 11:37:27 +0530 Message-ID: Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API To: Owen Hilyard Cc: dpdk-dev , Honnappa Nagarahalli , Thomas Monjalon Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard wrote: >> >> scheme is probably over-engineered > > > I tried my hardest to keep this as simple as possible. The requirements i= mposed by DTS being a distributed system in Python restricted what I could = do a lot. Needing to be compatible with DPDK's license also got rid of a lo= t of options. Binding generators are made for simple projects, and DPDK is = not a simple project. There were some other options related to choice in th= e RPC framework, but very few RPC protocols seem to work well with C and be= usable from Python, which is why I ended up using C++ with gRPC. Most of t= he code in api_impl.cc is taken from /app/test-acl/main.c, and the new part= is mostly the C++ class at the bottom. Overall, this proposal comes out to= ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100 lin= es of Meson. gRPC may be able to do a lot more than I need it to for the pr= oof of concept, but many of the features that are not used, like bi-directi= onal streaming, become very useful in writing more complicated tests. Overa= ll, this solution is probably more capable than we need it to be, but I thi= nk that those extra capabilities don't come at a very large cost. Now it is clear, I was carried away with the POC test application and I was not knowing existing DTS tests are based on python Is below a fair summary? 1) DPDK has interactive test cases and no interactive test cases. For The interactive test case like testpmd, I agree that we can enable RPC service via gRPC server in C++ as and client in Python, and something along the lines of exposing the existing test-pmd command line function as service to avoid command line parsing and reuse the existing python test suite. If so, I think, gRPC service would be along with existing testpmd functions, like start_packet_forwarding(). Also, We don't need to rewrite the existing testpmd, Instead, RPC service, we can add in existing app/test-pmd/ and hook to existing core testpmd functions to bypass the command-line parsing in C and control from python client as needed as service. Also, I agree that pulling in gRPC C++ server boilerplate and hooking to C functions is a good idea as it is the best C-based RPC scheme available today. 2)I think, DPDK has only one interactive test case which is testpmd, Remaining test cases are non-interactive, non-interactive test cases can simply run over ssh with passwordless login. Right? Do we need gRPC for that? Will the following scheme suffice? If not, How you are planning to do noninteractive test cases? i.e a)Copy test to target b) result=3D`ssh username@IP /path/to/testapp/in/target` I think, key should be leveraging existing test cases as much as possible and make easy for developers to add new test cases. >> >> Now that, Test code is also part of DPDK. > > > DTS is pure python. I tried to use FFI to call directly into DPDK from Py= thon and then use xmlrpc from the python standard library. As mentioned in = the writeup, I couldn't find a binding generator that would properly handle= DPDK's allocators, which made it so that anything passed to DPDK using pyt= hon was allocated using the system malloc. I don't think it is wise to atte= mpt to programmatically re-write the generated code to allow for custom all= ocators. The original reason for needing to have DTS and DPDK in the same r= epository was so that tests could be committed and run alongside the featur= e patch. > >> Interactive - Testpmd one, I believe, Feeding stdin programmatically wou= ld suffice to test all the combinations. > > > One of the issues this is trying to address is that human-readable string= s are a poor way to pass complex information between two programs. DTS is a= distributed system, and it can have up to 3 physical servers involved in a= ny given test. This means that it's not stdin via a pipe, it's an entire SS= H session. This adds a noticeable amount of overhead when trying to send an= d verify the result of sending 1,000+ packets, since the lack of structured= output means each packet must be checked before the next can be sent. This= might be solvable by adding a structured output mode to testpmd, but that = would involve committing to writing output twice for every function in test= pmd forever. > >> We need to add all test cases in this model and we need to maintain two = sets of programs.(Traditional tests and gRPC model-based tests). > > > Assuming by traditional tests you mean the unit tests run by Meson, I wou= ld argue that we are already maintaining 2 kinds of tests. The unit tests, = and the python-based DTS tests. My intention is to create a thin wrapper ar= ound DPDK that would be exposed via gRPC, like you see here, and use that a= s midware. Then, we would have two front-ends. Testpmd, which takes text an= d then calls midware as it does now, and the gRPC frontend, which parses me= ssages from the RPC server and runs the midware. This would enable testpmd = to still be used to sanity check a DPDK installation, but we would not need= to continually expand Testpmd. The primary issue is that, right now, anyth= ing not included in Testpmd is not really testable by DTS. This includes po= rtions of the RTE Flow API, which was part of my reason for proposing this.= The RTE Flow API would, in my estimation, if fully implemented into Testpm= d, probably add at least another 10,000 lines of code. As mentioned in my p= roposal, Testpmd already does more parsing and lexing than it does interact= ion with DPDK by line count. Also, since I am proposing making this a separ= ate application, we would be able to gradually migrate the tests inside of = DTS. This would have no effect on anything except for Testpmd, the new appl= ication and the addition of a flag to toggle the use of a C++ compiler. > > I'm not sure exactly what you mean by gRPC model-based tests. gRPC uses c= lasses to model services, but for this usecase we are essentially using it = to transfer function arguments across the internet and then pass the return= value back. Any RPC framework would function similarly if I ignored the re= strictions of which languages to use, and the choice is not important to ho= w tests are conducted. Put another way, how you write a test for DTS will n= ot change much if you are using this or testpmd, it's just how you transfer= data and get it back that I want to change.