From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C674A0509; Thu, 14 Apr 2022 22:10:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 170A04067E; Thu, 14 Apr 2022 22:10:10 +0200 (CEST) Received: from mail-oa1-f48.google.com (mail-oa1-f48.google.com [209.85.160.48]) by mails.dpdk.org (Postfix) with ESMTP id D23D94067C for <dev@dpdk.org>; Thu, 14 Apr 2022 22:10:08 +0200 (CEST) Received: by mail-oa1-f48.google.com with SMTP id 586e51a60fabf-df02f7e2c9so6337648fac.10 for <dev@dpdk.org>; Thu, 14 Apr 2022 13:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SfHmqAjmgWA8BRr2U+XFA4ay7sRhwYFsJSpT6+MT3fM=; b=XHksG57+eqyy33D9BsYBUMZmFXuJ3eKTYTyhzFJyE2zV5Y4yYFsK/AmA8nJSbypjRF 7vjzCPJaq/fyBTqwvSUOe9Y7DAQokfZ1z3q3XucMixrWsio3ThUo5u2EvmrmhK0yhn4W syHQWMk9eKWIbY5liRiErWoKtNBsKZCN6YfoE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SfHmqAjmgWA8BRr2U+XFA4ay7sRhwYFsJSpT6+MT3fM=; b=aJhOU9B1qh4fkWuVipuQAKPBiW6bXzePgOnDK7NvnqveiIBUkEfwhqILzraX5TiNKI tA8ZQiFZVEhN6otQEaQ4bVSGFKx46ElSMFMbLefS/Zsl1RpD8j440imKUEAyMIvco3hw TEXuduI1S7oAKrnvC3v1OHhVzHS/FgaQXgZLaLIqwpY2fyPeIsaGBaQwB8uB3B50Lgfe 48vnclt8SnH+1AmM1996s3l2TjMv5P4tZeDt7WSSSEX+wIBRtQNiTYnv4uWVpb42vf+U EqcTiRrxrOA34xtMJIVYe/dLZIkE6bLpWtMq7kZAIKH2y1ddKzUYZTZHdwrw7kBuzh1F Orag== X-Gm-Message-State: AOAM531AnUUbfCp7wIXOHb5o5FwmH0xuPs4wzIdM5UXJXp78lb61M2bB DkzXx+gpj66Dke1orWYQf8Bvu95vsRrdmUlAGFI5Uw== X-Google-Smtp-Source: ABdhPJxVWcroW/oUswlBNZGOAJODQV28H5lrAoJqsGqIFA/f/R4a6I+dteljs/ZOi+YpVdNUu/gWUluzhgcrDlOf3lk= X-Received: by 2002:a05:6870:61cc:b0:e1:9f6e:675a with SMTP id b12-20020a05687061cc00b000e19f6e675amr110125oah.213.1649967008076; Thu, 14 Apr 2022 13:10:08 -0700 (PDT) MIME-Version: 1.0 References: <20220407214707.29730-1-ohilyard@iol.unh.edu> <CALBAE1N2FAQPHJBoowpxqoVbEF3UcsTvCDQ2f_kgCr2n3wO0HQ@mail.gmail.com> <CAHx6DYAgLhNMupozoX3BQr6gqZUUWYH9UuSpRYUAvZdOsPTSMg@mail.gmail.com> <CALBAE1NGdgcauPS8Au1DcwfuLWSdxNmaNAO9NntefjJUuUUy2A@mail.gmail.com> <CAHx6DYCBFT0V37=2Qvknw-uZ3NGrqax6p25DYhUfei1zk5+SZA@mail.gmail.com> <DM6PR11MB449110A757F453293250B7089AEF9@DM6PR11MB4491.namprd11.prod.outlook.com> In-Reply-To: <DM6PR11MB449110A757F453293250B7089AEF9@DM6PR11MB4491.namprd11.prod.outlook.com> From: Owen Hilyard <ohilyard@iol.unh.edu> Date: Thu, 14 Apr 2022 16:09:32 -0400 Message-ID: <CAHx6DYA=w0-UqfcVf89smts+a1w1K5qcFoYCG9FPsf434pP7RA@mail.gmail.com> Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com> Cc: Jerin Jacob <jerinjacobk@gmail.com>, dpdk-dev <dev@dpdk.org>, Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, Thomas Monjalon <thomas@monjalon.net> Content-Type: multipart/alternative; boundary="00000000000068695b05dca2e136" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org --00000000000068695b05dca2e136 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > Though I think we shouldn=E2=80=99t remove existing CLI interface. > I agree, it's a very useful debugging tool for validating environments. I think having two "frontends", the CLI and API, which both consume one "backend" testpmd library would be the easiest way to go about doing that while keeping long-term maintenance low. Conditional compilation (new meson flag or so) is probably good enough for > this case. > One of the changes I made was an on-by-default meson flag to enable C++ compilation. If that flag is on, and all dependencies are present, then the application will be built. Would it be possible to try implement something more realistic with testpmd > itself I would consider it a "phase 2" version of this RFC. The hard part was getting gRPC working inside of Meson, which is why I picked a simple app to port. If this RFC moves forward, I can look at porting the functionality needed for the nic single core performance test ( http://git.dpdk.org/tools/dts/tree/test_plans/nic_single_core_perf_test_pla= n.rst ). On Thu, Apr 14, 2022 at 8:08 AM Ananyev, Konstantin < konstantin.ananyev@intel.com> wrote: > > Hi everyone, > > First of all thanks Owen for stepping forward with this RFC. > Few thoughts on this subject below. > Konstantin > > > -----Original Message----- > > From: Ananyev, Konstantin <konstantin.ananyev@intel.com> > > Sent: Thursday, April 14, 2022 12:59 PM > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com> > > Subject: FW: [PATCH v1 0/4] [RFC] Testpmd RPC API > > > > > > > > From: Owen Hilyard <ohilyard@iol.unh.edu> > > Sent: Wednesday, April 13, 2022 1:47 PM > > To: Jerin Jacob <jerinjacobk@gmail.com> > > Cc: dpdk-dev <dev@dpdk.org>; Honnappa Nagarahalli < > Honnappa.Nagarahalli@arm.com>; Thomas Monjalon <thomas@monjalon.net> > > Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API > > > > If so, I think, gRPC service would be along with existing testpmd > functions, like start_packet_forwarding(). > > > > It was my intention to re-use existing functions. I used the ACL tests > as an example because they are more self-contained then Testpmd, > > which made creating the proof of concept much easier. > > > > Also, We don't need to rewrite the existing testpmd, Instead, RPC > service, we can add in existing app/test-pmd/ > > > > The reason that I split out the services is that there doesn't seem to > be a way to produce multiple binaries without re-writing that section of > > the build system. I wanted to avoid the hard requirement of having a C+= + > compiler available in order to be able to use testpmd, since that > > may affect what platforms Testpmd can run on and I want to avoid this > being any kind of breaking change. If we decide to go the route of > > putting it all in a single application, we would need to conditionally > enable the gRPC service at build time. Meson's current lack of support > > for conditionally detecting compilers causes issues here. > > > > I think, DPDK has only one interactive test case which is testpmd, > > > > Could you point me to that test case? Either invocation or source is ok= . > I can't see anything that would lead me to assume use of testpmd in > > "meson test --list". To my knowledge, all of the test cases that use > testpmd are in DTS. If there is a test that uses testpmd but is not part = of > > DTS, I think it would be a candidate for moving into DTS assuming it's > not a unit test. > > > > How you are planning to do noninteractive test cases? > > > > I'm not planning to make any change to unit testing, you can read more > about how testing is currently conducted > > here: https://www.dpdk.org/blog/2021/07/05/dpdk-testing-approaches/ > > > > If there is a unit test that involves testpmd, there are two > possibilities. > > 1. If we are making a separate application for Testpmd with the gRPC > api, then nothing changes except for possibly changing where some > > of the testpmd source lives in order to enable code reuse between the > two applications. > > 2. If gRPC is being added to Testpmd, then the unit test should still > function as it previously did if I do any necessary refactoring as > correctly. > > > > I think, key should be leveraging existing test cases as much as > possible and make easy for developers to add new test cases. > > > > That is part of the reason why I want to be able to do this. Adding a > new test in DTS is very easy if the functionality needed already exists i= n > > Testpmd. If the functionality does not exist, then adding the test > becomes difficult, due to the required modifications to the Testpmd lexer > > and parser to accommodate the new command. My plan is to leave unit > testing in C, but help make it easier to expose C functions to Python > > for integration testing. This gives us the best of both worlds in terms > of access to DPDK and the ability to use a high-level language to write > > the tests. > > > > On Tue, Apr 12, 2022 at 2:07 AM Jerin Jacob <mailto: > jerinjacobk@gmail.com> wrote: > > On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <mailto: > ohilyard@iol.unh.edu> wrote: > > >> > > >> scheme is probably over-engineered > > > > > > > > > I tried my hardest to keep this as simple as possible. The > requirements imposed by DTS being a distributed system in Python restrict= ed > > what I could do a lot. Needing to be compatible with DPDK's license als= o > got rid of a lot of options. Binding generators are made for simple > > projects, and DPDK is not a simple project. There were some other > options related to choice in the RPC framework, but very few RPC > > protocols seem to work well with C and be usable from Python, which is > why I ended up using C++ with gRPC. Most of the code in > > api_impl.cc is taken from /app/test-acl/main.c, and the new part is > mostly the C++ class at the bottom. Overall, this proposal comes out to > > ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100 > lines of Meson. gRPC may be able to do a lot more than I need it to > > for the proof of concept, but many of the features that are not used, > like bi-directional streaming, become very useful in writing more > > complicated tests. Overall, this solution is probably more capable than > we need it to be, but I think that those extra capabilities don't come > > at a very large cost. > > > > > > Now it is clear, I was carried away with the POC test application and > > I was not knowing existing DTS tests are based on python > > > > Is below a fair summary? > > > > 1) DPDK has interactive test cases and no interactive test cases. > > > > For The interactive test case like testpmd, I agree that we can enable > > RPC service via gRPC server in C++ as and client in Python, and > > something along the lines of exposing the existing test-pmd command > > line function as service > > to avoid command line parsing and reuse the existing python test suite. > > > > If so, I think, gRPC service would be along with existing testpmd > > functions, like start_packet_forwarding(). Also, We don't need to > > rewrite the existing testpmd, > > Instead, RPC service, we can add in existing app/test-pmd/ and hook to > > existing core testpmd functions to bypass the command-line parsing in > > C and control from python client as needed as service. > > > > Also, I agree that pulling in gRPC C++ server boilerplate and hooking > > to C functions is a good idea as it is the best C-based RPC scheme > > available today. > > > > 2)I think, DPDK has only one interactive test case which is testpmd, > > Remaining test cases are non-interactive, non-interactive test cases > > can simply run over ssh with passwordless login. Right? > > Do we need gRPC for that? Will the following scheme suffice? If not, > > How you are planning to do noninteractive test cases? > > i.e > > a)Copy test to target > > b) result=3D`ssh username@IP /path/to/testapp/in/target` > > > > I think, key should be leveraging existing test cases as much as > > possible and make easy for developers to add new test cases. > > > > > > >> > > >> Now that, Test code is also part of DPDK. > > > > > > > > > DTS is pure python. I tried to use FFI to call directly into DPDK fro= m > Python and then use xmlrpc from the python standard library. As > > mentioned in the writeup, I couldn't find a binding generator that woul= d > properly handle DPDK's allocators, which made it so that anything > > passed to DPDK using python was allocated using the system malloc. I > don't think it is wise to attempt to programmatically re-write the > > generated code to allow for custom allocators. The original reason for > needing to have DTS and DPDK in the same repository was so that > > tests could be committed and run alongside the feature patch. > > > > > >> Interactive - Testpmd one, I believe, Feeding stdin programmatically > would suffice to test all the combinations. > > > > > > > > > One of the issues this is trying to address is that human-readable > strings are a poor way to pass complex information between two > > programs. DTS is a distributed system, and it can have up to 3 physical > servers involved in any given test. This means that it's not stdin via a > > pipe, it's an entire SSH session. This adds a noticeable amount of > overhead when trying to send and verify the result of sending 1,000+ > > packets, since the lack of structured output means each packet must be > checked before the next can be sent. This might be solvable by > > adding a structured output mode to testpmd, but that would involve > committing to writing output twice for every function in testpmd > > forever. > > > > > >> We need to add all test cases in this model and we need to maintain > two sets of programs.(Traditional tests and gRPC model-based > > tests). > > > > > > > > > Assuming by traditional tests you mean the unit tests run by Meson, I > would argue that we are already maintaining 2 kinds of tests. The > > unit tests, and the python-based DTS tests. My intention is to create a > thin wrapper around DPDK that would be exposed via gRPC, like you > > see here, and use that as midware. Then, we would have two front-ends. > Testpmd, which takes text and then calls midware as it does now, > > and the gRPC frontend, which parses messages from the RPC server and > runs the midware. This would enable testpmd to still be used to > > sanity check a DPDK installation, but we would not need to continually > expand Testpmd. The primary issue is that, right now, anything not > > included in Testpmd is not really testable by DTS. This includes > portions of the RTE Flow API, which was part of my reason for proposing > this. > > The RTE Flow API would, in my estimation, if fully implemented into > Testpmd, probably add at least another 10,000 lines of code. As > > mentioned in my proposal, Testpmd already does more parsing and lexing > than it does interaction with DPDK by line count. Also, since I am > > proposing making this a separate application, we would be able to > gradually migrate the tests inside of DTS. This would have no effect on > > anything except for Testpmd, the new application and the addition of a > flag to toggle the use of a C++ compiler. > > > > > > I'm not sure exactly what you mean by gRPC model-based tests. gRPC > uses classes to model services, but for this usecase we are > > essentially using it to transfer function arguments across the internet > and then pass the return value back. Any RPC framework would > > function similarly if I ignored the restrictions of which languages to > use, and the choice is not important to how tests are conducted. Put > > another way, how you write a test for DTS will not change much if you > are using this or testpmd, it's just how you transfer data and get it > > back that I want to change. > > - In general I think it is a good idea to adding gRPC binding to testpmd > to expose/improve testing automation. > Though I think we shouldn=E2=80=99t remove existing CLI interface. > Ideally I=E2=80=99d like to have both =E2=80=93 CLI and gRPC for all co= mmands. > Don=E2=80=99t know how realistic is that, but at least for major comman= ds - > port/queue configure, start/stop, etc. > - Conditional compilation (new meson flag or so) is probably good enough > for this case. > - About RFC itself - I understand that you choose testacl for simplicity, > but in fact, it is a standalone application > that has not much common with testpmd itself and the problems that you > mentioned: > interactive commands, parameter and results parsing, etc. > Would it be possible to try implement something more realistic with > testpmd itself, > like simple test-pmd port/queue configure, start, result collection, > etc.? > To get a better idea how it is going to work and how complicated it > would be. > > > --00000000000068695b05dca2e136 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">=C2=A0 T= hough I think we shouldn=E2=80=99t remove existing CLI interface.<br></bloc= kquote><div><br></div><div>I agree, it's a very useful debugging tool f= or validating environments. I think having two "frontends", the C= LI and API, which both consume one "backend" testpmd library woul= d be the easiest way to go about doing that while keeping long-term mainten= ance=C2=A0low.=C2=A0</div><div><br></div><blockquote class=3D"gmail_quote" = style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa= dding-left:1ex">Conditional compilation (new meson flag or so) is probably = good enough for this case.<br></blockquote><div><br></div><div>One of the c= hanges I made was an on-by-default meson flag to enable C++ compilation. If= that flag is on, and all dependencies are present, then the application wi= ll be built.=C2=A0</div><div><br></div><blockquote class=3D"gmail_quote" st= yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd= ing-left:1ex">Would it be possible to try implement something more realisti= c with testpmd itself</blockquote><div><br></div><div>I would consider it a= "phase 2" version of this RFC. The hard part was getting gRPC wo= rking inside of Meson, which is why I picked a simple app to port. If this = RFC moves forward, I can look at porting the functionality needed for the n= ic single core performance=C2=A0test (<a href=3D"http://git.dpdk.org/tools/= dts/tree/test_plans/nic_single_core_perf_test_plan.rst">http://git.dpdk.org= /tools/dts/tree/test_plans/nic_single_core_perf_test_plan.rst</a>).</div></= div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On= Thu, Apr 14, 2022 at 8:08 AM Ananyev, Konstantin <<a href=3D"mailto:kon= stantin.ananyev@intel.com">konstantin.ananyev@intel.com</a>> wrote:<br><= /div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo= rder-left:1px solid rgb(204,204,204);padding-left:1ex"><br> Hi everyone,<br> <br> First of all thanks Owen for stepping forward with this RFC.<br> Few thoughts on this subject below.<br> Konstantin=C2=A0 =C2=A0<br> <br> > -----Original Message-----<br> > From: Ananyev, Konstantin <<a href=3D"mailto:konstantin.ananyev@int= el.com" target=3D"_blank">konstantin.ananyev@intel.com</a>><br> > Sent: Thursday, April 14, 2022 12:59 PM<br> > To: Ananyev, Konstantin <<a href=3D"mailto:konstantin.ananyev@intel= .com" target=3D"_blank">konstantin.ananyev@intel.com</a>><br> > Subject: FW: [PATCH v1 0/4] [RFC] Testpmd RPC API<br> > <br> > <br> > <br> > From: Owen Hilyard <<a href=3D"mailto:ohilyard@iol.unh.edu" target= =3D"_blank">ohilyard@iol.unh.edu</a>><br> > Sent: Wednesday, April 13, 2022 1:47 PM<br> > To: Jerin Jacob <<a href=3D"mailto:jerinjacobk@gmail.com" target=3D= "_blank">jerinjacobk@gmail.com</a>><br> > Cc: dpdk-dev <<a href=3D"mailto:dev@dpdk.org" target=3D"_blank">dev= @dpdk.org</a>>; Honnappa Nagarahalli <<a href=3D"mailto:Honnappa.Naga= rahalli@arm.com" target=3D"_blank">Honnappa.Nagarahalli@arm.com</a>>; Th= omas Monjalon <<a href=3D"mailto:thomas@monjalon.net" target=3D"_blank">= thomas@monjalon.net</a>><br> > Subject: Re: [PATCH v1 0/4] [RFC] Testpmd RPC API<br> > <br> > If so, I think, gRPC service would be along with existing testpmd func= tions, like start_packet_forwarding().<br> > <br> > It was my intention to re-use existing functions. I used the ACL tests= as an example because they are more self-contained then Testpmd,<br> > which made creating the proof of concept much easier.<br> > <br> > Also, We don't need to rewrite the existing testpmd, Instead, RPC = service, we can add in existing app/test-pmd/<br> > <br> > The reason that I split out the services is that there doesn't see= m to be a way to produce multiple binaries without re-writing that section = of<br> > the build system. I wanted to avoid the hard requirement of having a C= ++ compiler available in order to be able to use testpmd, since that<br> > may affect what platforms Testpmd can run on and I want to avoid this = being any kind of breaking change. If we decide to go the route of<br> > putting it all in a single application, we would need to conditionally= enable the gRPC service at build time. Meson's current lack of support= <br> > for conditionally detecting compilers causes issues here.<br> > <br> > I think, DPDK has only one interactive test case which is testpmd,<br> > <br> > Could you point me to that test case? Either invocation or source is o= k. I can't see anything that would lead me to assume use of testpmd in<= br> > "meson test --list". To my knowledge, all of the test cases = that use testpmd are in DTS. If there is a test that uses testpmd but is no= t part of<br> > DTS, I think it would be a candidate for moving into DTS assuming it&#= 39;s not a unit test.<br> > <br> > How you are planning to do noninteractive test cases?<br> > <br> > I'm not planning to make any change to unit testing, you can read = more about how testing is currently conducted<br> > here: <a href=3D"https://www.dpdk.org/blog/2021/07/05/dpdk-testing-app= roaches/" rel=3D"noreferrer" target=3D"_blank">https://www.dpdk.org/blog/20= 21/07/05/dpdk-testing-approaches/</a><br> > <br> > If there is a unit test that involves testpmd, there are two possibili= ties.<br> > 1. If we are making a separate application for Testpmd with the gRPC a= pi, then nothing changes except for possibly changing where some<br> > of the testpmd source lives in order to enable code reuse between the = two applications.<br> > 2. If gRPC is being added to Testpmd, then the unit test should still = function as it previously did if I do any necessary refactoring as correctl= y.<br> > <br> > I think, key should be leveraging existing test cases as much as possi= ble and make easy for developers to add new test cases.<br> > <br> > That is part of the reason why I want to be able to do this. Adding a = new test in DTS is very easy if the functionality needed already exists in<= br> > Testpmd. If the functionality does not exist, then adding the test bec= omes difficult, due to the required modifications to the Testpmd lexer<br> > and parser to accommodate the new command. My plan is to leave unit te= sting in C, but help make it easier to expose C functions to Python<br> > for integration testing. This gives us the best of both worlds in term= s of access to DPDK and the ability to use a high-level language to write<b= r> > the tests.<br> > <br> > On Tue, Apr 12, 2022 at 2:07 AM Jerin Jacob <mailto:<a href=3D"mail= to:jerinjacobk@gmail.com" target=3D"_blank">jerinjacobk@gmail.com</a>> w= rote:<br> > On Mon, Apr 11, 2022 at 11:19 PM Owen Hilyard <mailto:<a href=3D"ma= ilto:ohilyard@iol.unh.edu" target=3D"_blank">ohilyard@iol.unh.edu</a>> w= rote:<br> > >><br> > >> scheme is probably over-engineered<br> > ><br> > ><br> > > I tried my hardest to keep this as simple as possible. The requir= ements imposed by DTS being a distributed system in Python restricted<br> > what I could do a lot. Needing to be compatible with DPDK's licens= e also got rid of a lot of options. Binding generators are made for simple<= br> > projects, and DPDK is not a simple project. There were some other opti= ons related to choice in the RPC framework, but very few RPC<br> > protocols seem to work well with C and be usable from Python, which is= why I ended up using C++ with gRPC. Most of the code in<br> > api_impl.cc is taken from /app/test-acl/main.c, and the new part is mo= stly the C++ class at the bottom. Overall, this proposal comes out to<br> > ~100 lines of new C++, 9 lines of C, 12 lines of gRPC Protobuf and 100= lines of Meson. gRPC may be able to do a lot more than I need it to<br> > for the proof of concept, but many of the features that are not used, = like bi-directional streaming, become very useful in writing more<br> > complicated tests. Overall, this solution is probably more capable tha= n we need it to be, but I think that those extra capabilities don't com= e<br> > at a very large cost.<br> > <br> > <br> > Now it is clear, I was carried away with the POC test application and<= br> > I was not knowing existing DTS tests are based on python<br> > <br> > Is below a fair summary?<br> > <br> > 1) DPDK has interactive test cases and no interactive test cases.<br> > <br> > For The interactive test case like testpmd, I agree that we can enable= <br> > RPC service via gRPC server in C++ as=C2=A0 and client in Python, and<= br> > something along the lines of exposing the existing test-pmd command<br= > > line function as service<br> > to avoid command line parsing and reuse the existing python test suite= .<br> > <br> > If so, I think, gRPC service would be along with existing testpmd<br> > functions, like start_packet_forwarding(). Also, We don't need to<= br> > rewrite the existing testpmd,<br> > Instead, RPC service, we can add in existing app/test-pmd/ and hook to= <br> > existing core testpmd functions to bypass the command-line parsing in<= br> > C and control from python client as needed as service.<br> > <br> > Also, I agree that pulling in gRPC C++ server boilerplate and hooking<= br> > to C functions is a good idea as it is the best C-based RPC scheme<br> > available today.<br> > <br> > 2)I think, DPDK has only one interactive test case which is testpmd,<b= r> > Remaining test cases are non-interactive, non-interactive test cases<b= r> > can simply run over ssh with passwordless login. Right?<br> > Do we need gRPC for that? Will the following scheme suffice? If not,<b= r> > How you are planning to do noninteractive test cases?<br> > i.e<br> > a)Copy test to target<br> > b) result=3D`ssh username@IP /path/to/testapp/in/target`<br> > <br> > I think, key should be leveraging existing test cases as much as<br> > possible and make easy for developers to add new test cases.<br> > <br> > <br> > >><br> > >> Now that, Test code is also part of DPDK.<br> > ><br> > ><br> > > DTS is pure python. I tried to use FFI to call directly into DPDK= from Python and then use xmlrpc from the python standard library. As<br> > mentioned in the writeup, I couldn't find a binding generator that= would properly handle DPDK's allocators, which made it so that anythin= g<br> > passed to DPDK using python was allocated using the system malloc. I d= on't think it is wise to attempt to programmatically re-write the<br> > generated code to allow for custom allocators. The original reason for= needing to have DTS and DPDK in the same repository was so that<br> > tests could be committed and run alongside the feature patch.<br> > ><br> > >> Interactive - Testpmd one, I believe, Feeding stdin programma= tically would suffice to test all the combinations.<br> > ><br> > ><br> > > One of the issues this is trying to address is that human-readabl= e strings are a poor way to pass complex information between two<br> > programs. DTS is a distributed system, and it can have up to 3 physica= l servers involved in any given test. This means that it's not stdin vi= a a<br> > pipe, it's an entire SSH session. This adds a noticeable amount of= overhead when trying to send and verify the result of sending 1,000+<br> > packets, since the lack of structured output means each packet must be= checked before the next can be sent. This might be solvable by<br> > adding a structured output mode to testpmd, but that would involve com= mitting to writing output twice for every function in testpmd<br> > forever.<br> > ><br> > >> We need to add all test cases in this model and we need to ma= intain two sets of programs.(Traditional tests and gRPC model-based<br> > tests).<br> > ><br> > ><br> > > Assuming by traditional tests you mean the unit tests run by Meso= n, I would argue that we are already maintaining 2 kinds of tests. The<br> > unit tests, and the python-based DTS tests. My intention is to create = a thin wrapper around DPDK that would be exposed via gRPC, like you<br> > see here, and use that as midware. Then, we would have two front-ends.= Testpmd, which takes text and then calls midware as it does now,<br> > and the gRPC frontend, which parses messages from the RPC server and r= uns the midware. This would enable testpmd to still be used to<br> > sanity check a DPDK installation, but we would not need to continually= expand Testpmd. The primary issue is that, right now, anything not<br> > included in Testpmd is not really testable by DTS. This includes porti= ons of the RTE Flow API, which was part of my reason for proposing this.<br= > > The RTE Flow API would, in my estimation, if fully implemented into Te= stpmd, probably add at least another 10,000 lines of code. As<br> > mentioned in my proposal, Testpmd already does more parsing and lexing= than it does interaction with DPDK by line count. Also, since I am<br> > proposing making this a separate application, we would be able to grad= ually migrate the tests inside of DTS. This would have no effect on<br> > anything except for Testpmd, the new application and the addition of a= flag to toggle the use of a C++ compiler.<br> > ><br> > > I'm not sure exactly what you mean by gRPC model-based tests.= gRPC uses classes to model services, but for this usecase we are<br> > essentially using it to transfer function arguments across the interne= t and then pass the return value back. Any RPC framework would<br> > function similarly if I ignored the restrictions of which languages to= use, and the choice is not important to how tests are conducted. Put<br> > another way, how you write a test for DTS will not change much if you = are using this or testpmd, it's just how you transfer data and get it<b= r> > back that I want to change.<br> <br> - In general I think it is a good idea to adding gRPC binding to testpmd to= expose/improve testing automation.<br> =C2=A0 Though I think we shouldn=E2=80=99t remove existing CLI interface.<b= r> =C2=A0 Ideally I=E2=80=99d like to have both =E2=80=93 CLI and gRPC for all= commands.<br> =C2=A0 Don=E2=80=99t know how realistic is that, but at least for major com= mands -=C2=A0 port/queue configure, start/stop, etc.<br> - Conditional compilation (new meson flag or so) is probably good enough fo= r this case.=C2=A0 =C2=A0<br> - About RFC itself - I understand that you choose testacl for simplicity, b= ut in fact, it is a standalone application<br> =C2=A0 that has not much common with testpmd itself and the problems that y= ou mentioned:<br> =C2=A0 interactive commands, parameter and results parsing, etc.<br> =C2=A0 Would it be possible to try implement something more realistic with = testpmd itself,<br> =C2=A0 like simple test-pmd port/queue configure, start,=C2=A0 result colle= ction, etc.?<br> =C2=A0 To get a better idea how it is going to work and how complicated it = would be.<br> <br> <br> </blockquote></div> --00000000000068695b05dca2e136--