From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F176A04A9 for ; Tue, 8 Feb 2022 22:35:17 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 688A14114E; Tue, 8 Feb 2022 22:35:17 +0100 (CET) Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) by mails.dpdk.org (Postfix) with ESMTP id C750641143 for ; Tue, 8 Feb 2022 22:35:14 +0100 (CET) Received: by mail-oi1-f173.google.com with SMTP id r27so503748oiw.4 for ; Tue, 08 Feb 2022 13:35:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:from:date:message-id:subject:to; bh=xVGpb/dtOwqyF8O9wrLqSyYxcR6tGsJ0Ww7veDWbGi0=; b=HGDiFGcfplos3gxiEM/zsJeRnaga2KzExBEtN9vBk1nKVFTD5uxmg9BADVCeLaFI2m R9Rjr3Qlfap0PwPLeHpaVgkZCJsMZWU54jfrs12t4iXxLAWF9N3AFNklvX0QL0Zfn4wt 4jy+NB5LrPNJxoesC7Se/i5fuEXOWr9P9tjbI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=xVGpb/dtOwqyF8O9wrLqSyYxcR6tGsJ0Ww7veDWbGi0=; b=4AJMKHYl9DZBKFoJOwy3hSx7L0JsjgkfkEFBbo4gkB3qJCDyCVt8BG2qPM0tPcw3Wv deTUgabk1yQWCDY7RrYzRDyOYLfWjSWkMGr0PfM0M7plnUWxYZkNJnHlXidjlbUed3S7 EA5mTXHTIVxk1KB8RrPFXzokFyWQsU62PWIIvBOtXEjaVqPGBrGc7KIwaJtIGlyIuVpU zqxL94lZfNBLkdVadyMvcABXyNwQzbavJeemTCmMuYVeRsUtww/1/HkmRotRfbOgKDrJ FblnvA/VASECFpHl4BYIzI7MX1wOUudxJq8fg2Fi94pyQyeRrJBQGzVcC3cURxTBCehO hu9A== X-Gm-Message-State: AOAM532Vr3bnuL1HshqHEpgkzmj1AXwhNP0/bWVSOjfDpzYMtNetmayI TpG+lAGhqCKgMlOm/UIJI+PLrxWDc5NtDXBKRPKVWg== X-Google-Smtp-Source: ABdhPJw/p9vJWxINplO5eyM8+2NweCldT0XY6HgGEx3a5Pswf4bL8ltH7RVN0hwNy1QR/JZGPta/Uhcp4H/w0mFrH+w= X-Received: by 2002:a05:6808:1897:: with SMTP id bi23mr1493369oib.66.1644356113737; Tue, 08 Feb 2022 13:35:13 -0800 (PST) MIME-Version: 1.0 From: Owen Hilyard Date: Tue, 8 Feb 2022 16:34:38 -0500 Message-ID: Subject: [RFC] Testpmd Maintenance affecting DTS To: dts@dpdk.org, dev , ci@dpdk.org Content-Type: multipart/alternative; boundary="0000000000000b3d3905d7887ed3" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --0000000000000b3d3905d7887ed3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Currently, DTS tests DPDK via the example applications (/examples) and the apps (/app) included with DPDK. The DTS Working group has been trying to move to primarily using Testpmd, since it is the most complete and only using one application causes less maintenance work. Since Testpmd provides a text interface into DPDK, it must be continually updated to expose this new functionality, sometimes it may lag behind the current feature set of DPDK. For example, Testpmd currently does not support all of the options available for rte flow, which limits the testing coverage that can be provided by DTS for RTE flow. Because DTS is a Python application, it is not able to directly interface with DPDK. This means that whenever Testpmd falls behind DPDK, those unexposed parts of DPDK are functionally untestable as far as DTS is concerned. DTS is the tool used by the CI and labs to functionally test DPDK in a working configuration, compared to the unit testing, that may test individual components or parts of the stack. The DTS improvements WG and the Community Lab have a goal to continue the expansion of the test functional and performance test coverage of DPDK. This means that there must be some way for everything that the DPDK community wants to be tested to be exposed to DTS. I have a few possible solutions to this problem: 1. Continue to update Testpmd every time new functionality is added to DPDK to expose that functionality 2. Use a parser generator or some other method to make adding new options to Testpmd much easier, updating this every time new functionality is ad= ded to testpmd 3. Create a dedicated testing application for DPDK that uses a binding generator and Python=E2=80=99s XMLRPC to allow more programmatic access = to DPDK on the DUT, adding new RPC calls to expose new functionality. I personally think that option 3 is the best because it would involve a bit of work up front to make it much easier to expose parts of DPDK to DTS. This is because after the initial work is done, we would just need to write a wrapper function in C to expose the functionality we want, and then expose the wrapper function via RPC. If you have thoughts or suggestions, we will be discussing this at the DTS working group meeting. 2:00 PM UTC, Wednesday, February 16 https://armltd.zoom.us/j/97503259680?pwd=3DVVlmWnlnTXJkVGkwR2JOU3R3b3Vndz09= &from=3Daddon --0000000000000b3d3905d7887ed3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Currently, DTS tests DPDK via the example applications (/examples) = and the apps (/app) included with DPDK. The DTS Working group has been tryi= ng to move to primarily using Testpmd, since it is the most complete and on= ly using one application causes less maintenance work. Since Testpmd provid= es a text interface into DPDK, it must be continually updated to expose thi= s new functionality, sometimes it may lag behind the current feature set of= DPDK. For example, Testpmd currently does not support all of the options a= vailable for rte flow, which limits the testing coverage that can be provid= ed by DTS for RTE flow. Because DTS is a Python application, it is not able= to directly interface with DPDK. This means that whenever Testpmd falls be= hind DPDK, those unexposed parts of DPDK are functionally untestable as far= as DTS is concerned. DTS is the tool used by the CI and labs to functional= ly test DPDK in a working configuration, compared to the unit testing, that= may test individual components or parts of the stack.=C2=A0 The DTS improv= ements WG and the Community Lab have a goal to continue the expansion of th= e test functional and performance test coverage of DPDK.=C2=A0 This means t= hat there must be some way for everything that the DPDK community wants to = be tested to be exposed to DTS. I have a few possible solutions to this pro= blem:


  1. Continue to update Testpmd every time new f= unctionality is added to DPDK to expose that functionality

    2. <= li dir=3D"ltr" style=3D"list-style-type:decimal;font-size:11pt;font-family:= Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:no= rmal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre= ">

    Use a parser generator or some other = method to make adding new options to Testpmd much easier, updating this eve= ry time new functionality is added to testpmd

  2. Create a dedicated testing application for DPDK th= at uses a binding generator and Python=E2=80=99s XMLRPC to allow more progr= ammatic access to DPDK on the DUT, adding new RPC calls to expose new funct= ionality.


I personally think that option 3 is the best because it would invol= ve a bit of work up front to make it much easier to expose parts of DPDK to= DTS. This is because after the initial work is done, we would just need to= write a wrapper function in C to expose the functionality we want, and the= n expose the wrapper function via RPC.=C2=A0


If you have thoughts or suggestions, we wi= ll be discussing this at the DTS working group meeting.

2:00 PM UTC, Wednesday, February 16
https://armlt= d.zoom.us/j/97503259680?pwd=3DVVlmWnlnTXJkVGkwR2JOU3R3b3Vndz09&from=3Da= ddon

--0000000000000b3d3905d7887ed3--