From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98D4BA04A9; Tue, 8 Feb 2022 22:35:17 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7928441155; Tue, 8 Feb 2022 22:35:17 +0100 (CET) Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) by mails.dpdk.org (Postfix) with ESMTP id CCCB141144 for ; Tue, 8 Feb 2022 22:35:14 +0100 (CET) Received: by mail-oi1-f172.google.com with SMTP id v67so469144oie.9 for ; Tue, 08 Feb 2022 13:35:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:from:date:message-id:subject:to; bh=xVGpb/dtOwqyF8O9wrLqSyYxcR6tGsJ0Ww7veDWbGi0=; b=HGDiFGcfplos3gxiEM/zsJeRnaga2KzExBEtN9vBk1nKVFTD5uxmg9BADVCeLaFI2m R9Rjr3Qlfap0PwPLeHpaVgkZCJsMZWU54jfrs12t4iXxLAWF9N3AFNklvX0QL0Zfn4wt 4jy+NB5LrPNJxoesC7Se/i5fuEXOWr9P9tjbI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=xVGpb/dtOwqyF8O9wrLqSyYxcR6tGsJ0Ww7veDWbGi0=; b=YIUrwvSwPoSENVgUwnFOu4jG7xvaLui2FIJWtmsdYflD6EPOBJNI6PVcuLWt2ccCI6 /1ySMVCsn6WelSwFIHltBwGYgTqwGWEyXvH23Y1E9fIAVlwqoQzhq7FXaeSV4x27TtK5 20vE6P9aAFnJcglO4SjUX7KD6u2f4EM0eeM+C6wvH7Eimp72VUL/h5/W0Lx+UBqv+RA1 CurLfxclTHnxVdJDsW7vxAn5AP3ZD6iKDkZ1QYmZ/kSMjqPAwiNZfinU3cke1t045AWA Gza0H/kqqFqyFV5e2ztLIaGRkmmQ/792Bacj9nfvAIUZUYxxjs1L/9LxG25YMtksbOL9 atow== X-Gm-Message-State: AOAM5339X5IT6TBPWhqG6fxoy93Z56Ld05J+SxmRvdEZsivEPA/85v/B YwVw77zf0V7l0jQ76QjCHUdC32xq2NMlDGU8130xEa0N99Tmsw== X-Google-Smtp-Source: ABdhPJw/p9vJWxINplO5eyM8+2NweCldT0XY6HgGEx3a5Pswf4bL8ltH7RVN0hwNy1QR/JZGPta/Uhcp4H/w0mFrH+w= X-Received: by 2002:a05:6808:1897:: with SMTP id bi23mr1493369oib.66.1644356113737; Tue, 08 Feb 2022 13:35:13 -0800 (PST) MIME-Version: 1.0 From: Owen Hilyard Date: Tue, 8 Feb 2022 16:34:38 -0500 Message-ID: Subject: [RFC] Testpmd Maintenance affecting DTS To: dts@dpdk.org, dev , ci@dpdk.org Content-Type: multipart/alternative; boundary="0000000000000b3d3905d7887ed3" X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org --0000000000000b3d3905d7887ed3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Currently, DTS tests DPDK via the example applications (/examples) and the apps (/app) included with DPDK. The DTS Working group has been trying to move to primarily using Testpmd, since it is the most complete and only using one application causes less maintenance work. Since Testpmd provides a text interface into DPDK, it must be continually updated to expose this new functionality, sometimes it may lag behind the current feature set of DPDK. For example, Testpmd currently does not support all of the options available for rte flow, which limits the testing coverage that can be provided by DTS for RTE flow. Because DTS is a Python application, it is not able to directly interface with DPDK. This means that whenever Testpmd falls behind DPDK, those unexposed parts of DPDK are functionally untestable as far as DTS is concerned. DTS is the tool used by the CI and labs to functionally test DPDK in a working configuration, compared to the unit testing, that may test individual components or parts of the stack. The DTS improvements WG and the Community Lab have a goal to continue the expansion of the test functional and performance test coverage of DPDK. This means that there must be some way for everything that the DPDK community wants to be tested to be exposed to DTS. I have a few possible solutions to this problem: 1. Continue to update Testpmd every time new functionality is added to DPDK to expose that functionality 2. Use a parser generator or some other method to make adding new options to Testpmd much easier, updating this every time new functionality is ad= ded to testpmd 3. Create a dedicated testing application for DPDK that uses a binding generator and Python=E2=80=99s XMLRPC to allow more programmatic access = to DPDK on the DUT, adding new RPC calls to expose new functionality. I personally think that option 3 is the best because it would involve a bit of work up front to make it much easier to expose parts of DPDK to DTS. This is because after the initial work is done, we would just need to write a wrapper function in C to expose the functionality we want, and then expose the wrapper function via RPC. If you have thoughts or suggestions, we will be discussing this at the DTS working group meeting. 2:00 PM UTC, Wednesday, February 16 https://armltd.zoom.us/j/97503259680?pwd=3DVVlmWnlnTXJkVGkwR2JOU3R3b3Vndz09= &from=3Daddon --0000000000000b3d3905d7887ed3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Currently, DTS tests DPDK via the example applications (/examples) = and the apps (/app) included with DPDK. The DTS Working group has been tryi= ng to move to primarily using Testpmd, since it is the most complete and on= ly using one application causes less maintenance work. Since Testpmd provid= es a text interface into DPDK, it must be continually updated to expose thi= s new functionality, sometimes it may lag behind the current feature set of= DPDK. For example, Testpmd currently does not support all of the options a= vailable for rte flow, which limits the testing coverage that can be provid= ed by DTS for RTE flow. Because DTS is a Python application, it is not able= to directly interface with DPDK. This means that whenever Testpmd falls be= hind DPDK, those unexposed parts of DPDK are functionally untestable as far= as DTS is concerned. DTS is the tool used by the CI and labs to functional= ly test DPDK in a working configuration, compared to the unit testing, that= may test individual components or parts of the stack.=C2=A0 The DTS improv= ements WG and the Community Lab have a goal to continue the expansion of th= e test functional and performance test coverage of DPDK.=C2=A0 This means t= hat there must be some way for everything that the DPDK community wants to = be tested to be exposed to DTS. I have a few possible solutions to this pro= blem:


  1. Continue to update Testpmd every time new f= unctionality is added to DPDK to expose that functionality

    2. <= li dir=3D"ltr" style=3D"list-style-type:decimal;font-size:11pt;font-family:= Arial;color:rgb(0,0,0);background-color:transparent;font-variant-numeric:no= rmal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre= ">

    Use a parser generator or some other = method to make adding new options to Testpmd much easier, updating this eve= ry time new functionality is added to testpmd

  2. Create a dedicated testing application for DPDK th= at uses a binding generator and Python=E2=80=99s XMLRPC to allow more progr= ammatic access to DPDK on the DUT, adding new RPC calls to expose new funct= ionality.


I personally think that option 3 is the best because it would invol= ve a bit of work up front to make it much easier to expose parts of DPDK to= DTS. This is because after the initial work is done, we would just need to= write a wrapper function in C to expose the functionality we want, and the= n expose the wrapper function via RPC.=C2=A0


If you have thoughts or suggestions, we wi= ll be discussing this at the DTS working group meeting.

2:00 PM UTC, Wednesday, February 16
https://armlt= d.zoom.us/j/97503259680?pwd=3DVVlmWnlnTXJkVGkwR2JOU3R3b3Vndz09&from=3Da= ddon

--0000000000000b3d3905d7887ed3--