From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8778438F6; Thu, 18 Jan 2024 20:52:29 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C153F40E2D; Thu, 18 Jan 2024 20:52:29 +0100 (CET) Received: from mail-oo1-f42.google.com (mail-oo1-f42.google.com [209.85.161.42]) by mails.dpdk.org (Postfix) with ESMTP id 38253402D9 for ; Thu, 18 Jan 2024 20:52:28 +0100 (CET) Received: by mail-oo1-f42.google.com with SMTP id 006d021491bc7-59883168a83so3959eaf.2 for ; Thu, 18 Jan 2024 11:52:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1705607547; x=1706212347; darn=dpdk.org; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=12qOdkJc4xhA0Ob3DA+6lC1vF58by4dHl6Z7oSaWRR8=; b=Mowv05awEP2MvbGHx4NF/i9VqL5XyJeFDfoBOecAQ9q7dxfU0yUAepY2C3LI0ByQlX xSA6if9iE9D3DXITeJwLMePSLwgbS5ZBrcuU56dQ2F57wUhDMU37bJStKJqhm1frjQG4 xX8WU+fH6EsNFaqWe6D4AQWh2yyvmU73LG0/g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705607547; x=1706212347; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=12qOdkJc4xhA0Ob3DA+6lC1vF58by4dHl6Z7oSaWRR8=; b=mF3Nq0QUet7vuqqhgPvV8EC+b3kKfqUMhppgaQQw1QWLSoYkKcEzKRTwfShebTTH+C rSKYuAFD4SHInMbgy0LNYG8ZYjxz7gPT7vSHIKKOxA9M3vuc5MGznXpOswRAoH3N3MWE jSILeGEOOXExJSsCS5UBDyahXZBVh0dyXlU1KFBG8S4+Qrf9F38YAYWuAgiBdfSr4Ovb JeEitRLxEevFo3Qg8pkHuyR2dCaUe4/0MUZcRHUPA/tCTphIpjwILB80YKEadl9szzoO HMVjV/gARuJIs9amA4Hut2tTTGUIbz6WH1YCU86UyTsM4MAcqkenZrSP9rIt3tEIUwF7 /TwQ== X-Gm-Message-State: AOJu0YyWh2B0hcVJhQL5p9WFGghutqFQto0Y2Xvv4aA+aWi4VZIPYEvg kRrCVxjTEAwcd0AejVS90A7Fwrbvy5vdk7uJGpWl5MPHDDtn0CDONsmGSxRt0qw8GJr7hzu3asP vn6Dz/3j6ZI4he1CZpQBfftDQQsiBXF3KqHOyDA== X-Google-Smtp-Source: AGHT+IEag9wuVc48hdA1w0Jp6zUMiUSfMG53UNTjpHa5aoyUPzXJpopHXNVRU5CqhcsbIB9PN+DzysUtPEHvyIcrYiM= X-Received: by 2002:a4a:3c13:0:b0:599:4cca:3f9b with SMTP id d19-20020a4a3c13000000b005994cca3f9bmr510321ooa.13.1705607547479; Thu, 18 Jan 2024 11:52:27 -0800 (PST) MIME-Version: 1.0 From: Patrick Robb Date: Thu, 18 Jan 2024 14:52:16 -0500 Message-ID: Subject: DTS WG Meeting Minutes - January 17, 2024 To: dts@dpdk.org Cc: ci@dpdk.org, "NBU-Contact-Thomas Monjalon (EXTERNAL)" , =?UTF-8?Q?Juraj_Linke=C5=A1?= Content-Type: multipart/alternative; boundary="000000000000fe8170060f3db3a2" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --000000000000fe8170060f3db3a2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable January 17, 2024 ##################################################################### Attendees * Juraj Linke=C5=A1 * Gregory Etelson * Honnappa Nagarahalli * Jeremy Spewock * Luca Vizzarro * Paul Szczepanek ##################################################################### Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * 24.03 roadmap ##################################################################### Minutes =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Additions to the agenda * Nothing =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D YAML test suites * Greg wanted to automate his testing; started with test writtens in Python, but was not scalable; easily understandable by newcomers. * The idea is to take an application, send commands (interactive input), collect output and compare with expected strings. * The code was available as of two months ago, but no longer is (private on GitHub). Greg may be able to share it once taking care of it in his company= . * Gregory submitted an idea for writing test suites in yaml, which just passes values into a templated testpmd testsuite. * Do we want to support a secondary way of writing test suites? * Will this be usable for both functional and performance testing? * Will this coexist well with the current method? * The current method also aims to be minimalistic and intuitive * Coexistence makes sense as the yaml approach may not be able to cover more complicated cases * What are any limitations which this places on the testing framework? If there aren=E2=80=99t major downsides, then the benefits in terms of quic= kly adding new testpmd testsuites seems significant. * The traffic generator can't be configured here, we need an abstraction that works for all traffic generators; we can mark the test cases as functional/performance though, which could be enough * We can only specify test-specific testpmd cmdline options; shouldn't be a problem, but we have to keep in mind that configuration such as cores and pci addresses are configured elsewhere (the testbed configuration) * Using specific strings in testpmd is harder to maintain (if the same config is used in multiple places) * Are the phases for both setup/teardown and test cases? This could complicate results recording * Can we easily specify multiple test cases? I.e. we have a test method and we want to test different input combinations (the inputs could just be the number of cores/packet size for performance tests) --000000000000fe8170060f3db3a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
January 17, 2024

##################################= ###################################
Attendees
* Juraj Linke=C5=A1
= * Gregory Etelson
* Honnappa Nagarahalli
* Jeremy Spewock
* Luca V= izzarro
* Paul Szczepanek

#######################################= ##############################
Agenda
* Additions to the agenda
* = Patch discussions
* DTS Developer documentation
* 24.03 roadmap
#####################################################################
= Minutes

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DAdditions to the agenda
* Nothing

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D
YAML test suites
* Greg wanted to automat= e his testing; started with test writtens in Python, but was not scalable; = easily understandable by newcomers.
=C2=A0 =C2=A0* The idea is to take a= n application, send commands (interactive input), collect output and compar= e with expected strings.
* The code was available as of two months ago, = but no longer is (private on GitHub). Greg may be able to share it once tak= ing care of it in his company.
* Gregory submitted an idea for writing t= est suites in yaml, which just passes values into a templated testpmd tests= uite.
=C2=A0 =C2=A0* Do we want to support a secondary way of writing te= st suites?
=C2=A0 =C2=A0 =C2=A0 * Will this be usable for both functiona= l and performance testing?
=C2=A0 =C2=A0* Will this coexist well with th= e current method?
=C2=A0 =C2=A0 =C2=A0 * The current method also aims to= be minimalistic and intuitive
=C2=A0 =C2=A0 =C2=A0 * Coexistence makes = sense as the yaml approach may not be able to cover more complicated cases<= br>=C2=A0 =C2=A0* What are any limitations which this places on the testing= framework? If there aren=E2=80=99t major downsides, then the benefits in t= erms of quickly adding new testpmd testsuites seems significant.
=C2=A0 = =C2=A0 =C2=A0 * The traffic generator can't be configured here, we need= an abstraction that works for all traffic generators; we can mark the test= cases as functional/performance though, which could be enough
=C2=A0 = =C2=A0 =C2=A0 * We can only specify test-specific testpmd cmdline options; = shouldn't be a problem, but we have to keep in mind that configuration = such as cores and pci addresses are configured elsewhere (the testbed confi= guration)
=C2=A0 =C2=A0 =C2=A0 * Using specific strings in testpmd is ha= rder to maintain (if the same config is used in multiple places)
=C2=A0 = =C2=A0* Are the phases for both setup/teardown and test cases? This could c= omplicate results recording
=C2=A0 =C2=A0* Can we easily specify multipl= e test cases? I.e. we have a test method and we want to test different inpu= t combinations (the inputs could just be the number of cores/packet size fo= r performance tests)
--000000000000fe8170060f3db3a2--