From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1AAEE438F7; Thu, 18 Jan 2024 20:52:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5C1740EE7; Thu, 18 Jan 2024 20:52:29 +0100 (CET) Received: from mail-oo1-f54.google.com (mail-oo1-f54.google.com [209.85.161.54]) by mails.dpdk.org (Postfix) with ESMTP id 54AC940E2D for ; Thu, 18 Jan 2024 20:52:28 +0100 (CET) Received: by mail-oo1-f54.google.com with SMTP id 006d021491bc7-598cf18b259so9781eaf.1 for ; Thu, 18 Jan 2024 11:52:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1705607547; x=1706212347; darn=dpdk.org; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=12qOdkJc4xhA0Ob3DA+6lC1vF58by4dHl6Z7oSaWRR8=; b=Mowv05awEP2MvbGHx4NF/i9VqL5XyJeFDfoBOecAQ9q7dxfU0yUAepY2C3LI0ByQlX xSA6if9iE9D3DXITeJwLMePSLwgbS5ZBrcuU56dQ2F57wUhDMU37bJStKJqhm1frjQG4 xX8WU+fH6EsNFaqWe6D4AQWh2yyvmU73LG0/g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705607547; x=1706212347; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=12qOdkJc4xhA0Ob3DA+6lC1vF58by4dHl6Z7oSaWRR8=; b=gbcoDsMWAaHHx8BkIGnrVDH8AFH981fUERAiihl+xnhn8JrwOqBAJI6b23XqFaZj45 vrFGnPz2faJ9MhZRdfDM+39DunvITGS/QhbAhHi8rxdJTwKghr9hR+AeuxaziKILTyG8 okFpJ8y6hX/cvkCYN4bWHVYCiCLpF6NRV0nRrTBw6pdjqS6RtN5eCOwVLk2qznNlgcHz HqdHODemRdOF+6GIEiIjk6f8eupWohr/wfnu1GWq0Req79GjCvM2nJzuUJfDuHWshyOi /WEpT7L4dAmdqZXCIytPx3tTL2CRF1sPmd1Cg4a+NhYKr6XaFkALCqvhWsM4WYiuZTbf +ZYw== X-Gm-Message-State: AOJu0YyrRoqQ9TorMsmLvLBNReEqslh3NWeFLU1vaVVlO9cW06eZUEY2 JDIogvdk3rRJ1Rv/WE05oVFj8qFIVSt7iubLbN626dhPYkH/AOJe+izLjF683sggrnaO6YfvlP8 YGoRf9W8BAZ47z392kMKy2d0JLkxDl1IqggUDsUUJoOXGbyQAtQM= X-Google-Smtp-Source: AGHT+IEag9wuVc48hdA1w0Jp6zUMiUSfMG53UNTjpHa5aoyUPzXJpopHXNVRU5CqhcsbIB9PN+DzysUtPEHvyIcrYiM= X-Received: by 2002:a4a:3c13:0:b0:599:4cca:3f9b with SMTP id d19-20020a4a3c13000000b005994cca3f9bmr510321ooa.13.1705607547479; Thu, 18 Jan 2024 11:52:27 -0800 (PST) MIME-Version: 1.0 From: Patrick Robb Date: Thu, 18 Jan 2024 14:52:16 -0500 Message-ID: Subject: DTS WG Meeting Minutes - January 17, 2024 To: dts@dpdk.org Cc: ci@dpdk.org, "NBU-Contact-Thomas Monjalon (EXTERNAL)" , =?UTF-8?Q?Juraj_Linke=C5=A1?= Content-Type: multipart/alternative; boundary="000000000000fe8170060f3db3a2" X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org --000000000000fe8170060f3db3a2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable January 17, 2024 ##################################################################### Attendees * Juraj Linke=C5=A1 * Gregory Etelson * Honnappa Nagarahalli * Jeremy Spewock * Luca Vizzarro * Paul Szczepanek ##################################################################### Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * 24.03 roadmap ##################################################################### Minutes =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Additions to the agenda * Nothing =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D YAML test suites * Greg wanted to automate his testing; started with test writtens in Python, but was not scalable; easily understandable by newcomers. * The idea is to take an application, send commands (interactive input), collect output and compare with expected strings. * The code was available as of two months ago, but no longer is (private on GitHub). Greg may be able to share it once taking care of it in his company= . * Gregory submitted an idea for writing test suites in yaml, which just passes values into a templated testpmd testsuite. * Do we want to support a secondary way of writing test suites? * Will this be usable for both functional and performance testing? * Will this coexist well with the current method? * The current method also aims to be minimalistic and intuitive * Coexistence makes sense as the yaml approach may not be able to cover more complicated cases * What are any limitations which this places on the testing framework? If there aren=E2=80=99t major downsides, then the benefits in terms of quic= kly adding new testpmd testsuites seems significant. * The traffic generator can't be configured here, we need an abstraction that works for all traffic generators; we can mark the test cases as functional/performance though, which could be enough * We can only specify test-specific testpmd cmdline options; shouldn't be a problem, but we have to keep in mind that configuration such as cores and pci addresses are configured elsewhere (the testbed configuration) * Using specific strings in testpmd is harder to maintain (if the same config is used in multiple places) * Are the phases for both setup/teardown and test cases? This could complicate results recording * Can we easily specify multiple test cases? I.e. we have a test method and we want to test different input combinations (the inputs could just be the number of cores/packet size for performance tests) --000000000000fe8170060f3db3a2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
January 17, 2024

##################################= ###################################
Attendees
* Juraj Linke=C5=A1
= * Gregory Etelson
* Honnappa Nagarahalli
* Jeremy Spewock
* Luca V= izzarro
* Paul Szczepanek

#######################################= ##############################
Agenda
* Additions to the agenda
* = Patch discussions
* DTS Developer documentation
* 24.03 roadmap
#####################################################################
= Minutes

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DAdditions to the agenda
* Nothing

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D
YAML test suites
* Greg wanted to automat= e his testing; started with test writtens in Python, but was not scalable; = easily understandable by newcomers.
=C2=A0 =C2=A0* The idea is to take a= n application, send commands (interactive input), collect output and compar= e with expected strings.
* The code was available as of two months ago, = but no longer is (private on GitHub). Greg may be able to share it once tak= ing care of it in his company.
* Gregory submitted an idea for writing t= est suites in yaml, which just passes values into a templated testpmd tests= uite.
=C2=A0 =C2=A0* Do we want to support a secondary way of writing te= st suites?
=C2=A0 =C2=A0 =C2=A0 * Will this be usable for both functiona= l and performance testing?
=C2=A0 =C2=A0* Will this coexist well with th= e current method?
=C2=A0 =C2=A0 =C2=A0 * The current method also aims to= be minimalistic and intuitive
=C2=A0 =C2=A0 =C2=A0 * Coexistence makes = sense as the yaml approach may not be able to cover more complicated cases<= br>=C2=A0 =C2=A0* What are any limitations which this places on the testing= framework? If there aren=E2=80=99t major downsides, then the benefits in t= erms of quickly adding new testpmd testsuites seems significant.
=C2=A0 = =C2=A0 =C2=A0 * The traffic generator can't be configured here, we need= an abstraction that works for all traffic generators; we can mark the test= cases as functional/performance though, which could be enough
=C2=A0 = =C2=A0 =C2=A0 * We can only specify test-specific testpmd cmdline options; = shouldn't be a problem, but we have to keep in mind that configuration = such as cores and pci addresses are configured elsewhere (the testbed confi= guration)
=C2=A0 =C2=A0 =C2=A0 * Using specific strings in testpmd is ha= rder to maintain (if the same config is used in multiple places)
=C2=A0 = =C2=A0* Are the phases for both setup/teardown and test cases? This could c= omplicate results recording
=C2=A0 =C2=A0* Can we easily specify multipl= e test cases? I.e. we have a test method and we want to test different inpu= t combinations (the inputs could just be the number of cores/packet size fo= r performance tests)
--000000000000fe8170060f3db3a2--