From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C9095438F6; Thu, 18 Jan 2024 21:00:25 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BAA3F40ED0; Thu, 18 Jan 2024 21:00:25 +0100 (CET) Received: from mail-oo1-f41.google.com (mail-oo1-f41.google.com [209.85.161.41]) by mails.dpdk.org (Postfix) with ESMTP id 1467740E2D for ; Thu, 18 Jan 2024 21:00:25 +0100 (CET) Received: by mail-oo1-f41.google.com with SMTP id 006d021491bc7-58e256505f7so5968eaf.3 for ; Thu, 18 Jan 2024 12:00:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1705608024; x=1706212824; darn=dpdk.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=OA1MbrYE8H3U9MjjPUjzVxZS5gPjpz1UN712w1ed1VY=; b=QKh1zr8TK2kZZZM0SDg9h4xqiq1umU1y9shHUDmdqMkaxMSzij1DhP5dSVvohssItj objVUkroY4YsGaq+PQyqPMIZZmod9up9TYPCM5Tc0fc5GxzfiT2k8UOHk7PIidcdHqmZ ph5eR0h5+DVAEnpVoYsNQXZ6uJ3GuICLW2rnU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705608024; x=1706212824; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OA1MbrYE8H3U9MjjPUjzVxZS5gPjpz1UN712w1ed1VY=; b=sThaD0gSnmPqBhnHCW7kqDL6wjhw4s3Dgs18uNje8H1Yy4kt/pPE6kdBCakEQfK3FF d7DDSGJ4vJerPVTA7SIGl9mt8+VSn4yt3mMMWHzruBJb7Pp/mkApZcydMS3NuGyQcrkB q/KUCM7DbTRzrNfkgO3RR+iA+mHR4LqMHqXmK3V5BTJ3Vb5Q9mv7pwHTUzIdPgYLTD4r daPmTuoMwHoQFlAraYjhhe7lxp426TgcdJv6838fdzci16xDeK+awhWG4nwGGw8cgS7W UUMKeF+sz2Mt7+no7fZCpY7BmrWgtufE3Gd0fgBX1tdCTQOobW4JWDPYijVf4bbzNSfN y4GQ== X-Gm-Message-State: AOJu0YxEcEaHy/P1y6fpxcZunnZAOYY+zN7ABBjlUShhNJD+F44+NSsA aAnOHBhgkAO3USbvriOP70zmoSx9aNr/5z+hXruLM0KPhDyX8+NnQlkNT4Moe0ud/mo8Zjuwi2U Snt+QIXTkhqf4APERrojvepQ1IqWadXduDSFn1aBMM4Ujrw69 X-Google-Smtp-Source: AGHT+IGwiJUtr5EuWFtXl0Z4pILeNhlNQQxyDw6y+oXDPmQMGCBlFVErrnibU5ZBHR1ucZicqN5oaMuCraQwG5G6p3I= X-Received: by 2002:a05:6820:160f:b0:594:26c9:5537 with SMTP id bb15-20020a056820160f00b0059426c95537mr2052524oob.6.1705608024163; Thu, 18 Jan 2024 12:00:24 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Patrick Robb Date: Thu, 18 Jan 2024 15:00:13 -0500 Message-ID: Subject: Fwd: DTS WG Meeting Minutes - January 17, 2024 To: dev@dpdk.org Content-Type: multipart/alternative; boundary="000000000000681bae060f3dd0a7" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000681bae060f3dd0a7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I forgot to CC the dev mailing list ---------- Forwarded message --------- From: Patrick Robb Date: Thu, Jan 18, 2024 at 2:52=E2=80=AFPM Subject: DTS WG Meeting Minutes - January 17, 2024 To: Cc: , NBU-Contact-Thomas Monjalon (EXTERNAL) < thomas@monjalon.net>, Juraj Linke=C5=A1 January 17, 2024 ##################################################################### Attendees * Juraj Linke=C5=A1 * Gregory Etelson * Honnappa Nagarahalli * Jeremy Spewock * Luca Vizzarro * Paul Szczepanek ##################################################################### Agenda * Additions to the agenda * Patch discussions * DTS Developer documentation * 24.03 roadmap ##################################################################### Minutes =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Additions to the agenda * Nothing =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D YAML test suites * Greg wanted to automate his testing; started with test writtens in Python, but was not scalable; easily understandable by newcomers. * The idea is to take an application, send commands (interactive input), collect output and compare with expected strings. * The code was available as of two months ago, but no longer is (private on GitHub). Greg may be able to share it once taking care of it in his company= . * Gregory submitted an idea for writing test suites in yaml, which just passes values into a templated testpmd testsuite. * Do we want to support a secondary way of writing test suites? * Will this be usable for both functional and performance testing? * Will this coexist well with the current method? * The current method also aims to be minimalistic and intuitive * Coexistence makes sense as the yaml approach may not be able to cover more complicated cases * What are any limitations which this places on the testing framework? If there aren=E2=80=99t major downsides, then the benefits in terms of quic= kly adding new testpmd testsuites seems significant. * The traffic generator can't be configured here, we need an abstraction that works for all traffic generators; we can mark the test cases as functional/performance though, which could be enough * We can only specify test-specific testpmd cmdline options; shouldn't be a problem, but we have to keep in mind that configuration such as cores and pci addresses are configured elsewhere (the testbed configuration) * Using specific strings in testpmd is harder to maintain (if the same config is used in multiple places) * Are the phases for both setup/teardown and test cases? This could complicate results recording * Can we easily specify multiple test cases? I.e. we have a test method and we want to test different input combinations (the inputs could just be the number of cores/packet size for performance tests) --000000000000681bae060f3dd0a7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I forgot to CC the dev mailing list

---------- Forwarded mess= age ---------
From: Patr= ick Robb <probb@iol.unh.edu>
Date: Thu, Jan 18, 2024 at 2:52=E2= =80=AFPM
Subject: DTS WG Meeting Minutes - January 17, 2024
To: <= dts@dpdk.org>
Cc: <ci@dpdk.org>, NBU-Contact-Thomas Monjalon (EXTE= RNAL) <thomas@monjalon.net>= ;, Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech>


January 17, 2024

####################################= #################################
Attendees
* Juraj Linke=C5=A1
* = Gregory Etelson
* Honnappa Nagarahalli
* Jeremy Spewock
* Luca Viz= zarro
* Paul Szczepanek

#########################################= ############################
Agenda
* Additions to the agenda
* Pa= tch discussions
* DTS Developer documentation
* 24.03 roadmap

= #####################################################################
Mi= nutes

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3DAdditions to the agenda
* Nothing

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D
YAML test suites
* Greg wanted to automat= e his testing; started with test writtens in Python, but was not scalable; = easily understandable by newcomers.
=C2=A0 =C2=A0* The idea is to take a= n application, send commands (interactive input), collect output and compar= e with expected strings.
* The code was available as of two months ago, = but no longer is (private on GitHub). Greg may be able to share it once tak= ing care of it in his company.
* Gregory submitted an idea for writing t= est suites in yaml, which just passes values into a templated testpmd tests= uite.
=C2=A0 =C2=A0* Do we want to support a secondary way of writing te= st suites?
=C2=A0 =C2=A0 =C2=A0 * Will this be usable for both functiona= l and performance testing?
=C2=A0 =C2=A0* Will this coexist well with th= e current method?
=C2=A0 =C2=A0 =C2=A0 * The current method also aims to= be minimalistic and intuitive
=C2=A0 =C2=A0 =C2=A0 * Coexistence makes = sense as the yaml approach may not be able to cover more complicated cases<= br>=C2=A0 =C2=A0* What are any limitations which this places on the testing= framework? If there aren=E2=80=99t major downsides, then the benefits in t= erms of quickly adding new testpmd testsuites seems significant.
=C2=A0 = =C2=A0 =C2=A0 * The traffic generator can't be configured here, we need= an abstraction that works for all traffic generators; we can mark the test= cases as functional/performance though, which could be enough
=C2=A0 = =C2=A0 =C2=A0 * We can only specify test-specific testpmd cmdline options; = shouldn't be a problem, but we have to keep in mind that configuration = such as cores and pci addresses are configured elsewhere (the testbed confi= guration)
=C2=A0 =C2=A0 =C2=A0 * Using specific strings in testpmd is ha= rder to maintain (if the same config is used in multiple places)
=C2=A0 = =C2=A0* Are the phases for both setup/teardown and test cases? This could c= omplicate results recording
=C2=A0 =C2=A0* Can we easily specify multipl= e test cases? I.e. we have a test method and we want to test different inpu= t combinations (the inputs could just be the number of cores/packet size fo= r performance tests)


--000000000000681bae060f3dd0a7--