From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60B8A43092; Thu, 17 Aug 2023 19:03:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3AF9D40ED9; Thu, 17 Aug 2023 19:03:02 +0200 (CEST) Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by mails.dpdk.org (Postfix) with ESMTP id DBE6A4013F for ; Thu, 17 Aug 2023 19:03:00 +0200 (CEST) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-589f784a972so598367b3.0 for ; Thu, 17 Aug 2023 10:03:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1692291780; x=1692896580; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=yv3eU0eb219V5urnUTOc45GymPLjUcCYsMpgjTtsdk8=; b=c4/WEjeanSLlZW7FJjtChsaCDonl94/91gskKXSZYHj5nbVEbPMa/irSXd+NVcV4D+ seRBXxVMol1xDA48q8UXQq+RN7lrf3n9HGWJXrl9LPc2a9KS9tN9G9ay2t5N8kfWsrCh Ne9TTETur0Vn0GNljOWNrMT9IgPGXnX/Vt1vU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692291780; x=1692896580; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yv3eU0eb219V5urnUTOc45GymPLjUcCYsMpgjTtsdk8=; b=KUitoCdu/SycJeZy7dU5gm/w3cvruOaUdddGKr4xgTnxDjGlWI0yU0NGtWds6oMTtu h8vSiHGbaRNv/JoHsq23/ClW1fYSxh53rs8fjrtEJGNBHfsJ8wKRurFv+Xp3DGsQ8621 tX5QkQKjK66/a5XSSkNY6rxaOXgAxbCofRhfZ71ObPDJ14Yropajar+lNv3rGefpVw1c sivYWRhGMnVobincIQnTIUvc4t37xJSTUlBEnwl5uzjScjz6NK4pj6PGJGQKzpr+pOuJ WsYRNNBNB6xMnyi+p5dzDABcJXJbAGLWS5+oAJqqyMTVF1MHBx40jzflAhtazF1oOCEM aEpw== X-Gm-Message-State: AOJu0YxQeAMZBIHxtEAbiBifniyoJWeOhVTjr7on4qVVhQrQGfGAT3uv D3Aypg4nlU6s3HiPWttUTHF85EK5pUgE3t7102z4lA== X-Google-Smtp-Source: AGHT+IGRKEBlVGQN6KTnTp8EecnxdNBtDuKO9cl/VVtosxAwpjP3hDx0fFEi1giMIOZIbxtU+kzojmWSgTc6dKtkvb4= X-Received: by 2002:a81:4811:0:b0:577:60d4:a879 with SMTP id v17-20020a814811000000b0057760d4a879mr5538977ywa.33.1692291779964; Thu, 17 Aug 2023 10:02:59 -0700 (PDT) MIME-Version: 1.0 References: <35340484-1d7e-7e5f-cad4-c965ba541397@oktetlabs.ru> In-Reply-To: <35340484-1d7e-7e5f-cad4-c965ba541397@oktetlabs.ru> From: Adam Hassick Date: Thu, 17 Aug 2023 13:03:20 -0400 Message-ID: Subject: Re: Setting up DPDK PMD Test Suite To: Andrew Rybchenko Cc: Patrick Robb , Konstantin Ushakov , ci@dpdk.org Content-Type: multipart/alternative; boundary="00000000000066c6880603216271" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --00000000000066c6880603216271 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Andrew, I'm adding the CI mailing list to this conversation. Others in the community might find this conversation valuable. We do want to run testing on a regular basis. The Jenkins integration will be very useful for us, as most of our CI is orchestrated by Jenkins. I've conferred with the rest of the team, and we think it would be best to move forward with mainly option B. If you would like to know anything about our testbeds that would help you with creating an example ts-rigs repo, I'd be happy to answer any questions you have. We have multiple test rigs (we call these "DUT-tester pairs") that we run our existing hardware testing on, with differing network hardware and CPU architecture. I figured this might be an important detail. Thanks, Adam On Thu, Aug 17, 2023 at 11:44=E2=80=AFAM Andrew Rybchenko < andrew.rybchenko@oktetlabs.ru> wrote: > Greatings Adam, > > I'm happy to hear that you're trying to bring it up. > > As I understand the final goal is to run it on regular basis. So, we need > to make it properly from the very beginning. > Bring up of all features consists of 4 steps: > > 1. Create site-specific repository (we call it ts-rigs) which contains > information about test rigs and other site-specific information like wher= e > to send mails, where to store logs etc. It is required for manual executi= on > as well, since test rigs description is essential. I'll return to the top= ic > below. > > 2. Setup logs storage for automated runs. Basically it is a disk space > plus apache2 web server with few CGI scripts which help a lot to save dis= k > space. > > 3. Setup Bublik web application which provides web interface to view > testing results. Same as https://ts-factory.io/bublik > > 4. Setup Jenkins to run tests on regularly, save logs in log storage (2) > and import it to bublik (3). > > Last few month we spent on our homework to make it simpler to bring up > automated execution using Jenkins - > https://github.com/ts-factory/te-jenkins > Corresponding bits in dpdk-ethdev-ts will be available tomorrow. > > Let's return to the step (1). > > Unfortunately there is no publicly available example of the ts-rigs > repository since sensitive site-specific information is located there. Bu= t > I'm ready to help you to create it for UNH. I see two options here: > > (A) I'll ask questions and based on your answers will create the first > draft with my comments. > > (B) I'll make a template/example ts-rigs repo, publish it and you'll > create UNH ts-rigs based on it. > > Of course, I'll help to debug and finally bring it up in any case. > > (A) is a bit simpler for me and you, but (B) is a bit more generic and > will help other potential users to bring it up. > We can combine (A)+(B). I.e. start from (A). What do you think? > > Thanks, > Andrew. > > On 8/17/23 15:18, Konstantin Ushakov wrote: > > Greetings Adam, > > > Thanks for contacting us. I copy Andrew who would be happy to help > > Thanks, > Konstantin > > On 16 Aug 2023, at 21:50, Adam Hassick > wrote: > > =EF=BB=BF > Greetings Konstantin, > > I am in the process of setting up the DPDK Poll Mode Driver test suite as > an addition to our testing coverage for DPDK at the UNH lab. > > I have some questions about how to set the test suite arguments. > > I have been able to configure the Test Engine to connect to the hosts in > the testbed. The RCF, Configurator, and Tester all begin to run, however > the prelude of the test suite fails to run. > > https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters > > The documentation mentions that there are several test parameters for the > test suite, like for the IUT test link MAC, etc. These seem like they wou= ld > need to be set somewhere to run many of the tests. > > I see in the Test Engine documentation, there are instructions on how to > create new parameters for test suites in the Tester configuration, but > there is nothing in the user guide or in the Tester guide for how to set > the arguments for the parameters when running the test suite that I can > find. I'm not sure if I need to write my own Tester config, or if I shoul= d > be setting these in some other way. > > How should these values be set? > > I'm also not sure what environment variables/arguments are strictly > necessary or which are optional. > > Regards, > Adam > > -- > *Adam Hassick* > Senior Developer > UNH InterOperability Lab > ahassick@iol.unh.edu > iol.unh.edu > +1 (603) 475-8248 > > > --=20 *Adam Hassick* Senior Developer UNH InterOperability Lab ahassick@iol.unh.edu iol.unh.edu +1 (603) 475-8248 --00000000000066c6880603216271 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Andrew,

I'm adding the = CI mailing list to this conversation. Others in the community might find th= is conversation valuable.

We do want to run testing= on a regular basis. The Jenkins integration will be very useful for us, as= most of our CI is orchestrated by Jenkins.
I've conferre= d with the rest of the team, and we think it would be best to move forward = with mainly option B.
If you would like to know anything abou= t our testbeds that would help you with creating an example ts-rigs repo, I= 'd be happy to answer any questions you have.

<= div>We have multiple test rigs (we call these "DUT-tester pairs")= that we run our existing hardware testing on, with differing network hardw= are and CPU architecture. I figured this might be an important detail.
<= /div>

Thanks,
Adam

On Thu, Aug 17= , 2023 at 11:44=E2=80=AFAM Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru> wrote:
=20 =20 =20
Greatings Adam,

I'm happy to hear that you're trying to bring it up.

As I understand the final goal is to run it on regular basis. So, we need to make it properly from the very beginning.
Bring up of all features consists of 4 steps:

1. Create site-specific repository (we call it ts-rigs) which contains information about test rigs and other site-specific information like where to send mails, where to store logs etc. It is required for manual execution as well, since test rigs description is essential. I'll return to the topic below.

2. Setup logs storage for automated runs. Basically it is a disk space plus apache2 web server with few CGI scripts which help a lot to save disk space.

3. Setup Bublik web application which provides web interface to view testing results. Same as https://ts-factory.io/bublik

4. Setup Jenkins to run tests on regularly, save logs in log storage (2) and import it to bublik (3).

Last few month we spent on our homework to make it simpler to bring up automated execution using Jenkins - https://github.com/ts-factory/te-jenkins
Corresponding bits in dpdk-ethdev-ts will be available tomorrow.

Let's return to the step (1).

Unfortunately there is no publicly available example of the ts-rigs repository since sensitive site-specific information is located there. But I'm ready to help you to create it for UNH. I see two options here:

(A) I'll ask questions and based on your answers will create the first draft with my comments.

(B) I'll make a template/example ts-rigs repo, publish it and you'll create UNH ts-rigs based on it.

Of course, I'll help to debug and finally bring it up in any case= .

(A) is a bit simpler for me and you, but (B) is a bit more generic and will help other potential users to bring it up.
We can combine (A)+(B). I.e. start from (A). What do you think?

Thanks,
Andrew.

On 8/17/23 15:18, Konstantin Ushakov wrote:
=20 Greetings Adam,


Thanks for contacting us. I copy Andrew who would be happy to help

Thanks,
Konstantin

On 16 Aug 2023, at 21:50, Adam Hassick <a= hassick@iol.unh.edu> wrote:

=EF=BB=BF
Greetings Konstantin,

I am in the process of setting up the DPDK Poll Mode Driver test suite as an addition to our testing coverage for DPDK at the UNH lab.

I have some questions about how to set the test suite arguments.

I have been able to configure the Test Engine to connect to the hosts in the testbed. The RCF, Configurator, and Tester all begin to run, however the prelude of the test suite fails to run.
The documentation mentions that there are several test parameters for the test suite, like for the IUT test link MAC, etc. These seem like they would need to be set somewhere to run many of the tests.

I see in the Test Engine documentation, there are instructions on how to create new parameters for test suites in the Tester configuration, but there is nothing in the user guide or in the Tester guide for how to set the arguments for the parameters when running the test suite that I can find. I'm not sure if I need to write my own Tester config, or if I should be setting these in some other way.

How should these values be set?

I'm also not sure what environment variables/arguments are strictly necessary or which are optional.

Regards,
Adam

-- =
Adam Hassick
S= enior Developer
<= span style=3D"color:rgb(11,83,148)">UNH InterOperability Lab
<= a href=3D"mailto:ahassick@iol.unh.edu" target=3D"_blank">ahassick@iol.unh.e= du
<= a href=3D"https://www.iol.unh.edu/" target=3D"_blank">iol.unh.edu
+1 (603) 475-8248



--
Adam Hassick
Senior Developer
UNH InterOperability Lab
+1 (603) 475-8248
--00000000000066c6880603216271--