From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by dpdk.org (Postfix) with ESMTP id F150A1B48B for ; Fri, 4 Jan 2019 17:16:26 +0100 (CET) Received: by mail-oi1-f174.google.com with SMTP id m6so30714190oig.11 for ; Fri, 04 Jan 2019 08:16:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SzuP+Z0IixDj0uYd6JkvJC3zm5sBbpsToPkSe5YArSI=; b=L/a41Mcv2iMsBK8bDbfh1UhbJ1PfMu1kBq58fIgKQcRjcPcrf28kMjgFYVeRA2s8t0 DNgdr7nXThHoAK/n4QxkYQMdFqJKuZJ4K4VBWxe2iEGMudowf1xdxqrms1uLGGbptp9F G7vW1166Q3AC8KRhcLiSdBZXK3cf1AnclGblM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SzuP+Z0IixDj0uYd6JkvJC3zm5sBbpsToPkSe5YArSI=; b=kG4lqFj3Q9xsjzeeXmhlIN9keeIYWnlrxCTtQUKw5S0xmoQkGGRy/esGHM+VYX1Aao Ah2OPpydTVVKDAChL4dDO2Ga+PHzJ0oz/9EDMTGdvFkzuy4ohZ6ahwz5ijJteSsTFzcN Ebh6Uo0AUEKpwj3/C/JD7vvpV9ZWFucRX/e76wMT2t8KpTue+7AFKAxUJgd3ewFCbVld t1nlBcUV2cBbOBqiX+UTpkQ2S9Dhq1eJGv3WfMs4+9+FcfcEmtM6zgKnn5qNG/76thxu flGzz29lt5J4R8ZJXKp/WUVNecY5fpnmbu07Ptmz6ix2Lf9ooJfdoxY6uuZhlmgihNUz 851A== X-Gm-Message-State: AJcUukeqyEbGA9zZv+nIg5tEFSKg4zbycuSrTkOfdALdtVF6dn5Y/qt4 0b7f5egeV8FzNz6s5wMKFyXBPAjZMe/HmNd00K1H2Q== X-Google-Smtp-Source: ALg8bN5/cN/nh8Cqxc88/bHz1Bc8FjXQb9SdUKK9O/qAZxFQAEBmXMrhMz5W7mminSJcJz6slcfvZUp/WS1oVWDTGa8= X-Received: by 2002:aca:a805:: with SMTP id r5mr1528291oie.5.1546618585972; Fri, 04 Jan 2019 08:16:25 -0800 (PST) MIME-Version: 1.0 References: <2631230.hnArF6gTu5@xps> In-Reply-To: <2631230.hnArF6gTu5@xps> From: Jeremy Plsek Date: Fri, 4 Jan 2019 11:15:49 -0500 Message-ID: To: Thomas Monjalon , Rami Rosen Cc: ci@dpdk.org Content-Type: multipart/alternative; boundary="0000000000006ba5cd057ea4338d" Subject: Re: [dpdk-ci] Question about performance test X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 16:16:27 -0000 --0000000000006ba5cd057ea4338d Content-Type: text/plain; charset="UTF-8" > > I do not see the actual results of the DTS perf test in the links you > posted, only percentage of degradation or improvement, unless I miss > something. > This was done on purpose as requested by the Members participating in the effort. This CI is meant for the DPDK maintainers to make sure that a patch does not introduce significant performance regressions on various hardware platforms, or to show how a patch may improve the performance of DPDK on these platforms (such as driver updates or something in the core of DPDK itself). It is not to compare the performance between different devices. I believe it can be helpful if the baseline of the actual results will also > be shown > to enable comparing to other vendors besides Intel and Mellanox. > There is not a single baseline for all devices. The baselines are generated per device. Knowing the baseline would allow creating an absolute result, which we are trying to avoid. Do you plan to add such URL in the report sent to patchwork? > Created a ticket to add the URL to the emailed reports: https://bugs.dpdk.org/show_bug.cgi?id=180 Instead of filtering based on the label, you could filter based on > the paths of modified files. > Note that such filter depends on the test you run, > because you could also test the doc syntax in the CI. > Okay, so if a series only modified the doc folder, then don't include it for performance tests. Later on, we can introduce syntax checking for documentation when we introduce more unit testing / functional testing. https://bugs.dpdk.org/show_bug.cgi?id=181 On Fri, Jan 4, 2019 at 10:34 AM Thomas Monjalon wrote: > 04/01/2019 15:44, Jeremy Plsek: > > Hi Rami, > > > > I'm the current maintainer of the DPDK Performance CI. I realize that the > > performance results don't point to the website, so it's not obvious on > > where to find this information. You can find an overview of these tests > > here: https://lab.dpdk.org > > > > Most of this information can be either found on the detailed results of a > > test (such as https://lab.dpdk.org/results/dashboard/patchsets/4157/) > or on > > the about page (https://lab.dpdk.org/results/dashboard/about/). > > Do you plan to add such URL in the report sent to patchwork? > > > We don't apply the doc folder when applying the series, in case a patch > > included code unrelated to documentation. If others in the group feel > that > > it's still unnecessary to include "doc" labeled series, I can look into > > filtering them out. > > Instead of filtering based on the label, you could filter based on > the paths of modified files. > Note that such filter depends on the test you run, > because you could also test the doc syntax in the CI. > > > -- Jeremy Plsek UNH InterOperability Laboratory --0000000000006ba5cd057ea4338d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I do not see the actual results of the DTS perf test in the= links you=20 posted, only percentage of degradation or improvement, unless I miss=20 something.

This was done on purpose a= s requested by the Members participating in the effort. This CI is meant fo= r the DPDK maintainers to make sure that a patch does not introduce signifi= cant performance regressions on various hardware platforms, or to show how = a patch may improve the performance of DPDK on these platforms (such as dri= ver updates or something in the core of DPDK itself). It is not to compare = the performance between different devices.

I believe it can be helpful= if the baseline of the actual results will also be shown=C2=A0
to enable comparing to other vendors besides Intel and Mellanox.

There is not a single baseline for = all devices. The baselines are generated per device. Knowing the baseline w= ould allow creating an absolute result, which we are trying to avoid.

Do you plan to add such URL in the report sent to patchwork?

Cr= eated a ticket to add the URL to the emailed reports: https://bugs.dpdk.org/= show_bug.cgi?id=3D180

Instead of filtering base= d on the label, you could filter based on
the paths of modified files.
Note that such filter depends on the test you run,
because you could also test the doc syntax in the CI.

Okay, so if a series only modified the doc folder, then do= n't include it for performance tests. Later on, we can introduce syntax= checking for documentation when we introduce more unit testing / functiona= l testing. https://bugs.dpdk.org/show_bug.cgi?id=3D181
=

On Fri, Jan 4, 2019 a= t 10:34 AM Thomas Monjalon <thomas@monjalon.net> wrote:
04/01/2019 15:44, Jeremy Plsek:
> Hi Rami,
>
> I'm the current maintainer of the DPDK Performance CI. I realize t= hat the
> performance results don't point to the website, so it's not ob= vious on
> where to find this information. You can find an overview of these test= s
> here: https://lab.dpdk.org
>
> Most of this information can be either found on the detailed results o= f a
> test (such as https://lab.dpdk.org/result= s/dashboard/patchsets/4157/) or on
> the about page (https://lab.dpdk.org/results/dashb= oard/about/).

Do you plan to add such URL in the report sent to patchwork?

> We don't apply the doc folder when applying the series, in case a = patch
> included code unrelated to documentation. If others in the group feel = that
> it's still unnecessary to include "doc" labeled series, = I can look into
> filtering them out.

Instead of filtering based on the label, you could filter based on
the paths of modified files.
Note that such filter depends on the test you run,
because you could also test the doc syntax in the CI.




--
Jeremy Plsek
UNH InterOpe= rability Laboratory
--0000000000006ba5cd057ea4338d--