From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by dpdk.org (Postfix) with ESMTP id 0D05D1B49F for ; Fri, 4 Jan 2019 16:33:45 +0100 (CET) Received: by mail-ed1-f51.google.com with SMTP id p6so32195527eds.0 for ; Fri, 04 Jan 2019 07:33:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6Z/V3jaZDwubDjfgXB60ZbfyszeJ0mEcXdT4T3MO6VM=; b=STrGl3WF36UZPI+jJTmY80qaEDWJ9rbwsbR8NxPhICZ/mfOMTa2qQrbUxoI374ZFBQ 2hNPTMv6rjnhCuSCt9ApNn28OJ8QPjSYoyejaIjzkxm/MasRlCSJ1cIUHoVA04QJK/tK FdTCvcrPu8aOdrrYY3aXLl1nN03HewpezGd6IgfyarsXzz9qme2Zyunpj1JgeyA1hFOu qlgpGWpSCl/LhUKjjEO2ZC2sI0eQunq6cmSLoXr+I5ZcUUeECyTKDjd/WIiXnaOMsPcs OOLsn1PWBxBJPT6P/+Yce1r94bhJisdS/tElGWfOgyRnjJE8RLoB8kpF2F2tma4JnkTQ egFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6Z/V3jaZDwubDjfgXB60ZbfyszeJ0mEcXdT4T3MO6VM=; b=ZX2OTLP9BBeN0TcIMl2SbpsA3xc/Whxwn/juGOSCNCO6wQY1JOChbgbaeDWgEuWefq f2mw1+cD1MNrl3WJ1777pE6tBaN6BWFZmoF2KG4uIXKw5+gOGtn4V/rH+H6UokGVsOFb MAZUMR9SqaWTgRWUnWIB8drygpYeVEUpnuMk1KfSZ0WVRMw6pwNhd/nLHWFUKMGGZ0WR c23A9JdzI0r1755We/x+Rnr6QL6vDSshrmwszChmd4bydy1+Rg0oJmjGFuN1RkHF9a5X QSUeFwzIoquSA8o5JSXDqivvn6XyKvof1ceVQd1/22KvMQXZZaX3CqZ30kHCxN0PLd1e wDgw== X-Gm-Message-State: AJcUukdBf9ogw/fklcajb4CnJXw8u2bGPjcqS+8ZM+HKzPIQ2Cu8fX9D QoNw0Il3FrOY9XjPW/HOpzgiL1z4u5ibNI0+OFRNNYtf X-Google-Smtp-Source: ALg8bN5cSDBC3FhnqHFAmR+Yy1yR/LywA14aS31wukJ2tixqtfWT0zsO7DWnCfCi0udY8pvmkY7y3ZCBsSjBSVJIpWM= X-Received: by 2002:a17:906:4a0d:: with SMTP id w13-v6mr11484909eju.128.1546616024566; Fri, 04 Jan 2019 07:33:44 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Rami Rosen Date: Fri, 4 Jan 2019 17:33:34 +0200 Message-ID: To: Jeremy Plsek Cc: ci@dpdk.org Content-Type: multipart/alternative; boundary="000000000000bf9153057ea39a92" Subject: Re: [dpdk-ci] Question about performance test X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 15:33:45 -0000 --000000000000bf9153057ea39a92 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Jeremy, Thanks for your quick reply, it fully clarifies the query I had posted. I do not see the actual results of the DTS perf test in the links you posted, only percentage of degradation or improvement, unless I miss something. I believe it can be helpful if the baseline of the actual results will also be shown to enable comparing to other vendors besides Intel and Mellanox. Regards, Rami Rosen =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=95=D7=B3, 4 =D7= =91=D7=99=D7=A0=D7=95=D7=B3 2019, 16:44, =D7=9E=D7=90=D7=AA Jeremy Plsek : > Hi Rami, > > I'm the current maintainer of the DPDK Performance CI. I realize that the > performance results don't point to the website, so it's not obvious on > where to find this information. You can find an overview of these tests > here: https://lab.dpdk.org > > Most of this information can be either found on the detailed results of a > test (such as https://lab.dpdk.org/results/dashboard/patchsets/4157/) or > on the about page (https://lab.dpdk.org/results/dashboard/about/). > > But to answer your questions: > At the moment, we only run performance tests. Specifically the > nic_single_core_perf_test from the DPDK Test Suite with the TRex traffic > generator. > The devices we are testing are currently the Intel 82599ES 10G, the Intel > XL710-QDA2 40G, the Mellanox ConnectX-5 100G, and the ConnectX-4 Lx 25G a= nd > 40G. > > We don't apply the doc folder when applying the series, in case a patch > included code unrelated to documentation. If others in the group feel tha= t > it's still unnecessary to include "doc" labeled series, I can look into > filtering them out. > > Thanks. > > > On Fri, Jan 4, 2019 at 5:46 AM Rami Rosen wrote: > >> Hi, >> I have a question about DPDK CI process and the tests done when a patch >> is submitted to dpdk-dev mailing list. >> In DPDK patch work I see these response messages from the DPDK CI for al= l >> patches: >> >> ... >> ci/intel-Performance-Testing success Performance Testing PASS >> ci/mellanox-Performance-Testing success Performance Testing PASS >> ... >> >> My question is (I hope and believe the info is available publicly) : >> which tests are run in the ci, generating these messages? is it done wi= th >> IXIA and DTS ? (DPDK test suite, https://doc.dpdk.org/dts/gsg/) ? are >> these l2fwd/l3fwd performance tests? or more than that ? and on which >> Intel/Mellanox nics ? Are these merely performance tests, or also >> functional tests ? >> >> And BTW, I noticed that the CI runs a full performance cycle also for do= c >> patches (at least these messages are generated), which is a kind of >> redundant (unless there is some filter which checks that if a patch only >> affects modules under "doc", than such a cycle is not done but the messa= ges >> are still sent) >> >> Regards, >> Rami Rosen >> > > > -- > Jeremy Plsek > UNH InterOperability Laboratory > > --000000000000bf9153057ea39a92 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Jeremy,

<= div dir=3D"auto">
Thanks for your quick reply, i= t fully clarifies the query I had posted.

=
I do not see the actual results of the DTS perf test in t= he links you posted, only percentage of degradation or improvement, unless = I miss something. I believe it can be helpful if the baseline of the actual= results will also be shown=C2=A0
to enable comparin= g to other vendors besides Intel and Mellanox.

<= /div>
Regards,
Rami Rosen



<= /div>


=D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=95=D7=B3, 4 = =D7=91=D7=99=D7=A0=D7=95=D7=B3 2019, 16:44, =D7=9E=D7=90=D7=AA Jeremy Plsek= <jplsek@iol.unh.edu>:
<= /div>
Hi Rami,
=

I'm the current maintainer of the DPDK Performance = CI. I realize that the performance results don't point to the website, = so it's not obvious on where to find this information. You can find an = overview of these tests here: https://lab.dpdk.org

= Most of this information can be either found on the detailed results of a t= est (such as https://lab.dpdk.org/results/dash= board/patchsets/4157/) or on the about page (https:= //lab.dpdk.org/results/dashboard/about/).

But = to answer your questions:
At the moment, we only run performance = tests. Specifically the nic_single_core_perf_test from the DPDK Test Suite = with the TRex traffic generator.
The devices we are testing are c= urrently the Intel 82599ES 10G, the Intel XL710-QDA2 40G, the Mellanox Conn= ectX-5 100G, and the ConnectX-4 Lx 25G and 40G.

We= don't apply the doc folder when applying the series, in case a patch i= ncluded code unrelated to documentation. If others in the group feel that i= t's still unnecessary to include "doc" labeled series, I can = look into filtering them out.

Thanks.

On Fri, Jan 4, 2019 at 5:46 AM= Rami Rosen <ramirose@gmail.com> wrote:
Hi,
I have a question about DPDK CI process and the tests don= e when a patch is submitted to dpdk-dev mailing list.
In DPDK pat= ch work I see these response messages from the DPDK CI for all patches:

...
ci/intel-Performance-Testing success Performance Testing = PASS
ci/mellanox-= Performance-Testing success Performance Testing PASS=C2=A0
...

My question is (I hope and believe the = info is available publicly) : which tests are run in the ci, generating the= se messages?=C2=A0 is it done with IXIA and DTS ? (DPDK test suite,=C2=A0https://doc.dpdk.org/dts/gsg/) ? are these l2fwd/l3fwd performance te= sts? or more than that ? and on which Intel/Mellanox nics ?=C2=A0=C2=A0Are these merely perfor= mance tests, or also=C2=A0
functional tests ?=C2=A0

= And BTW, I noticed that the CI runs a full performance cycle also for doc p= atches (at least these messages are generated), which is a kind of redundan= t (unless there is some filter which checks that if a patch only affects mo= dules under "doc", than such a cycle is not done but the messages= are still sent)

Regards,
Rami Rosen
=


--
Jeremy Plsek
UNH InterOperability Laboratory

=
--000000000000bf9153057ea39a92--