From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <ramirose@gmail.com> Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by dpdk.org (Postfix) with ESMTP id 0D05D1B49F for <ci@dpdk.org>; Fri, 4 Jan 2019 16:33:45 +0100 (CET) Received: by mail-ed1-f51.google.com with SMTP id p6so32195527eds.0 for <ci@dpdk.org>; Fri, 04 Jan 2019 07:33:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6Z/V3jaZDwubDjfgXB60ZbfyszeJ0mEcXdT4T3MO6VM=; b=STrGl3WF36UZPI+jJTmY80qaEDWJ9rbwsbR8NxPhICZ/mfOMTa2qQrbUxoI374ZFBQ 2hNPTMv6rjnhCuSCt9ApNn28OJ8QPjSYoyejaIjzkxm/MasRlCSJ1cIUHoVA04QJK/tK FdTCvcrPu8aOdrrYY3aXLl1nN03HewpezGd6IgfyarsXzz9qme2Zyunpj1JgeyA1hFOu qlgpGWpSCl/LhUKjjEO2ZC2sI0eQunq6cmSLoXr+I5ZcUUeECyTKDjd/WIiXnaOMsPcs OOLsn1PWBxBJPT6P/+Yce1r94bhJisdS/tElGWfOgyRnjJE8RLoB8kpF2F2tma4JnkTQ egFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6Z/V3jaZDwubDjfgXB60ZbfyszeJ0mEcXdT4T3MO6VM=; b=ZX2OTLP9BBeN0TcIMl2SbpsA3xc/Whxwn/juGOSCNCO6wQY1JOChbgbaeDWgEuWefq f2mw1+cD1MNrl3WJ1777pE6tBaN6BWFZmoF2KG4uIXKw5+gOGtn4V/rH+H6UokGVsOFb MAZUMR9SqaWTgRWUnWIB8drygpYeVEUpnuMk1KfSZ0WVRMw6pwNhd/nLHWFUKMGGZ0WR c23A9JdzI0r1755We/x+Rnr6QL6vDSshrmwszChmd4bydy1+Rg0oJmjGFuN1RkHF9a5X QSUeFwzIoquSA8o5JSXDqivvn6XyKvof1ceVQd1/22KvMQXZZaX3CqZ30kHCxN0PLd1e wDgw== X-Gm-Message-State: AJcUukdBf9ogw/fklcajb4CnJXw8u2bGPjcqS+8ZM+HKzPIQ2Cu8fX9D QoNw0Il3FrOY9XjPW/HOpzgiL1z4u5ibNI0+OFRNNYtf X-Google-Smtp-Source: ALg8bN5cSDBC3FhnqHFAmR+Yy1yR/LywA14aS31wukJ2tixqtfWT0zsO7DWnCfCi0udY8pvmkY7y3ZCBsSjBSVJIpWM= X-Received: by 2002:a17:906:4a0d:: with SMTP id w13-v6mr11484909eju.128.1546616024566; Fri, 04 Jan 2019 07:33:44 -0800 (PST) MIME-Version: 1.0 References: <CAHLOa7TnitO=wrB69B3b8BEvrV_va1pXrAGwUe+0euioQy1PZQ@mail.gmail.com> <CA+xUZB66fvJtTpqL=6U2gkZ2B6P6q60iNCKoZNq5WA7LYYm1dA@mail.gmail.com> In-Reply-To: <CA+xUZB66fvJtTpqL=6U2gkZ2B6P6q60iNCKoZNq5WA7LYYm1dA@mail.gmail.com> From: Rami Rosen <ramirose@gmail.com> Date: Fri, 4 Jan 2019 17:33:34 +0200 Message-ID: <CAHLOa7QtqgCLe7hEtFmdqFnpdF4UAy1O=rkKpy4rkb6Mzz6+4w@mail.gmail.com> To: Jeremy Plsek <jplsek@iol.unh.edu> Cc: ci@dpdk.org Content-Type: multipart/alternative; boundary="000000000000bf9153057ea39a92" Subject: Re: [dpdk-ci] Question about performance test X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions <ci.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/ci>, <mailto:ci-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/ci/> List-Post: <mailto:ci@dpdk.org> List-Help: <mailto:ci-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/ci>, <mailto:ci-request@dpdk.org?subject=subscribe> X-List-Received-Date: Fri, 04 Jan 2019 15:33:45 -0000 --000000000000bf9153057ea39a92 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Jeremy, Thanks for your quick reply, it fully clarifies the query I had posted. I do not see the actual results of the DTS perf test in the links you posted, only percentage of degradation or improvement, unless I miss something. I believe it can be helpful if the baseline of the actual results will also be shown to enable comparing to other vendors besides Intel and Mellanox. Regards, Rami Rosen =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=95=D7=B3, 4 =D7= =91=D7=99=D7=A0=D7=95=D7=B3 2019, 16:44, =D7=9E=D7=90=D7=AA Jeremy Plsek <j= plsek@iol.unh.edu>: > Hi Rami, > > I'm the current maintainer of the DPDK Performance CI. I realize that the > performance results don't point to the website, so it's not obvious on > where to find this information. You can find an overview of these tests > here: https://lab.dpdk.org > > Most of this information can be either found on the detailed results of a > test (such as https://lab.dpdk.org/results/dashboard/patchsets/4157/) or > on the about page (https://lab.dpdk.org/results/dashboard/about/). > > But to answer your questions: > At the moment, we only run performance tests. Specifically the > nic_single_core_perf_test from the DPDK Test Suite with the TRex traffic > generator. > The devices we are testing are currently the Intel 82599ES 10G, the Intel > XL710-QDA2 40G, the Mellanox ConnectX-5 100G, and the ConnectX-4 Lx 25G a= nd > 40G. > > We don't apply the doc folder when applying the series, in case a patch > included code unrelated to documentation. If others in the group feel tha= t > it's still unnecessary to include "doc" labeled series, I can look into > filtering them out. > > Thanks. > > > On Fri, Jan 4, 2019 at 5:46 AM Rami Rosen <ramirose@gmail.com> wrote: > >> Hi, >> I have a question about DPDK CI process and the tests done when a patch >> is submitted to dpdk-dev mailing list. >> In DPDK patch work I see these response messages from the DPDK CI for al= l >> patches: >> >> ... >> ci/intel-Performance-Testing success Performance Testing PASS >> ci/mellanox-Performance-Testing success Performance Testing PASS >> ... >> >> My question is (I hope and believe the info is available publicly) : >> which tests are run in the ci, generating these messages? is it done wi= th >> IXIA and DTS ? (DPDK test suite, https://doc.dpdk.org/dts/gsg/) ? are >> these l2fwd/l3fwd performance tests? or more than that ? and on which >> Intel/Mellanox nics ? Are these merely performance tests, or also >> functional tests ? >> >> And BTW, I noticed that the CI runs a full performance cycle also for do= c >> patches (at least these messages are generated), which is a kind of >> redundant (unless there is some filter which checks that if a patch only >> affects modules under "doc", than such a cycle is not done but the messa= ges >> are still sent) >> >> Regards, >> Rami Rosen >> > > > -- > Jeremy Plsek > UNH InterOperability Laboratory > > --000000000000bf9153057ea39a92 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Hi Jeremy,<div dir=3D"auto"><br></div><div dir=3D"auto"><= div dir=3D"auto"><br></div><div dir=3D"auto">Thanks for your quick reply, i= t fully clarifies the query I had posted.</div><div dir=3D"auto"><br></div>= <div dir=3D"auto">I do not see the actual results of the DTS perf test in t= he links you posted, only percentage of degradation or improvement, unless = I miss something. I believe it can be helpful if the baseline of the actual= results will also be shown=C2=A0</div><div dir=3D"auto">to enable comparin= g to other vendors besides Intel and Mellanox.</div><div dir=3D"auto"><br><= /div><div dir=3D"auto">Regards,</div><div dir=3D"auto">Rami Rosen</div><div= dir=3D"auto"><br></div><div dir=3D"auto"><br></div><div dir=3D"auto"><br><= /div></div><br><br><div class=3D"gmail_quote" dir=3D"auto"><div dir=3D"ltr"= >=D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A =D7=99=D7=95=D7=9D =D7=95=D7=B3, 4 = =D7=91=D7=99=D7=A0=D7=95=D7=B3 2019, 16:44, =D7=9E=D7=90=D7=AA Jeremy Plsek= <<a href=3D"mailto:jplsek@iol.unh.edu">jplsek@iol.unh.edu</a>>:<br><= /div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-le= ft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><div = dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"= ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div>Hi Rami,</div>= <div><br></div><div>I'm the current maintainer of the DPDK Performance = CI. I realize that the performance results don't point to the website, = so it's not obvious on where to find this information. You can find an = overview of these tests here: <a href=3D"https://lab.dpdk.org" target=3D"_b= lank" rel=3D"noreferrer">https://lab.dpdk.org</a></div><div><br></div><div>= Most of this information can be either found on the detailed results of a t= est (such as <a href=3D"https://lab.dpdk.org/results/dashboard/patchsets/41= 57/" target=3D"_blank" rel=3D"noreferrer">https://lab.dpdk.org/results/dash= board/patchsets/4157/</a>) or on the about page (<a href=3D"https://lab.dpd= k.org/results/dashboard/about/" target=3D"_blank" rel=3D"noreferrer">https:= //lab.dpdk.org/results/dashboard/about/</a>).</div><div><br></div><div>But = to answer your questions:</div><div>At the moment, we only run performance = tests. Specifically the nic_single_core_perf_test from the DPDK Test Suite = with the TRex traffic generator.</div><div>The devices we are testing are c= urrently the Intel 82599ES 10G, the Intel XL710-QDA2 40G, the Mellanox Conn= ectX-5 100G, and the ConnectX-4 Lx 25G and 40G.</div><div><br></div><div>We= don't apply the doc folder when applying the series, in case a patch i= ncluded code unrelated to documentation. If others in the group feel that i= t's still unnecessary to include "doc" labeled series, I can = look into filtering them out.</div><div><br></div><div>Thanks.<br></div><di= v><br></div></div></div></div></div></div></div></div></div></div></div><br= ><div class=3D"gmail_quote"><div dir=3D"ltr">On Fri, Jan 4, 2019 at 5:46 AM= Rami Rosen <<a href=3D"mailto:ramirose@gmail.com" target=3D"_blank" rel= =3D"noreferrer">ramirose@gmail.com</a>> wrote:<br></div><blockquote clas= s=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid r= gb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><div di= r=3D"ltr">Hi,<div>I have a question about DPDK CI process and the tests don= e when a patch is submitted to dpdk-dev mailing list.</div><div>In DPDK pat= ch work I see these response messages from the DPDK CI for all patches:</di= v><div><br></div><div>...</div><div><div><div><font face=3D"Helvetica Neue,= Helvetica, Arial, sans-serif" color=3D"#333333"><span style=3D"font-size:1= 4px">ci/intel-Performance-Testing<span style=3D"white-space:pre-wrap"> </sp= an>success<span style=3D"white-space:pre-wrap"> </span>Performance Testing = PASS</span></font></div><div><font face=3D"Helvetica Neue, Helvetica, Arial= , sans-serif" color=3D"#333333"><span style=3D"font-size:14px">ci/mellanox-= Performance-Testing<span style=3D"white-space:pre-wrap"> </span>success<spa= n style=3D"white-space:pre-wrap"> </span>Performance Testing PASS=C2=A0</sp= an></font></div></div><div><font face=3D"Helvetica Neue, Helvetica, Arial, = sans-serif" color=3D"#333333"><span style=3D"font-size:14px">...</span></fo= nt></div><div><font face=3D"Helvetica Neue, Helvetica, Arial, sans-serif" c= olor=3D"#333333"><span style=3D"font-size:14px"><br></span></font></div><di= v><font face=3D"Helvetica Neue, Helvetica, Arial, sans-serif" color=3D"#333= 333"><span style=3D"font-size:14px">My question is (I hope and believe the = info is available publicly) : which tests are run in the ci, generating the= se messages?=C2=A0 is it done with IXIA and DTS ? (DPDK test suite,=C2=A0<a= href=3D"https://doc.dpdk.org/dts/gsg/" target=3D"_blank" rel=3D"noreferrer= ">https://doc.dpdk.org/dts/gsg/</a>) ? are these l2fwd/l3fwd performance te= sts? or more than that ? and on which Intel/Mellanox nics ?=C2=A0</span></f= ont><span style=3D"font-size:14px;color:rgb(51,51,51);font-family:"Hel= vetica Neue",Helvetica,Arial,sans-serif">=C2=A0Are these merely perfor= mance tests, or also=C2=A0</span></div><div><font face=3D"Helvetica Neue, H= elvetica, Arial, sans-serif" color=3D"#333333"><span style=3D"font-size:14p= x">functional tests ?=C2=A0</span></font></div><div><span style=3D"font-siz= e:14px;color:rgb(51,51,51);font-family:"Helvetica Neue",Helvetica= ,Arial,sans-serif"><br></span></div><div><font face=3D"Helvetica Neue, Helv= etica, Arial, sans-serif" color=3D"#333333"><span style=3D"font-size:14px">= And BTW, I noticed that the CI runs a full performance cycle also for doc p= atches (at least these messages are generated), which is a kind of redundan= t (unless there is some filter which checks that if a patch only affects mo= dules under "doc", than such a cycle is not done but the messages= are still sent)<br></span></font></div><div><br></div><div dir=3D"ltr" cla= ss=3D"m_8710140823469032295gmail-m_-6386449682181207695gmail_signature"><di= v dir=3D"ltr">Regards,<br><div>Rami Rosen<br></div></div></div></div></div>= </div></div> </blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"m= _8710140823469032295gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"= ><div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr"><div></d= iv>Jeremy Plsek<br></div><div>UNH InterOperability Laboratory<br><br></div>= </div></div></div></div></div></div></div></div></div> </blockquote></div></div> --000000000000bf9153057ea39a92--