DPDK CI discussions
 help / color / Atom feed
From: Rami Rosen <ramirose@gmail.com>
To: Jeremy Plsek <jplsek@iol.unh.edu>
Cc: ci@dpdk.org
Subject: Re: [dpdk-ci] Question about performance test
Date: Fri, 4 Jan 2019 17:33:34 +0200
Message-ID: <CAHLOa7QtqgCLe7hEtFmdqFnpdF4UAy1O=rkKpy4rkb6Mzz6+4w@mail.gmail.com> (raw)
In-Reply-To: <CA+xUZB66fvJtTpqL=6U2gkZ2B6P6q60iNCKoZNq5WA7LYYm1dA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2937 bytes --]

Hi Jeremy,


Thanks for your quick reply, it fully clarifies the query I had posted.

I do not see the actual results of the DTS perf test in the links you
posted, only percentage of degradation or improvement, unless I miss
something. I believe it can be helpful if the baseline of the actual
results will also be shown
to enable comparing to other vendors besides Intel and Mellanox.

Regards,
Rami Rosen





בתאריך יום ו׳, 4 בינו׳ 2019, 16:44, מאת Jeremy Plsek <jplsek@iol.unh.edu>:

> Hi Rami,
>
> I'm the current maintainer of the DPDK Performance CI. I realize that the
> performance results don't point to the website, so it's not obvious on
> where to find this information. You can find an overview of these tests
> here: https://lab.dpdk.org
>
> Most of this information can be either found on the detailed results of a
> test (such as https://lab.dpdk.org/results/dashboard/patchsets/4157/) or
> on the about page (https://lab.dpdk.org/results/dashboard/about/).
>
> But to answer your questions:
> At the moment, we only run performance tests. Specifically the
> nic_single_core_perf_test from the DPDK Test Suite with the TRex traffic
> generator.
> The devices we are testing are currently the Intel 82599ES 10G, the Intel
> XL710-QDA2 40G, the Mellanox ConnectX-5 100G, and the ConnectX-4 Lx 25G and
> 40G.
>
> We don't apply the doc folder when applying the series, in case a patch
> included code unrelated to documentation. If others in the group feel that
> it's still unnecessary to include "doc" labeled series, I can look into
> filtering them out.
>
> Thanks.
>
>
> On Fri, Jan 4, 2019 at 5:46 AM Rami Rosen <ramirose@gmail.com> wrote:
>
>> Hi,
>> I have a question about DPDK CI process and the tests done when a patch
>> is submitted to dpdk-dev mailing list.
>> In DPDK patch work I see these response messages from the DPDK CI for all
>> patches:
>>
>> ...
>> ci/intel-Performance-Testing success Performance Testing PASS
>> ci/mellanox-Performance-Testing success Performance Testing PASS
>> ...
>>
>> My question is (I hope and believe the info is available publicly) :
>> which tests are run in the ci, generating these messages?  is it done with
>> IXIA and DTS ? (DPDK test suite, https://doc.dpdk.org/dts/gsg/) ? are
>> these l2fwd/l3fwd performance tests? or more than that ? and on which
>> Intel/Mellanox nics ?  Are these merely performance tests, or also
>> functional tests ?
>>
>> And BTW, I noticed that the CI runs a full performance cycle also for doc
>> patches (at least these messages are generated), which is a kind of
>> redundant (unless there is some filter which checks that if a patch only
>> affects modules under "doc", than such a cycle is not done but the messages
>> are still sent)
>>
>> Regards,
>> Rami Rosen
>>
>
>
> --
> Jeremy Plsek
> UNH InterOperability Laboratory
>
>

[-- Attachment #2: Type: text/html, Size: 6182 bytes --]

<div dir="auto">Hi Jeremy,<div dir="auto"><br></div><div dir="auto"><div dir="auto"><br></div><div dir="auto">Thanks for your quick reply, it fully clarifies the query I had posted.</div><div dir="auto"><br></div><div dir="auto">I do not see the actual results of the DTS perf test in the links you posted, only percentage of degradation or improvement, unless I miss something. I believe it can be helpful if the baseline of the actual results will also be shown </div><div dir="auto">to enable comparing to other vendors besides Intel and Mellanox.</div><div dir="auto"><br></div><div dir="auto">Regards,</div><div dir="auto">Rami Rosen</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><br></div></div><br><br><div class="gmail_quote" dir="auto"><div dir="ltr">בתאריך יום ו׳, 4 בינו׳ 2019, 16:44, מאת Jeremy Plsek &lt;<a href="mailto:jplsek@iol.unh.edu">jplsek@iol.unh.edu</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Hi Rami,</div><div><br></div><div>I&#39;m the current maintainer of the DPDK Performance CI. I realize that the performance results don&#39;t point to the website, so it&#39;s not obvious on where to find this information. You can find an overview of these tests here: <a href="https://lab.dpdk.org" target="_blank" rel="noreferrer">https://lab.dpdk.org</a></div><div><br></div><div>Most of this information can be either found on the detailed results of a test (such as <a href="https://lab.dpdk.org/results/dashboard/patchsets/4157/" target="_blank" rel="noreferrer">https://lab.dpdk.org/results/dashboard/patchsets/4157/</a>) or on the about page (<a href="https://lab.dpdk.org/results/dashboard/about/" target="_blank" rel="noreferrer">https://lab.dpdk.org/results/dashboard/about/</a>).</div><div><br></div><div>But to answer your questions:</div><div>At the moment, we only run performance tests. Specifically the nic_single_core_perf_test from the DPDK Test Suite with the TRex traffic generator.</div><div>The devices we are testing are currently the Intel 82599ES 10G, the Intel XL710-QDA2 40G, the Mellanox ConnectX-5 100G, and the ConnectX-4 Lx 25G and 40G.</div><div><br></div><div>We don&#39;t apply the doc folder when applying the series, in case a patch included code unrelated to documentation. If others in the group feel that it&#39;s still unnecessary to include &quot;doc&quot; labeled series, I can look into filtering them out.</div><div><br></div><div>Thanks.<br></div><div><br></div></div></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Jan 4, 2019 at 5:46 AM Rami Rosen &lt;<a href="mailto:ramirose@gmail.com" target="_blank" rel="noreferrer">ramirose@gmail.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi,<div>I have a question about DPDK CI process and the tests done when a patch is submitted to dpdk-dev mailing list.</div><div>In DPDK patch work I see these response messages from the DPDK CI for all patches:</div><div><br></div><div>...</div><div><div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">ci/intel-Performance-Testing<span style="white-space:pre-wrap">	</span>success<span style="white-space:pre-wrap">	</span>Performance Testing PASS</span></font></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">ci/mellanox-Performance-Testing<span style="white-space:pre-wrap">	</span>success<span style="white-space:pre-wrap">	</span>Performance Testing PASS </span></font></div></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">...</span></font></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px"><br></span></font></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">My question is (I hope and believe the info is available publicly) : which tests are run in the ci, generating these messages?  is it done with IXIA and DTS ? (DPDK test suite, <a href="https://doc.dpdk.org/dts/gsg/" target="_blank" rel="noreferrer">https://doc.dpdk.org/dts/gsg/</a>) ? are these l2fwd/l3fwd performance tests? or more than that ? and on which Intel/Mellanox nics ? </span></font><span style="font-size:14px;color:rgb(51,51,51);font-family:&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif"> Are these merely performance tests, or also </span></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">functional tests ? </span></font></div><div><span style="font-size:14px;color:rgb(51,51,51);font-family:&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif"><br></span></div><div><font face="Helvetica Neue, Helvetica, Arial, sans-serif" color="#333333"><span style="font-size:14px">And BTW, I noticed that the CI runs a full performance cycle also for doc patches (at least these messages are generated), which is a kind of redundant (unless there is some filter which checks that if a patch only affects modules under &quot;doc&quot;, than such a cycle is not done but the messages are still sent)<br></span></font></div><div><br></div><div dir="ltr" class="m_8710140823469032295gmail-m_-6386449682181207695gmail_signature"><div dir="ltr">Regards,<br><div>Rami Rosen<br></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="m_8710140823469032295gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div></div>Jeremy Plsek<br></div><div>UNH InterOperability Laboratory<br><br></div></div></div></div></div></div></div></div></div></div>
</blockquote></div></div>

  reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-04 10:45 Rami Rosen
2019-01-04 14:44 ` Jeremy Plsek
2019-01-04 15:33   ` Rami Rosen [this message]
2019-01-04 15:34   ` Thomas Monjalon
2019-01-04 16:15     ` Jeremy Plsek
2019-01-04 16:19       ` Thomas Monjalon
2019-01-08  6:40   ` Rami Rosen
2019-01-08  7:06     ` Tu, Lijuan
2019-01-08  8:23       ` Rami Rosen

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHLOa7QtqgCLe7hEtFmdqFnpdF4UAy1O=rkKpy4rkb6Mzz6+4w@mail.gmail.com' \
    --to=ramirose@gmail.com \
    --cc=ci@dpdk.org \
    --cc=jplsek@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox