DPDK CI discussions
 help / color / Atom feed
From: Jeremy Plsek <jplsek@iol.unh.edu>
To: "O'Driscoll, Tim" <tim.odriscoll@intel.com>
Cc: thomas@monjalon.net, "Yigit, Ferruh" <ferruh.yigit@intel.com>,
	ci@dpdk.org, marketing@dpdk.org,
	Lincoln Lavoie <lylavoie@iol.unh.edu>,
	 Patrick MacArthur <pmacarth@iol.unh.edu>
Subject: Re: [dpdk-ci] [dpdk-marketing]  Results are now public
Date: Wed, 14 Nov 2018 16:07:21 -0500
Message-ID: <CA+xUZB63aEB4AetF0MKqRvsXf4QbPSmsWf38FgDKphvjj+1Usg@mail.gmail.com> (raw)
In-Reply-To: <CA+xUZB4=Yve-z5TdCRRPCKYf_BG4bgKKGmzR=X1Rqo77ac7M4w@mail.gmail.com>

With help from Lincoln and Patrick, below is an initial draft of the blog post.



DPDK Community Lab Publishes Relative Performance Testing Results

The DPDK Community Lab is an open, independent testing resource for
the DPDK project. Its purpose is to perform automated testing on
incoming patch submissions, to ensure the performance and quality of
DPDK is maintained. Participation in the lab is open to all DPDK
project participants.

For some time now, the DPDK Community Lab has been gathering
performance deltas using the single-core packet I/O layer 2 throughput
test from DTS for each patch series submitted to DPDK compared to the
master branch. We are pleased to announce that the Lab has recently
been allowed to make these results public. These results are also now
published to Patchwork as they are automatically generated. These
results currently contain Mellanox and Intel devices, and the lab is
able to support hardware from any DPDK participants wishing to support
these testing efforts.

To view these results, you can go to DPDK Community Lab Dashboard via
the following link: https://lab.dpdk.org. The dashboard lists an
overview of all active patch series and their results. Detailed
results can be viewed by clicking on the patch series. If a patch
fails to merge into master, a build log will show to help identify any
issues. If a patch cleanly merges into master, performance delta
results will show for each participating member.

The Lab is hosted by the University of New Hampshire InterOperability
Laboratory, as a neutral, third party location. This provides a secure
environment for hosting equipment and generating unbiased results for
all participating vendors. Lab participants, i.e. companies hosting
equipment in the testing, can securely access their equipment through
a VPN, allowing for maintenance and performance tuning, as the DPDK
project progresses.

The Lab works by polling the Patchwork API. When new patches are
submitted, the CI server merges them with the master branch and
generates a tarball. Each participating system unpacks and installs
the DPDK tarball and then runs the performance testing against this
DPDK build. When all systems have finished testing, the CI gathers the
results into our internal database to be shown on the Dashboard, and
sends final reports to Patchwork to show up on the submitted patch.
This allows patch submitters to utilize Patchwork to view their
individual results, while also allowing anyone to quickly see an
overview of results on the Dashboard. The system provides maintainers
with positive confirmation of the stability and performance of the
overall project.

In the future, we plan to open the Lab to more testing scenarios, such
as performance testing of other features, beyond single-core packet
I/O layer 2 throughput, and possibly running Unit Tests for DPDK.
Additional features will be added to the Dashboard, such as showing
graphs of the performance changes of master over time.

If your company would like to be involved, email the Continuous
Integration group at ci@dpdk.org and dpdklab@iol.unh.edu.



Thanks!

On Fri, Nov 9, 2018 at 10:03 AM Jeremy Plsek <jplsek@iol.unh.edu> wrote:
>
> I can try to get it done later today, but I expect it to be more likely done on Monday.
>
> On Fri, Nov 9, 2018, 9:08 AM O'Driscoll, Tim <tim.odriscoll@intel.com wrote:
>>
>>
>>
>> > -----Original Message-----
>> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
>> > Sent: Friday, November 9, 2018 2:02 PM
>> > To: O'Driscoll, Tim <tim.odriscoll@intel.com>
>> > Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Jeremy Plsek
>> > <jplsek@iol.unh.edu>; ci@dpdk.org; marketing@dpdk.org
>> > Subject: Re: [dpdk-marketing] [dpdk-ci] Results are now public
>> >
>> > 09/11/2018 14:58, O'Driscoll, Tim:
>> > > From: Thomas Monjalon
>> > > > 08/11/2018 14:01, Ferruh Yigit:
>> > > > > On 11/7/2018 7:30 AM, Thomas Monjalon wrote:
>> > > > > > 06/11/2018 16:33, Jeremy Plsek:
>> > > > > >> Hi all,
>> > > > > >>
>> > > > > >> This is just an update that all detailed results are now
>> > public.
>> > > > > >>
>> > > > > >> If there is anything out of place, feel free to let me know!
>> > > > > >
>> > > > > > It is really nice!
>> > > > > > Thank you
>> > > > >
>> > > > > Should we announce this in 'announce' or 'dev' mail lists? Many
>> > people
>> > > > not aware
>> > > > > of this.
>> > > >
>> > > > It deserves an announce.
>> > > > And even a blog post I guess.
>> > > > +Cc marketing team
>> > >
>> > > Agreed. I can do a quick post to announce@dpdk.org. I believe Jeremy
>> > is working on a blog post which can provide more detail.
>> >
>> > Should we wait to have a blog post and reference it in the announce?
>>
>> Good question. I think it depends how long the blog will take. If we can do it fairly quickly, then this is a good idea. If it's going to take a few weeks, then we should announce it now.
>>
>> Jeremy: When do you think the blog will be ready?
>>


--
Jeremy Plsek
UNH InterOperability Laboratory

  reply index

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-06 15:33 [dpdk-ci] " Jeremy Plsek
2018-11-07  7:30 ` Thomas Monjalon
2018-11-08 13:01   ` Ferruh Yigit
2018-11-08 13:10     ` Thomas Monjalon
2018-11-08 13:16       ` [dpdk-ci] [dpdk-marketing] " Trishan de Lanerolle
2018-11-08 13:37         ` Thomas Monjalon
2018-11-08 15:57         ` Jeremy Plsek
2018-11-09 13:58       ` O'Driscoll, Tim
2018-11-09 14:01         ` Thomas Monjalon
2018-11-09 14:07           ` Trishan de Lanerolle
2018-11-09 14:08           ` O'Driscoll, Tim
2018-11-09 15:03             ` Jeremy Plsek
2018-11-14 21:07               ` Jeremy Plsek [this message]
2018-11-14 22:16                 ` Thomas Monjalon
2018-11-15 22:13                   ` O'Driscoll, Tim
2018-11-15 22:30                     ` Jill Lovato
2018-11-16  2:29                     ` Tu, Lijuan
2018-11-16  2:48                       ` Lincoln Lavoie
2018-11-16 16:42                         ` Jill Lovato
2018-11-19 15:08                           ` Mara Bernazzani
2018-11-19 17:48                             ` Jill Lovato
2018-11-19 17:52                               ` Lincoln Lavoie
2018-11-19 17:55                                 ` Jill Lovato
2018-11-20  9:40                                   ` Thomas Monjalon
2018-11-20 13:58                                   ` O'Driscoll, Tim
2018-11-20 15:35                                     ` Jeremy Plsek

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+xUZB63aEB4AetF0MKqRvsXf4QbPSmsWf38FgDKphvjj+1Usg@mail.gmail.com \
    --to=jplsek@iol.unh.edu \
    --cc=ci@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=lylavoie@iol.unh.edu \
    --cc=marketing@dpdk.org \
    --cc=pmacarth@iol.unh.edu \
    --cc=thomas@monjalon.net \
    --cc=tim.odriscoll@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox