DPDK CI discussions
 help / color / Atom feed
From: Lincoln Lavoie <lylavoie@iol.unh.edu>
To: Aaron Conole <aconole@redhat.com>
Cc: "O'Driscoll, Tim" <tim.odriscoll@intel.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	 "ci@dpdk.org" <ci@dpdk.org>,
	"Stokes, Ian" <ian.stokes@intel.com>,
	Rashid Khan <rkhan@redhat.com>
Subject: Re: [dpdk-ci] Minutes of DPDK Lab Meeting, February 26th
Date: Mon, 4 Mar 2019 11:59:11 -0500
Message-ID: <CAOE1vsP9tqkN+J3fS5Xmum5xf_8UYQ7a6xiq6uN58jNCukKSXg@mail.gmail.com> (raw)
In-Reply-To: <f7th8cin60t.fsf@dhcp-25.97.bos.redhat.com>

[-- Attachment #1: Type: text/plain, Size: 3586 bytes --]

Hi All,

The reason we selection loaner machines (UNH provided) for the development
was to avoid interference with the existing setup, i.e. don't break or
degrade the performance tuned systems.

For the deployed testing (i.e. once we have the OVS developed and
integrated with the lab dashboard) can be done either on the existing
hardware, or a stand alone setup with multiple NICs.  I think this was
proposed, because function testing with multiple NICs would had more
hardware coverage than the two vendor performance systems right now.  That
might also be a lower bar for some hardware vendors to only provide a NIC,
etc.

In we choose the "option A" to use the existing performance setups, we
would serialize the testing, so the performance jobs run independently, but
I don't think that was really the question.

Cheers,
Lincoln

On Mon, Mar 4, 2019 at 8:06 AM Aaron Conole <aconole@redhat.com> wrote:

> "O'Driscoll, Tim" <tim.odriscoll@intel.com> writes:
>
> >> -----Original Message-----
> >> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> >> Sent: Thursday, February 28, 2019 3:20 PM
> >> To: ci@dpdk.org
> >> Cc: O'Driscoll, Tim <tim.odriscoll@intel.com>
> >> Subject: Re: [dpdk-ci] Minutes of DPDK Lab Meeting, February 26th
> >>
> >> Hi,
> >>
> >> 28/02/2019 15:49, O'Driscoll, Tim:
> >> > OVS Tests:
> >> > - Jeremy and Aaron are working on setup of the temporary hardware.
> >> > - There are two options for hardware to run this on when the setup is
> >> complete: 1) use existing vendor hardware; 2) obtain standalone servers
> >> for OVS testing. The OVS team's preference is option 2. It's not
> >> realistic to expect a vendor to provide hardware to run a competitor's
> >> products so we'd need to find a different way to procure this. Aaron
> >> will check with Rashid to see if budget is available from Red Hat. I'll
> >> check with Trishan to see if the DPDK project budget could cover this.
> >> > - The OVS focus is on functional tests, not performance tests. The
> >> DPDK lab is currently set up so that each vendor has complete control
> >> over performance tests & results on their hardware. If we use separate
> >> hardware for the OVS tests, we need to ensure that we restrict scope to
> >> functional tests so that it does not conflict with this principle in
> >> future.
> >>
> >> I am not sure to understand.
> >> In my opinion, the purpose of this lab is to have properly tuned
> >> hardware
> >> for running a large set of tests. We should be able to run various
> >> tests
> >> on the same machine. So the OVS tests, like any new test scenario,
> >> should be run on the same machine as the performance tests.
>
> This is definitely something I support as well.
>
> >> I think we just need to have a job queue to run tests one by one,
> >> avoiding a test to disturb results of another one.
> >>
> >> Why are we looking for additional machines?
>
> I think because there is no such kind of job queue available, currently?
> I don't recall if an exact reason was given other than the nebulous fear
> of "breaking the existing setups".
>
> > That was my assumption too. I believe the reason is that the OVS team
> > want to validate with multiple vendor NICs to be sure that nothing is
> > broken. We only have Intel and Mellanox hardware in our lab at
> > present, so we don't cover all vendors.
> >
> > Aaron and Ian can provide more details.
>


-- 
*Lincoln Lavoie*
Senior Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie@iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu/>

[-- Attachment #2: Type: text/html, Size: 5774 bytes --]

<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi All,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">The reason we selection loaner machines (UNH provided) for the development was to avoid interference with the existing setup, i.e. don&#39;t break or degrade the performance tuned systems.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">For the deployed testing (i.e. once we have the OVS developed and integrated with the lab dashboard) can be done either on the existing hardware, or a stand alone setup with multiple NICs.  I think this was proposed, because function testing with multiple NICs would had more hardware coverage than the two vendor performance systems right now.  That might also be a lower bar for some hardware vendors to only provide a NIC, etc.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">In we choose the &quot;option A&quot; to use the existing performance setups, we would serialize the testing, so the performance jobs run independently, but I don&#39;t think that was really the question.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Cheers,</div><div class="gmail_default" style="font-size:small">Lincoln</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 4, 2019 at 8:06 AM Aaron Conole &lt;<a href="mailto:aconole@redhat.com">aconole@redhat.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">&quot;O&#39;Driscoll, Tim&quot; &lt;<a href="mailto:tim.odriscoll@intel.com" target="_blank">tim.odriscoll@intel.com</a>&gt; writes:<br>
<br>
&gt;&gt; -----Original Message-----<br>
&gt;&gt; From: Thomas Monjalon [mailto:<a href="mailto:thomas@monjalon.net" target="_blank">thomas@monjalon.net</a>]<br>
&gt;&gt; Sent: Thursday, February 28, 2019 3:20 PM<br>
&gt;&gt; To: <a href="mailto:ci@dpdk.org" target="_blank">ci@dpdk.org</a><br>
&gt;&gt; Cc: O&#39;Driscoll, Tim &lt;<a href="mailto:tim.odriscoll@intel.com" target="_blank">tim.odriscoll@intel.com</a>&gt;<br>
&gt;&gt; Subject: Re: [dpdk-ci] Minutes of DPDK Lab Meeting, February 26th<br>
&gt;&gt; <br>
&gt;&gt; Hi,<br>
&gt;&gt; <br>
&gt;&gt; 28/02/2019 15:49, O&#39;Driscoll, Tim:<br>
&gt;&gt; &gt; OVS Tests:<br>
&gt;&gt; &gt; - Jeremy and Aaron are working on setup of the temporary hardware.<br>
&gt;&gt; &gt; - There are two options for hardware to run this on when the setup is<br>
&gt;&gt; complete: 1) use existing vendor hardware; 2) obtain standalone servers<br>
&gt;&gt; for OVS testing. The OVS team&#39;s preference is option 2. It&#39;s not<br>
&gt;&gt; realistic to expect a vendor to provide hardware to run a competitor&#39;s<br>
&gt;&gt; products so we&#39;d need to find a different way to procure this. Aaron<br>
&gt;&gt; will check with Rashid to see if budget is available from Red Hat. I&#39;ll<br>
&gt;&gt; check with Trishan to see if the DPDK project budget could cover this.<br>
&gt;&gt; &gt; - The OVS focus is on functional tests, not performance tests. The<br>
&gt;&gt; DPDK lab is currently set up so that each vendor has complete control<br>
&gt;&gt; over performance tests &amp; results on their hardware. If we use separate<br>
&gt;&gt; hardware for the OVS tests, we need to ensure that we restrict scope to<br>
&gt;&gt; functional tests so that it does not conflict with this principle in<br>
&gt;&gt; future.<br>
&gt;&gt; <br>
&gt;&gt; I am not sure to understand.<br>
&gt;&gt; In my opinion, the purpose of this lab is to have properly tuned<br>
&gt;&gt; hardware<br>
&gt;&gt; for running a large set of tests. We should be able to run various<br>
&gt;&gt; tests<br>
&gt;&gt; on the same machine. So the OVS tests, like any new test scenario,<br>
&gt;&gt; should be run on the same machine as the performance tests.<br>
<br>
This is definitely something I support as well.<br>
<br>
&gt;&gt; I think we just need to have a job queue to run tests one by one,<br>
&gt;&gt; avoiding a test to disturb results of another one.<br>
&gt;&gt; <br>
&gt;&gt; Why are we looking for additional machines?<br>
<br>
I think because there is no such kind of job queue available, currently?<br>
I don&#39;t recall if an exact reason was given other than the nebulous fear<br>
of &quot;breaking the existing setups&quot;.<br>
<br>
&gt; That was my assumption too. I believe the reason is that the OVS team<br>
&gt; want to validate with multiple vendor NICs to be sure that nothing is<br>
&gt; broken. We only have Intel and Mellanox hardware in our lab at<br>
&gt; present, so we don&#39;t cover all vendors.<br>
&gt;<br>
&gt; Aaron and Ian can provide more details.<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><b>Lincoln Lavoie</b><br></div><div>Senior Engineer, Broadband Technologies</div><div>21 Madbury Rd., Ste. 100, Durham, NH 03824</div><div><a href="mailto:lylavoie@iol.unh.edu" target="_blank">lylavoie@iol.unh.edu</a></div><div><a href="https://www.iol.unh.edu" target="_blank">https://www.iol.unh.edu</a></div><div>+1-603-674-2755 (m)<br></div><div><a href="https://www.iol.unh.edu/" target="_blank"><img src="http://homeautomation.lavoieholdings.com/_/rsrc/1390068882701/unh-iol-logo.png"></a></div></div></div></div></div></div></div></div></div></div></div></div>

  reply index

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-28 14:49 O'Driscoll, Tim
2019-02-28 15:20 ` Thomas Monjalon
2019-02-28 15:24   ` O'Driscoll, Tim
2019-03-04  8:06     ` Stokes, Ian
2019-03-04  8:40       ` Thomas Monjalon
2019-03-04  8:49         ` Stokes, Ian
2019-03-04 13:06     ` Aaron Conole
2019-03-04 16:59       ` Lincoln Lavoie [this message]
2019-03-04 17:40         ` Thomas Monjalon
2019-03-08 21:24           ` Aaron Conole
2019-03-08 23:22             ` Thomas Monjalon

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOE1vsP9tqkN+J3fS5Xmum5xf_8UYQ7a6xiq6uN58jNCukKSXg@mail.gmail.com \
    --to=lylavoie@iol.unh.edu \
    --cc=aconole@redhat.com \
    --cc=ci@dpdk.org \
    --cc=ian.stokes@intel.com \
    --cc=rkhan@redhat.com \
    --cc=thomas@monjalon.net \
    --cc=tim.odriscoll@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK CI discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox