test suite reviews and discussions
 help / color / mirror / Atom feed
From: David Liu <dliu@iol.unh.edu>
To: "Tu, Lijuan" <lijuan.tu@intel.com>
Cc: Owen Hilyard <ohilyard@iol.unh.edu>,
	Ali Alnubani <alialnu@nvidia.com>,  "dts@dpdk.org" <dts@dpdk.org>,
	"dpdklab@iol.unh.edu" <dpdklab@iol.unh.edu>
Subject: Re: [dts] [dpdklab] Re: [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage
Date: Tue, 30 Mar 2021 10:27:02 -0400
Message-ID: <CAAuqQpREftAQhXHaCOBwRPX0QeQoM7SQg+BrmnyhqTb9y0sCQw@mail.gmail.com> (raw)
In-Reply-To: <2388e0c11ce248aea86e00c9687b359f@intel.com>

[-- Attachment #1: Type: text/plain, Size: 4488 bytes --]

Hi Lijuan,

Just a quick update.

We had tested the patch on all our bare-metal machines.

And they all work as expected.

Best,
David



On Thu, Mar 25, 2021 at 11:34 PM Tu, Lijuan <lijuan.tu@intel.com> wrote:

> Hi Owen,
>
>
>
> I ‘d like to add 2 more changes in V2:
>
>
>
> 1,  test_interval  will be renamed to throughput_stat_sample_interval for
> better understanding.
>
> 2,  accepted_tolerance will be a float to algin with delta in json, both
> format are {:.2f}
>
>
>
>
>
> *From:* dts <dts-bounces@dpdk.org> *On Behalf Of * Tu, Lijuan
> *Sent:* 2021年3月25日 15:29
> *To:* Owen Hilyard <ohilyard@iol.unh.edu>
> *Cc:* David Liu <dliu@iol.unh.edu>; Ali Alnubani <alialnu@nvidia.com>;
> dts@dpdk.org; dpdklab@iol.unh.edu
> *Subject:* Re: [dts] [dpdklab] Re: [PATCH] tests/nic_single_core_perf:
> modify accepted_tolerance to percentage
>
>
>
> sure, I prefer to use a float too.
>
>
>
> *From:* Owen Hilyard <ohilyard@iol.unh.edu>
> *Sent:* 2021年3月24日 1:24
> *To:* Tu, Lijuan <lijuan.tu@intel.com>
> *Cc:* David Liu <dliu@iol.unh.edu>; Ali Alnubani <alialnu@nvidia.com>;
> dts@dpdk.org; dpdklab@iol.unh.edu
> *Subject:* Re: [dpdklab] Re: [dts] [PATCH] tests/nic_single_core_perf:
> modify accepted_tolerance to percentage
>
>
>
> Hello Lijuan,
>
>
>
> Would it be possible to have the delta be a float in the json
> representation? For example, a 20% delta would be encoded as "delta": .2 in
> json. Currently, most of the community lab's infrastructure expects to make
> pass/fail determinations on a numeric type, so having the delta as a float
> would reduce the required downtime to change everything over to percentages
> since we shouldn't need to patch all of our services to expect a string.
>
>
>
>
>
> Thanks,
>
> Owen Hilyard
>
>
>
> On Tue, Mar 23, 2021 at 10:53 AM David Liu <dliu@iol.unh.edu> wrote:
>
> Thank you for Ali and Lijuan.
>
> We had tested the patch on the Arm machines setup. Other than the value
> issues, the test is running as expected.
>
>
>
> On Tue, Mar 23, 2021 at 4:48 AM Tu, Lijuan <lijuan.tu@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Ali Alnubani <alialnu@nvidia.com>
> > Sent: 2021年3月23日 16:44
> > To: Tu, Lijuan <lijuan.tu@intel.com>
> > Cc: dts@dpdk.org; dpdklab@iol.unh.edu
> > Subject: RE: [dts] [PATCH] tests/nic_single_core_perf: modify
> > accepted_tolerance to percentage
> >
> > > -----Original Message-----
> > > From: dts <dts-bounces@dpdk.org> On Behalf Of Lijuan Tu
> > > Sent: Thursday, March 11, 2021 12:48 PM
> > > To: dpdklab@iol.unh.edu
> > > Cc: dts@dpdk.org; Lijuan Tu <lijuan.tu@intel.com>
> > > Subject: [dts] [PATCH] tests/nic_single_core_perf: modify
> > > accepted_tolerance to percentage
> > >
> > > * accepted_tolerance in conf/nic_single_core_perf.cfg are changed to
> > > percentage, e.g.
> > >     accepted_tolerance = '1%%'
> > >     if not set, it will be set to 2% by default.
> > > * delat in json are chagned to percentage too, e.g.
> > >     'delta': '1.5%'
> > > * expected_throughput will be forced to 1Mpps for compute delat in
> > > percentage
> > > * add new conf test_interval in conf/nic_single_core_perf.cfg, which
> > > indicates interval of getting throughput statistics in second, e.g.
> > >     test_interval = 10
> > >     if not set, it will be set to 5 seconds by default.
> > > * collect better statistics , get multiple throughput statistics
> > > during the test run, and compute the average as the final results.
> > >
> > > Signed-off-by: Lijuan Tu <lijuan.tu@intel.com>
> > > ---
> > <snipped>
> > > +accepted_tolerance = '1%%'
> > <snipped>
> > > +                    diff =
> > > + (self.throughput[fwd_config][frame_size][nb_desc] -
> > > +
> > > self.expected_throughput[fwd_config][frame_size][nb_desc])/ \
> > > +
> > > self.expected_throughput[fwd_config][frame_size][nb_desc]
> >
> > The value of `diff` here is still a decimal, and needs to be multiplied
> by a 100
> > before it can be compared to `accepted_tolerance`.
> >
> > I suggest splitting this patch into multiple ones. One for making the
> percentage
> > change and one for taking the mean for multiple readings since they are
> > unrelated changes.
> > I also see other unrelated changes not explained in the commit message.
>
> Really appreciate for your review, I will accept your advices.
>
> >
> > Thanks,
> > Ali
>
>

[-- Attachment #2: Type: text/html, Size: 9641 bytes --]

      reply	other threads:[~2021-03-30 14:27 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-11 10:48 [dts] " Lijuan Tu
2021-03-22 18:13 ` Ali Alnubani
2021-03-23  8:43 ` Ali Alnubani
2021-03-23  8:48   ` Tu, Lijuan
2021-03-23 14:52     ` David Liu
2021-03-23 17:23       ` [dts] [dpdklab] " Owen Hilyard
2021-03-25  7:28         ` Tu, Lijuan
2021-03-26  3:34           ` Tu, Lijuan
2021-03-30 14:27             ` David Liu [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAAuqQpREftAQhXHaCOBwRPX0QeQoM7SQg+BrmnyhqTb9y0sCQw@mail.gmail.com \
    --to=dliu@iol.unh.edu \
    --cc=alialnu@nvidia.com \
    --cc=dpdklab@iol.unh.edu \
    --cc=dts@dpdk.org \
    --cc=lijuan.tu@intel.com \
    --cc=ohilyard@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

test suite reviews and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ https://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git