test suite reviews and discussions
 help / color / mirror / Atom feed
From: "Tu, Lijuan" <lijuan.tu@intel.com>
To: "Tu, Lijuan" <lijuan.tu@intel.com>, Owen Hilyard <ohilyard@iol.unh.edu>
Cc: David Liu <dliu@iol.unh.edu>, Ali Alnubani <alialnu@nvidia.com>,
	"dts@dpdk.org" <dts@dpdk.org>,
	"dpdklab@iol.unh.edu" <dpdklab@iol.unh.edu>
Subject: Re: [dts] [dpdklab] Re: [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage
Date: Fri, 26 Mar 2021 03:34:55 +0000
Message-ID: <2388e0c11ce248aea86e00c9687b359f@intel.com> (raw)
In-Reply-To: <7af980019bc64098a093202578d96288@intel.com>

[-- Attachment #1: Type: text/plain, Size: 4390 bytes --]

Hi Owen,

I ‘d like to add 2 more changes in V2:

1,  test_interval  will be renamed to throughput_stat_sample_interval for better understanding.
2,  accepted_tolerance will be a float to algin with delta in json, both format are {:.2f}


From: dts <dts-bounces@dpdk.org> On Behalf Of Tu, Lijuan
Sent: 2021年3月25日 15:29
To: Owen Hilyard <ohilyard@iol.unh.edu>
Cc: David Liu <dliu@iol.unh.edu>; Ali Alnubani <alialnu@nvidia.com>; dts@dpdk.org; dpdklab@iol.unh.edu
Subject: Re: [dts] [dpdklab] Re: [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage

sure, I prefer to use a float too.

From: Owen Hilyard <ohilyard@iol.unh.edu<mailto:ohilyard@iol.unh.edu>>
Sent: 2021年3月24日 1:24
To: Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
Cc: David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>>; Ali Alnubani <alialnu@nvidia.com<mailto:alialnu@nvidia.com>>; dts@dpdk.org<mailto:dts@dpdk.org>; dpdklab@iol.unh.edu<mailto:dpdklab@iol.unh.edu>
Subject: Re: [dpdklab] Re: [dts] [PATCH] tests/nic_single_core_perf: modify accepted_tolerance to percentage

Hello Lijuan,

Would it be possible to have the delta be a float in the json representation? For example, a 20% delta would be encoded as "delta": .2 in json. Currently, most of the community lab's infrastructure expects to make pass/fail determinations on a numeric type, so having the delta as a float would reduce the required downtime to change everything over to percentages since we shouldn't need to patch all of our services to expect a string.


Thanks,
Owen Hilyard

On Tue, Mar 23, 2021 at 10:53 AM David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>> wrote:
Thank you for Ali and Lijuan.

We had tested the patch on the Arm machines setup. Other than the value issues, the test is running as expected.

On Tue, Mar 23, 2021 at 4:48 AM Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>> wrote:


> -----Original Message-----
> From: Ali Alnubani <alialnu@nvidia.com<mailto:alialnu@nvidia.com>>
> Sent: 2021年3月23日 16:44
> To: Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
> Cc: dts@dpdk.org<mailto:dts@dpdk.org>; dpdklab@iol.unh.edu<mailto:dpdklab@iol.unh.edu>
> Subject: RE: [dts] [PATCH] tests/nic_single_core_perf: modify
> accepted_tolerance to percentage
>
> > -----Original Message-----
> > From: dts <dts-bounces@dpdk.org<mailto:dts-bounces@dpdk.org>> On Behalf Of Lijuan Tu
> > Sent: Thursday, March 11, 2021 12:48 PM
> > To: dpdklab@iol.unh.edu<mailto:dpdklab@iol.unh.edu>
> > Cc: dts@dpdk.org<mailto:dts@dpdk.org>; Lijuan Tu <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
> > Subject: [dts] [PATCH] tests/nic_single_core_perf: modify
> > accepted_tolerance to percentage
> >
> > * accepted_tolerance in conf/nic_single_core_perf.cfg are changed to
> > percentage, e.g.
> >     accepted_tolerance = '1%%'
> >     if not set, it will be set to 2% by default.
> > * delat in json are chagned to percentage too, e.g.
> >     'delta': '1.5%'
> > * expected_throughput will be forced to 1Mpps for compute delat in
> > percentage
> > * add new conf test_interval in conf/nic_single_core_perf.cfg, which
> > indicates interval of getting throughput statistics in second, e.g.
> >     test_interval = 10
> >     if not set, it will be set to 5 seconds by default.
> > * collect better statistics , get multiple throughput statistics
> > during the test run, and compute the average as the final results.
> >
> > Signed-off-by: Lijuan Tu <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
> > ---
> <snipped>
> > +accepted_tolerance = '1%%'
> <snipped>
> > +                    diff =
> > + (self.throughput[fwd_config][frame_size][nb_desc] -
> > +
> > self.expected_throughput[fwd_config][frame_size][nb_desc])/ \
> > +
> > self.expected_throughput[fwd_config][frame_size][nb_desc]
>
> The value of `diff` here is still a decimal, and needs to be multiplied by a 100
> before it can be compared to `accepted_tolerance`.
>
> I suggest splitting this patch into multiple ones. One for making the percentage
> change and one for taking the mean for multiple readings since they are
> unrelated changes.
> I also see other unrelated changes not explained in the commit message.

Really appreciate for your review, I will accept your advices.

>
> Thanks,
> Ali

[-- Attachment #2: Type: text/html, Size: 10272 bytes --]

  reply	other threads:[~2021-03-26  3:35 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-11 10:48 [dts] " Lijuan Tu
2021-03-22 18:13 ` Ali Alnubani
2021-03-23  8:43 ` Ali Alnubani
2021-03-23  8:48   ` Tu, Lijuan
2021-03-23 14:52     ` David Liu
2021-03-23 17:23       ` [dts] [dpdklab] " Owen Hilyard
2021-03-25  7:28         ` Tu, Lijuan
2021-03-26  3:34           ` Tu, Lijuan [this message]
2021-03-30 14:27             ` David Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2388e0c11ce248aea86e00c9687b359f@intel.com \
    --to=lijuan.tu@intel.com \
    --cc=alialnu@nvidia.com \
    --cc=dliu@iol.unh.edu \
    --cc=dpdklab@iol.unh.edu \
    --cc=dts@dpdk.org \
    --cc=ohilyard@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

test suite reviews and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ https://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git