Hello Lijuan,

Would it be possible to have the delta be a float in the json
representation? For example, a 20% delta would be encoded as "delta": .2 in
json. Currently, most of the community lab's infrastructure expects to make
pass/fail determinations on a numeric type, so having the delta as a float
would reduce the required downtime to change everything over to percentages
since we shouldn't need to patch all of our services to expect a string.

Thanks,
Owen Hilyard

On Tue, Mar 23, 2021 at 10:53 AM David Liu <dliu@iol.unh.edu> wrote:

> Thank you for Ali and Lijuan.
>
> We had tested the patch on the Arm machines setup. Other than the value
> issues, the test is running as expected.
>
> On Tue, Mar 23, 2021 at 4:48 AM Tu, Lijuan <lijuan.tu@intel.com> wrote:
>
>>
>>
>> > -----Original Message-----
>> > From: Ali Alnubani <alialnu@nvidia.com>
>> > Sent: 2021年3月23日 16:44
>> > To: Tu, Lijuan <lijuan.tu@intel.com>
>> > Cc: dts@dpdk.org; dpdklab@iol.unh.edu
>> > Subject: RE: [dts] [PATCH] tests/nic_single_core_perf: modify
>> > accepted_tolerance to percentage
>> >
>> > > -----Original Message-----
>> > > From: dts <dts-bounces@dpdk.org> On Behalf Of Lijuan Tu
>> > > Sent: Thursday, March 11, 2021 12:48 PM
>> > > To: dpdklab@iol.unh.edu
>> > > Cc: dts@dpdk.org; Lijuan Tu <lijuan.tu@intel.com>
>> > > Subject: [dts] [PATCH] tests/nic_single_core_perf: modify
>> > > accepted_tolerance to percentage
>> > >
>> > > * accepted_tolerance in conf/nic_single_core_perf.cfg are changed to
>> > > percentage, e.g.
>> > >     accepted_tolerance = '1%%'
>> > >     if not set, it will be set to 2% by default.
>> > > * delat in json are chagned to percentage too, e.g.
>> > >     'delta': '1.5%'
>> > > * expected_throughput will be forced to 1Mpps for compute delat in
>> > > percentage
>> > > * add new conf test_interval in conf/nic_single_core_perf.cfg, which
>> > > indicates interval of getting throughput statistics in second, e.g.
>> > >     test_interval = 10
>> > >     if not set, it will be set to 5 seconds by default.
>> > > * collect better statistics , get multiple throughput statistics
>> > > during the test run, and compute the average as the final results.
>> > >
>> > > Signed-off-by: Lijuan Tu <lijuan.tu@intel.com>
>> > > ---
>> > <snipped>
>> > > +accepted_tolerance = '1%%'
>> > <snipped>
>> > > +                    diff =
>> > > + (self.throughput[fwd_config][frame_size][nb_desc] -
>> > > +
>> > > self.expected_throughput[fwd_config][frame_size][nb_desc])/ \
>> > > +
>> > > self.expected_throughput[fwd_config][frame_size][nb_desc]
>> >
>> > The value of `diff` here is still a decimal, and needs to be multiplied
>> by a 100
>> > before it can be compared to `accepted_tolerance`.
>> >
>> > I suggest splitting this patch into multiple ones. One for making the
>> percentage
>> > change and one for taking the mean for multiple readings since they are
>> > unrelated changes.
>> > I also see other unrelated changes not explained in the commit message.
>>
>> Really appreciate for your review, I will accept your advices.
>>
>> >
>> > Thanks,
>> > Ali
>>
>