From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74794A0525; Fri, 21 Feb 2020 08:57:27 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 685951BF9C; Fri, 21 Feb 2020 08:57:27 +0100 (CET) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 56155397D for ; Fri, 21 Feb 2020 08:57:25 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Feb 2020 23:57:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,467,1574150400"; d="scan'208";a="254750029" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga002.jf.intel.com with ESMTP; 20 Feb 2020 23:57:24 -0800 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 20 Feb 2020 23:57:24 -0800 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 20 Feb 2020 23:57:23 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 20 Feb 2020 23:57:22 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.222]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.201]) with mapi id 14.03.0439.000; Fri, 21 Feb 2020 15:57:20 +0800 From: "Tu, Lijuan" To: "Han, YingyaX" , "dts@dpdk.org" CC: "Han, YingyaX" Thread-Topic: [dts] [PATCH V1]tests/nic_single_core:Save standardized test results Thread-Index: AQHV6ImEp2ZI6ID3a0KndsoQcY5SJKglR/yg Date: Fri, 21 Feb 2020 07:57:20 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0BBCCBDE@SHSMSX101.ccr.corp.intel.com> References: <1582270835-100754-1-git-send-email-yingyax.han@intel.com> In-Reply-To: <1582270835-100754-1-git-send-email-yingyax.han@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1]tests/nic_single_core:Save standardized test results X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Applied, thanks > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of hanyingya > Sent: Friday, February 21, 2020 3:41 PM > To: dts@dpdk.org > Cc: Han, YingyaX > Subject: [dts] [PATCH V1]tests/nic_single_core:Save standardized test res= ults >=20 > Rich results in json format and remove redundant verification code. >=20 > Signed-off-by: hanyingya > --- > tests/TestSuite_nic_single_core_perf.py | 54 +++++++++++-------------- > 1 file changed, 23 insertions(+), 31 deletions(-) >=20 > diff --git a/tests/TestSuite_nic_single_core_perf.py > b/tests/TestSuite_nic_single_core_perf.py > index 5781bc4..5fafe5f 100644 > --- a/tests/TestSuite_nic_single_core_perf.py > +++ b/tests/TestSuite_nic_single_core_perf.py > @@ -197,20 +197,7 @@ class TestNicSingleCorePerf(TestCase): > self.verify(self.nb_ports >=3D 1, "At least 1 port is required t= o test") > self.perf_test(self.nb_ports) > self.handle_results() > - > - # check the gap between expected throughput and actual throughpu= t > - try: > - for frame_size in list(self.test_parameters.keys()): > - for nb_desc in self.test_parameters[frame_size]: > - cur_gap =3D (self.expected_throughput[frame_size][nb= _desc] - > self.throughput[frame_size][nb_desc]) > - self.verify(cur_gap < self.gap, "Beyond Gap, Possibl= e regression") > - except Exception as e: > - self.logger.error(e) > - self.handle_expected() > - raise VerifyFailure( > - "Possible regression, Check your configuration please") > - else: > - self.handle_expected() > + self.handle_expected() >=20 > def handle_expected(self): > """ > @@ -336,27 +323,32 @@ class TestNicSingleCorePerf(TestCase): > if self.save_result_flag is True > ''' > json_obj =3D dict() > - json_obj['nic_type'] =3D self.nic > - json_obj['results'] =3D list() > + case_name =3D self.running_case > + json_obj[case_name] =3D list() > + status_result =3D [] > for frame_size in list(self.test_parameters.keys()): > for nb_desc in self.test_parameters[frame_size]: > row_in =3D self.test_result[frame_size][nb_desc] > - row_dict =3D dict() > - row_dict['parameters'] =3D dict() > - row_dict['parameters']['frame_size'] =3D dict( > - value=3Drow_in['Frame Size'], unit=3D'bytes') > - row_dict['parameters']['txd/rxd'] =3D dict( > - value=3Drow_in['TXD/RXD'], unit=3D'descriptors') > - delta =3D (float(row_in['Throughput'].split()[0]) - > - float(row_in['Expected Throughput'].split()[0])= ) > - if delta >=3D -self.gap: > - result =3D 'PASS' > + row_dict0 =3D dict() > + row_dict0['performance'] =3D list() > + row_dict0['parameters'] =3D list() > + result_throughput =3D float(row_in['Throughput'].split()= [0]) > + expected_throughput =3D float(row_in['Expected > Throughput'].split()[0]) > + # delta value and accepted tolerance in percentage > + delta =3D result_throughput - expected_throughput > + if delta > -self.gap: > + row_dict0['status'] =3D 'PASS' > else: > - result =3D 'FAIL' > - row_dict['throughput'] =3D dict( > - delta=3Ddelta, unit=3Drow_in['Throughput'].split()[1= ], > - result=3Dresult) > - json_obj['results'].append(row_dict) > + row_dict0['status'] =3D 'FAIL' > + row_dict1 =3D dict(name=3D"Throughput", value=3Dresult_t= hroughput, > unit=3D"Mpps", delta=3Ddelta) > + row_dict2 =3D dict(name=3D"Txd/Rxd", value=3Drow_in["TXD= /RXD"], > unit=3D"descriptor") > + row_dict3 =3D dict(name=3D"frame_size", value=3Drow_in["= Frame Size"], > unit=3D"bytes") > + row_dict0['performance'].append(row_dict1) > + row_dict0['parameters'].append(row_dict2) > + row_dict0['parameters'].append(row_dict3) > + json_obj[case_name].append(row_dict0) > + status_result.append(row_dict0['status']) > + self.verify("FAIL" not in status_result, "Excessive gap between = test > results and expectations") > with open(os.path.join(rst.path2Result, > '{0:s}_single_core_perf.json'.format( > self.nic)), 'w') as fp: > -- > 2.17.2