From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id B56A01B0FC for ; Wed, 3 Apr 2019 16:09:16 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Apr 2019 07:09:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,304,1549958400"; d="scan'208";a="147710015" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by orsmga002.jf.intel.com with ESMTP; 03 Apr 2019 07:09:09 -0700 Received: from irsmsx155.ger.corp.intel.com (163.33.192.3) by IRSMSX109.ger.corp.intel.com (163.33.3.23) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 3 Apr 2019 15:08:47 +0100 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.230]) by irsmsx155.ger.corp.intel.com ([169.254.14.140]) with mapi id 14.03.0415.000; Wed, 3 Apr 2019 15:08:47 +0100 From: "Trahe, Fiona" To: Ayuj Verma , "akhil.goyal@nxp.com" , "De Lara Guarch, Pablo" , "Kusztal, ArkadiuszX" CC: Shally Verma , Sunila Sahu , Kanaka Durga Kotamarthy , Arvind Desai , "dev@dpdk.org" , "Trahe, Fiona" Thread-Topic: [PATCH v1] app/test: add check for tests skipped Thread-Index: AQHU6gVayobz+bUHqkuzo+WcbGNDn6YqcvXg Date: Wed, 3 Apr 2019 14:08:46 +0000 Message-ID: <348A99DA5F5B7549AA880327E580B43589729A7B@IRSMSX101.ger.corp.intel.com> References: <1554103916-31459-1-git-send-email-ayverma@marvell.com>, <1554103916-31459-2-git-send-email-ayverma@marvell.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODNkM2E1M2UtOTA0OS00ODJkLWI0NDgtOWE1MzUwYzNjYjQxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiVTJETnNMRTZuc1BlRkFOcElZVmZTOVRSSGh3d2lRVElpdExXYlwvSm0xUHVYa1dCZnZKY1cxa0hxd3lJUVl0RzMifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1] app/test: add check for tests skipped X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 14:09:17 -0000 Hi Ayuj, ________________________________________ From: Ayuj Verma Sent: 01 April 2019 13:01:56 To: akhil.goyal@nxp.com Cc: arkadiuszx.kusztal@intel.com; Shally Verma; Sunila Sahu; Kanaka Durga K= otamarthy; Arvind Desai; dev@dpdk.org; Ayuj Verma Subject: [PATCH v1] app/test: add check for tests skipped=20 =A0 Add skipped counter to count for number of skipped testcases. Signed-off-by: Ayuj Verma Signed-off-by: Shally Verma --- =A0app/test/test.c | 2 ++ =A01 file changed, 2 insertions(+) diff --git a/app/test/test.c b/app/test/test.c index d646f51..1e0113b 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -237,6 +237,8 @@ =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 succeeded++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 el= se if (test_success =3D=3D -ENOTSUP) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 unsupported++; +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 else if= (test_success =3D=3D TEST_SKIPPED) +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 skipped++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 el= se =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 failed++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } else if (test_success = =3D=3D -ENOTSUP) { --=20 1.8.3.1 [Fiona] I just tried it out. The patch itself is ok - it works if a test case at run-time decides to ret= urn TEST_SKIPPED. A compile-time decision to skip a test-case can be made by using TEST_CASE_= ST_DISABLED and this already counts as skipped. But I see the 2 places you're using TEST_SKIPPED are where the device doesn= 't support the capability needed for the test. In other similar cases (e.g.= sym_crypto ZUC test) -ENOTSUP is returned. This seems like a more appropri= ate return value. So unless there's some other reason other than Unsupported that a run-time = decision would be made to skip a test, I'd nack this patch as it leads to c= onfusion about which value should be returned. Fiona From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id AD7EBA0679 for ; Wed, 3 Apr 2019 16:09:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 088DA1B3B8; Wed, 3 Apr 2019 16:09:19 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id B56A01B0FC for ; Wed, 3 Apr 2019 16:09:16 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Apr 2019 07:09:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,304,1549958400"; d="scan'208";a="147710015" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by orsmga002.jf.intel.com with ESMTP; 03 Apr 2019 07:09:09 -0700 Received: from irsmsx155.ger.corp.intel.com (163.33.192.3) by IRSMSX109.ger.corp.intel.com (163.33.3.23) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 3 Apr 2019 15:08:47 +0100 Received: from irsmsx101.ger.corp.intel.com ([169.254.1.230]) by irsmsx155.ger.corp.intel.com ([169.254.14.140]) with mapi id 14.03.0415.000; Wed, 3 Apr 2019 15:08:47 +0100 From: "Trahe, Fiona" To: Ayuj Verma , "akhil.goyal@nxp.com" , "De Lara Guarch, Pablo" , "Kusztal, ArkadiuszX" CC: Shally Verma , Sunila Sahu , Kanaka Durga Kotamarthy , Arvind Desai , "dev@dpdk.org" , "Trahe, Fiona" Thread-Topic: [PATCH v1] app/test: add check for tests skipped Thread-Index: AQHU6gVayobz+bUHqkuzo+WcbGNDn6YqcvXg Date: Wed, 3 Apr 2019 14:08:46 +0000 Message-ID: <348A99DA5F5B7549AA880327E580B43589729A7B@IRSMSX101.ger.corp.intel.com> References: <1554103916-31459-1-git-send-email-ayverma@marvell.com>, <1554103916-31459-2-git-send-email-ayverma@marvell.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODNkM2E1M2UtOTA0OS00ODJkLWI0NDgtOWE1MzUwYzNjYjQxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiVTJETnNMRTZuc1BlRkFOcElZVmZTOVRSSGh3d2lRVElpdExXYlwvSm0xUHVYa1dCZnZKY1cxa0hxd3lJUVl0RzMifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1] app/test: add check for tests skipped X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190403140846.mQVuHNXeBnq-Frv5WfgEIdUkBY0GX71ox0xXEwyR2Qg@z> Hi Ayuj, ________________________________________ From: Ayuj Verma Sent: 01 April 2019 13:01:56 To: akhil.goyal@nxp.com Cc: arkadiuszx.kusztal@intel.com; Shally Verma; Sunila Sahu; Kanaka Durga K= otamarthy; Arvind Desai; dev@dpdk.org; Ayuj Verma Subject: [PATCH v1] app/test: add check for tests skipped=20 =A0 Add skipped counter to count for number of skipped testcases. Signed-off-by: Ayuj Verma Signed-off-by: Shally Verma --- =A0app/test/test.c | 2 ++ =A01 file changed, 2 insertions(+) diff --git a/app/test/test.c b/app/test/test.c index d646f51..1e0113b 100644 --- a/app/test/test.c +++ b/app/test/test.c @@ -237,6 +237,8 @@ =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 succeeded++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 el= se if (test_success =3D=3D -ENOTSUP) =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 unsupported++; +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 else if= (test_success =3D=3D TEST_SKIPPED) +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 skipped++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 el= se =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 failed++; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } else if (test_success = =3D=3D -ENOTSUP) { --=20 1.8.3.1 [Fiona] I just tried it out. The patch itself is ok - it works if a test case at run-time decides to ret= urn TEST_SKIPPED. A compile-time decision to skip a test-case can be made by using TEST_CASE_= ST_DISABLED and this already counts as skipped. But I see the 2 places you're using TEST_SKIPPED are where the device doesn= 't support the capability needed for the test. In other similar cases (e.g.= sym_crypto ZUC test) -ENOTSUP is returned. This seems like a more appropri= ate return value. So unless there's some other reason other than Unsupported that a run-time = decision would be made to skip a test, I'd nack this patch as it leads to c= onfusion about which value should be returned. Fiona