From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id 0ADB58D9D for ; Sun, 20 May 2018 21:21:12 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id D287421FD5; Sun, 20 May 2018 15:21:11 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sun, 20 May 2018 15:21:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=mesmtp; bh=203WuMKtvEt3zG3c99rHS5n/PQ gg1XhpEpBGLEZlqAY=; b=CwD+iT/zKYNBxjfRadimEFDTvj/0ooxSDO6MRyoUcP IAssh2M+feeVsFSgNRUjWQAlqb1Sb8pghWk2EctpTWIVgqw3jzWQXG011fEXmjY6 RVXnE8AlKhLFmAA77+OqlxtpRZIQQ41xxpS0KPkyyM3tvFr5wsxiipLxsNLTuvhQ E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=203WuM KtvEt3zG3c99rHS5n/PQgg1XhpEpBGLEZlqAY=; b=kvHV5vXIO8D5HyBX7G6op5 u+UU192cWPZFVLjJyZaeOv030I8OlsAIFGf905sEXApMx+wlMpV1WoM84wl9iTnU /QhxocmhLauVTgJ4O5/LeHgec7boOtG1VBJWAYtY1JhFOeZYLF/Qx/7fzpVOHkYA 4wfUR/H8mVHbsoajNoWNfSp4Xyea35vZ5TJ1JYU8n3LytpeU0RrzzBV0IbWj8KfI es2IA55mjWoE5B4V2j9NXKbtWUnz0pysjH49Cyy54z8l9pT0sCbNyJolxdgWapLU jNXrbldr71HBpY9wRwy/b7LDuK0yrpvLwWB6n9LQiacWWMb3o4sRcXHGX4NYeXRA == X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Proxy: X-ME-Sender: Received: from xps.localnet (114.149.6.93.rev.sfr.net [93.6.149.114]) by mail.messagingengine.com (Postfix) with ESMTPA id 0477D10262; Sun, 20 May 2018 15:21:10 -0400 (EDT) From: Thomas Monjalon To: "Parthasarathy, JananeeX M" Cc: "dev@dpdk.org" , "Burakov, Anatoly" , "Pattan, Reshma" Date: Sun, 20 May 2018 21:21:08 +0200 Message-ID: <15428018.4IKRziEG8p@xps> In-Reply-To: <7AE31235A30B41498D1C31348DC858BD5B3D64D6@IRSMSX103.ger.corp.intel.com> References: <1524835241-12778-1-git-send-email-reshma.pattan@intel.com> <12834353.uf7ZmsVvok@xps> <7AE31235A30B41498D1C31348DC858BD5B3D64D6@IRSMSX103.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2] app/test: enhance sanity script X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 May 2018 19:21:13 -0000 20/05/2018 19:41, Parthasarathy, JananeeX M: > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > 18/05/2018 15:32, Parthasarathy, JananeeX M: > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > 15/05/2018 18:14, Parthasarathy, JananeeX M: > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > > > > > I think we need to keep a fast subset of tests. > > > > > > > > Please check in mk/rte.sdktest.mk: > > > > PERFLIST=ring_perf,mempool_perf,memcpy_perf,hash_perf,ti > > > > mer_perf > > > > > > > > test-fast: BLACKLIST=-$(PERFLIST) > > > > > > We have added the below test cases to blacklist in mk/rte.sdktest.mk > > > reciprocal_division,reciprocal_division_perf,lpm_perf,red_all > > > > > > Now time taken is around 4mins. > > > Please let us know is there any minimum time limit for test execution. > > > Is this time duration fine? > > > > 4 minutes, it's a bit long for developer oriented tests. > > > > We need to distinguish which tests are enough for quick tests, > > and which ones can be in another category for full validation purpose. > > > > Please can you make a summary of which tests you are adding, > > and propose a priority classification. > > > > Thanks > > Please find below the test cases newly added to sanity script (autotest_data.py) with priority classification. > > P1 - Main test cases We can move some tests from P1. Examples: - bonding can be P2 with other drivers. - PMD perf and ring perf can P3 with other perf tests. > P2 - Cryptodev test cases which requires dependent libraries (Can be added if the environment is set accordingly) > P3 - Perf test cases and others which takes more time (> 10s) (these will be in the blacklist test cases) > P4 - Logging/Dump test cases (to be checked whether these are really necessary) I think this classification is good. We can adjust the categories in the makefile to reflect it. The goal is to have a really quick and core subset (equal P1 category), and other categories available with a different make command (drivers for P2, perf for P3 and debug for P4). > The blacklist is updated in makefile "rte.sdktest.mk" with below list > of test cases to reduce the total test execution time. > > Blacklist test cases are ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf, > reciprocal_division,reciprocal_division_perf,lpm_perf,red_all,barrier, > hash_multiwriter,timer_racecond,efd,hash_functions,eventdev_selftest_sw, > member_perf,efd_perf,lpm6_perf,red_perf,distributor_perf > > Currently after the above blacklist update, the total execution time of "make test-fast" is 54s. Less than a minute is a good limit! Maybe we can do even better with above suggestions. Thanks for working on it. Feel free to send a new patch when you think it is ready. We can try to have the new autotests when starting 18.08. Then we should communicate to enforce a better coverage of the autotests.