From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id E43781EBA5 for ; Wed, 13 Jun 2018 10:38:35 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Jun 2018 01:38:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,218,1526367600"; d="scan'208";a="63678612" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.66]) ([10.237.220.66]) by fmsmga001.fm.intel.com with ESMTP; 13 Jun 2018 01:38:33 -0700 To: Thomas Monjalon Cc: dev@dpdk.org, john.mcnamara@intel.com, reshma.pattan@intel.com, bruce.richardson@intel.com, Jananee Parthasarathy References: <8176896.7gnY6nVOUV@xps> From: "Burakov, Anatoly" Message-ID: <0191e8b6-1823-7004-2829-d8973e33a1ae@intel.com> Date: Wed, 13 Jun 2018 09:38:32 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <8176896.7gnY6nVOUV@xps> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 0/7] Make unit tests great again X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 08:38:36 -0000 On 12-Jun-18 2:07 PM, Thomas Monjalon wrote: > +Cc Jananee > > 07/06/2018 23:01, Anatoly Burakov: >> Previously, unit tests were running in groups. There were >> technical reasons why that was the case (mostly having to do >> with limiting memory), but it was hard to maintain and update >> the autotest script. >> >> In 18.05, limiting of memory at DPDK startup was no longer >> necessary, as DPDK allocates memory at runtime as needed. This >> has the implication that the old test grouping can now be >> retired and replaced with a more sensible way of running unit >> tests (using multiprocessing pool of workers and a queue of >> tests). This patchset accomplishes exactly that. >> >> This patchset conflicts with some of the earlier work on >> autotests [1] [2] [3], but i think it presents a cleaner >> solution for some of the problems highlighted by those patch >> series. I can integrate those patches into this series if >> need be. >> >> [1] http://dpdk.org/dev/patchwork/patch/40370/ >> [2] http://dpdk.org/dev/patchwork/patch/40371/ >> [3] http://dpdk.org/dev/patchwork/patch/40372/ > > It may be interesting to work on lists of tests as done > in the following patch: > http://dpdk.org/dev/patchwork/patch/40373/ > > The idea is to split tests in several categories: > - basic and short test > - longer and lower priority > - performance test > As a long term solution, we can think about making category an attribute > inside the test itself? > These test categories do not conflict with my patchset as they ultimately rely on white/blacklisting, which will continue to work as before. In my view, it really boils down to two things - either tests can be run in parallel with others (i.e. their result won't be affected by another independent DPDK test app instance), or not. On top of that, we can use blacklisting or whitelisting to define which tests will actually be run (i.e. define any "categories" we want), but their (non-)parallelism must always be respected to get good test results. -- Thanks, Anatoly