From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by dpdk.space (Postfix) with ESMTP id 430CFA00E6
	for <public@inbox.dpdk.org>; Sat, 15 Jun 2019 08:44:55 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 727F51D608;
	Sat, 15 Jun 2019 08:43:41 +0200 (CEST)
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28])
 by dpdk.org (Postfix) with ESMTP id 5CC441D5F7
 for <dev@dpdk.org>; Sat, 15 Jun 2019 08:43:37 +0200 (CEST)
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
 [10.5.11.16])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.redhat.com (Postfix) with ESMTPS id CE65D85539;
 Sat, 15 Jun 2019 06:43:36 +0000 (UTC)
Received: from dmarchan.remote.csb (unknown [10.40.205.71])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 3F6C95C1A1;
 Sat, 15 Jun 2019 06:43:35 +0000 (UTC)
From: David Marchand <david.marchand@redhat.com>
To: dev@dpdk.org
Cc: thomas@monjalon.net,
	aconole@redhat.com,
	msantana@redhat.com
Date: Sat, 15 Jun 2019 08:42:29 +0200
Message-Id: <1560580950-16754-15-git-send-email-david.marchand@redhat.com>
In-Reply-To: <1560580950-16754-1-git-send-email-david.marchand@redhat.com>
References: <1559638792-8608-1-git-send-email-david.marchand@redhat.com>
 <1560580950-16754-1-git-send-email-david.marchand@redhat.com>
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16
 (mx1.redhat.com [10.5.110.28]); Sat, 15 Jun 2019 06:43:36 +0000 (UTC)
Subject: [dpdk-dev] [PATCH v2 14/15] test: do not start tests in parallel
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

Running the tests in parallel has two drawbacks:
- the tests are racing on the hugepages allocations,
- the tests are sharing the cores to run their checks which results in
  undeterministic execution time,

This results in random failures.
For better reproducibility in CI, start them all in a serialised way.

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Changelog since v1:
- rebased on master

---
 app/test/meson.build | 25 +++++--------------------
 1 file changed, 5 insertions(+), 20 deletions(-)

diff --git a/app/test/meson.build b/app/test/meson.build
index f799c8e..f1db02f 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -148,8 +148,7 @@ test_deps = ['acl',
 	'timer'
 ]
 
-# All test cases in fast_parallel_test_names list are parallel
-fast_parallel_test_names = [
+fast_test_names = [
         'acl_autotest',
         'alarm_autotest',
         'atomic_autotest',
@@ -209,10 +208,6 @@ fast_parallel_test_names = [
         'timer_autotest',
         'user_delay_us',
         'version_autotest',
-]
-
-# All test cases in fast_non_parallel_test_names list are non-parallel
-fast_non_parallel_test_names = [
         'bitratestats_autotest',
         'crc_autotest',
         'delay_us_sleep_autotest',
@@ -236,7 +231,6 @@ fast_non_parallel_test_names = [
         'thash_autotest',
 ]
 
-# All test cases in perf_test_names list are non-parallel
 perf_test_names = [
         'ring_perf_autotest',
         'mempool_perf_autotest',
@@ -264,7 +258,6 @@ perf_test_names = [
         'stack_lf_perf_autotest',
 ]
 
-# All test cases in driver_test_names list are non-parallel
 driver_test_names = [
         'cryptodev_aesni_mb_autotest',
         'cryptodev_aesni_gcm_autotest',
@@ -286,7 +279,6 @@ driver_test_names = [
         'link_bonding_rssconf_autotest',
 ]
 
-# All test cases in dump_test_names list are non-parallel
 dump_test_names = [
         'dump_struct_sizes',
         'dump_mempool',
@@ -335,7 +327,7 @@ if dpdk_conf.has('RTE_LIBRTE_COMPRESSDEV')
 		test_dep_objs += compress_test_dep
 		test_sources += 'test_compressdev.c'
 		test_deps += 'compressdev'
-		fast_non_parallel_test_names += 'compressdev_autotest'
+		fast_test_names += 'compressdev_autotest'
 	endif
 endif
 
@@ -383,30 +375,23 @@ endif
 num_cores_arg = '-l ' + num_cores
 
 test_args = [num_cores_arg]
-foreach arg : fast_parallel_test_names
+foreach arg : fast_test_names
 	if host_machine.system() == 'linux'
 		test(arg, dpdk_test,
 			  env : ['DPDK_TEST=' + arg],
 			  args : test_args +
 				 ['--file-prefix=@0@'.format(arg)],
 		timeout : timeout_seconds_fast,
+		is_parallel : false,
 		suite : 'fast-tests')
 	else
 		test(arg, dpdk_test,
 			env : ['DPDK_TEST=' + arg],
 			args : test_args,
 		timeout : timeout_seconds_fast,
-		suite : 'fast-tests')
-	endif
-endforeach
-
-foreach arg : fast_non_parallel_test_names
-	test(arg, dpdk_test,
-		env : ['DPDK_TEST=' + arg],
-		args : test_args,
-		timeout : timeout_seconds_fast,
 		is_parallel : false,
 		suite : 'fast-tests')
+	endif
 endforeach
 
 foreach arg : perf_test_names
-- 
1.8.3.1