From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 74B1D2B99 for ; Mon, 1 Apr 2019 16:07:18 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D5F19308FBAF for ; Mon, 1 Apr 2019 14:07:17 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.61]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5039A5D9C8; Mon, 1 Apr 2019 14:07:15 +0000 (UTC) From: Aaron Conole To: David Marchand Cc: dev References: <20190329172241.11916-1-aconole@redhat.com> <20190329172241.11916-2-aconole@redhat.com> Date: Mon, 01 Apr 2019 10:07:14 -0400 In-Reply-To: (David Marchand's message of "Mon, 1 Apr 2019 15:48:25 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Mon, 01 Apr 2019 14:07:17 +0000 (UTC) Subject: Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2019 14:07:18 -0000 David Marchand writes: Thanks for the review, David! > On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole wrote: > > Some environments do not provide a minimum 4 cores for running tests. This > allows those environments to still execute 'ninja test' without causing > multiple failures. > > Signed-off-by: Aaron Conole > --- > app/test/meson.build | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/app/test/meson.build b/app/test/meson.build > index ddb4d09ae..975b38daa 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -337,10 +337,15 @@ if get_option('tests') > timeout_seconds = 600 > timeout_seconds_fast = 10 > > + # Retreive the number of CPU cores > + num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1] > + num_cores_arg = '-l 0-' + num_cores > > Seeing how we can reduce the core number, we can at least cap it to 4 and not use all the cores on bigger > systems. > But, thinking again, do you know if there is a need for more than 2 cores in the existing tests ? Probably not. At least, the systems in the travis environment only provide 2 cores and most of the tests pass. OTOH, some still fail and I need to investigate them a bit more. Some of the test case failures are ex. eal flags failures and maybe they do pass some '-c f' as part of the test case, which would fail on systems without 4 cores. > + > + test_args = [num_cores_arg, '-n 4'] Whoops! I think, this '-n 4' option can probably be removed as well. I think the memory channel config is optional, and I believe there's no need to try and tune it. > foreach arg : fast_parallel_test_names > test(arg, dpdk_test, > env : ['DPDK_TEST=' + arg], > - args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)], > + args : test_args + ['--file-prefix=@0@'.format(arg)], > timeout : timeout_seconds_fast, > suite : 'fast-tests') > endforeach > @@ -348,6 +353,7 @@ if get_option('tests') > foreach arg : fast_non_parallel_test_names > test(arg, dpdk_test, > env : ['DPDK_TEST=' + arg], > + args : test_args + ['--file-prefix=@0@'.format(arg)], > > I don't understand this part. > > (How|) was it working before ? It did work before. > Is there some default args ? I could not find it. I think it does try to do some auto-detection. I did extend this because it seemed to make sense, and I have some future work to allow passing '--no-huge' (in case we have a CI system that doesn't allow allocating hugepages). Maybe it doesn't make sense to add that right now, though. I'll try without it. > timeout : timeout_seconds_fast, > is_parallel : false, > suite : 'fast-tests') From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 6F8F4A0679 for ; Mon, 1 Apr 2019 16:07:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5B1FF34F3; Mon, 1 Apr 2019 16:07:19 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 74B1D2B99 for ; Mon, 1 Apr 2019 16:07:18 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D5F19308FBAF for ; Mon, 1 Apr 2019 14:07:17 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.61]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5039A5D9C8; Mon, 1 Apr 2019 14:07:15 +0000 (UTC) From: Aaron Conole To: David Marchand Cc: dev References: <20190329172241.11916-1-aconole@redhat.com> <20190329172241.11916-2-aconole@redhat.com> Date: Mon, 01 Apr 2019 10:07:14 -0400 In-Reply-To: (David Marchand's message of "Mon, 1 Apr 2019 15:48:25 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Mon, 01 Apr 2019 14:07:17 +0000 (UTC) Subject: Re: [dpdk-dev] [RFC 1/3] test/meson: auto detect number of cores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190401140714.CFL8-kY4z5--IhFDNOeMDP3vQ0Jt8a0SzCpAImr_EKc@z> David Marchand writes: Thanks for the review, David! > On Fri, Mar 29, 2019 at 6:23 PM Aaron Conole wrote: > > Some environments do not provide a minimum 4 cores for running tests. This > allows those environments to still execute 'ninja test' without causing > multiple failures. > > Signed-off-by: Aaron Conole > --- > app/test/meson.build | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/app/test/meson.build b/app/test/meson.build > index ddb4d09ae..975b38daa 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -337,10 +337,15 @@ if get_option('tests') > timeout_seconds = 600 > timeout_seconds_fast = 10 > > + # Retreive the number of CPU cores > + num_cores = run_command('lscpu', '-p=cpu').stdout().strip().split('\n')[-1] > + num_cores_arg = '-l 0-' + num_cores > > Seeing how we can reduce the core number, we can at least cap it to 4 and not use all the cores on bigger > systems. > But, thinking again, do you know if there is a need for more than 2 cores in the existing tests ? Probably not. At least, the systems in the travis environment only provide 2 cores and most of the tests pass. OTOH, some still fail and I need to investigate them a bit more. Some of the test case failures are ex. eal flags failures and maybe they do pass some '-c f' as part of the test case, which would fail on systems without 4 cores. > + > + test_args = [num_cores_arg, '-n 4'] Whoops! I think, this '-n 4' option can probably be removed as well. I think the memory channel config is optional, and I believe there's no need to try and tune it. > foreach arg : fast_parallel_test_names > test(arg, dpdk_test, > env : ['DPDK_TEST=' + arg], > - args : ['-c f','-n 4', '--file-prefix=@0@'.format(arg)], > + args : test_args + ['--file-prefix=@0@'.format(arg)], > timeout : timeout_seconds_fast, > suite : 'fast-tests') > endforeach > @@ -348,6 +353,7 @@ if get_option('tests') > foreach arg : fast_non_parallel_test_names > test(arg, dpdk_test, > env : ['DPDK_TEST=' + arg], > + args : test_args + ['--file-prefix=@0@'.format(arg)], > > I don't understand this part. > > (How|) was it working before ? It did work before. > Is there some default args ? I could not find it. I think it does try to do some auto-detection. I did extend this because it seemed to make sense, and I have some future work to allow passing '--no-huge' (in case we have a CI system that doesn't allow allocating hugepages). Maybe it doesn't make sense to add that right now, though. I'll try without it. > timeout : timeout_seconds_fast, > is_parallel : false, > suite : 'fast-tests')