From: "Wiles, Keith" <keith.wiles@intel.com>
To: Neil Horman <nhorman@redhat.com>
Cc: Neil Horman <nhorman@tuxdriver.com>,
Thomas Monjalon <thomas.monjalon@6wind.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Mcnamara, John" <john.mcnamara@intel.com>
Subject: Re: [dpdk-dev] [PATCH] validate_abi: build faster by augmenting make with job count
Date: Thu, 21 Jul 2016 15:22:45 +0000 [thread overview]
Message-ID: <64F83F10-008B-4DB0-9662-989A8F57BE80@intel.com> (raw)
In-Reply-To: <20160721150604.GC10032@hmsreliant.think-freely.org>
> On Jul 21, 2016, at 10:06 AM, Neil Horman <nhorman@redhat.com> wrote:
>
> On Thu, Jul 21, 2016 at 02:09:19PM +0000, Wiles, Keith wrote:
>>
>>> On Jul 21, 2016, at 8:54 AM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>>
>>> On Wed, Jul 20, 2016 at 10:32:28PM +0000, Wiles, Keith wrote:
>>>>
>>>>> On Jul 20, 2016, at 3:16 PM, Neil Horman <nhorman@tuxdriver.com> wrote:
>>>>>
>>>>> On Wed, Jul 20, 2016 at 07:47:32PM +0000, Wiles, Keith wrote:
>>>>>>
>>>>>>> On Jul 20, 2016, at 12:48 PM, Neil Horman <nhorman@redhat.com> wrote:
>>>>>>>
>>>>>>> On Wed, Jul 20, 2016 at 07:40:49PM +0200, Thomas Monjalon wrote:
>>>>>>>> 2016-07-20 13:09, Neil Horman:
>>>>>>>>> From: Neil Horman <nhorman@redhat.com>
>>>>>>>>>
>>>>>>>>> John Mcnamara and I were discussing enhacing the validate_abi script to build
>>>>>>>>> the dpdk tree faster with multiple jobs. Theres no reason not to do it, so this
>>>>>>>>> implements that requirement. It uses a MAKE_JOBS variable that can be set by
>>>>>>>>> the user to limit the job count. By default the job count is set to the number
>>>>>>>>> of online cpus.
>>>>>>>>
>>>>>>>> Please could you use the variable name DPDK_MAKE_JOBS?
>>>>>>>> This name is already used in scripts/test-build.sh.
>>>>>>>>
>>>>>>> Sure
>>>>>>>
>>>>>>>>> +if [ -z "$MAKE_JOBS" ]
>>>>>>>>> +then
>>>>>>>>> + # This counts the number of cpus on the system
>>>>>>>>> + MAKE_JOBS=`lscpu -p=cpu | grep -v "#" | wc -l`
>>>>>>>>> +fi
>>>>>>>>
>>>>>>>> Is lscpu common enough?
>>>>>>>>
>>>>>>> I'm not sure how to answer that. lscpu is part of the util-linux package, which
>>>>>>> is part of any base install. Theres a variant for BSD, but I'm not sure how
>>>>>>> common it is there.
>>>>>>> Neil
>>>>>>>
>>>>>>>> Another acceptable default would be just "-j" without any number.
>>>>>>>> It would make the number of jobs unlimited.
>>>>>>
>>>>>> I think the best is just use -j as it tries to use the correct number of jobs based on the number of cores, right?
>>>>>>
>>>>> -j with no argument (or -j 0), is sort of, maybe what you want. With either of
>>>>> those options, make will just issue jobs as fast as it processes dependencies.
>>>>> Dependent on how parallel the build is, that can lead to tons of waiting process
>>>>> (i.e. more than your number of online cpus), which can actually hurt your build
>>>>> time.
>>>>
>>>> I read the manual and looked at the code, which supports your statement. (I think I had some statement on stack overflow and the last time I believe anything on the internet :-) I have not seen a lot of differences in compile times with -j on my system. Mostly I suspect it is the number of paths in the dependency, cores and memory on the system.
>>>>
>>>> I have 72 lcores or 2 sockets, 18 cores per socket. Xeon 2.3Ghz cores.
>>>>
>>>> $ export RTE_TARGET=x86_64-native-linuxapp-gcc
>>>>
>>>> $ time make install T=${RTE_TARGET}
>>>> real 0m59.445s user 0m27.344s sys 0m7.040s
>>>>
>>>> $ time make install T=${RTE_TARGET} -j
>>>> real 0m26.584s user 0m14.380s sys 0m5.120s
>>>>
>>>> # Remove the x86_64-native-linuxapp-gcc
>>>>
>>>> $ time make install T=${RTE_TARGET} -j 72
>>>> real 0m23.454s user 0m10.832s sys 0m4.664s
>>>>
>>>> $ time make install T=${RTE_TARGET} -j 8
>>>> real 0m23.812s user 0m10.672s sys 0m4.276s
>>>>
>>>> cd x86_64-native-linuxapp-gcc
>>>> $ make clean
>>>> $ time make
>>>> real 0m28.539s user 0m9.820s sys 0m3.620s
>>>>
>>>> # Do a make clean between each build.
>>>>
>>>> $ time make -j
>>>> real 0m7.217s user 0m6.532s sys 0m2.332s
>>>>
>>>> $ time make -j 8
>>>> real 0m8.256s user 0m6.472s sys 0m2.456s
>>>>
>>>> $ time make -j 72
>>>> real 0m6.866s user 0m6.184s sys 0m2.216s
>>>>
>>>> Just the real time numbers in the following table.
>>>>
>>>> processes real Time depdirs
>>>> no -j 59.4s Yes
>>>> -j 8 23.8s Yes
>>>> -j 72 23.5s Yes
>>>> -j 26.5s Yes
>>>>
>>>> no -j 28.5s No
>>>> -j 8 8.2s No
>>>> -j 72 6.8s No
>>>> -j 7.2s No
>>>>
>>>> Looks like the depdirs build time on my system:
>>>> $ make clean -j
>>>> $ rm .depdirs
>>>> $ time make -j
>>>> real 0m23.734s user 0m11.228s sys 0m4.844s
>>>>
>>>> About 16 seconds, which is not a lot of savings. Now the difference from no -j to -j is a lot, but the difference between -j and -j <cpu_count> is not a huge saving. This leads me back to over engineering the problem when ‘-j’ would work just as well here.
>>>>
>>>> Even on my MacBook Pro i7 system the difference is not that much 1m8s without depdirs build for -j in a VirtualBox with all 4 cores 8G RAM. Compared to 1m13s with -j 4 option.
>>>>
>>>> I just wonder if it makes a lot of sense to use cpuinfo in this given case if it turns out to be -j works with the 80% rule?
>>>>
>>> It may, but that seems to be reason to me to just set DPDK_MAKE_JOBS=0, and
>>> you'll get that behavior
>>
>> Just to be sure, ‘make -j 0’ is not a valid argument to the -j option. It looks like you have to do ‘-j’ or ‘-j N’ or no option where N != 0
>>
>> I think we just use -j which gets us the 80% rule and the best performance without counting cores.
>>
> Thats odd, specifying 0 works for me. If it doesn't for you, specify $MAX_INT
> or some other huge number would be comparable
rkwiles@supermicro (master):~/.../dpdk/x86_64-native-linuxapp-gcc$ make --version
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
rkwiles@supermicro (master):~/.../dpdk/x86_64-native-linuxapp-gcc$ make -j 0
make: the '-j' option requires a positive integer argument
rkwiles@supermicro (master):~/.../dpdk/x86_64-native-linuxapp-gcc$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial
>
> Neil
>
>>>
>>> Neil
>>>
>>>> On some other project with a lot more files like the FreeBSD or Linux distro, yes it would make a fair amount of real time difference.
>>>>
>>>> Keith
>>>>
>>>>>
>>>>> While its fine in los of cases, its not always fine, and with this
>>>>> implementation you can still opt in to that behavior by setting DPDK_MAKE_JOBS=0
>>>>>
>>>>> Neil
next prev parent reply other threads:[~2016-07-21 15:23 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-20 17:09 Neil Horman
2016-07-20 17:40 ` Thomas Monjalon
2016-07-20 17:48 ` Neil Horman
2016-07-20 19:47 ` Wiles, Keith
2016-07-20 20:15 ` Thomas Monjalon
2016-07-20 20:16 ` Neil Horman
2016-07-20 22:32 ` Wiles, Keith
2016-07-21 13:54 ` Neil Horman
2016-07-21 14:09 ` Wiles, Keith
2016-07-21 15:06 ` Neil Horman
2016-07-21 15:22 ` Wiles, Keith [this message]
2016-07-21 18:34 ` Neil Horman
2016-07-24 18:08 ` Wiles, Keith
2016-08-01 11:49 ` Neil Horman
2016-08-01 16:16 ` Wiles, Keith
2016-08-01 18:08 ` Neil Horman
2016-07-20 19:02 ` [dpdk-dev] [PATCH v2] " Neil Horman
2016-07-22 10:46 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=64F83F10-008B-4DB0-9662-989A8F57BE80@intel.com \
--to=keith.wiles@intel.com \
--cc=dev@dpdk.org \
--cc=john.mcnamara@intel.com \
--cc=nhorman@redhat.com \
--cc=nhorman@tuxdriver.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).