From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 705825586 for ; Thu, 21 Jul 2016 17:06:08 +0200 (CEST) Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8A067C05AA5C; Thu, 21 Jul 2016 15:06:07 +0000 (UTC) Received: from hmsreliant.think-freely.org (unused [10.10.51.23] (may be forged)) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u6LF65mi004436 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 21 Jul 2016 11:06:06 -0400 Date: Thu, 21 Jul 2016 11:06:04 -0400 From: Neil Horman To: "Wiles, Keith" Cc: Neil Horman , Thomas Monjalon , "dev@dpdk.org" , "Mcnamara, John" Message-ID: <20160721150604.GC10032@hmsreliant.think-freely.org> References: <1469034588-1847-1-git-send-email-nhorman@tuxdriver.com> <6330653.K6A9CLrOZy@xps13> <20160720174849.GE28844@hmsreliant.think-freely.org> <20160720201617.GA23515@hmsreliant.think-freely.org> <9309C1B3-D86C-4265-97A4-58E3D7958944@intel.com> <20160721135433.GA9628@hmsreliant.think-freely.org> <9B6D664A-6C02-4982-BBA1-628C94514164@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9B6D664A-6C02-4982-BBA1-628C94514164@intel.com> User-Agent: Mutt/1.6.2 (2016-07-01) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 21 Jul 2016 15:06:07 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH] validate_abi: build faster by augmenting make with job count X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jul 2016 15:06:08 -0000 On Thu, Jul 21, 2016 at 02:09:19PM +0000, Wiles, Keith wrote: > > > On Jul 21, 2016, at 8:54 AM, Neil Horman wrote: > > > > On Wed, Jul 20, 2016 at 10:32:28PM +0000, Wiles, Keith wrote: > >> > >>> On Jul 20, 2016, at 3:16 PM, Neil Horman wrote: > >>> > >>> On Wed, Jul 20, 2016 at 07:47:32PM +0000, Wiles, Keith wrote: > >>>> > >>>>> On Jul 20, 2016, at 12:48 PM, Neil Horman wrote: > >>>>> > >>>>> On Wed, Jul 20, 2016 at 07:40:49PM +0200, Thomas Monjalon wrote: > >>>>>> 2016-07-20 13:09, Neil Horman: > >>>>>>> From: Neil Horman > >>>>>>> > >>>>>>> John Mcnamara and I were discussing enhacing the validate_abi script to build > >>>>>>> the dpdk tree faster with multiple jobs. Theres no reason not to do it, so this > >>>>>>> implements that requirement. It uses a MAKE_JOBS variable that can be set by > >>>>>>> the user to limit the job count. By default the job count is set to the number > >>>>>>> of online cpus. > >>>>>> > >>>>>> Please could you use the variable name DPDK_MAKE_JOBS? > >>>>>> This name is already used in scripts/test-build.sh. > >>>>>> > >>>>> Sure > >>>>> > >>>>>>> +if [ -z "$MAKE_JOBS" ] > >>>>>>> +then > >>>>>>> + # This counts the number of cpus on the system > >>>>>>> + MAKE_JOBS=`lscpu -p=cpu | grep -v "#" | wc -l` > >>>>>>> +fi > >>>>>> > >>>>>> Is lscpu common enough? > >>>>>> > >>>>> I'm not sure how to answer that. lscpu is part of the util-linux package, which > >>>>> is part of any base install. Theres a variant for BSD, but I'm not sure how > >>>>> common it is there. > >>>>> Neil > >>>>> > >>>>>> Another acceptable default would be just "-j" without any number. > >>>>>> It would make the number of jobs unlimited. > >>>> > >>>> I think the best is just use -j as it tries to use the correct number of jobs based on the number of cores, right? > >>>> > >>> -j with no argument (or -j 0), is sort of, maybe what you want. With either of > >>> those options, make will just issue jobs as fast as it processes dependencies. > >>> Dependent on how parallel the build is, that can lead to tons of waiting process > >>> (i.e. more than your number of online cpus), which can actually hurt your build > >>> time. > >> > >> I read the manual and looked at the code, which supports your statement. (I think I had some statement on stack overflow and the last time I believe anything on the internet :-) I have not seen a lot of differences in compile times with -j on my system. Mostly I suspect it is the number of paths in the dependency, cores and memory on the system. > >> > >> I have 72 lcores or 2 sockets, 18 cores per socket. Xeon 2.3Ghz cores. > >> > >> $ export RTE_TARGET=x86_64-native-linuxapp-gcc > >> > >> $ time make install T=${RTE_TARGET} > >> real 0m59.445s user 0m27.344s sys 0m7.040s > >> > >> $ time make install T=${RTE_TARGET} -j > >> real 0m26.584s user 0m14.380s sys 0m5.120s > >> > >> # Remove the x86_64-native-linuxapp-gcc > >> > >> $ time make install T=${RTE_TARGET} -j 72 > >> real 0m23.454s user 0m10.832s sys 0m4.664s > >> > >> $ time make install T=${RTE_TARGET} -j 8 > >> real 0m23.812s user 0m10.672s sys 0m4.276s > >> > >> cd x86_64-native-linuxapp-gcc > >> $ make clean > >> $ time make > >> real 0m28.539s user 0m9.820s sys 0m3.620s > >> > >> # Do a make clean between each build. > >> > >> $ time make -j > >> real 0m7.217s user 0m6.532s sys 0m2.332s > >> > >> $ time make -j 8 > >> real 0m8.256s user 0m6.472s sys 0m2.456s > >> > >> $ time make -j 72 > >> real 0m6.866s user 0m6.184s sys 0m2.216s > >> > >> Just the real time numbers in the following table. > >> > >> processes real Time depdirs > >> no -j 59.4s Yes > >> -j 8 23.8s Yes > >> -j 72 23.5s Yes > >> -j 26.5s Yes > >> > >> no -j 28.5s No > >> -j 8 8.2s No > >> -j 72 6.8s No > >> -j 7.2s No > >> > >> Looks like the depdirs build time on my system: > >> $ make clean -j > >> $ rm .depdirs > >> $ time make -j > >> real 0m23.734s user 0m11.228s sys 0m4.844s > >> > >> About 16 seconds, which is not a lot of savings. Now the difference from no -j to -j is a lot, but the difference between -j and -j is not a huge saving. This leads me back to over engineering the problem when ‘-j’ would work just as well here. > >> > >> Even on my MacBook Pro i7 system the difference is not that much 1m8s without depdirs build for -j in a VirtualBox with all 4 cores 8G RAM. Compared to 1m13s with -j 4 option. > >> > >> I just wonder if it makes a lot of sense to use cpuinfo in this given case if it turns out to be -j works with the 80% rule? > >> > > It may, but that seems to be reason to me to just set DPDK_MAKE_JOBS=0, and > > you'll get that behavior > > Just to be sure, ‘make -j 0’ is not a valid argument to the -j option. It looks like you have to do ‘-j’ or ‘-j N’ or no option where N != 0 > > I think we just use -j which gets us the 80% rule and the best performance without counting cores. > Thats odd, specifying 0 works for me. If it doesn't for you, specify $MAX_INT or some other huge number would be comparable Neil > > > > Neil > > > >> On some other project with a lot more files like the FreeBSD or Linux distro, yes it would make a fair amount of real time difference. > >> > >> Keith > >> > >>> > >>> While its fine in los of cases, its not always fine, and with this > >>> implementation you can still opt in to that behavior by setting DPDK_MAKE_JOBS=0 > >>> > >>> Neil >