From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 87A93378B for ; Wed, 20 Jul 2016 22:16:32 +0200 (CEST) Received: from cpe-2606-a000-111b-40ed-7aac-c0ff-fec2-933b.dyn6.twc.com ([2606:a000:111b:40ed:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1bPxup-0004GG-ET; Wed, 20 Jul 2016 16:16:24 -0400 Date: Wed, 20 Jul 2016 16:16:17 -0400 From: Neil Horman To: "Wiles, Keith" Cc: Neil Horman , Thomas Monjalon , "dev@dpdk.org" , "Mcnamara, John" Message-ID: <20160720201617.GA23515@hmsreliant.think-freely.org> References: <1469034588-1847-1-git-send-email-nhorman@tuxdriver.com> <6330653.K6A9CLrOZy@xps13> <20160720174849.GE28844@hmsreliant.think-freely.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Score: -1.0 (-) X-Spam-Status: No Subject: Re: [dpdk-dev] [PATCH] validate_abi: build faster by augmenting make with job count X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jul 2016 20:16:32 -0000 On Wed, Jul 20, 2016 at 07:47:32PM +0000, Wiles, Keith wrote: > > > On Jul 20, 2016, at 12:48 PM, Neil Horman wrote: > > > > On Wed, Jul 20, 2016 at 07:40:49PM +0200, Thomas Monjalon wrote: > >> 2016-07-20 13:09, Neil Horman: > >>> From: Neil Horman > >>> > >>> John Mcnamara and I were discussing enhacing the validate_abi script to build > >>> the dpdk tree faster with multiple jobs. Theres no reason not to do it, so this > >>> implements that requirement. It uses a MAKE_JOBS variable that can be set by > >>> the user to limit the job count. By default the job count is set to the number > >>> of online cpus. > >> > >> Please could you use the variable name DPDK_MAKE_JOBS? > >> This name is already used in scripts/test-build.sh. > >> > > Sure > > > >>> +if [ -z "$MAKE_JOBS" ] > >>> +then > >>> + # This counts the number of cpus on the system > >>> + MAKE_JOBS=`lscpu -p=cpu | grep -v "#" | wc -l` > >>> +fi > >> > >> Is lscpu common enough? > >> > > I'm not sure how to answer that. lscpu is part of the util-linux package, which > > is part of any base install. Theres a variant for BSD, but I'm not sure how > > common it is there. > > Neil > > > >> Another acceptable default would be just "-j" without any number. > >> It would make the number of jobs unlimited. > > I think the best is just use -j as it tries to use the correct number of jobs based on the number of cores, right? > -j with no argument (or -j 0), is sort of, maybe what you want. With either of those options, make will just issue jobs as fast as it processes dependencies. Dependent on how parallel the build is, that can lead to tons of waiting process (i.e. more than your number of online cpus), which can actually hurt your build time. While its fine in los of cases, its not always fine, and with this implementation you can still opt in to that behavior by setting DPDK_MAKE_JOBS=0 Neil >