From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <thomas@monjalon.net> Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id AACA32C2B for <dev@dpdk.org>; Fri, 14 Jul 2017 17:54:20 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 586E020BA6; Fri, 14 Jul 2017 11:54:20 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Fri, 14 Jul 2017 11:54:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=OAZzu7c6Dj2EjK6 QIj6WQ1/YzkTWotUF4th8OrGV9jI=; b=r4GwKMcK+3T1Pp+m2ZGIGHRpw27fzuC ETvQdzNgPcAy8GO4SLLByfBslYb3I4FwjG55s6EpML/5mC5JzPH61BDoNIRQ+lYf oUhV3Ot01Xrm4vVbZon4GcxE2xl+TmFMJrxK8xbKUSD1yMYmIvdN3vrYzHXUE6Nu rBxExufrcCEc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=OAZzu7c6Dj2EjK6QIj6WQ1/YzkTWotUF4th8OrGV9jI=; b=XnWZbiGN oCnbMnn/doUlg+zCWL2fBnspv9StUUiMIuWDfY88ydoGOBJOfOqEGu4t2eJT73Eq 0Ak//CypiSVRVgP2LTfI1hbYCS+ajCX7x4u/KPFaKnEd2sLsc5JhvCyUXqzhKlD/ K65AjJEFA+dDZOiS01jh3CLVfGmKNwuMPggT5t4xGtB1Hnw92Ur1gTc1y8BYTfY9 6ss+lquMz4LjiWoveCyak+6Adews+Nhd/rpwcXxtGw6mHNfusiPoR+l2btrb9Lqf Chb9/ph9VSwX4rZML/e3F8MokaOliz9za4Tskgapc6NCTW5hbmVYCblyOiwO/1Rd NWNNfYmLESObNg== X-ME-Sender: <xms:LOloWc474uy9j12xxH4RVNTkgh4fgdDXifjw6QxYifm8TeJssXM4aQ> X-Sasl-enc: wwW15p1gjArFl688j4te0l6gBXqdv0HymseYRxWEOc9Q 1500047659 Received: from xps.localnet (190.84.136.77.rev.sfr.net [77.136.84.190]) by mail.messagingengine.com (Postfix) with ESMTPA id EDB817E4EE; Fri, 14 Jul 2017 11:54:19 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: Stephen Hemminger <stephen@networkplumber.org> Cc: dev@dpdk.org Date: Fri, 14 Jul 2017 17:54:18 +0200 Message-ID: <62513523.FtO1afHum7@xps> In-Reply-To: <12511927.qublii9NrP@xps> References: <20170711185546.26138-1-stephen@networkplumber.org> <7177425.p9cP4Rfg1a@xps> <12511927.qublii9NrP@xps> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH 01/19] devtools: add simple script to find duplicate includes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Fri, 14 Jul 2017 15:54:21 -0000 14/07/2017 17:39, Thomas Monjalon: > 13/07/2017 08:56, Thomas Monjalon: > > 12/07/2017 23:59, Stephen Hemminger: > > > On Tue, 11 Jul 2017 22:33:55 +0200 > > > Thomas Monjalon <thomas@monjalon.net> wrote: > > > > > > > Thank you for this script, but... it is written in Perl! > > > > I don't think it is a good idea to add yet another language to DPDK. > > > > We already have shell and python scripts. > > > > And I am not sure a lot of (young) people are able to parse it ;) > > > > > > > > I would like to propose this shell script: > [...] > > > > > plus shell is 7x slower. > > > > > > $ time bash -c "find . -name '*.c' | xargs /tmp/dupinc.sh" > > > real 0m0.765s > > > user 0m1.220s > > > sys 0m0.155s > > > $time bash -c "find . -name '*.c' | xargs ~/bin/dup_inc.pl" > > > real 0m0.131s > > > user 0m0.118s > > > sys 0m0.014s > > > > I don't think speed is really relevant here :) > > I did my own benchmark (recreation time): > > # time sh -c 'for file in $(git ls-files app buildtools drivers examples lib test) ; do devtools/dup_include.pl $file ; done' > 4,41s user 1,32s system 101% cpu 5,667 total > # time devtools/check-duplicate-includes.sh > 5,48s user 1,00s system 153% cpu 4,222 total > > The shell version is reported as faster on my computer! > > It is faster when filtering only .c and .h files: > > for file in $(git ls-files '*.[ch]') ; do > dups=$(sed -rn "s,$pattern,\1,p" $file | sort | uniq -d) > [ -z "$dups" ] || echo "$dups" | sed "s,^,$file: duplicated include: ," > done > > # time sh -c 'for file in $(git ls-files "*.[ch]") ; do devtools/dup_include.pl $file ; done' > 3,65s user 1,05s system 100% cpu 4,668 total > # time devtools/check-duplicate-includes.sh > 4,72s user 0,80s system 153% cpu 3,603 total > > I prefer this version using only pipes, which is well parallelized: > > for file in $(git ls-files '*.[ch]') ; do > sed -rn "s,$pattern,\1,p" $file | sort | uniq -d | > sed "s,^,$file: duplicated include: ," > done > > 7,40s user 1,49s system 231% cpu 3,847 total And now, the big shell optimization: export LC_ALL=C Result is impressive: 2,99s user 0,72s system 258% cpu 1,436 total I'm sure you will agree to integrate my version now :)